main
56
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Rayhaan JaufeerallyandClaude Fable 5.1
|
db8edaae8e |
policy, config, web: the TCP gauges named once, one revalidator, one peer-naming rule
A code-quality pass over policy, configuration and the status API. The twenty TCP gauges were spelled out in the struct, the constructor, the removal, the sampler in the web crate and the status view; one macro names each once with its help text, `PeerTcp::set` fills them from a connection, and the view is a `From`. Every metric and the list that forces them at start come from one declaration, with a test that every declared family is registered. The two revalidators, for next hops and for RPKI, share the task body that scans the table off the sessions' threads and fans the findings out to every session; the RPKI scan judges a route with the same `Checks` the import path uses. Which peer a query names is one rule for the received, rejected and TCP endpoints, so `/api/tcp` takes a name or an address like the others. The config parser says proto3's "zero is unset" in three helpers instead of a dozen shapes, and `parse_peer` is three named parts shorter. The policy's test-only next-hop flag is named for what it does, `accept_own_host_next_hop`; the "listed, or else ours" prefix rule and the mark-then-prefer step each exist once. The status page derives its spanning rows from the table's header, shares one key-value grid and one detail-row rule, and defines its helpers before their use. The web crate no longer re-exports views nothing uses, and the TCP endpoint sits above the test module. Testing: the TCP view's arithmetic (a share of no busy time is zero, not NaN), the peer-naming rule, the RPKI scan finding only what a change can have made invalid, and `/api/tcp` by address, by name and for an unknown peer. The server and web tests, the HTTP, roles, RPKI and import-policy integration tests, the docs build and every lab phase pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
4cddd6fb04 |
export: one rule each for admission, identifiers and what the peer had
A code-quality pass over the exporter. The resync path (gather then stream) had grown beside the steady-state path (judge then flush) with its own copies of the admission predicate, the path identifier rule, the reconciliation of a route the peer already had, the parser contexts, the withdrawal loop and the advertised gauge. Each is one function now: `prefix_may_go`, `lowest_free_id`, `Sent::renewed`, `parser_contexts`, `send_withdrawals`, `within_limit` and `note_held`. The 3,072-octet ceiling on a route's attributes, which is what keeps an UPDATE within its size, is a named constant where it is relied on. The UPDATEs are framed with the packet crate's helpers instead of a bare type code and a length written by hand. The positional tuples for a prefix's paths are structs with names. A mismatched kind of bookkeeping, which nothing should produce, is asserted in tests and logged in release instead of dropped silently. Types used only inside the crate are `pub(crate)`. Five doc comments that had drifted onto neighbours or named former designs are put right; the exporter's reference encoder no longer breaks the CI doc build with a link to an item that exists only under test. Testing: ADD-PATH export has unit tests at last: two routes for a prefix go with their own identifiers and keep them, and the withdrawal of one carries its identifier; what is not heard of again after a reset is withdrawn after the table. The byte-for-byte test of the encoded UPDATEs covers IPv4 routes with an IPv6 next hop (RFC 8950) too. The test fixtures are shared helpers. The server tests and the ADD-PATH, transit, limits, roles and deadlock integration tests pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
bef875e50b |
session: the way out to the peer as a type of its own
A code-quality pass over the per-peer session. The outgoing side of a session, eight fields served from six places in a two-thousand-line file, is `Outgoing`: the write buffer, the batches waiting for the writer task, the task and its word on why it ended. One real race went with it: when the writer task failed with the queue full, the session took whichever of two ready branches the select chose, and half the time ended with an invented send hold timer expiry instead of the connection's error, which decides whether the peer's routes are kept over a restart; the writer's own word is taken now. The run loop's handling of what the rest of the server has to send is a method that goes by the state machine's state, so that pieces after a NOTIFICATION in the same batch do not follow it out; `execute_all` reads one line an action with the sending, delivering and after-failure steps named. A peer's routes have one way onto the table at a session's end, whether removed or kept for a restart. An UPDATE whose attributes cannot be kept ends the session under an error of its own, `Unstorable`, where a message of ours that cannot be encoded is dropped. What is kept of a restarting peer is a value, not seven arguments. The dialer no longer turns a session's end into a Result and back. The server's screening of a connection is called that, apart from the collision check's `admit`. Testing: the back-pressure machinery has unit tests at last: batches wait while the writer has no room, room goes to the oldest batch, a piece is counted as the messages in it, and a lost connection is reported as one while batches wait. The routes module gains tests of holding at its bound, forgetting and revalidation. The server tests and the whole integration suite pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
bcfd6e013f |
rib: the lock-free number in a slot of its own, and the reader in named steps
A code-quality pass over the table. The latest sequence number of each journal, which readers compare their cursors with without the shard's lock, was an atomic inside the journal under the lock with the table holding the other end of an Arc to it, stored on every route; it is now in a slot beside each shard's lock, `ShardSlot`, stored once a shard a change while the write guard is still held and before the readers are woken. "One of each kind of journal" is one type, `ByKind<T>`, where three hand-written dispatchers and an array index were. `Reader::poll` is the loop skeleton over `resync_step` and `follow_pass`; the journal's `tidy` is three named stages; `live`'s nested Option is an enum that says what an entry says; the place of a cursor in a journal is found by one function for both the time chosen for a pass and the pass itself. The attribute set's four byte-readers share one walk over the raw attributes, with the next hop decoded once for both of its readers. `best` returns a `Route` like the other lookups. Docs that described an earlier design (the reader's `seen`, "every method takes &self", a `Routes::is_empty` that could not be true) are put right. Testing: new tests pin the lock-free skip, which nothing checked directly before: the number equals the journal's after every kind of change, with and without readers of each kind; a shard passed over is read again once it has news, from a reader of the whole table and from one that starts now. The OTC flag and the received-attributes interning gain tests. The crate's tests, both proptests included, the server tests, the integration tests, the kernel tests and every lab phase pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
c70a223d0d |
rib: a reader counts the table once a poll, not once a shard
A router with many peers has a reader of the table's journal for each session that is sent learned routes, seventy on ddln, and every change to the table wakes them all. Each pass of a reader went round every shard, and for each shard took a count on the table (an Arc clone) to hold it while the shard was read, and gave it back after: two atomic adds on one cache line, by seventy readers, 256 shards a pass. A profile of ddln taking a table from the other router had the reader's poll as the top function at two to three cores, and in the Linux VM with 64 such sessions half of all CPU time was in the two atomic add instructions. The sessions writing to the table waited behind it: the table came in at 58,000 routes a second. The count is now taken once a poll. And each journal keeps its latest sequence number in an atomic the table holds the other end of, so that a reader can tell a shard with nothing new for it from one with something without the shard's lock, and pass over it: on a quiet table, which is what the readers see most of the time, a pass is a load a shard and no lock. An exporter that is sent little of a table, as one to a peer at an exchange is, no longer makes a verdict of nothing for a prefix the peer does not hold, which cost a map insert and a lookup on the way out, and tells a route that came down or across (RFC 9234) from its attributes' bytes before parsing them. Testing: the mesh benchmark can have any number of collector sessions of the first router (BENCH_COLLECTORS): peers in the sense of RFC 9234, sent learned routes, with the feeders' routes marked as having come across from a peer, so that each has a reader following the table while it fills and passes nearly all of it over, as a router with many peers at exchanges has. In the Linux VM with 64 of them, three rounds alternating with the previous commit, the routers' exchange of their tables takes 11 s instead of 20 s, at 86 s of CPU time instead of 155 s; without them it is unchanged. The table's tests, the server tests, the integration tests, the kernel tests and every lab phase pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
a15efaa26c |
export: give the leave to gather back once the table is gathered
The last change held a session's leave to gather a table, of which there are four, until its table had gone out. A peer that reads at the pace of its receive window takes hours over a table, and two such peers held two of the four for that long, keeping other sessions from gathering theirs: they waited for a leave before their first UPDATE. The leave is for the memory a gather takes; what is still held once the table is gathered is let go of as it goes out, as it was before the gather was reworked. The mesh benchmark's new peer can also come once the first router has its own peers' routes and the other router's are still arriving (BENCH_RECEIVER_EARLY=2), as a router's sessions come up after a restart while its peers' tables arrive; the walk of a full table while the table fills costs the arriving routes nothing measurable in the VM. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
b5ca42244d |
export: a lighter gather, and UPDATEs to the session by the piece
Two things kept an exporter from its peer. A table being gathered to go out anew kept a map of every prefix heard, to let a later word on a prefix supersede an earlier one, and dressed every set of attributes before the first UPDATE went: a map insert and a parse-and-encode a route that the peer waited on. And the session's queue took UPDATEs one by one, a thousand at most, so the exporter stood at the queue once a hundred kilobytes were in it, behind a writer that takes a megabyte: the two took turns instead of running together. The journal gives a table out shard by shard, each prefix once, and a change meanwhile comes after the table as news of its own, so there is no second word to keep track of while gathering: the map is gone, and a prefix the peer is not to have is simply not gathered, as what it held of it is withdrawn once the table is through. Each set is dressed as its turn to go comes, so the first UPDATE goes once the table is gathered; only when the gathered routes alone would be over the peer's limit are the sets dressed first, to know which cannot be and take their routes with them, so that none goes if too many would. The UPDATEs for a set now go to the session in pieces of 64 KiB or so, each of whole messages, and the queue takes 64 of those: a few megabytes ahead of the writer, not a hundred kilobytes. The session counts the messages in a piece by their headers. Testing: the byte-for-byte test of the encoded UPDATEs now encodes thirty thousand routes and checks the pieces are of whole messages and of the size meant. The server tests, the integration tests, the kernel tests and every lab phase pass. In the Linux VM, three rounds alternating with the previous commit, a new peer's first UPDATE comes 1.8 s after its session is up instead of 3.1 s, and it has its four million routes in 3.8 s instead of 4.1 s; the smaller of the two routers' tables crosses in 3.0 s instead of 4.2 s and the larger in 5.0 s instead of 5.2 s, at the same CPU time. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
7425b101ed |
export: gather a table by attributes, then stream it
A table given out anew to a session, at its start or when the journal had let changes go, was gathered as a verdict for each prefix, each with the route dressed for the session, and sent once the table was all there. Dressing a route parses its attributes and encodes them for the session, so a table of five million routes took seconds before the first UPDATE went, on one worker thread without a yield. And the routes that share attributes went out in separate UPDATEs after all: they were grouped by the dressed form, of which only so many are kept, and the routes of a set are spread over the whole table, so by the time the next was reached the form had been let go of and made anew. The mesh benchmark's new peer got four million routes in 3.8 million UPDATEs; on a RIPE RIS table, grouped whole, a full table is 160,000 UPDATEs and 21 MB, and grouped in slices of 4,096 routes it is 845,000 UPDATEs and 72 MB, so slices are not the answer either. The table is now gathered by what dresses its routes: the attribute set as the table has it, who the route is from, and its IP version. Nothing is dressed while gathering, which is a map insert a route. Once the table is all there, each set is dressed once, with the thread lent to the other tasks every few thousand, and then the sets go out one after the other, each as few UPDATEs as its routes fit in, with the peer reading the first while the rest are sent. A prefix heard of again while the table is read supersedes what was heard before: each word on a prefix is numbered, and only the latest goes. Path identifiers are given at gathering, in the order the table has a prefix's routes, so that they are as before. What was announced before and not heard of again is withdrawn after the table, as before. The exporter never blocks the thread on a table: the steady state's batch is small, and the gathered table is gone through with yields. Testing: a new test gives a session a table of 300 routes sharing three sets of attributes, spread over the shards, and checks they go in three UPDATEs. The mesh benchmark has a new phase in which a peer takes the first router's table once the mesh is complete, timing the first UPDATE and the whole; its feeders now negotiate no hold timer, since they say nothing after their tables, and `BENCH_LOG` prints the routers' logs. A new example, packing, counts the UPDATEs and octets a RIS table takes grouped whole and in slices. The server tests, the integration tests, the kernel tests and every lab phase pass. In the Linux VM, three rounds alternating with the previous commit, the new peer's first UPDATE comes 2.9 s after its session is up instead of 5.3 s, and it has all four million routes in 4.0 s instead of 8.0 s, in 667,000 UPDATEs instead of 3.76 million; the whole run takes 22.4 s of CPU time instead of 26.2 s. The routers' exchange of their tables, which does not go through this path once their sessions are up, is unchanged within its noise. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
92b9bdd3c5 |
export: keep the peer reading while the next batch is dressed
An exporter following the table took up to 65,536 changes at a time, dressed them all, and only then sent them; and before each batch it slept the 50 ms that lets a run of changes be gathered, whether or not the table was already far ahead of it. The session's queue holds about a thousand UPDATEs, so once the batch went out the exporter waited on the peer reading it, and while the next batch was dressed the peer had nothing to read. A profile of two routers exchanging their tables, sampled in 25 ms windows, showed the two stages taking turns: the exporters at two cores while the sessions were at none, then the other way round, with a gap of 50 to 75 ms at the start of each cycle for the sleep. Each task ran at about half a core. A batch is now 4,096 changes, so that what one batch sends is being read by the peer while the next is being dressed, and the gathering sleep is skipped while the reader is behind the table: a run is in hand already. A reader says whether its last poll caught up. Testing: the server tests, the integration tests, the kernel tests and every lab phase pass. On the mesh benchmark in the Linux VM, three rounds each alternating with the previous commit, the larger table (4.9 million routes) is across in 5.6 s instead of 6.7 s and the smaller (2.5 million) in 4.4 s instead of 6.4 s; the whole exchange takes 7.3 s of wall-clock time instead of 8.6 s, and 18.9 s of CPU time instead of 20.5 s. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
381eabd886 |
export: encode a route's attributes once a session
The exporter built each UPDATE as a message: the attributes of the route's set, parsed from the table, were cloned, had the next hop added, were sorted, and were handed to the session, which encoded them to the wire with the rest of the message. For a table whose attribute sets each cover a few routes that is an encoding of the same octets for nearly every route, and a copy of the message's bytes on the way to the writer besides. A profile of a production router taking the other router's table with ADD-PATH had the exporters at 55% of the CPU time, their dressing of routes at 23%, and memcpy at 12%. A route's attributes are now encoded once for a session, when it is first dressed for it, and kept with where in the octets a NEXT_HOP or an MP_REACH_NLRI belongs so that the attributes stay in order of type code. An UPDATE is then its header, the encoded octets with the next hop or the MP_REACH_NLRI spliced in, and the prefixes: written straight into a buffer that goes to the session as it is, and from the session to the writer without a copy. The session's queue takes either a message or an encoded UPDATE; an encoded one for a session that is no longer established is dropped, as the exporter made it for the session as it was. What is kept for a route, now a few hundred octets instead of a kilobyte, lets twice as many be kept. The exporter's maps hash with foldhash instead of SipHash, and the parser context for an UPDATE is made once a flush instead of once an UPDATE. Testing: a new test encodes a thousand routes with each of the forms an UPDATE takes (IPv4 with the next hop in its own attribute, IPv6 in MP_REACH_NLRI, with and without path identifiers, to two-octet and four-octet AS speakers, over several messages) both ways and checks the octets are the same byte for byte; the message-building form is kept for the test. The exporter tests decode what the session is handed. The server tests, the integration tests, the kernel tests and every lab phase pass. On the mesh benchmark in the Linux VM, three rounds each alternating with the previous commit, the two routers' exchange of 7.4 million routes takes 20.6 s of CPU time instead of 25.4 s (-19%) over 9.0 s of wall-clock time instead of 11.1 s, and the larger of the two tables is across in 7.1 s instead of 9.3 s. The smaller table's time varies too much from run to run (5-8 s) for a difference to be read from it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
cbc1dc7e44 |
bench: two routers exchanging their tables with ADD-PATH
The full-table benchmark measures one router taking in tables from its peers. What two routers of one AS do with each other, each sending the other every path it has (ADD-PATH both ways with all_paths, as a mesh of routers that each hold the network's whole view is configured), was measured nowhere: the exporter following the table's journal while the sessions fill the table, at the size of the Internet. In production that exchange took 70 seconds for a table that arrives from the Internet in 20. A new benchmark, mesh, starts two servers in one AS with distinct router IDs, peers them over IPv6 loopback with ADD-PATH both ways and all_paths, and feeds the first from four synthetic peers and the second from two over IPv4 loopback (BENCH_PEERS, BENCH_PEERS_B, BENCH_SCALE_DOWN). It prints how long each router takes to hold its own peers' routes and then the other's, with the rate of the latter. BENCH_DEADLINE prints the sessions' state if a router is not there in time. The benchmarks now allocate with jemalloc, as the daemon does, except when memory is being counted: the C library's allocator put its own locks in the profiles. Two routers on one host have each other's addresses as next hops, loopback addresses, which the import check refuses as it refuses a next hop that is this machine; the permissive import policy, which only tests use and no configuration can ask for, now takes them. Testing: the next-hop test covers both policies; the benchmark runs on macOS and in the Linux VM under perf. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
d0f092ff06 |
session: write to the peer from a task of its own
A session read from and wrote to its socket in one loop: while it was writing, it was not reading. Writing waits when the peer's receive window is closed, and the peer's window closes when the peer is not reading, which it is not while it is writing to us and waiting on our window in turn. Two routers sending each other every path at once, as they do with ADD-PATH both ways after a restart, each stopped reading within seconds and waited on the other until the send hold timer ended both sessions eight minutes later. The status page's TCP row showed it: both windows zero, the send queue full, the receive queue unread. The batching of writes in the previous change made a session spend far longer writing and so made this certain; it was possible before. Now each session has a writer task. The session encodes its messages into batches and hands them to the task through a bounded queue without ever waiting: a batch the queue has no room for is kept, and while any is kept the session takes no more messages from the rest of the server (which then waits, as before, in its own task) but goes on reading the peer. The writer task's failure, or a write that makes no progress for the send hold time, comes back to the session as a local failure and ends it. When the session ends, what is still to go out, a NOTIFICATION among it, is handed over and the writer shuts our side once it has written it. Testing: a new integration test has one peer's table in the server and a second peer send a table of its own while reading nothing of what the server sends it, and checks that the server takes the second table all the same; against the previous commit the test hangs until its timeout. The send hold timer test now reads the writer's failure. The session tests, the integration tests, the kernel tests and every lab phase pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
9276bc9f7a |
session: write messages in batches, with TCP_NODELAY
A session wrote every message to its socket as it came: one write and, with Nagle's algorithm, mostly one segment for each UPDATE, a hundred or two octets. On the IPv6 cross-connect between the two routers, whose round trip is 50 microseconds, a million routes took 42 seconds to go out: the status page's new TCP row showed the connection busy for that long with neither window nor buffer limiting it, a congestion window stuck at ten segments, and a smoothed round-trip time of 18 ms. That is the receiver's delayed acknowledgements, 40 ms at a time, on a stream of small segments, holding the sender to a few segments a round trip. Now a session gathers what is waiting for it, up to the outbound queue's length, in one wake, encodes the messages into one buffer, and writes the buffer when it reaches 64 KiB or when the actions of the wake are done; the socket has TCP_NODELAY, so a batch's last, partial segment is not held back for the acknowledgement of the one before. The NOTIFICATION that ends a session, including one sent after a panic, is flushed on its own. Measured in the Linux VM with four peers' tables going to a collector over a veth pair, 151 MiB: 121,850 segments instead of 313,773, the connection busy 0.8 s instead of 2.0 s, and the congestion window grown to 47 segments instead of 13. Over loopback the transit benchmark is unchanged at 2.5 s for the second neighbor to have the table: there the receiver's parsing sets the pace, not the segments. Testing: the session tests, including the one that sends a NOTIFICATION after a panic; the integration tests; the kernel tests and every lab phase. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
b0dd357986 |
export: a kilobyte less a route advertised with all_paths; memory accounted for
A router feeding a collector every path (all_paths) was 4.8 GB resident
with a table that should take a third of that. A heap profile of the
daemon in the Linux VM, with four full tables and such a collector,
showed where it went: the exporter kept, for every route it had
advertised, the route's outgoing form, the parsed attribute list it was
dressed with, through an `Arc<Dressed>` in the per-route record. One
such form lived for every distinct attribute set advertised, about a
kilobyte each, for the life of the session, on top of the exporter's
cache of 262,144 of them. The record needs only to tell whether a route
changed, and the table's own attribute set, which is in memory anyway,
tells it that: the outgoing form is a function of the set, the peer the
route came from and the policy.
This change:
* `Sent` holds an `Arc<AttrSet>` instead of an `Arc<Dressed>`, and a
route is sent again when its set differs. `Verdict::All` carries both
while a flush is on.
* `DRESSED_KEPT` goes from 262,144 to 65,536: a kilobyte a set, so 64 MB
rather than 256 MB a session at most; what a flush is sending holds
its own.
* The status page's header shows the daemon's memory as jemalloc
accounts for it (allocated and resident), `/api/status` has the six
figures (`memory`: allocated, active, metadata, resident, mapped,
retained), and `bgp_memory_bytes{kind}` exports them. jemalloc is built
with statistics for this, and `bgp-web` reads them through
`tikv-jemalloc-ctl`.
* `scripts/profile/full-table.sh` gains `COLLECTOR=1`, a collector
session over IPv6 that takes both families with ADD-PATH, and `HEAP=1`,
which builds the daemon with jemalloc's heap profiler (the new
`heap-profiling` feature of the daemon crate, not for production) and
turns the profile into a flame graph with jeprof; and it reports
memory with the table in and again 30 s later. The feeder
(`crates/bench/examples/feed.rs`) plays the collector with
`FEED_COLLECTOR=1`.
Measured in the VM, four peers' full tables and a collector sent every
path (4.9 million routes advertised): 1,723 MiB resident before, 1,286
MiB after, with jemalloc's allocated figure from 1,500 MiB to 946 MiB.
Without the collector the same tables take 630 MiB, so the collector's
bookkeeping is now about 50 bytes a route advertised. Of the resident
figure, some 400 MiB was pages jemalloc had freed and not yet returned:
it returns them on allocator activity only, and a router that has its
tables in allocates little. jemalloc is now built with its background
thread on (the `background_threads` feature), which returns them on
time: 1,020 MiB resident 30 s after the load, against 1,286 MiB without
it.
Documentation: `memory` in the HTTP API reference, `bgp_memory_bytes`
in the metrics reference and the monitoring guide, the script's new
settings in the testing guide.
Testing: the ADD-PATH integration tests pass, including the collector
ones; the heap profile and the figures above are from
`PEERS=4 COLLECTOR=1 HEAP=1 scripts/profile/full-table.sh` in the VM.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
d7d7d39bb9 |
metrics: export each session's TCP connection figures
The kernel's view of a session's TCP connection, which the status page shows on request since the last change, could not be looked at after the fact: a feed that crawled overnight left no record of whether the peer's window or our own side held it up. This change samples it at every scrape of `/metricz`: for each established session, one `NETLINK_SOCK_DIAG` round trip, and twenty gauges labelled by peer, `bgp_peer_tcp_*`: smoothed, least and variance of the round-trip time; the congestion window and segment size; the peer's receive window and ours; the send queue, the part of it not yet sent, the receive queue and the segments in flight; retransmitted and lost segments; the busy time and the time held up by the peer's window and by our send buffer, cumulative, so that their rates give the share of time a session is held up; octets sent and received; and the delivery and pacing rates. A session the kernel has no connection for loses its series, as does one that ends. `SessionHandle::metric_label` is the one place a peer's metric label is made (its name, or its address), used by the advertiser's gauge and here. Documentation: the metrics reference lists them; the monitoring guide gives the query for the share of time a session is held up by the neighbor's receive window, which is the question the overnight feed raised. Testing: a daemon in the Linux VM with one session shows the series and drops them when the session ends; the lab's basics phase checks the scrape for the FRR session's round-trip time; the workspace tests pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
9b211b61e9 |
export: count routes as they go out, and say when the table is walked again
The number of routes advertised to a peer was written once a flush was over, after every UPDATE of it had been handed to the session. The first flush to a route collector is the whole table, two million routes or more to a peer that reads at the pace of the Internet, so the status page showed the peer holding one route for as long as that took, which read as nothing going out. Now the count is written after each batch of a flush, so that the page and `bgp_peer_advertised_routes` show the table going out. Also, an exporter that had the whole table and is given it again, because the table changed faster than the exporter read it and the journal let the oldest changes go, now logs that it is going through the table again, at info. Until now that was invisible, and a collector session during a full-table load from several peers is where it can happen: the exporter is held up sending while the shards' journals, cut back to twice their rows, run past its place. Testing: a new integration test has a collector connected before two feeders send 40,000 prefixes each, and checks it is told all 80,000 routes with their identifiers, which takes the exporter's initial walk of the table through many batches; the advertiser and ADD-PATH tests pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
ec5c9be454 |
export: advertise every route for a prefix with ADD-PATH (all_paths)
Before this change, a session on which ADD-PATH (RFC 7911) was
negotiated in the sending direction was sent path identifiers, but only
the best route for each prefix. A route collector such as bgp.tools
wants every route a router has accepted, from every neighbor, and from
one session: without this the collector saw one path per prefix, nearly
always the same neighbor's.
This change adds `all_paths` to the export policy. On a session with it,
and ADD-PATH send negotiated for a family, every accepted route for a
prefix of that family is advertised, each with a path identifier of its
own; for a family without ADD-PATH send the best route goes as before.
`all_paths` needs `learned_routes` (a configuration error otherwise) and
the peer's `add_path` to offer sending. The export policy, the role
rules, the well-known communities and the export limits apply to every
route alike, and the export limits count every route.
Implementation, in the advertiser:
* An exporter with `all_paths` follows the table's journal of all routes
(`RibTable::reader_of_all_routes`, until now unused) instead of the
best-route journal, and is given all the routes of a prefix whenever
one of them changes (`News::Routes`).
* What the peer holds is `Announced`: the set of prefixes for a
best-route session, as before; or for an all-paths session, for each
prefix the routes announced, keyed by the peer they came from and that
peer's own identifier, with the identifier they went out with and the
form they went out in. A route keeps its identifier for as long as it
is announced, goes out again only if it changed, and is withdrawn by
its identifier; a new route gets the lowest identifier not in use for
the prefix, from 1.
* Verdicts carry either kind (`Verdict::Best`, `Verdict::All`), and a
reset of the table puts what was announced in doubt until heard of
again, as before, with the same identifiers kept across the reset.
* The record for an all-paths session is about 50 bytes a route, some
tens of megabytes for a full table from several neighbors; the
best-route bookkeeping is unchanged in size.
Also, so that a collector session can be watched: the number of routes
advertised to each peer and not withdrawn since, every route counted
where several are sent for a prefix. The exporter keeps it in the
session's counters after every change; it is `advertised_routes` in
`/api/peers`, the "sent" column of the status page's neighbor table,
and the gauge `bgp_peer_advertised_routes{peer}`.
Documentation: a "Feeding a Route Collector" section in the guide to
passing routes on, with the configuration for a collector; `all_paths`
in the configuration reference; the ADD-PATH sections of the neighbor
guide and the standards reference; the limitation that was removed;
the new field, column and metric in the HTTP API, monitoring and
metrics references.
Testing: an integration test with two feeders and a collector that
negotiated ADD-PATH receive, which checks the two routes with their
identifiers, the withdrawal of one by its identifier, a changed route
keeping its identifier and a new route taking the free one; a
configuration test for `all_paths` without `learned_routes`; a new
`collector` phase of the interoperability lab in which FRR, as a route
collector, is told both of BIRD's routes for a prefix from BIRD's two
sessions and holds them apart, and loses one when its session is
disabled, with the advertised count checked at each step. The workspace tests, the kernel tests and every lab phase pass.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
48da8cc7d2 |
rib: size the routing table for the routes the peers are expected to send
The routing table's shards started empty and grew as routes came in, each growth moving every row of the shard into a table twice the size. In a profile of the daemon taking in a full table, the growth showed as up to a seventh of the session's time, and the daemon's resident memory had the old and the new table of each shard side by side for a moment. This change adds `RibTable::expect_prefixes`, which reserves room for that many prefixes spread over the shards, and has the server call it at startup with the largest `expected_routes` among the configured peers: the prefixes of several full tables are mostly the same ones. Without `expected_routes` nothing changes. The `full_table` benchmark does the same unless `BENCH_PRESIZE=0`. Measured, the saving in time is within the run-to-run noise: in the Linux VM one peer's table is in the routing table in 0.81 s instead of 0.89 s, and the kernel is in step at about the same time as before; three pairs of runs of the benchmark on a Mac average 0.33 s against 0.35 s. The profile's attribution to the growth was mostly sampling skid. The memory saving is real: 442 MiB resident with one peer's table in instead of 483 MiB, and 626 MiB instead of 632 MiB with four. Also: the profiling script printed nothing for the kernel's change counts because it asked `/metrics` for them; the endpoint is `/metricz`. With that fixed, a four-peer load shows 1.25 million installs for 1.23 million prefixes: the journal already gives the kernel task only the latest change of a prefix when it lags, so no further coalescing of installs is called for. Documentation: `expected_routes` in the configuration reference, the neighbor guide and the proto comment say that the table is sized by it; the testing guide's VM results are brought up to date. Testing: a table test checks that the room is there and is kept; the workspace tests pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
6d8f89e509 |
policy, fib: blackhole routing for routes with the BLACKHOLE community
Before this change, a route carrying the BLACKHOLE community (65535:666,
RFC 7999) from an EBGP neighbor was always rejected with the reason
"blackhole request", and nothing discarded traffic. A customer under a
denial-of-service attack had no way to ask this router to drop traffic
to an address.
This change adds `accept_blackholes` to the import policy. When it is
set, a route with the BLACKHOLE community is accepted if:
* its prefix is a single address, a /32 or a /128 (otherwise it is
rejected with the new reason "blackhole for more than one address");
the `max_ipv4_prefix_length` and `max_ipv6_prefix_length` limits do
not apply, as RFC 7999 section 3.2 recommends;
* the address is inside a prefix the neighbor may announce, if
`prefixes` is configured, regardless of the entry's `max_length`
(new `PrefixSet::covers`); without a prefix list, an address inside
the local AS's own prefixes is rejected as before ("prefix is our
own");
* the AS path, origin and RPKI checks that apply to every route pass.
The next hop of an accepted blackhole route is not checked for
third-party addresses and is not resolved, because traffic is never
forwarded to it. The route takes part in best path selection as usual;
a host route is normally the active route for its address.
The kernel task installs the active route as a kernel blackhole route
(`RTN_BLACKHOLE`, shown by `ip route` as `blackhole PREFIX proto bgp`).
Blackhole routes left by an earlier run are adopted at startup like
other routes of ours. The route is advertised to other neighbors as the
export policy and export limits permit, with the community intact, so a
provider that honors BLACKHOLE can discard the traffic upstream.
Implementation:
* `plan::Target` becomes an enum: `Via { gateway, interface }` or
`Discard`. `plan::resolve` takes a `blackhole` flag and returns
`Discard` for such routes without looking at the next hop (the
own-network check still applies). No nexthop object is made for a
discard target.
* `AttrSet::is_blackhole` reads the community off the stored attribute
bytes, like `next_hop` and `as_path`, so the kernel task, the
revalidator and the RPKI revalidation can ask it of every route.
* `ImportPolicy::is_blackhole_request` and `check_blackhole_prefix`
carry the import side; the import flow (`routes.rs`) computes once per
UPDATE whether its routes are an accepted blackhole request and skips
the next hop checks for them.
* `Status::blackholes` and the `blackholes` field of `/api/kernel` count
the installed blackhole routes; the status page shows the count next
to the installed routes.
Not covered: a prefix this router originates with the BLACKHOLE
community is advertised but not installed as a blackhole route. The
documentation says to add a kernel blackhole route by hand for that
case.
Documentation: the "Blackhole Routes" section of the import policy
guide replaces "Blackhole Requests", the kernel guide gains a "Blackhole
Routes" section, the configuration reference describes
`accept_blackholes`, the HTTP API reference describes `blackholes`, and
the RFC 7999 rows of the standards reference are updated.
Testing: unit tests for the attribute check, the prefix checks and the
plan; integration tests that a blackhole request is refused by default
and, with `accept_blackholes`, taken for one address inside the peer's
space and refused otherwise, with the recorded reasons; a kernel test
that a blackhole request becomes a kernel blackhole route whatever its
next hop, is removed on withdrawal and is adopted after a restart. The
existing kernel tests, the interoperability lab and the workspace tests
pass.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
3608897245 |
fib: resolve next hops through the kernel's routing table
Before this change, Ferrix only installed a route if its BGP next hop was
directly connected: an address in the subnet of an interface that is up.
A route with any other next hop was held at import and never installed.
This made routes from iBGP neighbors unusable unless the neighbor set
itself as the next hop, and it is why the route reflector rewrites the
next hop on every reflected route.
Ferrix now resolves a next hop that is not directly connected through the
kernel's own routes, as other BGP implementations do:
* The route is looked up in one kernel table, the "resolution table"
(`kernel { resolution_table: N }`, default 254 = main). The longest
prefix that contains the next hop wins; of several routes for that
prefix, the one with the lowest metric.
* Because the Linux kernel does not resolve gateways recursively, the BGP
route is installed with the gateway and interface of the kernel route
that covers the next hop. A kernel route without a gateway (a route
onto a point-to-point or tunnel interface) makes the next hop itself
the gateway on that interface. A kernel route that names a nexthop
object is resolved through the object.
* Routes with protocol `bgp` never resolve a next hop, whether Ferrix's
own or another daemon's. A next hop that is reachable only through a
BGP route depends on another BGP next hop, and such chains are not
predictable.
* The default route does not resolve a next hop unless
`kernel { resolve_via_default_route: true }` is set. On a router the
default route points at a transit provider, and a next hop that is
reachable only through it is not usable.
* Blackhole, unreachable and prohibit routes make the next hop
unresolvable, as does a route whose interface is down.
The same resolution is used by the import policy and the revalidator, so
a route whose next hop cannot be resolved is still held rather than
selected (RFC 4271 section 9.1.2.1), and is taken when a kernel route
makes its next hop resolvable.
Implementation:
* `bgp_fib::routes::KernelRoutes` holds the routes of the resolution
table and answers longest-prefix lookups.
* `bgp_fib::topology::Topology` bundles the interfaces, the kernel routes
and the kernel's nexthop objects into one value. The kernel task
publishes it through `Fib::topology()` (formerly `Fib::interfaces()`),
and `plan::resolve` takes it in place of `Interfaces`.
* The kernel task learns the resolution table with a dump of that table
alone, using NETLINK_GET_STRICT_CHK (Linux 4.20 and later; older
kernels get a dump of every table, filtered in user space). It asks
again, debounced, whenever a dedicated netlink socket that is a member
of the route multicast groups receives anything. That socket is given a
64 KiB receive buffer on purpose: the kernel reports Ferrix's own route
installs on it too, up to a million at a time, and an ENOBUFS overflow
is treated as the same hint as a message. Its contents are never
parsed. An earlier attempt to receive route notifications on the
socket that writes routes failed, because the kernel only excludes the
requesting socket from a notification when NLM_F_ECHO was requested.
* `Unresolved::NotConnected` is replaced by `NoRoute`, `OnlyDefault` and
`Discarded`, which appear as the reasons in `/api/kernel` and on the
status page.
Configuration: two new `kernel` fields, `resolution_table` and
`resolve_via_default_route`, documented in the configuration reference
and in the kernel guide, whose "Next Hop Resolution" section is rewritten.
Existing configurations behave as before except that routes with far
next hops may now be installed where a static or connected route covers
the next hop.
Testing: unit tests for `KernelRoutes`, `Topology` and resolution
through static, tunnel, nexthop-object, blackhole and default routes; a
kernel test (`a_far_next_hop_goes_the_way_the_kernels_routes_say`) that
checks the installed routes follow static, tunnel and nexthop-object
routes as they are added, replaced and removed; the existing kernel
tests, the interoperability lab with FRR and BIRD, and the workspace
tests, all passing. Installing 200,000 routes still runs at about
186,000 routes per second.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
277b973f1a |
Status page: what moved, what is missing, and a dark room
The attention strip's names lead to the peer's row, which lights up for a moment. The IPv4 and IPv6 sessions of one neighbour (NAME-v4, NAME-v6) are listed together, joined down the side, the name said once in full. A route count that moved by more than 2% between refreshes carries ▲ or ▼ for two minutes; a row whose state or count changed while the tab was in the background carries ● until it is clicked. A compact mode drops the next hop and session columns, and is remembered by the browser. Colours are variables, with a second set for prefers-color-scheme: dark. A new per-peer `expected_routes` says about how many routes a peer sends when all is well. The daemon only passes it to the API; the page lists an established peer with under half of it as needing attention, a minute after the session came up, with the expected count beside the actual one. An established session with next to no routes looked, until now, exactly like one that was fine. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
1ce7eef27e |
Status page: what needs attention first, glyphs for what repeats
The page opened with every session in bold green, the normal state, and the brightest numbers on it were the rejection counts, which RPKI doing its job produces by the thousand. The two sessions that were actually down were in the same red, thirty rows down. Now: - A status line at the top says what needs attention, by name and with the reason: sessions down, peers restarting, the kernel behind or refusing routes, RPKI data missing or about to expire. When nothing does, it says so. A second line gives the normal state in grey. - Colour means one thing each. Red: failed, needs a hand. Amber: degraded, worth watching. Green, set small: a system doing its job. Grey: inactive and expected. Normal is black; "Established" is no longer green, and a rejection count no longer red. - Peers that need attention come first, then the others grouped by what they are to us, with a "problems only" switch. Numbers are right aligned with thousands separators; the established time and uptime are one cell; the session's properties are glyphs (v4, v6, GR, ENH, AP, 2B) with the meaning as a tooltip, as are the roles. - Received routes: the best route leads, marked ★; attributes without a column are glyphs (OTC, AA, AGG, AS4, OID, CL); validity is ✓ ✗ –. - Rejected routes: those alike (same prefix, reason and path) are one row with a count and the last time; each peer's reasons are a bar. - Communities with a configured name (community_names, new) are badges with that name, the number as the tooltip: "comrack", "SwissIX", "interconnection 2011" rather than 210036:100:2 and so on. A name for a pattern shows the open part after it. The daemon does nothing with the names but pass them to the page. - One lookup box at the top, which takes a peer's name too; a peer's name anywhere on the page leads to its routes; table headers and the status line stay in view while scrolling. - The host name comes from gethostname(2), not from running `hostname`, which a static binary on a bare machine could not find: comrack said "unknown host". Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
23e8b4aadb |
One word for a route that is not taken: rejected
The API, the metrics, the status page and the documentation say a route is "rejected" (/api/routes/rejected, rejected_routes, bgp_routes_rejected_total, "Rejected routes"), and the reason type is Rejection. The code that keeps the record said "refused": RefusalLog, PeerRefusals, RefusedRoute, RefusalsView, RefusedRouteView, the refusals module. Those are RejectionLog, PeerRejections, RejectedRoute, RejectionsView, RejectedRouteView and rejections now, and the log line says "Rejected a route". Nothing the API or the metrics expose changes. Refusing stays what it was: a connection turned away at the door (server::Refusal), a kernel that would not take a route, an OPEN we would not have. The two aliases of SendReceive, AddPathSendRecv and OrfSendRecv, are gone; it is one type. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
673920d48b |
Names that said the same thing twice, or two things once
- PathAttribute::UnknownPathAttribute and Capability::UnknownCapability are PathAttribute::Unknown and Capability::Unknown: the enum already says what kind of thing is unknown. - The capability entry types are all named XxxEntry now (ExtendedNextHopEntry, LongLivedGracefulRestartEntry), rather than two of them sharing a name with the variant that holds a list of them; and MultiLabelCompat is MultiLabel, since nothing explained the "Compat". - Two different things were both called Reflection: what a session needs to reflect routes (export.rs, still Reflection) and the identifiers a reflected route must not carry back to us (policy.rs, now ReflectorIds). - well_known_allow's peer_is_internal is ibgp, as everywhere else. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
0829f873f2 |
Server: the long functions in named steps
Five functions did several things each in one body of 120 to 190 lines. Each is now a list of steps with a name and a doc comment of their own, with nothing changed in what they do: - run_session: hold_down_after, keep_for and release_routes. - Session::establish: session_handle, validation and route_source. A session keeps the peer's settings as one PeerSettings rather than eight fields copied out of it one by one; the sweeps' two channel ends are named fields, not .0 and .1; whether the peer can be asked to send its routes again is a bool, not a copy of the families. - Fsm::on_open: negotiate_role and negotiate_add_path. - BgpServer::run: log_startup and spawn_tasks. - Config::from_proto: parse_listen_endpoints, parse_peers, check_roles, parse_originate and Ours::tell_peers, with the three per-peer defaults in one PeerDefaults. - Egress::outgoing: may_go, rewrite and next_hop_for. - Exporter::follow: Learned::drain, judge and flush, with the three things called "announced" told apart (own_announced, learned.announced, batches) and the verdicts given a type. The hold-downs are a module of their own, holddown.rs; policy.rs is about policy again. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
3b22e318a2 |
Server: one way to work on the table off-thread, and smaller tidying
Removing a peer's routes, sweeping its stale ones and sweeping the attribute sets were each written out as their own spawn_blocking, and each dealt with a failed task differently: the periodic sweep dropped a panic on the floor, and a restarting peer whose kept routes could not be removed was logged as "removed 0 routes". off_thread does it once, and every caller logs a failure. Also: - the hard-reset wrapping of a NOTIFICATION is one function in the FSM; - the two ways to close every session are one, and the registry has one leave() rather than leave() and shutdown(); - the dialer says why a session ended with SessionEnd::reason, and no longer turns a result into a result; - as_path_prepend is parsed in one place, and the limits in error texts come from the constants rather than being written out; - router_id and cluster_id are parsed by one function; - the exporter builds its Egress in one place; - the fields a session keeps only to hand to its handle are fewer: route_refresh is a bool, not a copy of the families; - counting the stale routes removed cannot underflow; - docs that had gone stale: the proto still said there was no route reflection, the advertiser's module doc said it announced only our own routes, from_server listed two of its five sources, and so on. Tests for what had none: which session ends count as restarts, the Hard Reset wrapping, the graceful restart settings, the dialer's jitter, withdrawals in MP_UNREACH_NLRI, and AS_TRANS on a two-octet session. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
befc292215 |
Tidy the export limits, passive peers and status page code
Readability, no change in behaviour, but for one log line: - The routes a peer holds from us were counted twice over, differently, in the exporter: once per learned-route batch and once per originated route. Learned::held_by_peer counts the learned ones in one place, and ExportLimits::exceeded_by returns the limit that was passed instead of a bool, so that ending the session no longer has to dig it out again with a default that could not happen. - The two hold-downs for a limit passed, ours or the peer's, were two near-identical blocks; they are one match. - How a session ended is put into words in one place (SessionEnd::reason), for the dialer's backoff and the passive peer's state alike. - parse_export_limits parses one block; what a missing block means (no limits at the top, the top's limits for a peer) is said where it differs, instead of Option in and Option out. - The startup warnings about export limits are a function of their own, and the peers without limits are named in one line rather than one line each: a router with eighty sessions does not need eighty warnings. The warning about a peer without a role is gone: the configuration now refuses one. - Hex formatting of attribute octets is one function; the text for an unknown attribute is built in three plain steps. - The exporter's test fixture returns a named struct, not a tuple. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Opus 5.5
|
c9751883c4 |
Passive peers: configured, and left to connect
A session could only be configured for a peer we would then dial, every few seconds to two minutes, for as long as it was not up. A new peer whose side was not ready yet filled the log with failed attempts, and a peer that opens its sessions itself was dialled for nothing. A peer with `passive: true` is not dialled. A connection from its address is taken as for any configured peer, and the session comes up with all that is configured for it: role, import policy, export limits, hold-downs. Until then the status page and /api/peers show it as `Passive`, waiting for the peer to connect; after a session, with how that session ended, which nothing else would say since nobody dials it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Opus 5.5
|
9cfd3f67f0 |
Export limits, and a role for every peer where routes are passed on
What other networks were told of rested on one thing: the Only to Customer mark that routes from providers, peers and route servers get as they come in. A route without it, from a session without a role or by a mistake anywhere along the way, went to every peer, from every router of ours, and nothing of ours stood in the way. export_limits are a second check, independent of the roles, the marks and the export policy. For peers above or beside us (PROVIDER, PEER, ROUTE_SERVER, and any in another AS without a role): - prefixes: only routes for these go out, learned or our own. With our and our customers' space listed, a leak of anyone else's routes goes nowhere above us. - max_prefixes: if more routes were to go out than this, none of them does. The session ends with a Cease / Administrative Shutdown whose Shutdown Communication says why (RFC 9003), and the peer is held down for its import.hold_down_seconds, so that it is told nothing rather than too much. The whole table is counted before any of it is sent. They are set at the top and hold for every such peer that has none of its own; unlike `export`, a peer's own export policy does not drop them. A peer's own export_limits take their place, and an empty one lifts them. Customers, route server clients and our own routers are not limited. A configuration in which learned routes are passed on and a peer in another AS has no role is now refused, as is accept_unconfigured_peers with learned routes: without a role, a peer's routes go to everyone and it is told of everything. This used to be a warning. At startup, a warning for each peer the limits apply to that has none, and for each originated route the limits leave out. A metric, bgp_export_limit_exceeded_total, counts the sessions ended for it. The interop lab has a phase in which a customer leaks: a route outside its space never reaches BIRD, and when more were to go than the limit, BIRD's session ends and its log has our reason. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Opus 5.5
|
b16227c51f |
Route reflection, with ourselves as next hop
A route learned from one router of our AS went to no other (RFC 4271 §9.2.1), so every router needed a session with every other. A router at the end of a single link, with one session to its neighbour inside the AS, heard only of what that neighbour learned itself, and nothing of what the neighbour had from the routers behind it. A peer in our own AS can now be made a route reflector client (`route_reflector_client`, RFC 4456). What a client sends goes to all our routers; what the others send goes to the clients. A reflected route gets ORIGINATOR_ID (where it started, if it does not say) and our cluster ID at the front of CLUSTER_LIST; it is not sent back to the router it started at, nor round to a cluster it has been through, and one that comes back anyway is refused as "route reflected back to us". The cluster ID is the router ID unless `cluster_id` says otherwise. The two attributes are now decoded as such rather than kept as bytes, and are never sent to other networks. Unlike a standard reflector, this one is the next hop of what it reflects, as it is of everything it announces: there is no resolving a next hop that is not a neighbour, so a client reaches what lies behind another router through the reflector. Only peers in our own AS can be clients; the configuration refuses the setting for any other. Nothing changes in what other networks are told: Only to Customer marks go along with reflected routes, and every receiving session keeps its own max_prefixes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Opus 5.5
|
4e945b278f |
Import rules: preference and communities for some routes only
The import policy did the same to every route of a session. Now `rules` can do more to some of them: those with certain communities, those for certain prefixes, or both. Each rule can set the local preference and put communities on; each route takes the first rule that holds for it. Rules see a route as the session's own statements leave it, with our communities taken off and the session's marks put on. So a router can act on where another router of ours learned a route, by the mark that router put on it, and a customer's single prefix can be marked for a prepend. Routes of one UPDATE that different rules hold for go into the table with attribute sets of their own. An UPDATE that no rule touches is taken as before, without its attributes being written out again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Opus 5.5
|
72444c2706 |
Control communities: session groups, prepends, and only ours to give
Control communities could say which peers a route goes to, by AS. Now
they can also name sessions by group, and ask for prepends:
L:11:P .. L:13:P our AS once to three times more towards AS P
(P 0: towards every peer in another AS)
L:20:G / L:21:G not to / only to the sessions of group G
L:31:G .. L:33:G our AS once to three times more towards group G
A group is a number given to sessions in the configuration (`groups` of
a peer): those at one exchange, or those over one link. The same network
at two exchanges is one AS but two sessions, which is what steering
inbound traffic between exchanges needs. Of several prepends that apply
the most counts, on top of the session's own as_path_prepend. Functions
2 to 9, as the documentation already promised, and 100 and up stay
informational; 10 to 99 are kept for actions to come.
They are acted on where a route leaves our AS, and taken off there; our
informational communities are not. To a peer in our own AS a route goes
with them, unchanged, so that a mark put on at one router is acted on at
another. Before, they were acted on and taken off on internal sessions
too, so a route marked "to nobody" did not even reach our own routers.
And they are ours to give: the communities of our own AS (L:n, L:n:m,
and 0:n where control communities are on) are taken off every route
from a peer in another AS as it comes in, before import marks are put
on. A peer at an exchange could otherwise decide where its routes go
from us, and set our informational marks. A customer that is to steer
its own routes is given import.accept_control_communities. An UPDATE
without any of ours is still taken as it came, without being written
out again.
This changes what a customer that set control communities gets: it now
needs accept_control_communities. The transit example and the interop
lab say so; the lab also has FRR, as a customer, ask for a prepend
towards BIRD's group, and BIRD, as a provider, have its "not to" taken
off.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
Rayhaan JaufeerallyandClaude Opus 5.5
|
3609451bdc |
Ignore our own prefixes from our own routers, and say why a route for a network of this machine is held
Two routers of an AS that both originate a prefix (anycast) each announce it to the other over their internal session. Each took the other's route, could not install it in the kernel, since the prefix is on one of its own interfaces, and so held it, counted as rejected with "next hop unreachable": a permanent rejection on a healthy session, for a next hop that was in fact fine. A route from a peer in our own AS for a prefix this router originates is now ignored: not taken, not held, not counted. Only that prefix itself; what is inside it is judged as usual. From other networks such routes are refused as our own prefixes, as before. A route that is held because its prefix is a network of this machine, or inside one (an exchange's LAN prefix, say), is now counted as "prefix is a network of this router", apart from routes whose next hop cannot be reached. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Opus 5.5
|
65f136048e |
Graceful restart (RFC 4724, RFC 8538)
A peer that restarts keeps its routes with us, and we keep ours with our peers while we restart, so that a restart withdraws nothing across the network and the machine goes on forwarding. On by default for every peer; `graceful_restart` sets the restart time we give, caps how long a peer's routes are kept, or turns it off. When a peer restarts: its routes are marked stale in the routing table, and chosen as before, for at most its restart time. When it is back, what it sends again is fresh, and the stale routes of a family go at its End-of-RIB marker, or after the cap; at once for a family whose forwarding it did not keep. Only a connection that failed counts as a restart, and, with RFC 8538 agreed, an expired hold timer or a NOTIFICATION other than a Hard Reset. The stale mark is a bit of the route's word that selection never looks at. When we restart: SIGUSR1 leaves the kernel's routes where they are, closes the sessions without a word where graceful restart is agreed, and exits with status 75, on which the systemd unit starts us again. The next start finds the routes in the kernel, says so in its OPENs (R, and F where forwarding was kept), and has them go once every peer has sent its End-of-RIB. Our own End-of-RIB markers for learned routes wait for that too. SIGTERM still stops for good, now with a Hard Reset where RFC 8538 is agreed, as are errors of ours. The status page and API show what was agreed with each peer, the routes kept while a peer restarts, and which routes are stale. Tested against FRR and BIRD in a new phase of the interoperability lab, and with routes kept in a real kernel over a restart. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Opus 5.5
|
efb2d81a50 |
Reject blackhole requests with a reason of their own
A route with the BLACKHOLE community (RFC 7999) asks for the traffic to its prefix to be dropped. Exchange route servers pass these on as host routes with the exchange's blackhole address as next hop, and they were rejected as "prefix too specific", which hid what they are. Routes from EBGP neighbors with BLACKHOLE are now rejected as "blackhole request", whatever their length. One short enough to pass would have been taken as an ordinary route, sending the traffic to the blackhole next hop: the request honoured without anyone having chosen to. Acting on blackhole requests is not supported; the import policy page and the standards list say so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Opus 5.5
|
bd674002a2 |
Keep the path attributes of rejected routes, and show them
A rejected route was recorded with its prefix, reason and AS path, which is not enough to tell why a neighbor sent it: whether a /32 is a blackhole request, which communities a leaked route carries, which network marked it Only to Customer. Each recorded rejection now keeps the attributes of the UPDATE it came in, without the NLRI of MP_REACH_NLRI and MP_UNREACH_NLRI, and its next hop. The attributes are copied once per UPDATE, only when a route of it is rejected, and shared by its rejected routes. A route rejected later, when the RPKI data changes, has the attributes it was stored with. /api/routes/rejected gives them as `summary` and `attributes`, as for received routes, and the status page shows them under "show" in each row of the refused routes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Opus 5.5
|
dc275f99d7 |
Add a documentation website, built with Zola
The documentation moves from READMEs into a site in docs/: getting started, a user guide, operations, a reference, and development pages, written in the style of network vendors' documentation (docs/STYLE.md). bin/bgp/README.md and docs/RFCs.md are folded into it. - A theme of its own in the International Typographic Style, in blues, with self-hosted Inter and JetBrains Mono, search over Zola's index, dark mode, and no resources loaded from other hosts. - Example configurations in docs/examples/, shown on the pages and checked with `bgp --check` in CI. - A test that fails when a field of config.proto is missing from the configuration reference. - A CI job that builds the site; Zola fails on broken links and anchors. - A hardened systemd unit, tried under systemd with kernel routes on. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
1e2e8f2d86 |
Pass learned routes on to peers, with marks and control communities
What was to be scripted is configuration, done natively:
- import { add_communities, add_large_communities, remove_communities,
local_pref }: every route taken from a peer is marked (where it was
learned, say), with nobody else's word for our marks taken, and given
its preference, which selection goes by.
- export { add_communities, add_large_communities, remove_communities,
as_path_prepend, learned_routes }, for a peer or for all that have none
of their own.
- control_communities: the communities by which routes say which peers
they are passed on to, as exchanges' route servers have them (0:P, L:P,
0:L, and the large forms L:0:P, L:1:P, L:0:0), taken off as the route
goes on.
And what the last of these is for: with export.learned_routes a peer is
told the best route of every prefix the others announced. The rules are
BGP's and cannot be turned off: not back where it came from, not to a
peer whose AS is on the path, not between two internal peers, not past
NO_EXPORT or NO_ADVERTISE; our AS on the path and ourselves as next hop
to another network, LOCAL_PREF to our own; to a route server's client
the path, MED and next hop as they came (RFC 7947). Which routes go where
follows from the roles of RFC 9234: a route that came down or across goes
on down only, so that with roles on every peer this makes us transit for
our customers and nothing else. A peer without a role is told everything,
and the daemon says so at startup.
Each session follows the table's best routes with a reader and keeps the
prefixes it has announced, so that it can withdraw them. Routes that go
out alike go in one UPDATE: changes are gathered for 50 ms first, and a
table that is given out anew is gathered whole (at most four sessions at
a time; it is tens of megabytes). For that to work the table's journal
had to give out together what was written together: every change now
carries a time on a table-wide clock, a reader's pass reads every shard
up to one time, no further than the changes that are complete in every
shard, and a resync takes its places in the journals as of one time.
One peer sending 1.23 M routes over a session, another told of them:
in our table after 0.47 s, all heard after 2.1 s, four routes an UPDATE.
The lab has a phase, transit: FRR as our customer and BIRD as our
provider, routes each way with the right paths and marks, one kept back
by a community, and pings between their networks through us.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
9367c38907 |
Remove policy scripts
Rhai made a decision about a route cost one to three microseconds, some two hundred times what the same costs in Rust. That was bearable where a function is asked once for an UPDATE, and is not where every route is weighed against every peer, as it is when routes are passed on. What is wanted of a policy (mark routes by where they came from, mark what is announced, let communities say who is told what) needs no language; it is to be configuration, done natively. Kept from that work: route selection goes by a LOCAL_PREF that policy set on a route from an external peer, and the lab's configuration can be added to by a phase. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
cbf0df1eef |
Policy scripts: the operator's own rules for routes, in Rhai
A new crate, bgp-policy, runs functions of a script on routes: on their
way in, after the general checks of a peer's import and before the
checks that can change (RPKI, next hop reachability); and on their way
out to a peer. A function is given the route and the peer, may refuse
the route, and may change its communities, large communities,
LOCAL_PREF and MED, and on the way out prepend our AS:
fn from_peer(route, peer) {
if route.has_community("65535:666") { return false; }
route.add_community(LEARNED_FROM[peer.role]);
route.local_pref = PREFERENCE[peer.role];
true
}
Configuration: policy { path | source, import_function, export_function }
and, for a peer, import { function } and export { function }. The script
is compiled at startup; `bgp --check` reads it and says which function
is whose. A function that fails on a route has refused it. Scripts have
no files, network or clock, and are stopped after 200,000 steps.
Route selection now goes by a LOCAL_PREF on a route from an external
peer: the decoder never lets a peer's own through, so one that is there
was set by policy.
What it costs was measured before it was built on (benches/policy.rs):
a call that tests a few communities and changes two takes 1 to 3
microseconds, some two hundred times what Rust would. So a function is
asked once for the routes of an UPDATE, which share their attributes,
unless it asks for the prefix. One peer sending 1.23 M routes over a
session: 0.37 s without a function, 0.80 s with one that marks and
ranks every route, 3.4 s if it also looks at the prefix.
Export applies to what is announced, which today is what the daemon
originates; learned routes are not passed on yet.
Tested by unit tests of the crate, sessions end to end, a fuzz target on
the example script (testdata/policy.rhai), and a lab phase in which a
script changes which of FRR's and BIRD's route is chosen and what each
of them is told.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
f1d80c66df |
Do not choose a route whose next hop cannot be reached (RFC 4271 §9.1.2.1)
Where routes go into the kernel, a route the kernel could not be given is no longer taken into the table: its next hop is no neighbor on any interface, or its prefix is a network the machine is itself on. Such a route used to be able to win selection and then leave its prefix without a route in the kernel, though another peer had one that worked. It is held aside the way an RPKI-invalid route is, and by the same means: judged on arrival against the interfaces as the kernel last described them, looked for in the table when they change, and taken from what is held when its next hop can be reached. The fib publishes the interfaces for this; the next hop is resolved once for an UPDATE and not for each of its routes. The lab has a phase, nexthops: FRR's better route with a next hop that is nowhere is held and BIRD's is installed, and they change places as an address comes and goes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
3d946c32ce |
Code health after review: docs, metrics, structure, one more fuzz target
What the code-health review found, and what a fuzz target written for it
found in its first seconds.
- A set of attributes with the same attribute twice is refused by the
routing table. Its readers (the selection key, the path, the next hop)
could each take a different one of the two. The decoder never lets one
through; the new fuzz target `attrs` found that the table would.
- bgp_peer_up{peer}, to alert on a peer going away, and bgp_build_info.
Every metric is there from the first scrape. Alerts for a peer down,
hijack attempts and kernel errors are in the README.
- `bgp --check config` validates a configuration without starting, and
`--version` says which build it is. CI runs the first on the example
configuration, which a test now also parses with its commented fields
put in; the example had to learn about the next hop check.
- Path attributes have a module of their own, packet::attributes,
re-exported from packet::message as before.
- policy::From, which shadowed the trait, is RouteSource. The server's
connection, dialer and fsm modules are private. Logging arguments live
in the binary; the server library no longer depends on clap.
- One is_unicast_link_local (the standard library's) instead of two of
our own; SelectionKey::origin(); one place where an Import is made.
- READMEs say what the daemon does today rather than what it did not do
a year ago. Documentation links are checked in CI. Unused dependencies
and bgp_web::serve are gone; the Docker build is --locked and does not
ship the fuzz corpus.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
9ca86ede2f |
Harden import, the kernel task and the RTR client after review
Three independent reviews (hijack resistance, reliability, code health) of the whole tree. This is what they found that could let a bad route in or take the daemon down. Hijacks: - A route had two paths on a two-octet session, AS_PATH and AS4_PATH, read in two places by two rules. A peer could make the origin checked on import differ from the one checked at revalidation, and get an RPKI-invalid route into the table. AS4_PATH is now merged into AS_PATH once, on receipt (RFC 6793 §4.2.3), and nothing else knows of it. - The next hop is judged on import: it must be usable, agreed on (RFC 8950), and by default the peer itself. - Our own originated prefixes are not taken from other networks. - A route for a network the machine is on is never put in the kernel. - Paths with an AS_SET are refused (RFC 9774). - A configured peer gets one session, not one per BGP Identifier. - More IPv6 special-purpose prefixes are bogons. Reliability: - A batch of requests the kernel refused could overflow the netlink socket and end the daemon. The socket has room now, a lost answer or a silent kernel means asking what is there, and neither is fatal. - Working the routes out again within the grace period after a restart no longer drops the routes being kept. - That work can no longer be put off for ever by interface changes. - Peers whose sessions keep failing are slowed down. - Unused attribute sets are swept when they pile up, not only each minute. - Log lines a peer can cause are said once in a while. - An RTR answer is bounded in size and time; data cannot expire between two refreshes. - The shard of a key is taken from a stirred hash: neighbouring bits of the hash proved alike for prefixes in a row. The test that showed it, failing one run in sixteen, is deterministic now. The lab has a phase, hijacks, in which FRR tries the first three. route::apply is a struct with methods instead of nested closures, and the kernel task's loop is shorter. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
fd2248feb5 |
Peer at link-local addresses: fe80::1%eth0
A peer's endpoint may be a link-local IPv6 address with the interface it is on. Such a peer is the address and the interface together, wherever peers are told apart: admission, collisions, hold-downs, the refusal log, the status page. The interface is looked up by name each time it is needed, so one that appears later or is made anew is found. Our routes go out with our link-local address in both places of MP_REACH_NLRI's next hop, as FRR sends its own. A peer's link-local next hop is taken alone, twice, or after "::" (BIRD), and resolved in the kernel on the interface of the session rather than by looking fe80::/64 up, which every interface has. The lab has a new phase, linklocal: sessions with FRR (by interface, with radvd beside the daemon) and BIRD over a LAN with link-local addresses only, both families, pings across, the interface down and up. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
92e013ca01 |
Carry IPv4 routes over IPv6 sessions with IPv6 next hops (RFC 8950)
For a link that has only IPv6 addresses. - The Extended Next Hop capability is offered on every session over IPv6 whose families include IPv4 unicast, and used if the peer offers it too. Nothing to configure. - On such a session our IPv4 routes go out in MP_REACH_NLRI with the session's IPv6 address as next hop, and are withdrawn in MP_UNREACH_NLRI. Without the agreement they are not sent, as before. - IPv4 routes are read with an IPv6 next hop, global or global with link-local. - The kernel installer resolves an IPv4 route's IPv6 next hop like any other and installs it: by way of an IPv6 nexthop object, or with RTA_VIA where routes carry their own gateway. Tested against a real kernel in both modes, with a link-local next hop too, and in the lab against FRR and BIRD: the new `v6only` phase takes the IPv4 addresses off the exchange LAN, runs one IPv6 session with each router for both families, and pings IPv4 across. Sessions between link-local addresses are not supported yet. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
321af51e4b |
Add an interoperability lab with FRR, BIRD and Routinator
scripts/interop/lab.sh runs the daemon against FRR, BIRD and Routinator, each in a network namespace on one exchange LAN, and checks what every side ends up with: sessions and routes over IPv4 and IPv6 with their attributes, four-octet AS numbers, ADD-PATH, RFC 9234 roles and Only to Customer, RTR version 2 and invalid routes, route refresh, session resets, killed peers, a cut wire, the daemon stopped and killed, a table of a million routes and a restart under it, role mismatch and the prefix limit. All of it passes against FRR 10.5.1, BIRD 2.18 and Routinator 0.15.2. scripts/interop/real-rpki.sh checks the RTR client against a Routinator with the real RPKI: 713,258 IPv4 and 187,479 IPv6 VRPs and 3,379 ASPAs, the same numbers on both sides. What the lab found, fixed here: - A daemon restarted with its routes in the kernel removed them all at once, its table being empty until the peers had sent theirs again, and then put them back: no forwarding meanwhile. Routes of an earlier run are now kept for `stale_seconds` (120 by default) and only those nobody announced again are removed. A restart under a million routes takes 1.1 s to be in step, with nothing removed. - FRR ends a session it loses to a collision with a state machine error without a subcode, which was shown as "error code 5 subcode 0". CI gets a job that runs the lab. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
1de75fc610 |
Install a full table in the kernel in about a second
A million IPv4 routes took 1.6 s to install (release build); they take 1.1 s now, and 0.5 s to remove instead of 1.15 s. Most of what is left is the kernel's own work on each route. - Nexthop objects (Linux 5.3 and later): a gateway is put in the kernel once and routes name it, so the kernel checks it once and not for every route. The daemon makes its own objects, uses nobody else's, is told when one of its own is removed and puts it back, and removes them when it stops, which takes all the routes that name them along at once. `without_nexthop_objects` turns this off; a kernel without them is noticed and served as before. - Writing to the kernel is a task of its own, fed through a short queue, so the next batch is worked out while the kernel is busy with the last. "In step" is said when the kernel has done the batch that completes the table, not when it was handed over. - What a next hop resolves to is kept, by next hop: it was looked up among the machine's addresses for every route. - A quicker hash for the plan's maps, and no lookups in the ones that are empty. Fixes a hole found on the way: when an interface went down and came back within the second it takes us to react, the kernel had dropped our routes and we did not put them back, since nothing about them had changed for us. After such a change the kernel is now asked what it actually has. scripts/in-namespace.sh runs one command in a scratch namespace; FERRIX_ROUTES sets the size of the timing test. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
1fc44798b7 |
Install best routes in the Linux kernel over netlink
New crate bgp-fib. It follows the routing table's best routes and keeps
the kernel's routing table the same, and learns the machine's interfaces
and addresses from the kernel, which is what says whether a next hop can
be reached.
- netlink: the rtnetlink messages used (routes, addresses, links, errors),
as bytes and back. No I/O; property tested and fuzzed.
- interfaces: what is directly reachable, and from which interface.
- plan: what to ask the kernel for, given the best routes and what the
kernel has. No I/O; tested against a model kernel.
- kernel (Linux only): the sockets and the task. Requests go in batches;
routes of ours found at startup are kept if still right and removed if
not; a change of interfaces has every route worked out again; a clean
stop removes our routes.
Only routes marked proto bgp with the configured table and metric are
ever touched. A best route is installed if its next hop is a neighbor;
next hops further away are not resolved through other routes yet, and
such prefixes are counted and shown instead.
Configured with `kernel { table metric }`; off by default. The daemon
needs CAP_NET_ADMIN and stops if the kernel refuses it. On shutdown it
closes sessions and then removes its routes.
/api/kernel and the status page show installed, unresolved and refused
counts, the last error and the interfaces; five bgp_kernel_* metrics.
Testing: scripts/kernel-tests.sh runs, in network namespaces of their
own and with only the capabilities the daemon needs, the installer
against `ip route`, the daemon with a peer, and two daemons that route
for each other with pings across over IPv4 and IPv6.
scripts/linux-vm.sh runs that, or anything else, in a Lima VM from
macOS. CI runs the kernel tests and fuzzes the netlink decoder.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
f9ef781d51 |
Check roles in OPEN and prevent route leaks with Only to Customer (RFC 9234)
A peer's configured role now does three more things: - Our OPEN carries the BGP Role capability with what we are to the peer. A peer whose own says it is something that does not pair with that, or that sends two different roles, is refused with OPEN Message Error / Role Mismatch. A peer that says nothing is accepted unless require_role is set. A role on a peer in our own AS is a configuration error. - Routes from a provider, a peer or a route server are marked Only to Customer on the way in if they are not already, and our routes are marked on the way out to a customer, a peer or a route server client. - A marked route from a customer or route server client, or from a peer with someone else's mark, is a leak and is refused. No policy setting turns that off. The packet crate gains the Only to Customer attribute (malformed: treat-as-withdraw), and the Role Mismatch subcode is corrected from 8 to 11. The status page shows whether the peer confirmed its role. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
8086aa994f |
Verify AS paths with ASPAs, over RTR version 2
Origin validation says who may originate a prefix and nothing about the
rest of the path: it does not catch a route leak, nor a hijack that keeps
the right origin at the end of a made-up path. ASPAs, in which a network
lists its providers, do.
bgp-rpki:
- aspa: the ASPAs and the upstream and downstream verification procedures
of draft-ietf-sidrops-aspa-verification-24, tested against the draft's
wording as written.
- rtr: protocol version 2 (draft-ietf-sidrops-8210bis-23) with the ASPA
PDU and the new error codes.
- client: asks for version 2 and falls back to whatever older version the
cache speaks; keeps ASPAs with the VRPs, applied whole or not at all;
publishes which customers' ASPAs changed; tells the cache why it is
being left (Error Report); does not ask one version's serial of another.
Server:
- A peer has a role (provider, customer, peer, route server, route server
client), which decides how its paths are verified. Without one they are
not.
- A route whose path is invalid is refused ("ASPA invalid") and held, like
one with an invalid origin; accept_aspa_invalid exempts a peer.
- When ASPAs change, routes in the table whose path has a changed customer
on it are verified again, and held routes are released when they pass
both checks.
- The revalidator finds a route's peer by index instead of searching a
list for every route.
The status page and API show each peer's role, each route's path validity,
the number of ASPAs and the RTR version in use; bgp_rpki_aspas is on
/metricz.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|