9caaf5018ea0cd8fc2073e71fde896f86d37bb43
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Rayhaan JaufeerallyandClaude Fable 5.1
|
a8aba52562 |
Review pass on the codec and the table: bugs fixed, encoders simplified
A review of the wire codec and the routing table, with the fixes. Bugs in the codec: - A NOTIFICATION with a code or subcode we have no name for (subcode 0, "unspecific", is common) failed to decode and was answered as a bad message. Every pair now reads, as BgpError::Other. - Optional parameters in the extended form of RFC 9072 were read and written with one-octet parameter lengths; they are two. - An ORF capability listing more than one address family made the whole OPEN malformed. - Bytes after an OPEN's optional parameters were accepted. - Extended communities were split into fields by rules that were wrong for the common types. They are kept as their eight octets. - Several length fields were computed in u16 and could wrap on encode. Simplifications in the codec: - Lengths are filled in after the value is written, for messages, attributes, capabilities and the UPDATE's two fields. The parallel wire_len functions, which had to be kept in step by hand, are gone. Message::to_wire no longer encodes into a second buffer. - Type codes for messages, attributes and capabilities are each defined once. The strum discriminant enums and their hand-written conversions are removed. - One function splits an attribute off a buffer; the UPDATE decoder, the plain attribute parser and the table's key reader all use it. - The UPDATE decoder's 185-line loop is an AttributeReader with one method that decides what becomes of an attribute. Duplicate detection is a bitset, not a HashSet allocated per UPDATE. - ParserContext::four_octet_asn is a bool. The "context not set" error path, which treated a programming error as a malformed route, is gone. - Codec functions take a context only if they read it, and return a Result only if they can fail. - MP_REACH_NLRI and MP_UNREACH_NLRI hold an AfiSafi. - AddressFamilyId is IpVersion, which is what it was. - Deserialize is no longer derived on codec types: nothing used it, and it bypassed IpPrefix's invariants. IpPrefix::to_wire, which was broken and unused, is removed. Bugs in the table: - The index of a row's best route was sixteen bits. A peer using ADD-PATH could send more routes for a prefix than that counts, and the index wrapped. It is 32 bits, at no cost in row size. - Shards and attribute-store stripes were picked from the top bits of the hash, which are the bits hashbrown keeps as the tag it compares first. With 64 shards each shard's table had 2 of 128 tag values. They are now picked from bits the table does not use. - remove_peer held the peer list's lock for the whole scan of the table, so no session could establish while a peer was removed. - When more prefixes went at once than were left, the forced trim dropped a fixed few entries, reset every reader, and re-ran the tidy on every later write. It drops the older half. - Readers were woken by every change, including a repeated announcement and changes their journal does not record. - Reader::poll with a budget of zero never made progress. Simplifications in the table: - The two journals of a shard are one Journals value written through three calls; the five places that kept them in step by hand are gone. Whether an entry is still live is decided in one place. - The shortcuts of route selection moved to select.rs, next to the invariant they depend on, which is now written down. - announce and withdraw take routes with or without path identifiers; the _paths variants are removed. - The selection key is read off the wire form only; the second implementation over parsed attributes is removed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
d49bab58a5 |
Add ADD-PATH (RFC 7911): several routes for a prefix from one peer
A peer can now send several routes for one prefix, told apart by a path identifier, and each is a route of its own in the table. Wire: - The ADD-PATH capability lists several address families. It was read as exactly one, so the OPEN of a router offering it for IPv4 and IPv6 was malformed and the session never came up. - Prefixes in UPDATEs are `Nlri`: a prefix and a path identifier. The identifier is on the wire, in the body and in MP_REACH/MP_UNREACH, for the families the session uses ADD-PATH for in that direction. Sending and receiving are settled separately, so a session has one codec context for each. Negotiation: - `add_path` on a peer in the configuration: off (the default), receive, send or both, offered for every family offered to the peer. - In use for a family and a direction when one side offered to send and the other to receive. Shown on /statusz and in /api/peers. Table: - A route is known by its peer and path identifier together. Identifiers are stored only in rows that have a non-zero one, so routes of peers that send one route each cost nothing more. - Equal routes from one peer are chosen between by the lower identifier. - /api/routes/received gives each route's path_id. Journal: - A second journal in each shard records every change to any route, for followers that want all the routes of a prefix and not only the best: RibTable::reader_of_all_routes and Event::Routes. This is what sending several routes to a peer will read. It costs nothing while nobody follows it. Sending: prefixes go out with a path identifier where that was agreed, but there is still one route for each prefix to send: only originated routes are announced. Tested with the table and journal model tests extended to path identifiers and to followers of all routes, the codec property tests and fuzz targets extended to ADD-PATH sessions, and end to end in both directions and for the case where only one side offers it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
31aa3a4563 |
Use the sharded table in the server: sessions choose best routes
The server now keeps what it learns in table::RibTable. The Adj-RIB-In actor is gone, and with it the queue every UPDATE from every session went through. - A session takes a number in the table when it is established and writes its UPDATEs into the table itself. Its routes are removed when it ends, on a blocking thread since that looks at every prefix; a session torn down without cleaning up still has them removed. - For every prefix the best route is chosen (RFC 4271 §9.1.2.2) as routes come and go. Nothing acts on the choice yet beyond showing it. - The UPDATE decoder hands over the bytes of the attributes it kept, so the table finds the attribute set from them without writing the attributes out again. On a session with two-octet AS numbers the bytes are not in the table's form, and the parsed attributes are used. - /api/routes/received marks the best route of each prefix and returns at most `limit` routes (default 1000): the table can hold hundreds of millions. /api/status has the totals, and the status page says how many it is showing. The time a route was received is no longer kept or shown. - The event stream reports changes of best route (best_changed, best_withdrawn) in place of every route received. Each client follows the table's journal with a reader of its own, from the moment it connects: table::RibTable::reader_from_now. - Metrics: bgp_rib_routes, bgp_rib_prefixes, bgp_rib_attribute_sets and bgp_rib_journal_entries replace bgp_adj_rib_in_routes. - A line is no longer logged at INFO for every UPDATE sent or received; a full table is a million of them. They are at DEBUG. - A table with no room for another peer ends the session with Cease / Out of Resources. Four peers sending a synthetic full table each over TCP, 4.9 million routes: 0.80 s, against 4.76 s before. One peer: 0.45 s against 0.69 s. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
3b20576971 |
Add the journal: following the table's best routes at one's own pace
Whatever acts on the best routes (export to peers, the forwarding table, the status page) reads their changes from a per-shard journal through a table::Reader. Nothing uses it yet. - The journal records that a prefix's best route changed, not what to. A reader that comes to an entry is given the route as it is then. A prefix that changes a thousand times between reads is one event, and the journal never holds more live entries than the table has prefixes. - Each row keeps the sequence number of its latest change; an entry with another number is dead and is dropped when the journal is tidied. - Entries go once every reader has passed them. A reader that does not read cannot hold them back for ever: past a limit the oldest half is dropped, and that reader is given the whole table again (Reset, every best route, Synced). A new reader starts the same way. - With no reader nothing is recorded. - Reader::changed waits for there to be something to read. Tested against a mirror kept from the events: after any interleaving of announcements, withdrawals, session flaps, reads with small budgets and readers coming and going, with journals small enough to be tidied and overrun constantly, a reader that has caught up holds exactly the table's best routes. Also with writers on four threads and a reader on a fifth. The property test found one fault (a pass that resumed part-way claimed to have caught up) and catches three more put in on purpose. On the AMS-IX dump, 12 threads: loading takes 2.29 s with a follower reading throughout against 2.17 s with none, the journal peaking at 170,000 entries for 1.39 million prefixes. With a follower that does not read, the journal ends at 1.74 million entries, and the follower then catches up in 0.08 s. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
eab891567b |
Add the sharded routing table: Adj-RIBs-In and Loc-RIB in one
bgp_rib::table::RibTable holds every route from every peer and the best
one for each prefix. It is not used by the server yet; this adds it with
its tests and benchmarks.
- Organised by prefix. A row holds the prefix's routes from all peers in
one array; choosing again after a change reads that row and nothing else.
- A route is sixteen bytes: a pointer to its attribute set and one word
holding the peer and the first three steps of selection (LOCAL_PREF, AS
path length, ORIGIN) as a single number. That number alone settles most
changes; a tie goes on to MED, compared only within a neighbouring AS
and in a way that does not depend on arrival order, and the tie-breaks.
- Attribute sets are stored once, in wire form, with what selection needs
from them worked out when they are stored.
- Sharded by hash of the prefix, a lock per shard. A session applies its
own UPDATEs; there is no queue or owning thread in front of the table.
- Readers take one shard at a time for reading and look at rows in place.
Route selection is checked against a plain implementation with no
shortcuts by a property test over random announcements, withdrawals and
session flaps, for both the chosen route and the count of changes.
On a RIPE RIS dump from AMS-IX (148 peers, 39.3 million routes), against
the Adj-RIB-In actor, which does no selection:
actor table
memory 11.0 GiB 1.8 GiB (279 -> 48 bytes a route)
load, 1 thread 10.2 s 12.3 s
load, 12 threads (no gain) 2.1 s (19 million routes a second)
read every route 1.6 s, copying 8 GiB, table stopped
0.2 s, in place, table running
The benchmark crate counts memory only with --features count-memory: the
counters are shared between threads and distort the timings.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|