b5ca42244dc407376e4402cc8a6c644a9f546c1e
9
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Rayhaan JaufeerallyandClaude Fable 5.1
|
b5ca42244d |
export: a lighter gather, and UPDATEs to the session by the piece
Two things kept an exporter from its peer. A table being gathered to go out anew kept a map of every prefix heard, to let a later word on a prefix supersede an earlier one, and dressed every set of attributes before the first UPDATE went: a map insert and a parse-and-encode a route that the peer waited on. And the session's queue took UPDATEs one by one, a thousand at most, so the exporter stood at the queue once a hundred kilobytes were in it, behind a writer that takes a megabyte: the two took turns instead of running together. The journal gives a table out shard by shard, each prefix once, and a change meanwhile comes after the table as news of its own, so there is no second word to keep track of while gathering: the map is gone, and a prefix the peer is not to have is simply not gathered, as what it held of it is withdrawn once the table is through. Each set is dressed as its turn to go comes, so the first UPDATE goes once the table is gathered; only when the gathered routes alone would be over the peer's limit are the sets dressed first, to know which cannot be and take their routes with them, so that none goes if too many would. The UPDATEs for a set now go to the session in pieces of 64 KiB or so, each of whole messages, and the queue takes 64 of those: a few megabytes ahead of the writer, not a hundred kilobytes. The session counts the messages in a piece by their headers. Testing: the byte-for-byte test of the encoded UPDATEs now encodes thirty thousand routes and checks the pieces are of whole messages and of the size meant. The server tests, the integration tests, the kernel tests and every lab phase pass. In the Linux VM, three rounds alternating with the previous commit, a new peer's first UPDATE comes 1.8 s after its session is up instead of 3.1 s, and it has its four million routes in 3.8 s instead of 4.1 s; the smaller of the two routers' tables crosses in 3.0 s instead of 4.2 s and the larger in 5.0 s instead of 5.2 s, at the same CPU time. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
381eabd886 |
export: encode a route's attributes once a session
The exporter built each UPDATE as a message: the attributes of the route's set, parsed from the table, were cloned, had the next hop added, were sorted, and were handed to the session, which encoded them to the wire with the rest of the message. For a table whose attribute sets each cover a few routes that is an encoding of the same octets for nearly every route, and a copy of the message's bytes on the way to the writer besides. A profile of a production router taking the other router's table with ADD-PATH had the exporters at 55% of the CPU time, their dressing of routes at 23%, and memcpy at 12%. A route's attributes are now encoded once for a session, when it is first dressed for it, and kept with where in the octets a NEXT_HOP or an MP_REACH_NLRI belongs so that the attributes stay in order of type code. An UPDATE is then its header, the encoded octets with the next hop or the MP_REACH_NLRI spliced in, and the prefixes: written straight into a buffer that goes to the session as it is, and from the session to the writer without a copy. The session's queue takes either a message or an encoded UPDATE; an encoded one for a session that is no longer established is dropped, as the exporter made it for the session as it was. What is kept for a route, now a few hundred octets instead of a kilobyte, lets twice as many be kept. The exporter's maps hash with foldhash instead of SipHash, and the parser context for an UPDATE is made once a flush instead of once an UPDATE. Testing: a new test encodes a thousand routes with each of the forms an UPDATE takes (IPv4 with the next hop in its own attribute, IPv6 in MP_REACH_NLRI, with and without path identifiers, to two-octet and four-octet AS speakers, over several messages) both ways and checks the octets are the same byte for byte; the message-building form is kept for the test. The exporter tests decode what the session is handed. The server tests, the integration tests, the kernel tests and every lab phase pass. On the mesh benchmark in the Linux VM, three rounds each alternating with the previous commit, the two routers' exchange of 7.4 million routes takes 20.6 s of CPU time instead of 25.4 s (-19%) over 9.0 s of wall-clock time instead of 11.1 s, and the larger of the two tables is across in 7.1 s instead of 9.3 s. The smaller table's time varies too much from run to run (5-8 s) for a difference to be read from it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
ec5c9be454 |
export: advertise every route for a prefix with ADD-PATH (all_paths)
Before this change, a session on which ADD-PATH (RFC 7911) was
negotiated in the sending direction was sent path identifiers, but only
the best route for each prefix. A route collector such as bgp.tools
wants every route a router has accepted, from every neighbor, and from
one session: without this the collector saw one path per prefix, nearly
always the same neighbor's.
This change adds `all_paths` to the export policy. On a session with it,
and ADD-PATH send negotiated for a family, every accepted route for a
prefix of that family is advertised, each with a path identifier of its
own; for a family without ADD-PATH send the best route goes as before.
`all_paths` needs `learned_routes` (a configuration error otherwise) and
the peer's `add_path` to offer sending. The export policy, the role
rules, the well-known communities and the export limits apply to every
route alike, and the export limits count every route.
Implementation, in the advertiser:
* An exporter with `all_paths` follows the table's journal of all routes
(`RibTable::reader_of_all_routes`, until now unused) instead of the
best-route journal, and is given all the routes of a prefix whenever
one of them changes (`News::Routes`).
* What the peer holds is `Announced`: the set of prefixes for a
best-route session, as before; or for an all-paths session, for each
prefix the routes announced, keyed by the peer they came from and that
peer's own identifier, with the identifier they went out with and the
form they went out in. A route keeps its identifier for as long as it
is announced, goes out again only if it changed, and is withdrawn by
its identifier; a new route gets the lowest identifier not in use for
the prefix, from 1.
* Verdicts carry either kind (`Verdict::Best`, `Verdict::All`), and a
reset of the table puts what was announced in doubt until heard of
again, as before, with the same identifiers kept across the reset.
* The record for an all-paths session is about 50 bytes a route, some
tens of megabytes for a full table from several neighbors; the
best-route bookkeeping is unchanged in size.
Also, so that a collector session can be watched: the number of routes
advertised to each peer and not withdrawn since, every route counted
where several are sent for a prefix. The exporter keeps it in the
session's counters after every change; it is `advertised_routes` in
`/api/peers`, the "sent" column of the status page's neighbor table,
and the gauge `bgp_peer_advertised_routes{peer}`.
Documentation: a "Feeding a Route Collector" section in the guide to
passing routes on, with the configuration for a collector; `all_paths`
in the configuration reference; the ADD-PATH sections of the neighbor
guide and the standards reference; the limitation that was removed;
the new field, column and metric in the HTTP API, monitoring and
metrics references.
Testing: an integration test with two feeders and a collector that
negotiated ADD-PATH receive, which checks the two routes with their
identifiers, the withdrawal of one by its identifier, a changed route
keeping its identifier and a new route taking the free one; a
configuration test for `all_paths` without `learned_routes`; a new
`collector` phase of the interoperability lab in which FRR, as a route
collector, is told both of BIRD's routes for a prefix from BIRD's two
sessions and holds them apart, and loses one when its session is
disabled, with the advertised count checked at each step. The workspace tests, the kernel tests and every lab phase pass.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
673920d48b |
Names that said the same thing twice, or two things once
- PathAttribute::UnknownPathAttribute and Capability::UnknownCapability are PathAttribute::Unknown and Capability::Unknown: the enum already says what kind of thing is unknown. - The capability entry types are all named XxxEntry now (ExtendedNextHopEntry, LongLivedGracefulRestartEntry), rather than two of them sharing a name with the variant that holds a list of them; and MultiLabelCompat is MultiLabel, since nothing explained the "Compat". - Two different things were both called Reflection: what a session needs to reflect routes (export.rs, still Reflection) and the identifiers a reflected route must not carry back to us (policy.rs, now ReflectorIds). - well_known_allow's peer_is_internal is ibgp, as everywhere else. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
0829f873f2 |
Server: the long functions in named steps
Five functions did several things each in one body of 120 to 190 lines. Each is now a list of steps with a name and a doc comment of their own, with nothing changed in what they do: - run_session: hold_down_after, keep_for and release_routes. - Session::establish: session_handle, validation and route_source. A session keeps the peer's settings as one PeerSettings rather than eight fields copied out of it one by one; the sweeps' two channel ends are named fields, not .0 and .1; whether the peer can be asked to send its routes again is a bool, not a copy of the families. - Fsm::on_open: negotiate_role and negotiate_add_path. - BgpServer::run: log_startup and spawn_tasks. - Config::from_proto: parse_listen_endpoints, parse_peers, check_roles, parse_originate and Ours::tell_peers, with the three per-peer defaults in one PeerDefaults. - Egress::outgoing: may_go, rewrite and next_hop_for. - Exporter::follow: Learned::drain, judge and flush, with the three things called "announced" told apart (own_announced, learned.announced, batches) and the verdicts given a type. The hold-downs are a module of their own, holddown.rs; policy.rs is about policy again. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
3b22e318a2 |
Server: one way to work on the table off-thread, and smaller tidying
Removing a peer's routes, sweeping its stale ones and sweeping the attribute sets were each written out as their own spawn_blocking, and each dealt with a failed task differently: the periodic sweep dropped a panic on the floor, and a restarting peer whose kept routes could not be removed was logged as "removed 0 routes". off_thread does it once, and every caller logs a failure. Also: - the hard-reset wrapping of a NOTIFICATION is one function in the FSM; - the two ways to close every session are one, and the registry has one leave() rather than leave() and shutdown(); - the dialer says why a session ended with SessionEnd::reason, and no longer turns a result into a result; - as_path_prepend is parsed in one place, and the limits in error texts come from the constants rather than being written out; - router_id and cluster_id are parsed by one function; - the exporter builds its Egress in one place; - the fields a session keeps only to hand to its handle are fewer: route_refresh is a bool, not a copy of the families; - counting the stale routes removed cannot underflow; - docs that had gone stale: the proto still said there was no route reflection, the advertiser's module doc said it announced only our own routes, from_server listed two of its five sources, and so on. Tests for what had none: which session ends count as restarts, the Hard Reset wrapping, the graceful restart settings, the dialer's jitter, withdrawals in MP_UNREACH_NLRI, and AS_TRANS on a two-octet session. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Opus 5.5
|
b16227c51f |
Route reflection, with ourselves as next hop
A route learned from one router of our AS went to no other (RFC 4271 §9.2.1), so every router needed a session with every other. A router at the end of a single link, with one session to its neighbour inside the AS, heard only of what that neighbour learned itself, and nothing of what the neighbour had from the routers behind it. A peer in our own AS can now be made a route reflector client (`route_reflector_client`, RFC 4456). What a client sends goes to all our routers; what the others send goes to the clients. A reflected route gets ORIGINATOR_ID (where it started, if it does not say) and our cluster ID at the front of CLUSTER_LIST; it is not sent back to the router it started at, nor round to a cluster it has been through, and one that comes back anyway is refused as "route reflected back to us". The cluster ID is the router ID unless `cluster_id` says otherwise. The two attributes are now decoded as such rather than kept as bytes, and are never sent to other networks. Unlike a standard reflector, this one is the next hop of what it reflects, as it is of everything it announces: there is no resolving a next hop that is not a neighbour, so a client reaches what lies behind another router through the reflector. Only peers in our own AS can be clients; the configuration refuses the setting for any other. Nothing changes in what other networks are told: Only to Customer marks go along with reflected routes, and every receiving session keeps its own max_prefixes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Opus 5.5
|
72444c2706 |
Control communities: session groups, prepends, and only ours to give
Control communities could say which peers a route goes to, by AS. Now
they can also name sessions by group, and ask for prepends:
L:11:P .. L:13:P our AS once to three times more towards AS P
(P 0: towards every peer in another AS)
L:20:G / L:21:G not to / only to the sessions of group G
L:31:G .. L:33:G our AS once to three times more towards group G
A group is a number given to sessions in the configuration (`groups` of
a peer): those at one exchange, or those over one link. The same network
at two exchanges is one AS but two sessions, which is what steering
inbound traffic between exchanges needs. Of several prepends that apply
the most counts, on top of the session's own as_path_prepend. Functions
2 to 9, as the documentation already promised, and 100 and up stay
informational; 10 to 99 are kept for actions to come.
They are acted on where a route leaves our AS, and taken off there; our
informational communities are not. To a peer in our own AS a route goes
with them, unchanged, so that a mark put on at one router is acted on at
another. Before, they were acted on and taken off on internal sessions
too, so a route marked "to nobody" did not even reach our own routers.
And they are ours to give: the communities of our own AS (L:n, L:n:m,
and 0:n where control communities are on) are taken off every route
from a peer in another AS as it comes in, before import marks are put
on. A peer at an exchange could otherwise decide where its routes go
from us, and set our informational marks. A customer that is to steer
its own routes is given import.accept_control_communities. An UPDATE
without any of ours is still taken as it came, without being written
out again.
This changes what a customer that set control communities gets: it now
needs accept_control_communities. The transit example and the interop
lab say so; the lab also has FRR, as a customer, ask for a prepend
towards BIRD's group, and BIRD, as a provider, have its "not to" taken
off.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
1e2e8f2d86 |
Pass learned routes on to peers, with marks and control communities
What was to be scripted is configuration, done natively:
- import { add_communities, add_large_communities, remove_communities,
local_pref }: every route taken from a peer is marked (where it was
learned, say), with nobody else's word for our marks taken, and given
its preference, which selection goes by.
- export { add_communities, add_large_communities, remove_communities,
as_path_prepend, learned_routes }, for a peer or for all that have none
of their own.
- control_communities: the communities by which routes say which peers
they are passed on to, as exchanges' route servers have them (0:P, L:P,
0:L, and the large forms L:0:P, L:1:P, L:0:0), taken off as the route
goes on.
And what the last of these is for: with export.learned_routes a peer is
told the best route of every prefix the others announced. The rules are
BGP's and cannot be turned off: not back where it came from, not to a
peer whose AS is on the path, not between two internal peers, not past
NO_EXPORT or NO_ADVERTISE; our AS on the path and ourselves as next hop
to another network, LOCAL_PREF to our own; to a route server's client
the path, MED and next hop as they came (RFC 7947). Which routes go where
follows from the roles of RFC 9234: a route that came down or across goes
on down only, so that with roles on every peer this makes us transit for
our customers and nothing else. A peer without a role is told everything,
and the daemon says so at startup.
Each session follows the table's best routes with a reader and keeps the
prefixes it has announced, so that it can withdraw them. Routes that go
out alike go in one UPDATE: changes are gathered for 50 ms first, and a
table that is given out anew is gathered whole (at most four sessions at
a time; it is tens of megabytes). For that to work the table's journal
had to give out together what was written together: every change now
carries a time on a table-wide clock, a reader's pass reads every shard
up to one time, no further than the changes that are complete in every
shard, and a resync takes its places in the journals as of one time.
One peer sending 1.23 M routes over a session, another told of them:
in our table after 0.47 s, all heard after 2.1 s, four routes an UPDATE.
The lab has a phase, transit: FRR as our customer and BIRD as our
provider, routes each way with the right paths and marks, one kept back
by a community, and pings between their networks through us.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|