Pass learned routes on to peers, with marks and control communities

What was to be scripted is configuration, done natively:

- import { add_communities, add_large_communities, remove_communities,
  local_pref }: every route taken from a peer is marked (where it was
  learned, say), with nobody else's word for our marks taken, and given
  its preference, which selection goes by.
- export { add_communities, add_large_communities, remove_communities,
  as_path_prepend, learned_routes }, for a peer or for all that have none
  of their own.
- control_communities: the communities by which routes say which peers
  they are passed on to, as exchanges' route servers have them (0:P, L:P,
  0:L, and the large forms L:0:P, L:1:P, L:0:0), taken off as the route
  goes on.

And what the last of these is for: with export.learned_routes a peer is
told the best route of every prefix the others announced. The rules are
BGP's and cannot be turned off: not back where it came from, not to a
peer whose AS is on the path, not between two internal peers, not past
NO_EXPORT or NO_ADVERTISE; our AS on the path and ourselves as next hop
to another network, LOCAL_PREF to our own; to a route server's client
the path, MED and next hop as they came (RFC 7947). Which routes go where
follows from the roles of RFC 9234: a route that came down or across goes
on down only, so that with roles on every peer this makes us transit for
our customers and nothing else. A peer without a role is told everything,
and the daemon says so at startup.

Each session follows the table's best routes with a reader and keeps the
prefixes it has announced, so that it can withdraw them. Routes that go
out alike go in one UPDATE: changes are gathered for 50 ms first, and a
table that is given out anew is gathered whole (at most four sessions at
a time; it is tens of megabytes). For that to work the table's journal
had to give out together what was written together: every change now
carries a time on a table-wide clock, a reader's pass reads every shard
up to one time, no further than the changes that are complete in every
shard, and a resync takes its places in the journals as of one time.

One peer sending 1.23 M routes over a session, another told of them:
in our table after 0.47 s, all heard after 2.1 s, four routes an UPDATE.
The lab has a phase, transit: FRR as our customer and BIRD as our
provider, routes each way with the right paths and marks, one kept back
by a community, and pings between their networks through us.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

bin/bgp/README.md

@@ -42,8 +42,8 @@ connections, with RFC 4271 collision detection), announces its originated
 routes to every peer with itself as next hop, judges what peers send by each
 peer's import policy, keeps what passes, chooses a best route for each
 prefix, and on Linux can put those best routes in the kernel so that the
-machine forwards by them. It does not yet pass routes learned from one peer
-on to another, and has no export policy.
+machine forwards by them. Peers whose `export` says so are told the best of
+what the others announced, as the roles of RFC 9234 allow.
 
 ## Who may connect
 
@@ -89,6 +89,88 @@ A refused route is dropped, along with the route it would have replaced.
 `bgp_routes_rejected_total` counts them by reason, and the status page shows
 how many routes each peer has and how many were refused.
 
+## Passing routes on
+
+A peer whose `export` has `learned_routes: true` is told the best route of
+every prefix the others announced, not only what we originate. The rules
+are the usual ones and cannot be turned off: not back to the peer a route
+came from, not to a peer whose AS is on its path already, not from one
+peer in our own AS to another, not past NO_EXPORT or NO_ADVERTISE. A
+route goes out with our AS added to its path and ourselves as next hop,
+without the MED it came with; to a peer in our own AS the path is as it
+was and LOCAL_PREF goes along; to a `ROUTE_SERVER_CLIENT` path, MED and
+(where the client can use it) next hop are left as they came, as a route
+server sends them (RFC 7947).
+
+**What goes where is decided by roles (RFC 9234).** A route learned from a
+provider, a peer or a route server is marked "only to customer" on the way
+in and goes on to customers only; a customer's route goes to everyone. So
+with roles on every peer, `learned_routes` makes us transit for our
+customers and nothing else. A peer without a role has no such rule, and is
+told every route; the daemon warns about that at startup. The top-level
+`export` is for every peer that has none of its own:
+
+```
+export { learned_routes: true }
+peers { name: "customer-a" role: CUSTOMER ... }
+peers { name: "transit-a" role: PROVIDER ... }
+```
+
+**Marks.** `import` puts communities on every route taken from a peer, and
+takes some off first; `export` does the same for every route announced to
+one. Where a route was learned, which router it went through:
+
+```
+peers {
+  name: "ixp-a-rs"
+  import {
+    remove_communities: "64601:*"        # nobody else's word for our marks
+    add_communities: "64601:1001"        # learned at IXP A
+    add_large_communities: "64601:1:1001"
+    local_pref: 200
+  }
+}
+export { add_large_communities: "64601:2:1" }   # went through router 1
+```
+
+Patterns for `remove_communities` are `64601:100`, `64601:*`, `*:*`, or
+with three parts for large communities. `import.local_pref` is what route
+selection goes by: of two routes for a prefix the one with the higher is
+chosen, whatever else they have.
+
+**Control communities.** With `control_communities: true`, a route's
+communities say which peers it is passed on to, in the form Internet
+exchanges' route servers use (`L` our AS, `P` a peer's):
+
+| Standard | Large | |
+|---|---|---|
+| `0:P` | `L:0:P` | not to the peer with AS `P` |
+| `L:P` | `L:1:P` | to the peer with AS `P`, and then only to peers named so |
+| `0:L` | `L:0:0` | to nobody |
+
+"Not" wins over "to"; a route with none of them goes to everyone it
+otherwise would. Standard communities only have room for AS numbers below
+65536; the large forms work for any. They are taken off a route as it is
+passed on, and they hold for routes we originate too. With this on,
+standard communities `L:n` mean "to the peer with AS n", so mark routes
+with large communities (`L:2:n` and up) instead.
+
+**Cost.** A decision is a few comparisons on the route's communities and
+path. Routes that go out alike go in one UPDATE: changes are gathered for
+50 ms before they go, and a peer that is given the whole table (when it
+comes up, or asks for it again) has it gathered first, so that the
+prefixes that share attributes, which are spread over the table's shards,
+go together. At most four sessions gather a table at once; the others wait
+their turn. `cargo bench -p bgp-bench --bench transit`, with one peer
+sending 1.23 million routes over a session and another being told of them,
+on a laptop: in our table after 0.47 s, all of it heard by the other peer
+after 2.1 s, in UPDATEs of four routes each on average.
+
+**Not there yet.** Per-client best path selection for a route server (every
+client is told the one best route); ADD-PATH for learned routes (only the
+best is sent); route reflection; a way to send pre-built messages to
+several peers at once.
+
 ## Route origin validation (RPKI)
 
 With `rpki { caches { address: "..." port: 3323 } }` in the configuration,
@@ -369,6 +451,7 @@ Routinator 0.15.2. What it covers:
 | `limits` | A peer whose role does not fit gets no session; a peer over its prefix limit gets Cease / Maximum Number of Prefixes and is held down |
 | `hijacks` | FRR announces our own prefix, a route whose next hop is BIRD, and the exchange LAN's own prefix: none is taken |
 | `nexthops` | A better route from FRR whose next hop cannot be reached is held and BIRD's is chosen and installed; they swap when the next hop becomes a neighbor, and swap back when it stops being one |
+| `transit` | FRR as our customer and BIRD as our provider: FRR's routes reach BIRD with our AS on the path, BIRD's reach FRR marked only to customer, one of FRR's is kept from BIRD by a community, and packets go between their networks through us |
 | `v6only` | The exchange LAN without IPv4 addresses: one IPv6 session with each router for both families, IPv4 routes with IPv6 next hops (RFC 8950) in BGP and in the kernel, IPv4 pings across |
 | `linklocal` | The exchange LAN with link-local addresses only, and the same ones on a second interface: sessions with FRR (by interface, with radvd beside the daemon) and BIRD at `fe80::…%ix0`, both families in BGP and in the kernel, pings across, the interface taken down and up |
 
@@ -416,6 +499,7 @@ For alerting, `/metricz` has:
 | `bgp_peer_max_prefixes{peer}` | The peer's limit, where it has one |
 | `bgp_prefix_limit_exceeded_total{peer}` | Sessions ended for going over the limit |
 | `bgp_connections_refused_total{reason}` | Connections closed without a session |
+| `bgp_routes_exported_total{change}` | Learned routes announced to, and withdrawn from, other peers |
 | `bgp_peer_up{peer}` | 1 while the session with a configured peer is Established, else 0 |
 | `bgp_build_info{version}` | Always 1; says which version is running |