Pass learned routes on to peers, with marks and control communities

What was to be scripted is configuration, done natively:

- import { add_communities, add_large_communities, remove_communities,
  local_pref }: every route taken from a peer is marked (where it was
  learned, say), with nobody else's word for our marks taken, and given
  its preference, which selection goes by.
- export { add_communities, add_large_communities, remove_communities,
  as_path_prepend, learned_routes }, for a peer or for all that have none
  of their own.
- control_communities: the communities by which routes say which peers
  they are passed on to, as exchanges' route servers have them (0:P, L:P,
  0:L, and the large forms L:0:P, L:1:P, L:0:0), taken off as the route
  goes on.

And what the last of these is for: with export.learned_routes a peer is
told the best route of every prefix the others announced. The rules are
BGP's and cannot be turned off: not back where it came from, not to a
peer whose AS is on the path, not between two internal peers, not past
NO_EXPORT or NO_ADVERTISE; our AS on the path and ourselves as next hop
to another network, LOCAL_PREF to our own; to a route server's client
the path, MED and next hop as they came (RFC 7947). Which routes go where
follows from the roles of RFC 9234: a route that came down or across goes
on down only, so that with roles on every peer this makes us transit for
our customers and nothing else. A peer without a role is told everything,
and the daemon says so at startup.

Each session follows the table's best routes with a reader and keeps the
prefixes it has announced, so that it can withdraw them. Routes that go
out alike go in one UPDATE: changes are gathered for 50 ms first, and a
table that is given out anew is gathered whole (at most four sessions at
a time; it is tens of megabytes). For that to work the table's journal
had to give out together what was written together: every change now
carries a time on a table-wide clock, a reader's pass reads every shard
up to one time, no further than the changes that are complete in every
shard, and a resync takes its places in the journals as of one time.

One peer sending 1.23 M routes over a session, another told of them:
in our table after 0.47 s, all heard after 2.1 s, four routes an UPDATE.
The lab has a phase, transit: FRR as our customer and BIRD as our
provider, routes each way with the right paths and marks, one kept back
by a community, and pings between their networks through us.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

crates/bench/Cargo.toml

@@ -38,5 +38,9 @@ harness = false
 name = "real_table"
 harness = false
 
+[[bench]]
+name = "transit"
+harness = false
+
 [lints]
 workspace = true

crates/bench/benches/transit.rs

@@ -0,0 +1,152 @@
+//! What passing routes on costs: one peer sends the table over a session,
+//! and another is told of every route, over its own. The time is from the
+//! first byte sent to the last route heard; alongside, the time for the
+//! same table to be in our own table and no further.
+
+use std::net::Ipv4Addr;
+use std::sync::Arc;
+use std::time::{Duration, Instant};
+
+use bgp_bench::{Prefixes, Shape, announcements, encoded};
+use bgp_integration_tests::test_peer::TestPeer;
+use bgp_integration_tests::test_server::TestServer;
+use bgp_packet::message::PathAttribute;
+use bgp_server::marks::{Control, Marks};
+use bgp_server::policy::ExportPolicy;
+use bytes::BytesMut;
+
+fn env(name: &str, default: usize) -> usize {
+    std::env::var(name)
+        .ok()
+        .and_then(|v| v.parse().ok())
+        .unwrap_or(default)
+}
+
+/// How a server passes routes on in one run.
+#[derive(Clone, Copy)]
+enum Passing {
+    Not,
+    Plain,
+    /// With the communities that say who is told, and a mark put on every
+    /// route.
+    Dressed,
+}
+
+/// Sends the table from one peer and, where it is passed on, hears all of
+/// it at another. Returns when the table was in, and when it was all
+/// heard.
+fn run(
+    wire: &[BytesMut],
+    routes: usize,
+    heard_routes: usize,
+    passing: Passing,
+) -> (Duration, Option<Duration>) {
+    let runtime = tokio::runtime::Builder::new_multi_thread()
+        .enable_all()
+        .build()
+        .expect("a runtime");
+    runtime.block_on(async {
+        let server = TestServer::start_with(4_200_000_000, |config| {
+            config.export = Arc::new(ExportPolicy {
+                learned_routes: !matches!(passing, Passing::Not),
+                marks: match passing {
+                    Passing::Dressed => Marks::parse(&[], &["64601:7".to_string()], &[]).unwrap(),
+                    _ => Marks::default(),
+                },
+                prepend: 0,
+            });
+            config.control = matches!(passing, Passing::Dressed).then_some(Control {
+                local_asn: 4_200_000_000,
+            });
+        })
+        .await
+        .expect("a server");
+        let mut hearer = TestPeer::connect(server.addr).await.expect("a connection");
+        hearer
+            // An AS number that is on none of the generated paths: a route
+            // is not passed to a peer whose AS it has been through.
+            .handshake(4_100_000_000, Ipv4Addr::new(10, 0, 0, 100), 90)
+            .await
+            .expect("a session");
+        let mut sender = TestPeer::connect(server.addr).await.expect("a connection");
+        sender
+            .handshake(64513, Ipv4Addr::new(10, 0, 0, 1), 90)
+            .await
+            .expect("a session");
+        let stream: Vec<u8> = wire.iter().flat_map(|m| m.iter().copied()).collect();
+        let started = Instant::now();
+        let sending = tokio::spawn(async move {
+            sender.send_raw(&stream).await.expect("the server reads");
+            sender
+        });
+        let hearing = tokio::spawn(async move {
+            if matches!(passing, Passing::Not) {
+                return (hearer, None, 0);
+            }
+            let mut heard = 0usize;
+            let mut messages = 0usize;
+            while heard < heard_routes {
+                let update = hearer.recv_update().await.expect("an UPDATE");
+                messages += 1;
+                heard += update.announced_routes.len();
+                heard += update
+                    .path_attributes
+                    .iter()
+                    .map(|attribute| match attribute {
+                        PathAttribute::MpReachNlri(mp) => mp.prefixes.len(),
+                        _ => 0,
+                    })
+                    .sum::<usize>();
+            }
+            (hearer, Some(started.elapsed()), messages)
+        });
+        while server.rib.len() < routes {
+            tokio::time::sleep(Duration::from_millis(2)).await;
+        }
+        let in_table = started.elapsed();
+        let (hearer, heard, messages) = hearing.await.expect("the hearer");
+        if messages > 0 {
+            println!(
+                "    ({messages} UPDATEs, {:.1} routes each)",
+                heard_routes as f64 / messages as f64
+            );
+        }
+        drop(sending.await);
+        drop(hearer);
+        (in_table, heard)
+    })
+}
+
+fn main() {
+    let shape = Shape::INTERNET.scaled_down(env("BENCH_SCALE_DOWN", 1));
+    let prefixes = Prefixes::generate(shape);
+    let wire = encoded(&announcements(&prefixes, shape, 1));
+    let routes = shape.prefixes();
+    // Over a session on IPv4 only the IPv4 routes go.
+    let heard_routes = shape.ipv4_prefixes;
+    println!(
+        "== One peer sends {routes} routes in {} UPDATEs over TCP; another, over IPv4, is told of the {heard_routes} IPv4 ones",
+        wire.len()
+    );
+    for (what, passing) in [
+        ("not passed on", Passing::Not),
+        ("passed on", Passing::Plain),
+        (
+            "passed on, with control communities and a mark",
+            Passing::Dressed,
+        ),
+    ] {
+        let (in_table, heard) = run(&wire, routes, heard_routes, passing);
+        let heard = heard.map_or(String::new(), |heard| {
+            format!(
+                "   all heard by the other peer after {:>6.3} s ({:.2} M routes/s)",
+                heard.as_secs_f64(),
+                heard_routes as f64 / heard.as_secs_f64() / 1e6
+            )
+        });
+        println!(
+            "{what:<48} in the table after {:>6.3} s{heard}",
+            in_table.as_secs_f64()
+        );
+    }
+}

crates/integration-tests/src/test_server.rs

@@ -67,6 +67,8 @@ impl TestServer {
             http_endpoint: None,
             rpki: None,
             kernel: None,
+            export: Default::default(),
+            control: None,
         };
         customize(&mut config);
 

crates/integration-tests/tests/add_path.rs

@@ -50,6 +50,7 @@ async fn server_offering(
             import: ImportPolicy::permissive(),
             role: None,
             require_role: false,
+            export: Default::default(),
         }];
         c.originate = originate;
         // Keep the dialer out of the way.

crates/integration-tests/tests/admission.rs

@@ -28,6 +28,7 @@ fn peer_at(endpoint: SocketAddr) -> PeerConfig {
         import: ImportPolicy::permissive(),
         role: None,
         require_role: false,
+        export: Default::default(),
     }
 }
 

crates/integration-tests/tests/aspa.rs

@@ -67,6 +67,7 @@ async fn server_with(
             import,
             role,
             require_role: false,
+            export: Default::default(),
         }];
     })
     .await?;

crates/integration-tests/tests/collision.rs

@@ -46,6 +46,7 @@ async fn server_dialing(listener: &TcpListener) -> Result<TestServer> {
             import: ImportPolicy::permissive(),
             role: None,
             require_role: false,
+            export: Default::default(),
         }];
     })
     .await

crates/integration-tests/tests/debugging.rs

@@ -180,6 +180,7 @@ async fn test_refused_routes_can_be_seen_and_counted() -> Result<()> {
             },
             role: None,
             require_role: false,
+            export: Default::default(),
         }];
     })
     .await?;

crates/integration-tests/tests/dialer.rs

@@ -37,6 +37,7 @@ async fn server_dialing(listener: &TcpListener, remote_asn: u32) -> Result<TestS
             import: ImportPolicy::permissive(),
             role: None,
             require_role: false,
+            export: Default::default(),
         }];
     })
     .await
@@ -194,6 +195,7 @@ async fn test_inbound_from_configured_address_uses_config() -> Result<()> {
             import: ImportPolicy::permissive(),
             role: None,
             require_role: false,
+            export: Default::default(),
         }];
     })
     .await?;
@@ -245,6 +247,7 @@ async fn test_local_address_is_used_as_the_source() -> Result<()> {
             import: ImportPolicy::permissive(),
             role: None,
             require_role: false,
+            export: Default::default(),
         }];
     })
     .await?;
@@ -265,6 +268,7 @@ async fn test_local_address_is_used_as_the_source() -> Result<()> {
             import: ImportPolicy::permissive(),
             role: None,
             require_role: false,
+            export: Default::default(),
         }];
     })
     .await?;

crates/integration-tests/tests/import_policy.rs

@@ -51,6 +51,7 @@ async fn server_with(import: ImportPolicy) -> Result<(TestServer, TcpListener)>
             import,
             role: None,
             require_role: false,
+            export: Default::default(),
         }];
     })
     .await?;
@@ -281,3 +282,63 @@ async fn test_a_peer_over_its_limit_is_cut_off_and_kept_down() -> Result<()> {
         .await?;
     Ok(())
 }
+
+/// Routes from a peer are marked with where they were learned and given
+/// a preference, as the peer's `import` says; and marks that mean
+/// something to us are not taken on the peer's word.
+#[tokio::test]
+async fn test_a_peers_routes_are_marked_and_ranked_as_configured() -> Result<()> {
+    use bgp_packet::message::{CommunitiesPathAttribute, LocalPrefPathAttribute};
+    use bgp_server::marks::Marks;
+    let (server, _listener) = server_with(ImportPolicy {
+        marks: Marks::parse(
+            &["1299:*".to_string()],
+            &["1299:1001".to_string()],
+            &["1299:9:7".to_string()],
+        )
+        .unwrap(),
+        local_pref: Some(250),
+        ..default_policy()
+    })
+    .await?;
+    let mut peer = established(&server).await?;
+    peer.send(&Message::Update(UpdateMessage {
+        withdrawn_routes: vec![],
+        path_attributes: vec![
+            PathAttribute::Origin(OriginPathAttribute::IGP),
+            AsPathAttribute::from_asns(vec![PEER_ASN]),
+            PathAttribute::NextHop(NextHopPathAttribute(Ipv4Addr::new(192, 0, 2, 1))),
+            PathAttribute::Communities(CommunitiesPathAttribute(vec![(3333, 5), (1299, 2002)])),
+        ],
+        announced_routes: vec![prefix("193.0.0.0/21").into(), prefix("193.0.8.0/22").into()],
+    }))
+    .await?;
+    table_after(&server, 2, 0).await?;
+    let routes = server.received_routes();
+    for route in &routes {
+        assert!(
+            route
+                .attributes
+                .contains(&PathAttribute::Communities(CommunitiesPathAttribute(vec![
+                    (3333, 5),
+                    (1299, 1001)
+                ]))),
+            "{route:?}"
+        );
+        assert!(
+            route
+                .attributes
+                .contains(&PathAttribute::LocalPref(LocalPrefPathAttribute(250))),
+            "{route:?}"
+        );
+        assert!(
+            route.attributes.iter().any(|attribute| matches!(
+                attribute,
+                PathAttribute::LargeCommunities(large) if large.communities.len() == 1
+            )),
+            "{route:?}"
+        );
+    }
+    assert_eq!(server.rib.attrs().len(), 1, "they share their attributes");
+    Ok(())
+}

crates/integration-tests/tests/negotiation.rs

@@ -193,6 +193,7 @@ async fn test_no_family_in_common_refuses_the_session() -> Result<()> {
             import: ImportPolicy::permissive(),
             role: None,
             require_role: false,
+            export: Default::default(),
         }];
         // Keep the dialer out of the way.
         c.connect_retry = Duration::from_secs(3600);

crates/integration-tests/tests/roles.rs

@@ -58,6 +58,7 @@ async fn server_with(role: Option<PeerRole>, require_role: bool) -> Result<Setup
             import: default_policy(),
             role,
             require_role,
+            export: Default::default(),
         }];
     })
     .await?;

crates/integration-tests/tests/rpki.rs

@@ -77,6 +77,7 @@ async fn server_with(
             import,
             role: None,
             require_role: false,
+            export: Default::default(),
         }];
     })
     .await?;

crates/integration-tests/tests/transit.rs

@@ -0,0 +1,389 @@
+//! Passing routes on: what one peer announced reaches the others, as far
+//! as the rules for that let it.
+
+use std::collections::BTreeSet;
+use std::net::Ipv4Addr;
+use std::sync::Arc;
+use std::time::Duration;
+
+use bgp_integration_tests::test_peer::TestPeer;
+use bgp_integration_tests::test_server::TestServer;
+use bgp_packet::message::{
+    AsPathAttribute, CommunitiesPathAttribute, Message, NextHopPathAttribute, OriginPathAttribute,
+    PathAttribute, RouteRefreshMessage, UpdateMessage,
+};
+use bgp_primitives::constants::AfiSafi;
+use bgp_primitives::ip_prefix::IpPrefix;
+use bgp_rib::{LocalRoute, LocalRouteAttrs};
+use bgp_server::marks::{Control, Marks};
+use bgp_server::policy::ExportPolicy;
+use eyre::Result;
+
+const LOCAL_ASN: u32 = 64601;
+const TIMEOUT: Duration = Duration::from_secs(5);
+
+fn prefix(s: &str) -> IpPrefix {
+    IpPrefix::try_from(s).unwrap()
+}
+
+/// A server that passes on to every peer what it learned from the others.
+async fn transit(customize: impl FnOnce(&mut bgp_server::config::Config)) -> Result<TestServer> {
+    TestServer::start_with(LOCAL_ASN, |config| {
+        config.export = Arc::new(ExportPolicy {
+            learned_routes: true,
+            ..ExportPolicy::default()
+        });
+        customize(config);
+    })
+    .await
+}
+
+/// The peer with AS `64700 + n`, its session up.
+async fn peer(server: &TestServer, n: u8) -> Result<TestPeer> {
+    let before = server.registry.list().len();
+    let mut peer = TestPeer::connect(server.addr).await?;
+    peer.handshake(64700 + u32::from(n), Ipv4Addr::new(10, 0, 0, n), 90)
+        .await?;
+    tokio::time::timeout(TIMEOUT, async {
+        while server.registry.list().len() != before + 1 {
+            tokio::time::sleep(Duration::from_millis(5)).await;
+        }
+    })
+    .await?;
+    Ok(peer)
+}
+
+async fn announce(
+    peer: &mut TestPeer,
+    prefixes: &[&str],
+    path: &[u32],
+    communities: &[(u16, u16)],
+) -> Result<()> {
+    let mut path_attributes = vec![
+        PathAttribute::Origin(OriginPathAttribute::IGP),
+        AsPathAttribute::from_asns(path.to_vec()),
+        PathAttribute::NextHop(NextHopPathAttribute(Ipv4Addr::new(192, 0, 2, 1))),
+    ];
+    if !communities.is_empty() {
+        path_attributes.push(PathAttribute::Communities(CommunitiesPathAttribute(
+            communities.to_vec(),
+        )));
+    }
+    peer.send(&Message::Update(UpdateMessage {
+        withdrawn_routes: vec![],
+        path_attributes,
+        announced_routes: prefixes.iter().map(|p| prefix(p).into()).collect(),
+    }))
+    .await
+}
+
+async fn withdraw(peer: &mut TestPeer, prefixes: &[&str]) -> Result<()> {
+    peer.send(&Message::Update(UpdateMessage {
+        withdrawn_routes: prefixes.iter().map(|p| prefix(p).into()).collect(),
+        path_attributes: vec![],
+        announced_routes: vec![],
+    }))
+    .await
+}
+
+/// What a peer has been told so far, as the prefixes it would now have
+/// and the path each came with.
+#[derive(Default, Debug)]
+struct Told {
+    routes: std::collections::BTreeMap<String, Vec<u32>>,
+    updates: usize,
+}
+
+impl Told {
+    fn take(&mut self, update: &UpdateMessage) {
+        self.updates += 1;
+        for route in &update.withdrawn_routes {
+            self.routes.remove(&route.prefix.to_string());
+        }
+        let path: Vec<u32> = update
+            .path_attributes
+            .iter()
+            .find_map(|attribute| match attribute {
+                PathAttribute::AsPath(path) => Some(
+                    path.segments
+                        .iter()
+                        .flat_map(|segment| segment.path.clone())
+                        .collect(),
+                ),
+                _ => None,
+            })
+            .unwrap_or_default();
+        for route in &update.announced_routes {
+            self.routes.insert(route.prefix.to_string(), path.clone());
+        }
+    }
+
+    fn prefixes(&self) -> BTreeSet<&str> {
+        self.routes.keys().map(String::as_str).collect()
+    }
+}
+
+/// Reads what the peer is sent until it has exactly `expected`.
+async fn told_becomes(peer: &mut TestPeer, told: &mut Told, expected: &[&str]) -> Result<()> {
+    let expected: BTreeSet<&str> = expected.iter().copied().collect();
+    let reading = async {
+        while told.prefixes() != expected {
+            let update = peer.recv_update().await?;
+            told.take(&update);
+        }
+        Ok::<(), eyre::Report>(())
+    };
+    match tokio::time::timeout(TIMEOUT, reading).await {
+        Ok(read) => read,
+        Err(_) => Err(eyre::eyre!(
+            "the peer has {:?}; expected {expected:?}",
+            told.prefixes()
+        )),
+    }
+}
+
+/// Nothing more is sent to the peer for a while.
+async fn told_nothing(peer: &mut TestPeer) {
+    let more = tokio::time::timeout(Duration::from_millis(300), peer.recv_update()).await;
+    assert!(more.is_err(), "the peer was sent {more:?}");
+}
+
+/// What one peer announces the others are told, with our AS on the path
+/// and ourselves as next hop; and what it takes back, they are told too.
+#[tokio::test]
+async fn test_routes_from_one_peer_reach_the_others() -> Result<()> {
+    let server = transit(|_| {}).await?;
+    let mut one = peer(&server, 1).await?;
+    let mut two = peer(&server, 2).await?;
+
+    announce(
+        &mut one,
+        &["10.1.0.0/16", "10.2.0.0/16", "10.3.0.0/16"],
+        &[64701, 65000],
+        &[],
+    )
+    .await?;
+    let mut told = Told::default();
+    told_becomes(
+        &mut two,
+        &mut told,
+        &["10.1.0.0/16", "10.2.0.0/16", "10.3.0.0/16"],
+    )
+    .await?;
+    assert_eq!(told.routes["10.1.0.0/16"], [LOCAL_ASN, 64701, 65000]);
+    assert_eq!(
+        told.updates, 1,
+        "routes that go out alike go in one message"
+    );
+    // Not back to where they came from.
+    told_nothing(&mut one).await;
+
+    // The next hop is ourselves: the address of our end of the session.
+    announce(&mut one, &["10.4.0.0/16"], &[64701], &[]).await?;
+    let update = tokio::time::timeout(TIMEOUT, two.recv_update()).await??;
+    assert!(
+        update
+            .path_attributes
+            .contains(&PathAttribute::NextHop(NextHopPathAttribute(
+                Ipv4Addr::LOCALHOST
+            ))),
+        "{update:?}"
+    );
+    told.take(&update);
+
+    // One is taken back; then the peer goes, and its routes with it.
+    withdraw(&mut one, &["10.2.0.0/16"]).await?;
+    told_becomes(
+        &mut two,
+        &mut told,
+        &["10.1.0.0/16", "10.3.0.0/16", "10.4.0.0/16"],
+    )
+    .await?;
+    drop(one);
+    told_becomes(&mut two, &mut told, &[]).await?;
+    Ok(())
+}
+
+/// A peer is told of the best route, and of the next best when that one
+/// goes; and a peer that comes later is told what there is.
+#[tokio::test]
+async fn test_a_peer_is_kept_at_the_best_route() -> Result<()> {
+    let server = transit(|_| {}).await?;
+    let mut near = peer(&server, 1).await?;
+    let mut far = peer(&server, 2).await?;
+    let mut watcher = peer(&server, 3).await?;
+    let mut told = Told::default();
+
+    announce(&mut far, &["10.1.0.0/16"], &[64702, 65000, 65001], &[]).await?;
+    told_becomes(&mut watcher, &mut told, &["10.1.0.0/16"]).await?;
+    assert_eq!(told.routes["10.1.0.0/16"], [LOCAL_ASN, 64702, 65000, 65001]);
+
+    // A better route for it comes: that is the one told now.
+    announce(&mut near, &["10.1.0.0/16"], &[64701, 65001], &[]).await?;
+    let update = tokio::time::timeout(TIMEOUT, watcher.recv_update()).await??;
+    told.take(&update);
+    assert_eq!(told.routes["10.1.0.0/16"], [LOCAL_ASN, 64701, 65001]);
+    assert!(
+        update.withdrawn_routes.is_empty(),
+        "replaced, not withdrawn"
+    );
+
+    // And goes again: back to the other, with no gap.
+    withdraw(&mut near, &["10.1.0.0/16"]).await?;
+    let update = tokio::time::timeout(TIMEOUT, watcher.recv_update()).await??;
+    told.take(&update);
+    assert_eq!(told.routes["10.1.0.0/16"], [LOCAL_ASN, 64702, 65000, 65001]);
+
+    // The other peer was told of the better route while it was the best,
+    // and then that it is gone: the one left is its own, which it is not
+    // told of.
+    let mut told_far = Told::default();
+    told_becomes(&mut far, &mut told_far, &["10.1.0.0/16"]).await?;
+    let gone = tokio::time::timeout(TIMEOUT, far.recv_update()).await??;
+    assert_eq!(gone.withdrawn_routes, vec![prefix("10.1.0.0/16")]);
+    told_nothing(&mut far).await;
+
+    // A peer that comes now is told what there is; and again if it asks.
+    let mut late = peer(&server, 4).await?;
+    let mut late_told = Told::default();
+    told_becomes(&mut late, &mut late_told, &["10.1.0.0/16"]).await?;
+    late.send(&Message::RouteRefresh(RouteRefreshMessage::request(
+        AfiSafi::IPV4_UNICAST,
+    )))
+    .await?;
+    let again = tokio::time::timeout(TIMEOUT, late.recv_update()).await??;
+    assert_eq!(again.announced_routes, vec![prefix("10.1.0.0/16")]);
+    Ok(())
+}
+
+/// A route is not sent to a peer whose AS is on its path already, nor
+/// anywhere if it says NO_EXPORT; and what we originate goes to everyone
+/// beside what was learned.
+#[tokio::test]
+async fn test_routes_stop_where_they_must() -> Result<()> {
+    let server = transit(|config| {
+        config.originate = vec![LocalRoute::new(
+            prefix("10.99.0.0/16"),
+            LocalRouteAttrs::default(),
+        )];
+    })
+    .await?;
+    let mut one = peer(&server, 1).await?;
+    let mut two = peer(&server, 2).await?;
+    let mut told = Told::default();
+    told_becomes(&mut two, &mut told, &["10.99.0.0/16"]).await?;
+    assert_eq!(told.routes["10.99.0.0/16"], [LOCAL_ASN]);
+
+    // Through the second peer's AS already; and not for other networks.
+    announce(&mut one, &["10.1.0.0/16"], &[64701, 64702, 65000], &[]).await?;
+    announce(&mut one, &["10.2.0.0/16"], &[64701], &[(0xFFFF, 0xFF01)]).await?;
+    announce(&mut one, &["10.3.0.0/16"], &[64701], &[]).await?;
+    told_becomes(&mut two, &mut told, &["10.99.0.0/16", "10.3.0.0/16"]).await?;
+    told_nothing(&mut two).await;
+    assert_eq!(server.rib.len(), 3, "all three were taken");
+    Ok(())
+}
+
+/// Communities on a route say which peers it is passed on to, and are
+/// gone when it is; and every route to a peer gets the marks of `export`.
+#[tokio::test]
+async fn test_communities_say_who_is_told() -> Result<()> {
+    let server = transit(|config| {
+        config.control = Some(Control {
+            local_asn: LOCAL_ASN,
+        });
+        config.export = Arc::new(ExportPolicy {
+            learned_routes: true,
+            marks: Marks::parse(&[], &["64601:7".to_string()], &[]).unwrap(),
+            prepend: 1,
+        });
+        config.originate = vec![LocalRoute::new(
+            prefix("10.99.0.0/16"),
+            LocalRouteAttrs {
+                // Our own route, and not for the second peer.
+                communities: vec![(0, 64702)],
+                ..LocalRouteAttrs::default()
+            },
+        )];
+    })
+    .await?;
+    let mut one = peer(&server, 1).await?;
+    let mut two = peer(&server, 2).await?;
+    let mut three = peer(&server, 3).await?;
+    let (mut told_two, mut told_three) = (Told::default(), Told::default());
+
+    // To everyone but the second; to the second only; to nobody; and
+    // with nothing said.
+    announce(
+        &mut one,
+        &["10.1.0.0/16"],
+        &[64701],
+        &[(0, 64702), (3356, 3)],
+    )
+    .await?;
+    announce(&mut one, &["10.2.0.0/16"], &[64701], &[(64601, 64702)]).await?;
+    announce(&mut one, &["10.3.0.0/16"], &[64701], &[(0, 64601)]).await?;
+    announce(&mut one, &["10.4.0.0/16"], &[64701], &[]).await?;
+
+    told_becomes(&mut two, &mut told_two, &["10.2.0.0/16", "10.4.0.0/16"]).await?;
+    told_becomes(
+        &mut three,
+        &mut told_three,
+        &["10.99.0.0/16", "10.1.0.0/16", "10.4.0.0/16"],
+    )
+    .await?;
+    told_nothing(&mut two).await;
+    told_nothing(&mut three).await;
+    // Our AS once for passing it on, and once more by `export`.
+    assert_eq!(
+        told_three.routes["10.1.0.0/16"],
+        [LOCAL_ASN, LOCAL_ASN, 64701]
+    );
+    assert_eq!(told_three.routes["10.99.0.0/16"], [LOCAL_ASN, LOCAL_ASN]);
+
+    // What was for us to act on is gone, and the router's mark is on.
+    announce(
+        &mut one,
+        &["10.5.0.0/16"],
+        &[64701],
+        &[(0, 64702), (3356, 3)],
+    )
+    .await?;
+    let update = tokio::time::timeout(TIMEOUT, three.recv_update()).await??;
+    assert!(
+        update
+            .path_attributes
+            .contains(&PathAttribute::Communities(CommunitiesPathAttribute(vec![
+                (3356, 3),
+                (64601, 7)
+            ]))),
+        "{update:?}"
+    );
+    Ok(())
+}
+
+/// Without `learned_routes` a peer is told what we originate and nothing
+/// else, as before there was such a thing.
+#[tokio::test]
+async fn test_nothing_learned_is_passed_on_unless_asked_for() -> Result<()> {
+    let server = TestServer::start_with(LOCAL_ASN, |config| {
+        config.originate = vec![LocalRoute::new(
+            prefix("10.99.0.0/16"),
+            LocalRouteAttrs::default(),
+        )];
+    })
+    .await?;
+    let mut one = peer(&server, 1).await?;
+    let mut two = peer(&server, 2).await?;
+    let mut told = Told::default();
+    told_becomes(&mut two, &mut told, &["10.99.0.0/16"]).await?;
+    announce(&mut one, &["10.1.0.0/16"], &[64701], &[]).await?;
+    tokio::time::timeout(TIMEOUT, async {
+        while server.rib.len() != 1 {
+            tokio::time::sleep(Duration::from_millis(5)).await;
+        }
+    })
+    .await?;
+    told_nothing(&mut two).await;
+    Ok(())
+}

crates/rib/src/table/journal.rs

@@ -87,6 +87,11 @@ impl Default for Limits {
 struct Entry {
     seq: u64,
     prefix: IpPrefix,
+    /// When, by the table's clock: the same for every change one call
+    /// made, in whatever shard. A reader gives out all the changes up to
+    /// a time or none of them, so that what was changed together is read
+    /// together.
+    stamp: u64,
 }
 
 /// One journal of one shard.
@@ -111,6 +116,8 @@ struct Journal {
     /// How many reader slots are in use.
     readers: usize,
     limits: Limits,
+    /// The time the entries written now are given; see [`Entry::stamp`].
+    stamp: u64,
 }
 
 impl Journal {
@@ -124,6 +131,7 @@ impl Journal {
             gone: HashMap::default(),
             cursors: Vec::new(),
             readers: 0,
+            stamp: 0,
         }
     }
 
@@ -135,6 +143,7 @@ impl Journal {
             self.entries.push_back(Entry {
                 seq: self.seq,
                 prefix,
+                stamp: self.stamp,
             });
         }
         self.seq
@@ -156,6 +165,7 @@ impl Journal {
             self.entries.push_back(Entry {
                 seq: self.seq,
                 prefix,
+                stamp: self.stamp,
             });
             self.gone.insert(prefix, self.seq);
         }
@@ -289,6 +299,13 @@ impl Journals {
         }
     }
 
+    /// The time by the table's clock that the changes made from now on
+    /// are given; see [`Entry::stamp`].
+    pub(super) fn set_stamp(&mut self, stamp: u64) {
+        self.best.stamp = stamp;
+        self.all.stamp = stamp;
+    }
+
     /// Notes that `prefix` has its first route, and so a row. Returns the
     /// sequence numbers for the new row.
     pub(super) fn appeared(&mut self, prefix: IpPrefix) -> Seqs {
@@ -409,10 +426,22 @@ pub struct Progress {
 
 enum Phase {
     /// Giving out the whole table; `next` is the shard to do next, and
-    /// `announced` whether [`Event::Reset`] has been given.
-    Resync { next: usize, announced: bool },
-    /// Following the journals; `next` is the shard to read next.
-    Follow { next: usize },
+    /// `announced` whether [`Event::Reset`] has been given. `upto` is the
+    /// time on the table's clock the table is given out as of: the shards
+    /// are gone through one at a time while the table changes, and the
+    /// place taken in each shard's journal is as of that one time, so that
+    /// what follows is consistent from shard to shard. A change after it
+    /// that a later shard's rows already show is read again from the
+    /// journal; that is harmless.
+    Resync {
+        next: usize,
+        announced: bool,
+        upto: u64,
+    },
+    /// Following the journals; `next` is the shard to read next, and
+    /// `upto` the time by the table's clock up to which this pass reads
+    /// (see [`Entry::stamp`]).
+    Follow { next: usize, upto: u64 },
 }
 
 impl Phase {
@@ -420,6 +449,7 @@ impl Phase {
     const RESYNC: Self = Self::Resync {
         next: 0,
         announced: false,
+        upto: 0,
     };
 }
 
@@ -453,7 +483,7 @@ impl RibTable {
     /// those who want the news and not the table. If it falls too far
     /// behind it is reset like any other, and is then given the table.
     pub fn reader_from_now(self: &Arc<Self>) -> Reader {
-        let mut reader = self.register(Kind::Best, Phase::Follow { next: 0 });
+        let mut reader = self.register(Kind::Best, Phase::Follow { next: 0, upto: 0 });
         for (index, shard) in self.shards.iter().enumerate() {
             let shard = read(shard);
             reader.set_cursor(index, &shard, shard.journals.best.seq);
@@ -567,54 +597,65 @@ impl Reader {
         let mut events = 0;
         loop {
             match self.phase {
-                Phase::Resync { next, announced } => {
-                    if !announced {
+                Phase::Resync {
+                    next,
+                    announced,
+                    upto,
+                } => {
+                    let upto = if announced {
+                        upto
+                    } else {
                         self.forget_place();
                         visit(Event::Reset);
                         events += 1;
-                    }
+                        self.table.settled()
+                    };
                     if next == shards {
                         visit(Event::Synced);
                         events += 1;
-                        self.phase = Phase::Follow { next: 0 };
+                        self.phase = Phase::Follow { next: 0, upto: 0 };
                         continue;
                     }
-                    events += self.scan(next, &mut visit);
+                    events += self.scan(next, upto, &mut visit);
                     self.phase = Phase::Resync {
                         next: next + 1,
                         announced: true,
+                        upto,
                     };
                     if events >= budget {
                         return self.progress(events, false);
                     }
                 }
-                Phase::Follow { next } => {
+                Phase::Follow { next, upto } => {
                     // Read before the pass: a change made during it shows as
                     // a newer version, and `changed` then returns at once.
-                    let version = if next == 0 {
-                        self.table.version.load(Ordering::Acquire)
+                    // And the pass reads up to one time in every shard
+                    // alike, chosen so that about the budget is read.
+                    let (version, upto) = if next == 0 {
+                        (
+                            self.table.version.load(Ordering::Acquire),
+                            self.time_for(budget - events),
+                        )
                     } else {
-                        self.seen
+                        (self.seen, upto)
                     };
                     self.seen = version;
                     let mut shard = next;
                     let mut lagged = false;
+                    let mut more_somewhere = false;
                     while shard < shards {
                         if events >= budget {
-                            self.phase = Phase::Follow { next: shard };
+                            self.phase = Phase::Follow { next: shard, upto };
                             return self.progress(events, false);
                         }
-                        match self.drain(shard, budget - events, &mut visit) {
+                        match self.drain(shard, upto, &mut visit) {
                             Drained::Lagged => {
                                 lagged = true;
                                 break;
                             }
                             Drained::Read { events: read, more } => {
                                 events += read;
-                                if more {
-                                    self.phase = Phase::Follow { next: shard };
-                                    return self.progress(events, false);
-                                }
+                                more_somewhere |= more;
                             }
                         }
                         shard += 1;
@@ -623,11 +664,16 @@ impl Reader {
                         self.phase = Phase::RESYNC;
                         continue;
                     }
-                    self.phase = Phase::Follow { next: 0 };
-                    if next != 0 {
-                        // This pass took up where an earlier call stopped.
-                        // The shards before that point have not been looked
-                        // at in this call, so go round again from the first.
+                    self.phase = Phase::Follow { next: 0, upto: 0 };
+                    if next != 0 || more_somewhere {
+                        // This pass took up where an earlier call stopped,
+                        // so the shards before that point have not been
+                        // looked at in this call; or some shard had more
+                        // than its share, or changes later than this
+                        // pass's time. Go round again from the first.
+                        if events >= budget {
+                            return self.progress(events, false);
+                        }
                         continue;
                     }
                     return self.progress(events, true);
@@ -691,22 +737,68 @@ impl Reader {
         shard.journals.of(self.kind).cursors[self.slot].store(cursor, Ordering::Release);
     }
 
-    /// Gives out every prefix in a shard and takes a place at the end of
-    /// its journal, in one hold of the shard: what changes afterwards is in
-    /// the journal after that place.
-    fn scan(&mut self, index: usize, visit: &mut impl FnMut(Event<'_>)) -> usize {
+    /// Gives out every prefix in a shard and takes a place in its journal
+    /// as of `upto` on the table's clock, in one hold of the shard: what
+    /// changed after that time is in the journal after that place, whether
+    /// or not the rows given out show it already.
+    fn scan(&mut self, index: usize, upto: u64, visit: &mut impl FnMut(Event<'_>)) -> usize {
         let table = Arc::clone(&self.table);
         let shard = read(&table.shards[index]);
         for row in &shard.rows {
             visit(self.event_for(row));
         }
-        self.set_cursor(index, &shard, shard.journals.of(self.kind).seq);
+        let journal = shard.journals.of(self.kind);
+        let beyond = journal.entries.partition_point(|entry| entry.stamp <= upto);
+        let cursor = match (beyond.checked_sub(1), journal.entries.get(beyond)) {
+            // Nothing in the journal is later than `upto`.
+            (_, None) => journal.seq,
+            // The last entry that is not later; or, if every entry is
+            // later, just before the first.
+            (Some(last), Some(_)) => journal.entries[last].seq,
+            (None, Some(first)) => first.seq - 1,
+        };
+        self.set_cursor(index, &shard, cursor);
         shard.rows.len()
     }
 
-    /// Reads a shard's journal from this reader's place, for at most `max`
-    /// events.
-    fn drain(&mut self, index: usize, max: usize, visit: &mut impl FnMut(Event<'_>)) -> Drained {
+    /// The time on the table's clock up to which a pass over the shards
+    /// is to read, for about `budget` events in all: changes made together
+    /// are read together, in whatever shards they are (see
+    /// [`Entry::stamp`]). Taken shard by shard to the budget, the first
+    /// shards would be read far ahead of the last, and the routes of one
+    /// UPDATE, spread over the shards by their prefixes, given out far
+    /// apart.
+    fn time_for(&self, budget: usize) -> u64 {
+        let shards = self.table.shards.len();
+        let share = (budget / shards).max(1);
+        // No further than the changes that are in every shard already.
+        let settled = self.table.settled();
+        let mut upto = settled;
+        // The earliest change not yet read: at least that one is, however
+        // many changes were made with it.
+        let mut earliest = None;
+        for (index, shard) in self.table.shards.iter().enumerate() {
+            let shard = read(shard);
+            let journal = shard.journals.of(self.kind);
+            let cursor = self.cursors[index];
+            if cursor < journal.floor || cursor == journal.seq {
+                continue;
+            }
+            let start = journal.entries.partition_point(|entry| entry.seq <= cursor);
+            if let Some(next) = journal.entries.get(start) {
+                earliest = Some(earliest.map_or(next.stamp, |e: u64| e.min(next.stamp)));
+            }
+            // Everything before the change that would go over the share.
+            if let Some(beyond) = journal.entries.get(start + share) {
+                upto = upto.min(beyond.stamp - 1);
+            }
+        }
+        upto.max(earliest.unwrap_or(upto)).min(settled)
+    }
+
+    /// Reads a shard's journal from this reader's place, no further than
+    /// the changes made by `upto` on the table's clock.
+    fn drain(&mut self, index: usize, upto: u64, visit: &mut impl FnMut(Event<'_>)) -> Drained {
         let table = Arc::clone(&self.table);
         let shard = read(&table.shards[index]);
         let journal = shard.journals.of(self.kind);
@@ -725,7 +817,7 @@ impl Reader {
         let mut events = 0;
         let mut more = false;
         for entry in journal.entries.range(start..) {
-            if events == max {
+            if entry.stamp > upto {
                 more = true;
                 break;
             }

crates/rib/src/table/journal_tests.rs

@@ -711,3 +711,56 @@ proptest! {
         }
     }
 }
+
+/// Routes that were announced together, with one set of attributes, are
+/// spread over the shards by their prefixes. A reader that takes its
+/// budget from the shards evenly is given them together all the same,
+/// which is what lets whoever passes them on send them in one message.
+#[test]
+fn changes_made_together_are_read_together() {
+    let fixture = Fixture::on(RibTable::new(128), &[external(1)]);
+    let mut reader = fixture.table.reader();
+    assert!(reader.poll(10, |_| {}).caught_up);
+
+    // 6000 UPDATEs of six prefixes each, every one with attributes of its
+    // own.
+    let groups = 6000u32;
+    for group in 0..groups {
+        let attrs = fixture
+            .table
+            .attrs()
+            .intern(&Route::via(&[1, 100_000 + group]).attributes(), None)
+            .unwrap();
+        let prefixes: Vec<IpPrefix> = (0..6).map(|n| nth_prefix(group * 6 + n)).collect();
+        fixture.table.announce(fixture.peers[0], &attrs, &prefixes);
+    }
+
+    // Read in batches a tenth of that size: each should hold whole groups,
+    // but for the ones cut by its edges.
+    let mut batches = 0;
+    let mut split_groups = 0;
+    let mut seen_total = 0;
+    while seen_total < groups * 6 {
+        let mut groups_in_batch: HashMap<usize, usize> = HashMap::new();
+        let progress = reader.poll(3600, |event| {
+            if let Event::Best { attrs, .. } = event {
+                *groups_in_batch
+                    .entry(Arc::as_ptr(attrs) as usize)
+                    .or_default() += 1;
+            }
+        });
+        seen_total += groups_in_batch.values().sum::<usize>() as u32;
+        split_groups += groups_in_batch.values().filter(|n| **n < 6).count();
+        batches += 1;
+        if progress.events == 0 {
+            break;
+        }
+    }
+    assert_eq!(seen_total, groups * 6);
+    // A few are, where a batch's time falls inside one: the reader cannot
+    // know what was written together until it is all there.
+    assert!(
+        split_groups < 100,
+        "{split_groups} of {groups} groups were split over {batches} batches"
+    );
+}

crates/rib/src/table/mod.rs

@@ -30,6 +30,7 @@ mod select;
 #[cfg(test)]
 mod tests;
 
+use std::collections::BTreeSet;
 use std::hash::BuildHasher;
 use std::net::{IpAddr, Ipv4Addr};
 use std::sync::atomic::{AtomicU64, AtomicUsize};
@@ -301,6 +302,13 @@ pub struct RibTable {
     /// Goes up whenever something is written to a journal that has
     /// readers.
     version: AtomicU64,
+    /// Counts the calls that change the table, so that a change can say
+    /// when it was made relative to others (see the journal's `Entry`).
+    clock: AtomicU64,
+    /// The changes being made right now, by their time on the clock: a
+    /// reader does not read past the earliest of them, which may not have
+    /// reached every shard yet.
+    in_progress: Mutex<BTreeSet<u64>>,
     wake: Notify,
 }
 
@@ -339,6 +347,8 @@ impl RibTable {
             reader_slots: Mutex::default(),
             readers: AtomicUsize::new(0),
             version: AtomicU64::new(0),
+            clock: AtomicU64::new(0),
+            in_progress: Mutex::default(),
             wake: Notify::new(),
         }
     }
@@ -395,8 +405,10 @@ impl RibTable {
     pub fn remove_peer(&self, peer: PeerIndex) -> Outcome {
         let mut outcome = Outcome::default();
         let mut news = false;
+        let change = self.begin_change();
         for shard in &self.shards {
             let mut shard = write(shard);
+            shard.journals.set_stamp(change.stamp);
             let before = shard.journals.mark();
             outcome += shard.remove_peer(peer);
             // From here on the shard takes no more routes from the peer.
@@ -491,8 +503,11 @@ impl RibTable {
         placed.sort_unstable_by_key(|(hash, _)| self.shard_of(*hash));
 
         let mut news = false;
+        // One time for all of this call's changes, in every shard.
+        let change = self.begin_change();
         for run in placed.chunk_by(|a, b| self.shard_of(a.0) == self.shard_of(b.0)) {
             let mut shard = write(&self.shards[self.shard_of(run[0].0)]);
+            shard.journals.set_stamp(change.stamp);
             let before = shard.journals.mark();
             apply(&mut shard, run);
             shard.tidy_journals();
@@ -503,6 +518,30 @@ impl RibTable {
         }
     }
 
+    /// Takes a time on the clock for a change that is about to be made,
+    /// and notes that it is under way until the guard drops.
+    fn begin_change(&self) -> Change<'_> {
+        let stamp = self.clock.fetch_add(1, std::sync::atomic::Ordering::AcqRel) + 1;
+        self.in_progress
+            .lock()
+            .unwrap_or_else(PoisonError::into_inner)
+            .insert(stamp);
+        Change { table: self, stamp }
+    }
+
+    /// The time on the clock up to which every change is complete in every
+    /// shard: everything before the earliest one still under way.
+    pub(super) fn settled(&self) -> u64 {
+        let in_progress = self
+            .in_progress
+            .lock()
+            .unwrap_or_else(PoisonError::into_inner);
+        match in_progress.first() {
+            Some(earliest) => earliest - 1,
+            None => self.clock.load(std::sync::atomic::Ordering::Acquire),
+        }
+    }
+
     /// How many routes the table holds, from all peers.
     pub fn len(&self) -> usize {
         self.shards.iter().map(|shard| read(shard).routes).sum()
@@ -620,6 +659,22 @@ fn write(shard: &RwLock<Shard>) -> RwLockWriteGuard<'_, Shard> {
     shard.write().unwrap_or_else(PoisonError::into_inner)
 }
 
+/// A change to the table that is under way; see [`RibTable::begin_change`].
+struct Change<'a> {
+    table: &'a RibTable,
+    stamp: u64,
+}
+
+impl Drop for Change<'_> {
+    fn drop(&mut self) {
+        self.table
+            .in_progress
+            .lock()
+            .unwrap_or_else(PoisonError::into_inner)
+            .remove(&self.stamp);
+    }
+}
+
 /// Picks one of `count` buckets by a hash, for spreading keys over shards
 /// or lock stripes that each hold a hash table keyed by the same hash.
 ///

crates/server/proto/config.proto

@@ -53,6 +53,32 @@ message BgpConfig {
   // that the machine forwards packets by them. Left out, the daemon only
   // speaks BGP and the machine's forwarding is not touched.
   Kernel kernel = 11;
+
+  // What is announced, and how, to every peer that has no `export` of its
+  // own.
+  ExportPolicy export = 12;
+
+  // Let routes say, with communities, which peers they are passed on to,
+  // as the route servers of Internet exchanges do. With L our own AS
+  // number and P a peer's:
+  //
+  //   0:P   or  L:0:P   not to the peer with AS P
+  //   L:P   or  L:1:P   to the peer with AS P, and then only to peers
+  //                     named so
+  //   0:L   or  L:0:0   to nobody
+  //
+  // A route with none of these goes to everyone it otherwise would. "Not"
+  // wins over "to". The two-part form is a standard community, with room
+  // for AS numbers below 65536; the three-part form is a large community
+  // and works for any. They are taken off a route as it is passed on.
+  //
+  // They are honoured on routes we originate too (`communities` of
+  // `originate`), and on what import.add_communities puts on.
+  //
+  // With this on, standard communities L:n are taken for "to the peer
+  // with AS n": mark routes with something else (large communities
+  // L:2:n and up, say).
+  bool control_communities = 13;
 }
 
 // Installing routes in the kernel. Linux only; the daemon needs the
@@ -211,6 +237,55 @@ message Peer {
   // that says nothing (one that does not implement RFC 9234) is taken at
   // this configuration's word for it. Needs role.
   bool require_role = 10;
+
+  // What is announced to the peer, and how. Left out, the top-level
+  // `export` holds, and without that the routes we originate are
+  // announced as they are.
+  ExportPolicy export = 11;
+}
+
+// What is announced to a peer.
+//
+// The routes we originate are always announced. Routes learned from other
+// peers are passed on only if learned_routes says so, and then only as
+// far as these let them:
+//
+// - Never back to the peer a route came from, nor to a peer whose AS is
+//   on its path already.
+// - A route from a peer in our own AS is not passed to another such peer
+//   (there is no route reflection).
+// - Where peers have roles (RFC 9234): a route learned from a provider, a
+//   peer or a route server goes on to customers only. So with roles on
+//   all peers, turning this on makes us transit for our customers and for
+//   nobody else. Without roles there is no such rule, and every route
+//   goes to every peer: do not turn this on for a peer without a role
+//   unless that is what is wanted.
+// - The communities NO_EXPORT and NO_ADVERTISE, and control_communities.
+//
+// A route is passed on with our AS added to its path and ourselves as
+// next hop, without the MED it came with. To a peer in our own AS the
+// path is left as it is and LOCAL_PREF goes along. To a peer whose role
+// is ROUTE_SERVER_CLIENT the route goes as a route server sends it
+// (RFC 7947): path, MED and, where the client can use it, next hop
+// unchanged.
+message ExportPolicy {
+  // Pass on the best routes learned from other peers.
+  bool learned_routes = 1;
+
+  // Communities put on every route announced to the peer: which router
+  // it went through, say. "64601:100", or a well-known name.
+  repeated string add_communities = 2;
+
+  // Large communities put on, as "64601:1:2".
+  repeated string add_large_communities = 3;
+
+  // Communities taken off first: one ("64601:100"), all of an AS
+  // ("64601:*"), or large ones with three parts ("64601:1:*").
+  repeated string remove_communities = 4;
+
+  // How many more times our AS is put on the path of every route
+  // announced to the peer, to make them look further away. At most 16.
+  uint32 as_path_prepend = 5;
 }
 
 // What a peer is to us.
@@ -296,6 +371,23 @@ message ImportPolicy {
   // to. Not needed for a peer whose role is ROUTE_SERVER, whose routes
   // have its clients as next hops.
   bool accept_third_party_next_hop = 14;
+
+  // Communities put on every route taken from the peer: where it was
+  // learned, say, for other routers of ours and for `export` to go by.
+  // "64601:100", or a well-known name.
+  repeated string add_communities = 15;
+
+  // Large communities put on, as "64601:1:2".
+  repeated string add_large_communities = 16;
+
+  // Communities taken off first: one ("64601:100"), all of an AS
+  // ("64601:*"), or large ones with three parts ("64601:1:*"). Marks that
+  // mean something to us are not to be taken on a peer's word.
+  repeated string remove_communities = 17;
+
+  // The LOCAL_PREF of the peer's routes: of two routes for a prefix the
+  // one with the higher is chosen, whatever else they have. Left out, 100.
+  optional uint32 local_pref = 18;
 }
 
 // A prefix, and optionally the more specific prefixes inside it.

crates/server/src/advertise.rs

@@ -18,6 +18,7 @@
 use std::collections::{HashMap, HashSet};
 use std::net::{IpAddr, Ipv6Addr};
 use std::sync::Arc;
+use std::time::Duration;
 
 use bgp_packet::message::{
     AsPathAttribute, AsPathSegment, CommunitiesPathAttribute, LargeCommunitiesPathAttribute,
@@ -26,14 +27,19 @@ use bgp_packet::message::{
 };
 use bgp_primitives::constants::{AfiSafi, IpVersion};
 use bgp_primitives::ip_prefix::{IpPrefix, Nlri};
-use bgp_rib::table::DEFAULT_LOCAL_PREF;
+use bgp_primitives::parser::ParserContext;
+use bgp_rib::table::{AttrSet, DEFAULT_LOCAL_PREF, Event, PeerIndex, Reader, RibTable};
 use bgp_rib::{LocalRib, LocalRibEvent, LocalRoute};
-use tokio::sync::Notify;
+use bytes::BytesMut;
 use tokio::sync::broadcast::Receiver;
 use tokio::sync::broadcast::error::RecvError;
+use tokio::sync::{Notify, OwnedSemaphorePermit, Semaphore};
 use tokio::task::JoinHandle;
 use tracing::{debug, info, warn};
 
+use crate::export::{self, Egress, Outgoing, Source};
+use crate::marks::Control;
+use crate::metrics::ROUTES_EXPORTED;
 use crate::policy::PeerRole;
 use crate::session::{PeerId, SessionClosed, SessionEvent, SessionHandle, SessionRegistry};
 
@@ -41,14 +47,19 @@ use crate::session::{PeerId, SessionClosed, SessionEvent, SessionHandle, Session
 /// this returns, so call it before anything that can bring one up.
 pub fn spawn_advertiser(
     local_rib: LocalRib,
+    rib: Arc<RibTable>,
     registry: SessionRegistry,
     local_asn: u32,
+    control: Option<Control>,
 ) -> JoinHandle<()> {
     let advertiser = Advertiser {
         session_events: registry.subscribe(),
         local_rib,
+        rib,
         registry,
         local_asn,
+        control,
+        resyncs: Arc::new(Semaphore::new(RESYNCS_AT_ONCE)),
         exporters: HashMap::new(),
     };
     tokio::spawn(advertiser.run())
@@ -70,8 +81,14 @@ impl Drop for Running {
 
 struct Advertiser {
     local_rib: LocalRib,
+    /// The routes learned from peers, of which the best are passed on.
+    rib: Arc<RibTable>,
     registry: SessionRegistry,
     local_asn: u32,
+    control: Option<Control>,
+    /// Leave for a session to gather the whole table before it sends it;
+    /// see `Learned::gathered`.
+    resyncs: Arc<Semaphore>,
     session_events: Receiver<SessionEvent>,
     exporters: HashMap<PeerId, Running>,
 }
@@ -113,6 +130,15 @@ impl Advertiser {
             // routes, so that no change between the two is missed.
             changes: self.local_rib.subscribe(),
             local_rib: self.local_rib.clone(),
+            // A follower of the best routes, if the peer is to be told
+            // of them: it is given the whole table first.
+            learned: session
+                .export
+                .learned_routes
+                .then(|| Learned::new(self.rib.reader())),
+            rib: Arc::clone(&self.rib),
+            resyncs: Arc::clone(&self.resyncs),
+            control: self.control,
             session: session.clone(),
             local_asn: self.local_asn,
             resend: Arc::clone(&resend),
@@ -147,8 +173,13 @@ impl Advertiser {
     }
 }
 
-/// Keeps one session's peer up to date with the routes we originate.
+/// Keeps one session's peer up to date with the routes we originate and,
+/// if it is to be told of them, the best of those learned from others.
 struct Exporter {
+    rib: Arc<RibTable>,
+    resyncs: Arc<Semaphore>,
+    control: Option<Control>,
+    learned: Option<Learned>,
     local_rib: LocalRib,
     changes: Receiver<LocalRibEvent>,
     session: SessionHandle,
@@ -159,6 +190,137 @@ struct Exporter {
     announced: HashSet<IpPrefix>,
 }
 
+/// How many changes of best route are taken from the table at a time.
+const EXPORT_BATCH: usize = 65_536;
+
+/// How long changes are gathered before they go out, once there are any.
+const COALESCE: Duration = Duration::from_millis(50);
+
+/// How many sessions may hold a whole table in hand at once while it is
+/// given out to them anew: some tens of megabytes each for the Internet.
+const RESYNCS_AT_ONCE: usize = 4;
+
+/// How many routes ready to go out are remembered, by the attributes they
+/// came with. A table has a few hundred thousand distinct sets.
+const DRESSED_KEPT: usize = 262_144;
+
+fn version_index(version: IpVersion) -> usize {
+    match version {
+        IpVersion::Ipv4 => 0,
+        IpVersion::Ipv6 => 1,
+    }
+}
+
+/// A change of best route, as taken out of the table to be dealt with
+/// once the table is let go of.
+enum News {
+    Reset,
+    Synced,
+    Best {
+        prefix: IpPrefix,
+        peer: PeerIndex,
+        attrs: Arc<AttrSet>,
+    },
+    Withdrawn(IpPrefix),
+}
+
+/// What a learned route goes out as depends on: the attributes it came
+/// with (by where they are), who it came from, and its IP version.
+type DressedKey = (usize, Source, IpVersion);
+
+/// A learned route as this session announces it.
+struct Dressed {
+    outgoing: Outgoing,
+    /// How many octets its attributes take on this session's wire.
+    octets: usize,
+}
+
+/// What an exporter knows of the learned routes it has passed on.
+struct Learned {
+    reader: Reader,
+    /// The verdicts so far on a table that is being given out anew, kept
+    /// until the table is all there; and the leave to keep them.
+    gathered: Option<HashMap<IpPrefix, Option<Arc<Dressed>>>>,
+    permit: Option<OwnedSemaphorePermit>,
+    /// The prefixes announced to the peer and not withdrawn since.
+    announced: HashSet<IpPrefix>,
+    /// While the table is being gone through anew: what was announced
+    /// before and has not been heard of again yet.
+    stale: Option<HashSet<IpPrefix>>,
+    /// Routes as they go out, by the attributes they came with, who they
+    /// came from and their IP version: a table's worth of routes share a
+    /// few sets of attributes, and each is worked out once. The key holds
+    /// the attributes, so that it stays theirs.
+    dressed: HashMap<DressedKey, (Arc<AttrSet>, Option<Arc<Dressed>>)>,
+}
+
+impl Learned {
+    fn new(reader: Reader) -> Self {
+        Self {
+            reader,
+            gathered: None,
+            permit: None,
+            announced: HashSet::new(),
+            stale: None,
+            dressed: HashMap::new(),
+        }
+    }
+
+    /// The route with `attrs` from `source` as it goes out on `egress`
+    /// for prefixes of IP version `version`; `None` if it does not.
+    fn dressed(
+        &mut self,
+        attrs: &Arc<AttrSet>,
+        source: Source,
+        version: IpVersion,
+        egress: &Egress<'_>,
+    ) -> Option<Arc<Dressed>> {
+        let key = (Arc::as_ptr(attrs) as usize, source, version);
+        if let Some((_, dressed)) = self.dressed.get(&key) {
+            return dressed.clone();
+        }
+        let dressed = attrs
+            .attributes()
+            .and_then(|(attributes, next_hop)| {
+                egress.outgoing(attributes, next_hop, source, version)
+            })
+            .map(|outgoing| {
+                // As long as they will be on the wire, to know how many
+                // prefixes fit beside them.
+                let ctx = ParserContext {
+                    four_octet_asn: egress.towards.four_octet_asn,
+                    ibgp: egress.towards.ibgp,
+                    add_path: Vec::new(),
+                };
+                let mut wire = BytesMut::new();
+                let mut attributes = outgoing.attributes.clone();
+                export::finish(&mut attributes, &egress.towards);
+                let octets = attributes
+                    .iter()
+                    .try_for_each(|attribute| attribute.to_wire(&ctx, &mut wire))
+                    .map_or(usize::MAX, |()| wire.len());
+                Arc::new(Dressed { outgoing, octets })
+            })
+            // One that cannot be written is one that does not go.
+            .filter(|dressed| dressed.octets < 3072);
+        if self.dressed.len() >= DRESSED_KEPT {
+            self.dressed.clear();
+        }
+        self.dressed
+            .insert(key, (Arc::clone(attrs), dressed.clone()));
+        dressed
+    }
+}
+
+/// Waits until there may be changes of best route to pass on; for ever if
+/// the peer is not told of them.
+async fn changed(learned: &mut Option<Learned>) {
+    match learned {
+        Some(learned) => learned.reader.changed().await,
+        None => std::future::pending().await,
+    }
+}
+
 /// Why an exporter has nothing more to do.
 struct Finished;
 
@@ -193,11 +355,209 @@ impl Exporter {
                     }
                     Err(RecvError::Closed) => return Err(Finished),
                 },
-                () = self.resend.notified() => self.send_all().await?,
+                () = self.resend.notified() => {
+                    self.send_all().await?;
+                    // Everything, again: a new follower starts with the
+                    // whole table.
+                    if let Some(learned) = &mut self.learned {
+                        learned.reader = self.rib.reader();
+                    }
+                }
+                () = changed(&mut self.learned) => {
+                    // Changes come in runs: a table arriving, a peer
+                    // going. Gathered for a moment, the routes that go out
+                    // alike go in one message, and the whole run costs one
+                    // look at what each peer is told.
+                    tokio::time::sleep(COALESCE).await;
+                    self.follow().await?;
+                }
             }
         }
     }
 
+    /// The session as far as it decides what goes out on it and how.
+    fn towards(&self) -> Towards {
+        let session = &self.session;
+        Towards {
+            local_asn: self.local_asn,
+            next_hop: session.local_addr.ip(),
+            four_octet_asn: session.four_octet_asn,
+            ibgp: session.ibgp,
+            role: session.role,
+            extended_next_hop: session.extended_next_hop,
+        }
+    }
+
+    /// Takes the next changes of best route from the table and tells the
+    /// peer of those that are for it.
+    async fn follow(&mut self) -> Result<(), Finished> {
+        let towards = self.towards();
+        let Some(learned) = &mut self.learned else {
+            return Ok(());
+        };
+        // Only what is needed is taken while the table is held; and all
+        // of what there is, up to a limit, so that it goes out together.
+        let mut news = Vec::new();
+        while news.len() < EXPORT_BATCH {
+            let progress = learned.reader.poll(EXPORT_BATCH - news.len(), |event| {
+                news.push(match event {
+                    Event::Reset => News::Reset,
+                    Event::Synced => News::Synced,
+                    Event::Best {
+                        prefix,
+                        peer,
+                        attrs,
+                        ..
+                    } => News::Best {
+                        prefix,
+                        peer,
+                        attrs: Arc::clone(attrs),
+                    },
+                    Event::Withdrawn { prefix } => News::Withdrawn(prefix),
+                    // Not given to a follower of the best routes.
+                    Event::Routes { .. } => return,
+                });
+            });
+            if progress.caught_up || progress.events == 0 {
+                break;
+            }
+        }
+
+        let session = &self.session;
+        let egress = Egress {
+            towards,
+            peer_asn: session.peer.asn,
+            rib_peer: session.rib_peer,
+            policy: &session.export,
+            control: self.control,
+        };
+        // What the peer is to end up with for each prefix heard of: the
+        // last word on it counts. While the whole table is being given out
+        // anew, which is shard by shard, this is kept until it is all
+        // there: the routes that share attributes are in every shard, and
+        // are only sent together if gathered first.
+        let mut verdicts = learned.gathered.take().unwrap_or_default();
+        let mut synced = false;
+        // Who the routes are from, asked of the table once for each.
+        let mut sources: HashMap<PeerIndex, Option<bool>> = HashMap::new();
+        for item in news {
+            match item {
+                News::Reset => {
+                    // Whatever was announced and is not heard of again by
+                    // the time the table is through is withdrawn then.
+                    let known = std::mem::take(&mut learned.announced);
+                    learned.stale.get_or_insert_default().extend(known);
+                    verdicts.clear();
+                    synced = false;
+                    // Gathering a table takes memory: only so many sessions
+                    // do it at once, the others wait their turn.
+                    if learned.permit.is_none() {
+                        learned.permit = self.resyncs.clone().acquire_owned().await.ok();
+                    }
+                }
+                News::Synced => {
+                    synced = true;
+                    learned.permit = None;
+                }
+                News::Withdrawn(prefix) => {
+                    verdicts.insert(prefix, None);
+                }
+                News::Best {
+                    prefix,
+                    peer,
+                    attrs,
+                } => {
+                    let ibgp = *sources
+                        .entry(peer)
+                        .or_insert_with(|| self.rib.peer(peer).map(|peer| peer.ibgp));
+                    let dressed = match ibgp {
+                        // What we originate ourselves is announced as
+                        // that, and not as someone else's route for it.
+                        Some(ibgp)
+                            if session.families.carries(&prefix)
+                                && !self.announced.contains(&prefix) =>
+                        {
+                            let source = Source { peer, ibgp };
+                            learned.dressed(&attrs, source, prefix.address_family(), &egress)
+                        }
+                        // The peer has gone, and its routes with it.
+                        _ => None,
+                    };
+                    verdicts.insert(prefix, dressed);
+                }
+            }
+        }
+
+        if learned.permit.is_some() && !synced {
+            // Not all there yet.
+            learned.gathered = Some(verdicts);
+            return Ok(());
+        }
+
+        // Withdrawals first, then announcements, routes that go out alike
+        // together.
+        let mut withdrawn: [Vec<Nlri>; 2] = Default::default();
+        let mut announced: HashMap<(usize, IpVersion), (Arc<Dressed>, Vec<Nlri>)> = HashMap::new();
+        for (prefix, verdict) in verdicts {
+            let version = prefix.address_family();
+            // Heard of again: no longer in doubt, whichever way it goes.
+            let was_stale = learned
+                .stale
+                .as_mut()
+                .is_some_and(|stale| stale.remove(&prefix));
+            match verdict {
+                Some(dressed) => {
+                    learned.announced.insert(prefix);
+                    announced
+                        .entry((Arc::as_ptr(&dressed) as usize, version))
+                        .or_insert_with(|| (dressed, Vec::new()))
+                        .1
+                        .push(prefix.into());
+                }
+                None => {
+                    if learned.announced.remove(&prefix) || was_stale {
+                        withdrawn[version_index(version)].push(prefix.into());
+                    }
+                }
+            }
+        }
+        if synced && let Some(stale) = learned.stale.take() {
+            for prefix in stale {
+                withdrawn[version_index(prefix.address_family())].push(prefix.into());
+            }
+        }
+        for (version, routes) in [IpVersion::Ipv4, IpVersion::Ipv6]
+            .into_iter()
+            .zip(withdrawn)
+        {
+            if routes.is_empty() {
+                continue;
+            }
+            ROUTES_EXPORTED
+                .with_label_values(&["withdrawn"])
+                .inc_by(routes.len() as u64);
+            for update in export::withdrawals(version, &routes, &towards) {
+                session.send(Message::Update(update)).await?;
+            }
+        }
+        for ((_, version), (dressed, routes)) in announced {
+            ROUTES_EXPORTED
+                .with_label_values(&["announced"])
+                .inc_by(routes.len() as u64);
+            let updates = export::announcements(
+                &dressed.outgoing,
+                version,
+                &routes,
+                &towards,
+                dressed.octets,
+            );
+            for update in updates {
+                session.send(Message::Update(update)).await?;
+            }
+        }
+        Ok(())
+    }
+
     /// Brings the peer to the routes as they now are: announces every one
     /// of them, and withdraws what was announced and is no longer there.
     ///
@@ -227,15 +587,8 @@ impl Exporter {
             );
             return Ok(());
         }
-        let towards = Towards {
-            local_asn: self.local_asn,
-            next_hop: session.local_addr.ip(),
-            four_octet_asn: session.four_octet_asn,
-            ibgp: session.ibgp,
-            role: session.role,
-            extended_next_hop: session.extended_next_hop,
-        };
-        let Some(update) = build_announce(route, &towards) else {
+        let towards = self.towards();
+        let Some(mut update) = build_announce(route, &towards) else {
             debug!(
                 peer = %session.peer,
                 prefix = %route.prefix,
@@ -243,6 +596,24 @@ impl Exporter {
             );
             return Ok(());
         };
+        // What every route on its way to this peer goes through: whether
+        // its communities let it go there, and what it is marked with.
+        let egress = Egress {
+            towards,
+            peer_asn: session.peer.asn,
+            rib_peer: session.rib_peer,
+            policy: &session.export,
+            control: self.control,
+        };
+        if !egress.dress(&mut update.path_attributes) {
+            debug!(
+                peer = %session.peer,
+                prefix = %route.prefix,
+                "Not announcing route: its communities say not to this peer"
+            );
+            return Ok(());
+        }
+        export::finish(&mut update.path_attributes, &towards);
         debug!(peer = %session.peer, prefix = %route.prefix, "Announcing local route");
         session.send(Message::Update(update)).await?;
         self.announced.insert(route.prefix);
@@ -402,7 +773,7 @@ pub fn build_announce(route: &LocalRoute, towards: &Towards) -> Option<UpdateMes
 /// global address would be and where the link-local one is (RFC 2545 §3
 /// has no form for a link-local address alone). FRR sends it so, and takes
 /// the route for unreachable if it comes by itself; BIRD takes either.
-fn ipv6_next_hop(ours: Ipv6Addr) -> NlriNextHop {
+pub(crate) fn ipv6_next_hop(ours: Ipv6Addr) -> NlriNextHop {
     if ours.is_unicast_link_local() {
         NlriNextHop::Ipv6WithLl {
             global: ours,
@@ -845,7 +1216,13 @@ mod tests {
         fn new() -> Self {
             let registry = SessionRegistry::new();
             let local_rib = LocalRib::spawn();
-            spawn_advertiser(local_rib.clone(), registry.clone(), 65000);
+            spawn_advertiser(
+                local_rib.clone(),
+                Arc::new(RibTable::new(4)),
+                registry.clone(),
+                65000,
+                None,
+            );
             Self {
                 registry,
                 local_rib,

crates/server/src/config.rs

@@ -13,9 +13,10 @@ use bgp_rib::{LocalRoute, LocalRouteAttrs};
 use bgp_rpki::RpkiConfig;
 
 use crate::address::{Endpoint, PeerAddress};
-use crate::error::{ConfigError, ImportProblem, OriginateProblem, PeerProblem};
+use crate::error::{ConfigError, ExportProblem, ImportProblem, OriginateProblem, PeerProblem};
 use crate::family::FamilySet;
-use crate::policy::{ImportPolicy, PeerRole, PrefixSet};
+use crate::marks::{Control, Marks};
+use crate::policy::{ExportPolicy, ImportPolicy, PeerRole, PrefixSet};
 use crate::proto::config::{self as proto, BgpConfig, OriginatedRoute, Peer};
 
 const DEFAULT_BGP_PORT: u16 = 179;
@@ -52,6 +53,11 @@ pub struct Config {
     /// Where route origin validation gets its data; `None` for no
     /// validation.
     pub rpki: Option<RpkiConfig>,
+    /// What a peer is told that has no `export` of its own.
+    pub export: Arc<ExportPolicy>,
+    /// The communities by which routes say who is told of them, if they
+    /// are honoured.
+    pub control: Option<Control>,
 }
 
 #[derive(Debug, Clone)]
@@ -76,6 +82,8 @@ pub struct PeerConfig {
     /// Refuse the peer if it does not say what it is itself (RFC 9234
     /// §4.2, "strict mode").
     pub require_role: bool,
+    /// What the peer is told, and how.
+    pub export: Arc<ExportPolicy>,
 }
 
 impl Config {
@@ -111,6 +119,10 @@ impl Config {
                 .collect::<Result<_, _>>()?
         };
 
+        let export = Arc::new(match proto.export.as_ref() {
+            Some(export) => parse_export(export)?,
+            None => ExportPolicy::default(),
+        });
         let mut peers: Vec<PeerConfig> = Vec::with_capacity(proto.peers.len());
         for peer in proto.peers {
             let name = peer.name.clone();
@@ -118,7 +130,7 @@ impl Config {
                 peer: name.clone(),
                 problem,
             };
-            let peer = parse_peer(peer).map_err(rejected)?;
+            let peer = parse_peer(peer, &export).map_err(rejected)?;
             // RFC 9234 is about what networks are to each other.
             if peer.role.is_some() && peer.remote_asn == proto.local_asn {
                 return Err(rejected(PeerProblem::RoleInOwnAs));
@@ -194,6 +206,10 @@ impl Config {
             http_endpoint,
             kernel: proto.kernel.as_ref().map(parse_kernel),
             rpki,
+            export,
+            control: proto.control_communities.then_some(Control {
+                local_asn: proto.local_asn,
+            }),
         })
     }
 }
@@ -214,7 +230,22 @@ fn parse_kernel(kernel: &proto::Kernel) -> FibConfig {
     }
 }
 
-fn parse_peer(peer: Peer) -> Result<PeerConfig, PeerProblem> {
+fn parse_export(export: &proto::ExportPolicy) -> Result<ExportPolicy, ExportProblem> {
+    Ok(ExportPolicy {
+        learned_routes: export.learned_routes,
+        marks: Marks::parse(
+            &export.remove_communities,
+            &export.add_communities,
+            &export.add_large_communities,
+        )?,
+        prepend: u8::try_from(export.as_path_prepend)
+            .ok()
+            .filter(|prepend| u32::from(*prepend) <= MAX_AS_PATH_PREPEND)
+            .ok_or(ExportProblem::Prepend(export.as_path_prepend))?,
+    })
+}
+
+fn parse_peer(peer: Peer, export: &Arc<ExportPolicy>) -> Result<PeerConfig, PeerProblem> {
     let ep = peer.endpoint.as_ref().ok_or(PeerProblem::MissingEndpoint)?;
     let endpoint = parse_port(ep.port)
         .and_then(|port| Ok(Endpoint::parse(&ep.address, port)?))
@@ -299,6 +330,10 @@ fn parse_peer(peer: Peer) -> Result<PeerConfig, PeerProblem> {
         import,
         role,
         require_role: peer.require_role,
+        export: match peer.export.as_ref() {
+            Some(own) => Arc::new(parse_export(own)?),
+            None => Arc::clone(export),
+        },
     })
 }
 
@@ -387,6 +422,12 @@ fn parse_import(import: &proto::ImportPolicy) -> Result<ImportPolicy, ImportProb
         accept_as_sets: import.accept_as_sets,
         accept_third_party_next_hop: import.accept_third_party_next_hop,
         own_prefixes: Arc::default(),
+        marks: Marks::parse(
+            &import.remove_communities,
+            &import.add_communities,
+            &import.add_large_communities,
+        )?,
+        local_pref: import.local_pref,
         max_as_path_length: match import.max_as_path_length {
             0 => defaults.max_as_path_length,
             length => u16::try_from(length).map_err(|_| ImportProblem::PathLength(length))?,

crates/server/src/connection.rs

@@ -45,7 +45,7 @@ use crate::metrics::{
     SESSIONS_ESTABLISHED_TOTAL, UPDATE_ERRORS, message_type_name,
 };
 use crate::panic::contained;
-use crate::policy::{self, HeldDown, HoldDowns, ImportPolicy, PeerRole};
+use crate::policy::{self, ExportPolicy, HeldDown, HoldDowns, ImportPolicy, PeerRole};
 use crate::reach::{Reach, neighbor_at};
 use crate::refusals::{PeerRefusals, RefusalLog};
 use crate::routes::{self, Held};
@@ -107,13 +107,15 @@ pub struct PeerSettings {
     pub role: Option<PeerRole>,
     /// Refuse the peer if it does not say what it is (RFC 9234).
     pub require_role: bool,
+    /// What the peer is told, and how.
+    pub export: Arc<ExportPolicy>,
 }
 
 impl PeerSettings {
     /// For a peer that connected to us and is not in the configuration.
     /// Such peers are taken only where that was asked for, on a closed
     /// test network, and there everything they announce is taken.
-    pub fn unconfigured(hold_time_secs: u16) -> Self {
+    pub fn unconfigured(hold_time_secs: u16, export: Arc<ExportPolicy>) -> Self {
         Self {
             name: None,
             expected_asn: None,
@@ -123,6 +125,7 @@ impl PeerSettings {
             import: Arc::new(ImportPolicy::permissive()),
             role: None,
             require_role: false,
+            export,
         }
     }
 }
@@ -138,6 +141,7 @@ impl From<&PeerConfig> for PeerSettings {
             import: Arc::new(peer.import.clone()),
             role: peer.role,
             require_role: peer.require_role,
+            export: Arc::clone(&peer.export),
         }
     }
 }
@@ -448,6 +452,8 @@ struct Session {
     import: Arc<ImportPolicy>,
     /// What the peer is to us, if that is known.
     role: Option<PeerRole>,
+    /// What the peer is told, and how.
+    export: Arc<ExportPolicy>,
     /// Where what is refused is noted.
     refusals: Arc<PeerRefusals>,
 }
@@ -503,6 +509,7 @@ impl Session {
             peer_name: peer.name,
             import: peer.import,
             role: peer.role,
+            export: peer.export,
             said: Said::new(),
         };
         Ok((session, first_actions))
@@ -782,6 +789,7 @@ impl Session {
         );
         handle.role = self.role;
         handle.role_confirmed = negotiated.role_confirmed;
+        handle.export = Arc::clone(&self.export);
         // Routes from other networks are validated, where there is data
         // to validate against and the peer's policy does not say to take
         // invalid ones anyway.
@@ -1181,7 +1189,7 @@ mod tests {
             peer_addr,
             Direction::Inbound,
             server,
-            PeerSettings::unconfigured(90),
+            PeerSettings::unconfigured(90, Arc::default()),
         )
         .unwrap();
 
@@ -1230,7 +1238,7 @@ mod tests {
             peer_addr,
             Direction::Inbound,
             server,
-            PeerSettings::unconfigured(90),
+            PeerSettings::unconfigured(90, Arc::default()),
         )
         .unwrap();
         session.send_hold_time = Duration::from_millis(100);

crates/server/src/error.rs

@@ -176,6 +176,8 @@ pub enum ConfigError {
     HttpPort,
     #[error("rpki: no caches are listed")]
     RpkiNoCaches,
+    #[error("export: {0}")]
+    Export(#[from] ExportProblem),
     #[error("rpki: the port of cache {0:?} is required")]
     RpkiCachePort(String),
     #[error("peer {peer:?}: {problem}")]
@@ -218,6 +220,8 @@ pub enum PeerProblem {
     Role,
     #[error("import: {0}")]
     Import(#[from] ImportProblem),
+    #[error("export: {0}")]
+    Export(#[from] ExportProblem),
 }
 
 /// What is wrong with a peer's `import` policy.
@@ -244,6 +248,17 @@ pub enum ImportProblem {
     },
     #[error("max_as_path_length of {0} is more than a path can hold")]
     PathLength(u32),
+    #[error(transparent)]
+    Marks(#[from] crate::marks::MarksProblem),
+}
+
+/// What is wrong with an `export` policy.
+#[derive(Debug, Clone, PartialEq, Eq, Error)]
+pub enum ExportProblem {
+    #[error(transparent)]
+    Marks(#[from] crate::marks::MarksProblem),
+    #[error("as_path_prepend of {0} is more than 16")]
+    Prepend(u32),
 }
 
 /// What is wrong with one `originate` entry.

crates/server/src/export.rs

@@ -0,0 +1,777 @@
+//! Whether a route goes to a peer, and as what.
+//!
+//! This is asked for every route and every peer it might go to, so it is
+//! plain comparisons on what the route carries; nothing here waits or
+//! looks anything up. [`crate::advertise`] does the sending.
+//!
+//! A route we originate always may go, as far as the rules below let it.
+//! A route learned from a peer is passed on only to peers whose `export`
+//! says so, and then:
+//!
+//! - not back to the peer it came from, and not to a peer whose AS is on
+//!   its path already (it would refuse it as a loop);
+//! - not from one peer in our own AS to another (RFC 4271 §9.2.1: there is
+//!   no route reflection here);
+//! - not up or across if it came down or across: a route marked "only to
+//!   customer" goes to customers only (RFC 9234 §5, egress, 2);
+//! - not where its communities say it is not to go: NO_EXPORT and
+//!   NO_ADVERTISE (RFC 1997), and those of [`Control`].
+//!
+//! What goes is changed as BGP has it: our AS on the path and ourselves as
+//! next hop towards another network, the MED it came with dropped; the
+//! path as it was and a LOCAL_PREF towards a peer in our own AS. To a
+//! route server's client the route goes as it came (RFC 7947 §2.2).
+
+use std::net::IpAddr;
+
+use bgp_packet::message::{
+    AsPathAttribute, AsPathSegment, LocalPrefPathAttribute, MpReachNlriPathAttribute,
+    MpUnreachNlriPathAttribute, NextHopPathAttribute, NlriNextHop, PathAttribute, UpdateMessage,
+};
+use bgp_primitives::constants::{AfiSafi, IpVersion};
+use bgp_primitives::ip_prefix::Nlri;
+use bgp_rib::table::{DEFAULT_LOCAL_PREF, PeerIndex};
+
+use crate::advertise::{Towards, ipv6_next_hop};
+use crate::marks::{self, Control, communities_of, well_known_allow};
+use crate::policy::{ExportPolicy, PeerRole};
+
+/// The session a route may go out on, as far as that decides whether it
+/// does and as what.
+#[derive(Debug, Clone, Copy)]
+pub struct Egress<'a> {
+    pub towards: Towards,
+    pub peer_asn: u32,
+    /// The peer's number in the routing table: what its own routes there
+    /// are marked with.
+    pub rib_peer: PeerIndex,
+    pub policy: &'a ExportPolicy,
+    /// The communities by which routes say who is told of them, if they
+    /// are honoured.
+    pub control: Option<Control>,
+}
+
+/// Where a learned route came from.
+#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
+pub struct Source {
+    pub peer: PeerIndex,
+    /// Whether that peer is in our own AS.
+    pub ibgp: bool,
+}
+
+/// A route ready to be announced on a session: its attributes, but for
+/// the ones that depend on which prefixes go with them.
+#[derive(Debug, Clone, PartialEq, Eq)]
+pub struct Outgoing {
+    pub attributes: Vec<PathAttribute>,
+    pub next_hop: NlriNextHop,
+}
+
+impl Egress<'_> {
+    fn transparent(&self) -> bool {
+        !self.towards.ibgp && self.towards.role == Some(PeerRole::RouteServerClient)
+    }
+
+    /// What is common to every route on its way out, originated or
+    /// learned: whether the communities that are for us to act on let it
+    /// go to this peer, and what it is marked with when it does. Returns
+    /// whether it goes.
+    pub fn dress(&self, attributes: &mut Vec<PathAttribute>) -> bool {
+        let towards = &self.towards;
+        let (communities, large) = communities_of(attributes);
+        if let Some(control) = self.control {
+            if !control.allows(communities, large, self.peer_asn) {
+                return false;
+            }
+            control.strip(attributes);
+        }
+        self.policy.marks.apply(attributes);
+        if self.policy.prepend > 0 && !towards.ibgp {
+            let ours = std::iter::repeat_n(towards.local_asn, usize::from(self.policy.prepend));
+            for attribute in attributes.iter_mut() {
+                if let PathAttribute::AsPath(path) = attribute {
+                    prepend(path, ours);
+                    break;
+                }
+            }
+        }
+        // A route that goes down to a customer or across to a peer may not
+        // come up again from there: it is marked, for every network after
+        // us to see (RFC 9234 §5, egress, 1).
+        let marked = attributes
+            .iter()
+            .any(|attribute| matches!(attribute, PathAttribute::OnlyToCustomer(_)));
+        if !marked
+            && let Some(marked_by) = towards
+                .role
+                .filter(|_| !towards.ibgp)
+                .and_then(|role| role.only_to_customer_on_export(towards.local_asn))
+        {
+            marks::put(attributes, PathAttribute::OnlyToCustomer(marked_by));
+        }
+        true
+    }
+
+    /// The route with `attributes` and `next_hop`, learned from `source`,
+    /// as it is announced on this session for prefixes of IP version
+    /// `routes`; `None` if it is not to be.
+    pub fn outgoing(
+        &self,
+        mut attributes: Vec<PathAttribute>,
+        next_hop: Option<NlriNextHop>,
+        source: Source,
+        routes: IpVersion,
+    ) -> Option<Outgoing> {
+        let towards = &self.towards;
+        if !self.policy.learned_routes || source.peer == self.rib_peer {
+            return None;
+        }
+        if source.ibgp && towards.ibgp {
+            return None;
+        }
+        let mut came_with = next_hop;
+        let mut marked = false;
+        let mut on_path = false;
+        for attribute in &attributes {
+            match attribute {
+                PathAttribute::NextHop(next_hop) => {
+                    came_with = came_with.or(Some(NlriNextHop::Ipv4(next_hop.0)));
+                }
+                PathAttribute::OnlyToCustomer(_) => marked = true,
+                PathAttribute::AsPath(path) => {
+                    on_path = path
+                        .segments
+                        .iter()
+                        .any(|segment| segment.path.contains(&self.peer_asn));
+                }
+                _ => {}
+            }
+        }
+        // The peer would refuse it as having been there before.
+        if on_path && !towards.ibgp {
+            return None;
+        }
+        // Whoever sent it said how far it may go (RFC 1997). On a route
+        // of our own the same communities are for those who get it from
+        // us, and it goes out with them.
+        if !well_known_allow(communities_of(&attributes).0, towards.ibgp) {
+            return None;
+        }
+        // It came down or across once: it goes on down only.
+        if marked
+            && !towards.ibgp
+            && towards
+                .role
+                .is_some_and(|role| !role.may_be_sent_only_to_customer_routes())
+        {
+            return None;
+        }
+
+        let transparent = self.transparent();
+        attributes.retain_mut(|attribute| match attribute {
+            // Ours to say, each way.
+            PathAttribute::NextHop(_)
+            | PathAttribute::MpReachNlri(_)
+            | PathAttribute::MpUnreachNlri(_)
+            | PathAttribute::As4Path(_) => false,
+            // For the routers of the AS it was learned in, and for ours
+            // (RFC 4271 §5.1.4, §5.1.5).
+            PathAttribute::MultiExitDisc(_) => towards.ibgp || transparent,
+            PathAttribute::LocalPref(_) => towards.ibgp,
+            // What we do not understand goes on if it says it may, marked
+            // as having passed a router that did not look at it; and no
+            // further if not (RFC 4271 §5).
+            PathAttribute::UnknownPathAttribute { flags, .. } => {
+                let passes = flags.optional() && flags.transitive();
+                if passes {
+                    flags.set_partial(true);
+                }
+                passes
+            }
+            _ => true,
+        });
+        if towards.ibgp {
+            if !attributes
+                .iter()
+                .any(|attribute| matches!(attribute, PathAttribute::LocalPref(_)))
+            {
+                marks::put(
+                    &mut attributes,
+                    PathAttribute::LocalPref(LocalPrefPathAttribute(DEFAULT_LOCAL_PREF)),
+                );
+            }
+        } else if !transparent {
+            let ours = std::iter::once(towards.local_asn);
+            match attributes.iter_mut().find_map(|attribute| match attribute {
+                PathAttribute::AsPath(path) => Some(path),
+                _ => None,
+            }) {
+                Some(path) => prepend(path, ours),
+                None => marks::put(
+                    &mut attributes,
+                    AsPathAttribute::from_asns(vec![towards.local_asn]),
+                ),
+            }
+        }
+        if !self.dress(&mut attributes) {
+            return None;
+        }
+
+        // Ourselves, unless we are a route server and the client can use
+        // the next hop the route came with.
+        let ours = match (routes, towards.next_hop) {
+            (IpVersion::Ipv4, IpAddr::V4(ours)) => NlriNextHop::Ipv4(ours),
+            (IpVersion::Ipv6, IpAddr::V6(ours)) => ipv6_next_hop(ours),
+            (IpVersion::Ipv4, IpAddr::V6(ours)) if towards.extended_next_hop => ipv6_next_hop(ours),
+            _ => return None,
+        };
+        let same_kind = |theirs: &NlriNextHop| {
+            matches!(theirs, NlriNextHop::Ipv4(_)) == matches!(ours, NlriNextHop::Ipv4(_))
+        };
+        let next_hop = came_with
+            .filter(|theirs| transparent && same_kind(theirs))
+            .unwrap_or(ours);
+        Some(Outgoing {
+            attributes,
+            next_hop,
+        })
+    }
+}
+
+/// Puts `ours` at the front of `path`.
+fn prepend(path: &mut AsPathAttribute, ours: impl Iterator<Item = u32>) {
+    match path.segments.first_mut() {
+        Some(first) if first.ordered => {
+            first.path.splice(0..0, ours);
+        }
+        _ => path.segments.insert(
+            0,
+            AsPathSegment {
+                ordered: true,
+                path: ours.collect(),
+            },
+        ),
+    }
+}
+
+/// Makes `attributes` ready for the wire of a session: the path also as
+/// AS4_PATH where the session cannot carry its numbers (RFC 6793 §4.2.2),
+/// and everything in the order of the type codes.
+pub(crate) fn finish(attributes: &mut Vec<PathAttribute>, towards: &Towards) {
+    attributes.retain(|attribute| !matches!(attribute, PathAttribute::As4Path(_)));
+    if !towards.four_octet_asn {
+        let as4_path = attributes.iter().find_map(|attribute| match attribute {
+            PathAttribute::AsPath(path) => path.as4_path(),
+            _ => None,
+        });
+        attributes.extend(as4_path);
+    }
+    attributes.sort_by_key(PathAttribute::type_code);
+}
+
+/// How many octets a message has room for after its header and the
+/// lengths every UPDATE has.
+const UPDATE_ROOM: usize = 4096 - 19 - 4;
+
+/// The most octets a prefix takes in an UPDATE: a path identifier, a
+/// length, and the address.
+fn nlri_octets(routes: IpVersion) -> usize {
+    4 + 1 + usize::from(routes.bits()) / 8
+}
+
+/// The UPDATEs that announce `routes`, all of IP version `version`, as
+/// `outgoing`: as few as they fit in. `attributes_octets` is how long
+/// `outgoing`'s attributes are on this session's wire.
+pub(crate) fn announcements(
+    outgoing: &Outgoing,
+    version: IpVersion,
+    routes: &[Nlri],
+    towards: &Towards,
+    attributes_octets: usize,
+) -> Vec<UpdateMessage> {
+    // What the next hop takes, in its own attribute or with the prefixes,
+    // and a little to spare.
+    const NEXT_HOP_ROOM: usize = 64;
+    let room = UPDATE_ROOM.saturating_sub(attributes_octets + NEXT_HOP_ROOM);
+    let at_once = (room / nlri_octets(version)).max(1);
+    let in_body = matches!(outgoing.next_hop, NlriNextHop::Ipv4(_)) && version == IpVersion::Ipv4;
+    routes
+        .chunks(at_once)
+        .map(|routes| {
+            let mut attributes = outgoing.attributes.clone();
+            let mut announced_routes = Vec::new();
+            match outgoing.next_hop {
+                NlriNextHop::Ipv4(next_hop) if in_body => {
+                    attributes.push(PathAttribute::NextHop(NextHopPathAttribute(next_hop)));
+                    announced_routes = routes.to_vec();
+                }
+                next_hop => attributes.push(PathAttribute::MpReachNlri(MpReachNlriPathAttribute {
+                    family: AfiSafi::unicast(version),
+                    next_hop,
+                    prefixes: routes.to_vec(),
+                })),
+            }
+            finish(&mut attributes, towards);
+            UpdateMessage {
+                withdrawn_routes: vec![],
+                path_attributes: attributes,
+                announced_routes,
+            }
+        })
+        .collect()
+}
+
+/// The UPDATEs that withdraw `routes`, all of IP version `version`.
+pub(crate) fn withdrawals(
+    version: IpVersion,
+    routes: &[Nlri],
+    towards: &Towards,
+) -> Vec<UpdateMessage> {
+    let at_once = (UPDATE_ROOM - 16) / nlri_octets(version);
+    // IPv4 routes that went in MP_REACH_NLRI are withdrawn the same way.
+    let in_body =
+        version == IpVersion::Ipv4 && !(towards.extended_next_hop && towards.next_hop.is_ipv6());
+    routes
+        .chunks(at_once)
+        .map(|routes| {
+            if in_body {
+                UpdateMessage {
+                    withdrawn_routes: routes.to_vec(),
+                    path_attributes: vec![],
+                    announced_routes: vec![],
+                }
+            } else {
+                UpdateMessage {
+                    withdrawn_routes: vec![],
+                    path_attributes: vec![PathAttribute::MpUnreachNlri(
+                        MpUnreachNlriPathAttribute {
+                            family: AfiSafi::unicast(version),
+                            prefixes: routes.to_vec(),
+                        },
+                    )],
+                    announced_routes: vec![],
+                }
+            }
+        })
+        .collect()
+}
+
+#[cfg(test)]
+mod tests {
+    use std::net::Ipv4Addr;
+
+    use bgp_packet::message::{
+        CommunitiesPathAttribute, MultiExitDiscPathAttribute, OriginPathAttribute,
+        PathAttributeFlags,
+    };
+    use bgp_primitives::ip_prefix::IpPrefix;
+    use bgp_rib::table::{PeerInfo, RibTable};
+
+    use super::*;
+    use crate::marks::Marks;
+
+    const LOCAL: u32 = 64601;
+    const THEM: u32 = 64602;
+
+    /// Two peers' numbers in a table: the one a route is from, and the one
+    /// it might go to.
+    fn peers() -> (PeerIndex, PeerIndex) {
+        let table = RibTable::new(1);
+        let peer = |n: u8| {
+            table
+                .add_peer(PeerInfo {
+                    asn: 64600 + u32::from(n),
+                    router_id: Ipv4Addr::new(10, 0, 0, n),
+                    address: Ipv4Addr::new(192, 0, 2, n).into(),
+                    interface: 0,
+                    ibgp: false,
+                })
+                .unwrap()
+        };
+        (peer(1), peer(2))
+    }
+
+    fn towards(role: Option<PeerRole>, ibgp: bool) -> Towards {
+        Towards {
+            local_asn: LOCAL,
+            next_hop: Ipv4Addr::new(192, 0, 2, 1).into(),
+            four_octet_asn: true,
+            ibgp,
+            role,
+            extended_next_hop: false,
+        }
+    }
+
+    fn all_routes() -> ExportPolicy {
+        ExportPolicy {
+            learned_routes: true,
+            ..ExportPolicy::default()
+        }
+    }
+
+    /// A route from AS 64700 by way of AS 3356, next hop 192.0.2.9.
+    fn route(more: Vec<PathAttribute>) -> Vec<PathAttribute> {
+        let mut attributes = vec![
+            PathAttribute::Origin(OriginPathAttribute::IGP),
+            AsPathAttribute::from_asns(vec![3356, 64700]),
+            PathAttribute::NextHop(NextHopPathAttribute(Ipv4Addr::new(192, 0, 2, 9))),
+        ];
+        attributes.extend(more);
+        attributes
+    }
+
+    fn communities(communities: &[(u16, u16)]) -> PathAttribute {
+        PathAttribute::Communities(CommunitiesPathAttribute(communities.to_vec()))
+    }
+
+    struct Case<'a> {
+        policy: &'a ExportPolicy,
+        role: Option<PeerRole>,
+        to_ibgp: bool,
+        from_ibgp: bool,
+        control: Option<Control>,
+    }
+
+    impl Case<'_> {
+        fn out(&self, attributes: Vec<PathAttribute>) -> Option<Outgoing> {
+            let (from, to) = peers();
+            let egress = Egress {
+                towards: towards(self.role, self.to_ibgp),
+                peer_asn: if self.to_ibgp { LOCAL } else { THEM },
+                rib_peer: to,
+                policy: self.policy,
+                control: self.control,
+            };
+            let source = Source {
+                peer: from,
+                ibgp: self.from_ibgp,
+            };
+            egress.outgoing(attributes, None, source, IpVersion::Ipv4)
+        }
+    }
+
+    fn plain(policy: &ExportPolicy) -> Case<'_> {
+        Case {
+            policy,
+            role: None,
+            to_ibgp: false,
+            from_ibgp: false,
+            control: None,
+        }
+    }
+
+    /// To another network: our AS on the path, ourselves as next hop, and
+    /// nothing that was between the route's last AS and us.
+    #[test]
+    fn a_route_is_passed_on_to_another_network_as_bgp_has_it() {
+        let policy = all_routes();
+        let came = route(vec![
+            PathAttribute::MultiExitDisc(MultiExitDiscPathAttribute(50)),
+            PathAttribute::LocalPref(LocalPrefPathAttribute(300)),
+            communities(&[(3356, 3)]),
+            // Understood by nobody here: one that may go on, one that may
+            // not.
+            PathAttribute::UnknownPathAttribute {
+                flags: PathAttributeFlags::new(true, true, false, false),
+                type_code: 200,
+                payload: vec![1],
+            },
+            PathAttribute::UnknownPathAttribute {
+                flags: PathAttributeFlags::new(true, false, false, false),
+                type_code: 201,
+                payload: vec![2],
+            },
+        ]);
+        let outgoing = plain(&policy).out(came).unwrap();
+        assert_eq!(
+            outgoing.next_hop,
+            NlriNextHop::Ipv4(Ipv4Addr::new(192, 0, 2, 1))
+        );
+        assert_eq!(
+            outgoing.attributes,
+            [
+                PathAttribute::Origin(OriginPathAttribute::IGP),
+                AsPathAttribute::from_asns(vec![LOCAL, 3356, 64700]),
+                communities(&[(3356, 3)]),
+                PathAttribute::UnknownPathAttribute {
+                    // Marked as having passed a router that did not know it.
+                    flags: PathAttributeFlags::new(true, true, true, false),
+                    type_code: 200,
+                    payload: vec![1],
+                },
+            ]
+        );
+    }
+
+    #[test]
+    fn a_route_is_passed_on_only_where_it_may_go() {
+        let policy = all_routes();
+        let (from, _) = peers();
+        // Only if the peer is told of learned routes at all.
+        let originated_only = ExportPolicy::default();
+        assert!(plain(&originated_only).out(route(vec![])).is_none());
+        assert!(plain(&policy).out(route(vec![])).is_some());
+
+        // Not back to where it came from.
+        let egress = Egress {
+            towards: towards(None, false),
+            peer_asn: THEM,
+            rib_peer: from,
+            policy: &policy,
+            control: None,
+        };
+        let source = Source {
+            peer: from,
+            ibgp: false,
+        };
+        assert!(
+            egress
+                .outgoing(route(vec![]), None, source, IpVersion::Ipv4)
+                .is_none()
+        );
+
+        // Not to a peer whose AS it has been through.
+        let mut through_them = route(vec![]);
+        through_them[1] = AsPathAttribute::from_asns(vec![3356, THEM, 64700]);
+        assert!(plain(&policy).out(through_them).is_none());
+
+        // Not where the communities it came with say not to (RFC 1997).
+        let no_export = route(vec![communities(&[(0xFFFF, 0xFF01)])]);
+        assert!(plain(&policy).out(no_export.clone()).is_none());
+        let internal = Case {
+            to_ibgp: true,
+            ..plain(&policy)
+        };
+        assert!(internal.out(no_export).is_some());
+        let no_advertise = route(vec![communities(&[(0xFFFF, 0xFF02)])]);
+        assert!(internal.out(no_advertise).is_none());
+    }
+
+    /// Inside our own AS the path is as it came and LOCAL_PREF goes along;
+    /// and a route does not go from one internal peer to another.
+    #[test]
+    fn a_route_goes_to_a_peer_in_our_own_as_unchanged_but_for_the_next_hop() {
+        let policy = all_routes();
+        let internal = Case {
+            to_ibgp: true,
+            ..plain(&policy)
+        };
+        let came = route(vec![PathAttribute::MultiExitDisc(
+            MultiExitDiscPathAttribute(50),
+        )]);
+        let outgoing = internal.out(came).unwrap();
+        assert_eq!(
+            outgoing.attributes,
+            [
+                PathAttribute::Origin(OriginPathAttribute::IGP),
+                AsPathAttribute::from_asns(vec![3356, 64700]),
+                PathAttribute::MultiExitDisc(MultiExitDiscPathAttribute(50)),
+                PathAttribute::LocalPref(LocalPrefPathAttribute(DEFAULT_LOCAL_PREF)),
+            ]
+        );
+        // The preference import policy gave it is what the others hear.
+        let preferred = route(vec![PathAttribute::LocalPref(LocalPrefPathAttribute(300))]);
+        assert!(
+            internal
+                .out(preferred)
+                .unwrap()
+                .attributes
+                .contains(&PathAttribute::LocalPref(LocalPrefPathAttribute(300)))
+        );
+
+        let between_internal = Case {
+            to_ibgp: true,
+            from_ibgp: true,
+            ..plain(&policy)
+        };
+        assert!(between_internal.out(route(vec![])).is_none());
+        let from_internal = Case {
+            from_ibgp: true,
+            ..plain(&policy)
+        };
+        assert!(from_internal.out(route(vec![])).is_some());
+    }
+
+    /// RFC 9234 §5, egress: what came down or across goes on down only,
+    /// and what goes down or across is marked.
+    #[test]
+    fn a_route_only_for_customers_goes_to_customers_only() {
+        use PeerRole::*;
+        let policy = all_routes();
+        let marked = || route(vec![PathAttribute::OnlyToCustomer(3356)]);
+        for (role, goes) in [
+            (Customer, true),
+            (RouteServerClient, true),
+            (Provider, false),
+            (Peer, false),
+            (RouteServer, false),
+        ] {
+            let case = Case {
+                role: Some(role),
+                ..plain(&policy)
+            };
+            assert_eq!(case.out(marked()).is_some(), goes, "{role:?}");
+        }
+        // The mark it has is kept; one it lacks is put on, with our AS,
+        // on the way down or across.
+        let mark = |role, came: Vec<PathAttribute>| {
+            let case = Case {
+                role: Some(role),
+                ..plain(&policy)
+            };
+            case.out(came)
+                .unwrap()
+                .attributes
+                .iter()
+                .find_map(|attribute| match attribute {
+                    PathAttribute::OnlyToCustomer(asn) => Some(*asn),
+                    _ => None,
+                })
+        };
+        assert_eq!(mark(Customer, marked()), Some(3356));
+        assert_eq!(mark(Customer, route(vec![])), Some(LOCAL));
+        assert_eq!(mark(Peer, route(vec![])), Some(LOCAL));
+        assert_eq!(mark(Provider, route(vec![])), None);
+        // A peer that is nothing in particular: no rule, no mark.
+        assert!(plain(&policy).out(marked()).is_some());
+    }
+
+    /// RFC 7947 §2.2: a route server is not on the path, and not in the
+    /// way of the packets.
+    #[test]
+    fn a_route_server_passes_routes_on_as_they_came() {
+        let policy = all_routes();
+        let client = Case {
+            role: Some(PeerRole::RouteServerClient),
+            ..plain(&policy)
+        };
+        let came = route(vec![PathAttribute::MultiExitDisc(
+            MultiExitDiscPathAttribute(50),
+        )]);
+        let outgoing = client.out(came).unwrap();
+        assert_eq!(
+            outgoing.next_hop,
+            NlriNextHop::Ipv4(Ipv4Addr::new(192, 0, 2, 9))
+        );
+        assert_eq!(
+            outgoing.attributes,
+            [
+                PathAttribute::Origin(OriginPathAttribute::IGP),
+                AsPathAttribute::from_asns(vec![3356, 64700]),
+                PathAttribute::MultiExitDisc(MultiExitDiscPathAttribute(50)),
+                PathAttribute::OnlyToCustomer(LOCAL),
+            ]
+        );
+    }
+
+    /// The marks of `export`, and the communities that say who is told.
+    #[test]
+    fn communities_decide_and_are_changed_on_the_way_out() {
+        let strings =
+            |texts: &[&str]| -> Vec<String> { texts.iter().map(|text| text.to_string()).collect() };
+        let policy = ExportPolicy {
+            learned_routes: true,
+            marks: Marks::parse(&strings(&["64601:900"]), &strings(&["64601:5"]), &[]).unwrap(),
+            prepend: 2,
+        };
+        let control = Some(Control { local_asn: LOCAL });
+        let case = Case {
+            control,
+            ..plain(&policy)
+        };
+        // Not to this peer; to this peer only; and with nothing said.
+        assert!(case.out(route(vec![communities(&[(0, 64602)])])).is_none());
+        let outgoing = case
+            .out(route(vec![communities(&[
+                (64601, 64602),
+                (3356, 3),
+                (64601, 900),
+            ])]))
+            .unwrap();
+        assert_eq!(
+            outgoing.attributes,
+            [
+                PathAttribute::Origin(OriginPathAttribute::IGP),
+                // Ours once for passing it on, and twice more by policy.
+                AsPathAttribute::from_asns(vec![LOCAL, LOCAL, LOCAL, 3356, 64700]),
+                // What was for us is gone; the router's mark is on.
+                communities(&[(3356, 3), (64601, 5)]),
+            ]
+        );
+        assert!(
+            case.out(route(vec![communities(&[(64601, 64999)])]))
+                .is_none(),
+            "for another peer only"
+        );
+        // Without control communities the same are only communities.
+        assert!(
+            plain(&policy)
+                .out(route(vec![communities(&[(0, 64602)])]))
+                .is_some()
+        );
+    }
+
+    /// As many prefixes to a message as fit, and the path as AS4_PATH too
+    /// where the session cannot carry it.
+    #[test]
+    fn routes_that_go_out_alike_share_their_messages() {
+        let outgoing = Outgoing {
+            attributes: vec![
+                PathAttribute::Origin(OriginPathAttribute::IGP),
+                AsPathAttribute::from_asns(vec![LOCAL, 4_200_000_001]),
+            ],
+            next_hop: NlriNextHop::Ipv4(Ipv4Addr::new(192, 0, 2, 1)),
+        };
+        let routes: Vec<Nlri> = (0..3000u32)
+            .map(|n| {
+                let [_, _, a, b] = n.to_be_bytes();
+                IpPrefix::new(IpVersion::Ipv4, [100, a, b, 0], 24)
+                    .unwrap()
+                    .into()
+            })
+            .collect();
+        let narrow = Towards {
+            four_octet_asn: false,
+            ..towards(None, false)
+        };
+        let updates = announcements(&outgoing, IpVersion::Ipv4, &routes, &narrow, 40);
+        assert!(updates.len() < 8, "{} messages", updates.len());
+        let told: usize = updates.iter().map(|u| u.announced_routes.len()).sum();
+        assert_eq!(told, routes.len());
+        let ctx = bgp_primitives::parser::ParserContext {
+            four_octet_asn: false,
+            ibgp: false,
+            add_path: Vec::new(),
+        };
+        for update in &updates {
+            let codes: Vec<u8> = update
+                .path_attributes
+                .iter()
+                .map(PathAttribute::type_code)
+                .collect();
+            assert_eq!(codes, [1, 2, 3, 17], "AS4_PATH with it");
+            let mut wire = bytes::BytesMut::new();
+            bgp_packet::message::Message::Update(update.clone())
+                .to_wire(&ctx, &mut wire)
+                .expect("it fits a message");
+        }
+        // IPv6 routes, and IPv4 ones with an IPv6 next hop, go in
+        // MP_REACH_NLRI; and are withdrawn in MP_UNREACH_NLRI.
+        let v6 = Outgoing {
+            next_hop: NlriNextHop::Ipv6("2001:db8::1".parse().unwrap()),
+            ..outgoing
+        };
+        let update = &announcements(&v6, IpVersion::Ipv4, &routes[..2], &narrow, 40)[0];
+        assert!(update.announced_routes.is_empty());
+        assert!(update.path_attributes.iter().any(|attribute| matches!(
+            attribute,
+            PathAttribute::MpReachNlri(mp) if mp.prefixes.len() == 2
+        )));
+        let gone = withdrawals(IpVersion::Ipv4, &routes, &narrow);
+        assert_eq!(
+            gone.iter().map(|u| u.withdrawn_routes.len()).sum::<usize>(),
+            routes.len()
+        );
+        assert!(gone.len() < 8);
+    }
+}

crates/server/src/lib.rs

@@ -5,8 +5,10 @@ pub mod config;
 mod connection;
 mod dialer;
 pub mod error;
+pub mod export;
 pub mod family;
 mod fsm;
+pub mod marks;
 pub mod metrics;
 mod panic;
 pub mod policy;

crates/server/src/marks.rs

@@ -0,0 +1,497 @@
+//! What routes are marked with, and what marks tell us to do.
+//!
+//! A network marks routes with communities to remember things about
+//! them: where a route was learned, which router passed it on. And a
+//! route's communities can say who is to be told of it. Both are fixed
+//! rules with a few numbers in them, so they are configuration, and what
+//! they cost is a look through a route's communities:
+//!
+//! - [`Marks`]: communities taken off and put on every route from a peer
+//!   (`import`) or to one (`export`).
+//! - [`Control`]: communities on a route that say which peers it is, and
+//!   is not, passed on to.
+
+use bgp_packet::community::{CommunityError, parse_large, parse_standard};
+use bgp_packet::message::{
+    CommunitiesPathAttribute, LargeCommunitiesPathAttribute, LargeCommunity, PathAttribute,
+};
+
+/// A standard community, or all those with one half or both left open:
+/// `64601:100`, `64601:*`, `*:*`.
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub struct Pattern(Option<u16>, Option<u16>);
+
+/// The same for a large community: `64601:1:*`.
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub struct LargePattern(Option<u32>, Option<u32>, Option<u32>);
+
+fn fits<T: PartialEq>(wanted: Option<T>, has: T) -> bool {
+    wanted.is_none_or(|wanted| wanted == has)
+}
+
+impl Pattern {
+    fn matches(&self, community: (u16, u16)) -> bool {
+        fits(self.0, community.0) && fits(self.1, community.1)
+    }
+}
+
+impl LargePattern {
+    fn matches(&self, community: &LargeCommunity) -> bool {
+        fits(self.0, community.global_admin)
+            && fits(self.1, community.data_1)
+            && fits(self.2, community.data_2)
+    }
+}
+
+/// What is wrong with the communities of a policy.
+#[derive(Debug, Clone, PartialEq, Eq, thiserror::Error)]
+pub enum MarksProblem {
+    #[error(transparent)]
+    Community(#[from] CommunityError),
+    #[error(
+        "{0:?} is not a community, or a pattern of them: 64601:100, 64601:*, or with three \
+         numbers for large communities"
+    )]
+    Pattern(String),
+}
+
+/// One part of a pattern: a number, or `*` for any.
+fn part<T: std::str::FromStr>(text: &str, whole: &str) -> Result<Option<T>, MarksProblem> {
+    match text.trim() {
+        "*" => Ok(None),
+        number => number
+            .parse()
+            .map(Some)
+            .map_err(|_| MarksProblem::Pattern(whole.to_string())),
+    }
+}
+
+/// What is done to the communities of every route that goes one way on
+/// one session: some are taken off, then some are put on.
+#[derive(Debug, Clone, Default, PartialEq, Eq)]
+pub struct Marks {
+    remove: Vec<Pattern>,
+    remove_large: Vec<LargePattern>,
+    add: Vec<(u16, u16)>,
+    add_large: Vec<LargeCommunity>,
+}
+
+impl Marks {
+    /// From the configuration. `remove` has patterns of standard
+    /// communities (two parts) and of large ones (three).
+    pub fn parse(
+        remove: &[String],
+        add: &[String],
+        add_large: &[String],
+    ) -> Result<Self, MarksProblem> {
+        let mut marks = Self::default();
+        for text in remove {
+            let parts: Vec<&str> = text.split(':').collect();
+            match parts.as_slice() {
+                [high, low] => marks
+                    .remove
+                    .push(Pattern(part(high, text)?, part(low, text)?)),
+                [admin, first, second] => marks.remove_large.push(LargePattern(
+                    part(admin, text)?,
+                    part(first, text)?,
+                    part(second, text)?,
+                )),
+                _ => return Err(MarksProblem::Pattern(text.clone())),
+            }
+        }
+        for text in add {
+            let community = parse_standard(text)?;
+            if !marks.add.contains(&community) {
+                marks.add.push(community);
+            }
+        }
+        for text in add_large {
+            let community = parse_large(text)?;
+            if !marks.add_large.contains(&community) {
+                marks.add_large.push(community);
+            }
+        }
+        Ok(marks)
+    }
+
+    pub fn is_empty(&self) -> bool {
+        *self == Self::default()
+    }
+
+    /// Does it to `attributes`.
+    pub fn apply(&self, attributes: &mut Vec<PathAttribute>) {
+        if !self.remove.is_empty() || !self.add.is_empty() {
+            let mut communities = take_communities(attributes);
+            communities.retain(|community| !self.remove.iter().any(|p| p.matches(*community)));
+            for community in &self.add {
+                if !communities.contains(community) {
+                    communities.push(*community);
+                }
+            }
+            put_communities(attributes, communities);
+        }
+        if !self.remove_large.is_empty() || !self.add_large.is_empty() {
+            let mut large = take_large_communities(attributes);
+            large.retain(|community| !self.remove_large.iter().any(|p| p.matches(community)));
+            for community in &self.add_large {
+                if !large.contains(community) {
+                    large.push(community.clone());
+                }
+            }
+            put_large_communities(attributes, large);
+        }
+    }
+}
+
+/// Takes the standard communities out of `attributes`.
+fn take_communities(attributes: &mut Vec<PathAttribute>) -> Vec<(u16, u16)> {
+    let at = attributes
+        .iter()
+        .position(|attribute| matches!(attribute, PathAttribute::Communities(_)));
+    match at.map(|at| attributes.remove(at)) {
+        Some(PathAttribute::Communities(communities)) => communities.0,
+        _ => Vec::new(),
+    }
+}
+
+fn take_large_communities(attributes: &mut Vec<PathAttribute>) -> Vec<LargeCommunity> {
+    let at = attributes
+        .iter()
+        .position(|attribute| matches!(attribute, PathAttribute::LargeCommunities(_)));
+    match at.map(|at| attributes.remove(at)) {
+        Some(PathAttribute::LargeCommunities(large)) => large.communities,
+        _ => Vec::new(),
+    }
+}
+
+fn put_communities(attributes: &mut Vec<PathAttribute>, communities: Vec<(u16, u16)>) {
+    if !communities.is_empty() {
+        put(
+            attributes,
+            PathAttribute::Communities(CommunitiesPathAttribute(communities)),
+        );
+    }
+}
+
+fn put_large_communities(attributes: &mut Vec<PathAttribute>, communities: Vec<LargeCommunity>) {
+    if !communities.is_empty() {
+        put(
+            attributes,
+            PathAttribute::LargeCommunities(LargeCommunitiesPathAttribute { communities }),
+        );
+    }
+}
+
+/// Puts `attribute` among `attributes`, in place of one of its type if
+/// there is one, and else where it belongs in the order of type codes.
+pub(crate) fn put(attributes: &mut Vec<PathAttribute>, attribute: PathAttribute) {
+    let type_code = attribute.type_code();
+    match attributes
+        .iter()
+        .position(|other| other.type_code() >= type_code)
+    {
+        Some(at) if attributes[at].type_code() == type_code => attributes[at] = attribute,
+        Some(at) => attributes.insert(at, attribute),
+        None => attributes.push(attribute),
+    }
+}
+
+/// The communities by which a route says who is told of it, as Internet
+/// exchanges' route servers have them. With `L` our own AS number and `P`
+/// a peer's:
+///
+/// | Standard | Large | |
+/// |---|---|---|
+/// | `0:P` | `L:0:P` | not to the peer with AS `P` |
+/// | `L:P` | `L:1:P` | to the peer with AS `P`, and with that only to peers named so |
+/// | `0:L` | `L:0:0` | to nobody |
+///
+/// A route with none of them goes to everyone. What says "not" wins over
+/// what says "to". A standard community has room for AS numbers below
+/// 65536 only; the large ones are for the others, and work for all.
+///
+/// These are taken off a route as it is passed on: they were for us.
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub struct Control {
+    pub local_asn: u32,
+}
+
+/// The well-known communities that limit how far a route goes
+/// (RFC 1997).
+const NO_EXPORT: (u16, u16) = (0xFFFF, 0xFF01);
+const NO_ADVERTISE: (u16, u16) = (0xFFFF, 0xFF02);
+const NO_EXPORT_SUBCONFED: (u16, u16) = (0xFFFF, 0xFF03);
+
+/// Whether a route with these standard communities may go to a peer at
+/// all, as far as the well-known ones say: NO_ADVERTISE keeps it from
+/// everyone, NO_EXPORT from other networks. (There are no confederations
+/// here, so NO_EXPORT_SUBCONFED is the same as NO_EXPORT.)
+pub fn well_known_allow(communities: &[(u16, u16)], peer_is_internal: bool) -> bool {
+    !communities.iter().any(|community| {
+        *community == NO_ADVERTISE
+            || (!peer_is_internal && matches!(*community, NO_EXPORT | NO_EXPORT_SUBCONFED))
+    })
+}
+
+impl Control {
+    /// Our AS number as half a standard community, if it fits.
+    fn local_16(&self) -> Option<u16> {
+        u16::try_from(self.local_asn).ok()
+    }
+
+    /// Whether a route with these communities is passed on to the peer
+    /// with AS number `peer_asn`.
+    pub fn allows(
+        &self,
+        communities: &[(u16, u16)],
+        large: &[LargeCommunity],
+        peer_asn: u32,
+    ) -> bool {
+        let local_16 = self.local_16();
+        let peer_16 = u16::try_from(peer_asn).ok();
+        let (mut not_to_peer, mut to_peer, mut to_nobody, mut to_some) =
+            (false, false, false, false);
+        for (high, low) in communities {
+            if *high == 0 {
+                not_to_peer |= Some(*low) == peer_16;
+                to_nobody |= Some(*low) == local_16;
+            } else if Some(*high) == local_16 {
+                to_peer |= Some(*low) == peer_16;
+                // `L:L` is an exchange's "to everyone", which is what a
+                // route with nothing on it gets anyway.
+                to_some |= Some(*low) != local_16;
+            }
+        }
+        for community in large {
+            if community.global_admin != self.local_asn {
+                continue;
+            }
+            match (community.data_1, community.data_2) {
+                (0, 0) => to_nobody = true,
+                (0, asn) => not_to_peer |= asn == peer_asn,
+                (1, asn) => {
+                    to_peer |= asn == peer_asn;
+                    to_some = true;
+                }
+                _ => {}
+            }
+        }
+        !not_to_peer && (to_peer || !(to_nobody || to_some))
+    }
+
+    /// Takes off `attributes` the communities that were for us to act on.
+    pub fn strip(&self, attributes: &mut Vec<PathAttribute>) {
+        let local_16 = self.local_16();
+        let ours = |community: &(u16, u16)| community.0 == 0 || Some(community.0) == local_16;
+        let has = attributes.iter().any(|attribute| match attribute {
+            PathAttribute::Communities(communities) => communities.0.iter().any(ours),
+            _ => false,
+        });
+        if has {
+            let mut communities = take_communities(attributes);
+            communities.retain(|community| !ours(community));
+            put_communities(attributes, communities);
+        }
+        let ours_large = |community: &LargeCommunity| {
+            community.global_admin == self.local_asn && community.data_1 <= 1
+        };
+        let has_large = attributes.iter().any(|attribute| match attribute {
+            PathAttribute::LargeCommunities(large) => large.communities.iter().any(ours_large),
+            _ => false,
+        });
+        if has_large {
+            let mut large = take_large_communities(attributes);
+            large.retain(|community| !ours_large(community));
+            put_large_communities(attributes, large);
+        }
+    }
+}
+
+/// The communities of `attributes`, standard and large, for looking at.
+pub fn communities_of(attributes: &[PathAttribute]) -> (&[(u16, u16)], &[LargeCommunity]) {
+    let (mut standard, mut large): (&[(u16, u16)], &[LargeCommunity]) = (&[], &[]);
+    for attribute in attributes {
+        match attribute {
+            PathAttribute::Communities(communities) => standard = &communities.0,
+            PathAttribute::LargeCommunities(communities) => large = &communities.communities,
+            _ => {}
+        }
+    }
+    (standard, large)
+}
+
+// Used by the tests of other modules too.
+#[cfg(test)]
+pub(crate) fn large(global_admin: u32, data_1: u32, data_2: u32) -> LargeCommunity {
+    LargeCommunity {
+        global_admin,
+        data_1,
+        data_2,
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use bgp_packet::message::{AsPathAttribute, OriginPathAttribute};
+
+    use super::*;
+
+    fn strings(texts: &[&str]) -> Vec<String> {
+        texts.iter().map(|text| text.to_string()).collect()
+    }
+
+    fn route(
+        communities: &[(u16, u16)],
+        large_communities: &[LargeCommunity],
+    ) -> Vec<PathAttribute> {
+        let mut attributes = vec![
+            PathAttribute::Origin(OriginPathAttribute::IGP),
+            AsPathAttribute::from_asns(vec![64602]),
+        ];
+        put_communities(&mut attributes, communities.to_vec());
+        put_large_communities(&mut attributes, large_communities.to_vec());
+        attributes
+    }
+
+    /// Where a route was learned is put on it, and nobody else's word
+    /// for that is taken.
+    #[test]
+    fn marks_are_taken_off_and_put_on() {
+        let marks = Marks::parse(
+            &strings(&["64601:*", "64601:9:*"]),
+            &strings(&["64601:1001", "64601:1001"]),
+            &strings(&["64601:9:7"]),
+        )
+        .unwrap();
+        let mut attributes = route(
+            &[(3356, 3), (64601, 2002), (64601, 1001)],
+            &[large(64601, 9, 1), large(64601, 8, 1)],
+        );
+        marks.apply(&mut attributes);
+        assert_eq!(
+            attributes,
+            route(
+                &[(3356, 3), (64601, 1001)],
+                &[large(64601, 8, 1), large(64601, 9, 7)]
+            )
+        );
+
+        // On a route with none, in the order of the type codes.
+        let mut bare = route(&[], &[]);
+        marks.apply(&mut bare);
+        let codes: Vec<u8> = bare.iter().map(PathAttribute::type_code).collect();
+        assert_eq!(codes, [1, 2, 8, 32]);
+
+        // Taking off the last leaves no empty attribute behind.
+        let scrub = Marks::parse(&strings(&["*:*", "*:*:*"]), &[], &[]).unwrap();
+        let mut all = route(&[(1, 2)], &[large(1, 2, 3)]);
+        scrub.apply(&mut all);
+        assert_eq!(all, route(&[], &[]));
+
+        assert!(Marks::default().is_empty());
+        assert!(!marks.is_empty());
+    }
+
+    #[test]
+    fn communities_that_are_not_are_refused() {
+        for bad in ["64601", "64601:1:2:3", "x:1", "70000:1"] {
+            assert_eq!(
+                Marks::parse(&strings(&[bad]), &[], &[]),
+                Err(MarksProblem::Pattern(bad.to_string())),
+                "{bad}"
+            );
+        }
+        // What is put on is one community, not a pattern.
+        assert!(Marks::parse(&[], &strings(&["64601:*"]), &[]).is_err());
+        assert!(Marks::parse(&[], &[], &strings(&["64601:1"])).is_err());
+        // A well-known one by its name.
+        assert!(Marks::parse(&[], &strings(&["NO_EXPORT"]), &[]).is_ok());
+    }
+
+    /// A route says who is told of it.
+    #[test]
+    fn communities_say_which_peers_a_route_goes_to() {
+        let control = Control { local_asn: 64601 };
+        let to = |communities: &[(u16, u16)], large: &[LargeCommunity], peer: u32| {
+            control.allows(communities, large, peer)
+        };
+        // Nothing said: everyone.
+        assert!(to(&[(3356, 3)], &[], 64602));
+        // Not to one peer.
+        assert!(!to(&[(0, 64602)], &[], 64602));
+        assert!(to(&[(0, 64602)], &[], 64603));
+        // Only to those named.
+        assert!(to(&[(64601, 64602)], &[], 64602));
+        assert!(!to(&[(64601, 64602)], &[], 64603));
+        assert!(to(&[(64601, 64602), (64601, 64603)], &[], 64603));
+        // To nobody; and to nobody but one, as exchanges have it.
+        assert!(!to(&[(0, 64601)], &[], 64602));
+        assert!(to(&[(0, 64601), (64601, 64602)], &[], 64602));
+        assert!(!to(&[(0, 64601), (64601, 64602)], &[], 64603));
+        // "Not" wins over "to".
+        assert!(!to(&[(64601, 64602), (0, 64602)], &[], 64602));
+        // An exchange's "to everyone" changes nothing.
+        assert!(to(&[(64601, 64601)], &[], 64602));
+
+        // The same in large communities, which have room for any AS.
+        let wide = 4_200_000_001;
+        assert!(!to(&[], &[large(64601, 0, wide)], wide));
+        assert!(to(&[], &[large(64601, 0, wide)], 64602));
+        assert!(to(&[], &[large(64601, 1, wide)], wide));
+        assert!(!to(&[], &[large(64601, 1, wide)], 64602));
+        assert!(!to(&[], &[large(64601, 0, 0)], 64602));
+        assert!(to(
+            &[],
+            &[large(64601, 0, 0), large(64601, 1, 64602)],
+            64602
+        ));
+        // Someone else's, or ours for something else, say nothing.
+        assert!(to(
+            &[],
+            &[large(3356, 0, 64602), large(64601, 2, 64602)],
+            64602
+        ));
+        // A peer whose AS number does not fit a standard community is not
+        // meant by one.
+        assert!(to(&[(0, 1)], &[], 65537));
+
+        // Our own AS number may not fit either: then only the large ones
+        // can name us.
+        let control = Control { local_asn: wide };
+        assert!(control.allows(&[(0, 64602)], &[], 64603));
+        assert!(!control.allows(&[(0, 64602)], &[], 64602));
+        assert!(!control.allows(&[], &[large(wide, 0, 0)], 64602));
+    }
+
+    #[test]
+    fn what_was_for_us_is_taken_off_as_a_route_is_passed_on() {
+        let control = Control { local_asn: 64601 };
+        let mut attributes = route(
+            &[(0, 64602), (3356, 3), (64601, 64603)],
+            &[
+                large(64601, 0, 5),
+                large(64601, 1, 6),
+                large(64601, 2, 7),
+                large(3356, 0, 5),
+            ],
+        );
+        control.strip(&mut attributes);
+        assert_eq!(
+            attributes,
+            route(&[(3356, 3)], &[large(64601, 2, 7), large(3356, 0, 5)])
+        );
+        let mut all = route(&[(0, 1)], &[large(64601, 0, 0)]);
+        control.strip(&mut all);
+        assert_eq!(all, route(&[], &[]));
+    }
+
+    #[test]
+    fn the_well_known_communities_keep_a_route_in() {
+        assert!(well_known_allow(&[(3356, 3)], false));
+        assert!(!well_known_allow(&[NO_ADVERTISE], true));
+        assert!(!well_known_allow(&[NO_ADVERTISE], false));
+        assert!(well_known_allow(&[NO_EXPORT], true));
+        assert!(!well_known_allow(&[NO_EXPORT], false));
+        assert!(!well_known_allow(&[NO_EXPORT_SUBCONFED], false));
+    }
+}

crates/server/src/metrics.rs

@@ -164,6 +164,15 @@ pub static RIB_JOURNAL_ENTRIES: LazyLock<IntGauge> = LazyLock::new(|| {
     .expect("metric registration")
 });
 
+pub static ROUTES_EXPORTED: LazyLock<IntCounterVec> = LazyLock::new(|| {
+    register_int_counter_vec!(
+        "bgp_routes_exported_total",
+        "Routes learned from one peer and announced to, or withdrawn from, another",
+        &["change"]
+    )
+    .expect("metric registration")
+});
+
 /// Whether each configured peer's session is up: what to alert on when a
 /// peer goes away, which a count of sessions cannot say.
 pub static PEER_UP: LazyLock<IntGaugeVec> = LazyLock::new(|| {
@@ -207,6 +216,7 @@ pub fn init() {
     LazyLock::force(&RIB_PREFIXES);
     LazyLock::force(&RIB_ATTRIBUTE_SETS);
     LazyLock::force(&RIB_JOURNAL_ENTRIES);
+    LazyLock::force(&ROUTES_EXPORTED);
     LazyLock::force(&PEER_UP);
     BUILD_INFO
         .with_label_values(&[env!("CARGO_PKG_VERSION")])

crates/server/src/policy.rs

@@ -29,12 +29,15 @@ use std::net::{IpAddr, Ipv4Addr, Ipv6Addr};
 use std::sync::{Arc, LazyLock, Mutex};
 use std::time::{Duration, Instant};
 
-use bgp_packet::message::{AS_TRANS, AsPathAttribute, NlriNextHop, PathAttribute};
+use bgp_packet::message::{
+    AS_TRANS, AsPathAttribute, LocalPrefPathAttribute, NlriNextHop, PathAttribute,
+};
 use bgp_packet::open::BgpRole;
 use bgp_primitives::constants::IpVersion;
 use bgp_primitives::ip_prefix::IpPrefix;
 
 use crate::address::PeerAddress;
+use crate::marks::{self, Marks};
 use crate::sync::IgnorePoison;
 
 /// The longest prefixes routed between networks: anything more specific is
@@ -96,6 +99,11 @@ pub struct ImportPolicy {
     /// a customer with a part of our space. Empty for a peer in our own
     /// AS.
     pub own_prefixes: Arc<PrefixSet>,
+    /// Communities taken off and put on every route that is taken.
+    pub marks: Marks,
+    /// The LOCAL_PREF every route that is taken is given; `None` leaves
+    /// it at the default.
+    pub local_pref: Option<u32>,
 }
 
 impl Default for ImportPolicy {
@@ -118,6 +126,8 @@ impl Default for ImportPolicy {
             accept_as_sets: false,
             accept_third_party_next_hop: false,
             own_prefixes: Arc::default(),
+            marks: Marks::default(),
+            local_pref: None,
         }
     }
 }
@@ -143,6 +153,37 @@ impl ImportPolicy {
     }
 }
 
+impl ImportPolicy {
+    /// Whether routes are changed on their way in.
+    pub fn changes_routes(&self) -> bool {
+        !self.marks.is_empty() || self.local_pref.is_some()
+    }
+
+    /// Makes of `attributes` what the route is kept with: marked, and
+    /// with its preference.
+    pub fn change(&self, attributes: &mut Vec<PathAttribute>) {
+        self.marks.apply(attributes);
+        if let Some(local_pref) = self.local_pref {
+            marks::put(
+                attributes,
+                PathAttribute::LocalPref(LocalPrefPathAttribute(local_pref)),
+            );
+        }
+    }
+}
+
+/// What is announced to one peer, and how.
+#[derive(Debug, Clone, Default, PartialEq, Eq)]
+pub struct ExportPolicy {
+    /// Pass on the best routes learned from other peers, and not only
+    /// what we originate.
+    pub learned_routes: bool,
+    /// Communities taken off and put on every route announced.
+    pub marks: Marks,
+    /// How many more times our AS goes on the path.
+    pub prepend: u8,
+}
+
 /// Why a route was not taken.
 #[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
 pub enum Rejection {

crates/server/src/routes.rs

@@ -128,6 +128,7 @@ pub(crate) fn apply(
         path_invalid: OnceCell::new(),
         as_path: OnceCell::new(),
         mark: only_to_customer_mark(import, update),
+        changed: OnceCell::new(),
     };
     applying.withdraw();
     // Routes in the body have their next hop in the NEXT_HOP attribute,
@@ -181,6 +182,9 @@ struct Applying<'a> {
     as_path: OnceCell<Arc<[u32]>>,
     /// See [`only_to_customer_mark`].
     mark: Option<u32>,
+    /// The UPDATE's attributes as the policy changes them, if it does:
+    /// made when the first route is taken.
+    changed: OnceCell<Vec<PathAttribute>>,
 }
 
 impl Applying<'_> {
@@ -335,6 +339,27 @@ impl Applying<'_> {
     /// The attribute set the UPDATE's routes go into the table with.
     fn attributes(&self, next_hop: Option<&NlriNextHop>) -> Result<Arc<AttrSet>, EncodeError> {
         let store = self.table.attrs();
+        // What the policy puts on the peer's routes (communities that say
+        // where they were learned, a preference) is the same for all of
+        // an UPDATE's routes, and worked out once for them.
+        if self.import.policy.changes_routes() {
+            let changed = self.changed.get_or_init(|| {
+                let mut attributes = self.update.path_attributes.clone();
+                // Not the route's own: they hold the prefixes.
+                attributes.retain(|attribute| {
+                    !matches!(
+                        attribute,
+                        PathAttribute::MpReachNlri(_) | PathAttribute::MpUnreachNlri(_)
+                    )
+                });
+                self.import.policy.change(&mut attributes);
+                if let Some(marked_by) = self.mark {
+                    attributes.push(PathAttribute::OnlyToCustomer(marked_by));
+                }
+                attributes
+            });
+            return store.intern(changed, next_hop);
+        }
         let Some(marked_by) = self.mark else {
             if let Some(set) = self
                 .attributes_wire

crates/server/src/server.rs

@@ -179,6 +179,18 @@ impl BgpServer {
                     "No limit on the routes taken from this peer (import.max_prefixes)"
                 );
             }
+            // Without a role there is no rule about which learned routes
+            // go to the peer: all of them do.
+            if peer.export.learned_routes
+                && peer.role.is_none()
+                && peer.remote_asn != self.config.local_asn
+            {
+                warn!(
+                    peer = %peer.name,
+                    "Every route learned from any peer is passed on to this one: it has no role \
+                     (export.learned_routes without role)"
+                );
+            }
         }
         if self.config.accept_unconfigured_peers {
             warn!("Taking sessions and routes from any host (accept_unconfigured_peers)");
@@ -204,8 +216,10 @@ impl BgpServer {
             "advertiser",
             spawn_advertiser(
                 self.local_rib.clone(),
+                Arc::clone(&self.rib),
                 self.registry.clone(),
                 self.config.local_asn,
+                self.config.control,
             ),
         ));
         tasks.push(named("sweeper", spawn_sweeper(Arc::clone(&self.rib))));
@@ -354,7 +368,7 @@ fn admit(
     let peer = match config.peer_at(address) {
         Some(peer) => PeerSettings::from(peer),
         None if config.accept_unconfigured_peers => {
-            PeerSettings::unconfigured(config.hold_time_secs)
+            PeerSettings::unconfigured(config.hold_time_secs, Arc::clone(&config.export))
         }
         None => return Err(Refusal::NotAPeer),
     };

crates/server/src/session.rs

@@ -24,7 +24,7 @@ use crate::collision::Direction;
 use crate::family::FamilySet;
 use crate::fsm::Negotiated;
 use crate::metrics::{PEER_UP, SESSIONS_ESTABLISHED};
-use crate::policy::PeerRole;
+use crate::policy::{ExportPolicy, PeerRole};
 use crate::rpki::Subject;
 use crate::sync::IgnorePoisonRw;
 
@@ -145,6 +145,8 @@ pub struct SessionHandle {
     pub rib_peer: PeerIndex,
     /// How many routes the peer has, and how many were refused.
     pub counters: Arc<SessionCounters>,
+    /// What the peer is told, and how.
+    pub export: Arc<ExportPolicy>,
     outbound: mpsc::Sender<Message>,
     /// Which of the RPKI's checks the peer's routes are subject to, if
     /// any, and where the session is told to look at its routes again.
@@ -181,6 +183,7 @@ impl SessionHandle {
             add_path_send: negotiated.add_path_send.clone(),
             rib_peer,
             counters: Arc::default(),
+            export: Arc::default(),
             outbound,
             validation: None,
         }