Policy scripts: the operator's own rules for routes, in Rhai

A new crate, bgp-policy, runs functions of a script on routes: on their
way in, after the general checks of a peer's import and before the
checks that can change (RPKI, next hop reachability); and on their way
out to a peer. A function is given the route and the peer, may refuse
the route, and may change its communities, large communities,
LOCAL_PREF and MED, and on the way out prepend our AS:

    fn from_peer(route, peer) {
        if route.has_community("65535:666") { return false; }
        route.add_community(LEARNED_FROM[peer.role]);
        route.local_pref = PREFERENCE[peer.role];
        true
    }

Configuration: policy { path | source, import_function, export_function }
and, for a peer, import { function } and export { function }. The script
is compiled at startup; `bgp --check` reads it and says which function
is whose. A function that fails on a route has refused it. Scripts have
no files, network or clock, and are stopped after 200,000 steps.

Route selection now goes by a LOCAL_PREF on a route from an external
peer: the decoder never lets a peer's own through, so one that is there
was set by policy.

What it costs was measured before it was built on (benches/policy.rs):
a call that tests a few communities and changes two takes 1 to 3
microseconds, some two hundred times what Rust would. So a function is
asked once for the routes of an UPDATE, which share their attributes,
unless it asks for the prefix. One peer sending 1.23 M routes over a
session: 0.37 s without a function, 0.80 s with one that marks and
ranks every route, 3.4 s if it also looks at the prefix.

Export applies to what is announced, which today is what the daemon
originates; learned routes are not passed on yet.

Tested by unit tests of the crate, sessions end to end, a fuzz target on
the example script (testdata/policy.rhai), and a lab phase in which a
script changes which of FRR's and BIRD's route is chosen and what each
of them is told.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

crates/integration-tests/Cargo.toml

@@ -11,6 +11,7 @@ publish = false
 [dependencies]
 bgp-fib.workspace = true
 bgp-packet.workspace = true
+bgp-policy.workspace = true
 bgp-primitives.workspace = true
 bgp-rib.workspace = true
 bgp-rpki.workspace = true

crates/integration-tests/src/test_server.rs

@@ -67,6 +67,7 @@ impl TestServer {
             http_endpoint: None,
             rpki: None,
             kernel: None,
+            policy: None,
         };
         customize(&mut config);
 

crates/integration-tests/tests/add_path.rs

@@ -50,6 +50,8 @@ async fn server_offering(
             import: ImportPolicy::permissive(),
             role: None,
             require_role: false,
+            import_hook: None,
+            export_hook: None,
         }];
         c.originate = originate;
         // Keep the dialer out of the way.

crates/integration-tests/tests/admission.rs

@@ -28,6 +28,8 @@ fn peer_at(endpoint: SocketAddr) -> PeerConfig {
         import: ImportPolicy::permissive(),
         role: None,
         require_role: false,
+        import_hook: None,
+        export_hook: None,
     }
 }
 

crates/integration-tests/tests/aspa.rs

@@ -67,6 +67,8 @@ async fn server_with(
             import,
             role,
             require_role: false,
+            import_hook: None,
+            export_hook: None,
         }];
     })
     .await?;

crates/integration-tests/tests/collision.rs

@@ -46,6 +46,8 @@ async fn server_dialing(listener: &TcpListener) -> Result<TestServer> {
             import: ImportPolicy::permissive(),
             role: None,
             require_role: false,
+            import_hook: None,
+            export_hook: None,
         }];
     })
     .await

crates/integration-tests/tests/debugging.rs

@@ -180,6 +180,8 @@ async fn test_refused_routes_can_be_seen_and_counted() -> Result<()> {
             },
             role: None,
             require_role: false,
+            import_hook: None,
+            export_hook: None,
         }];
     })
     .await?;

crates/integration-tests/tests/dialer.rs

@@ -37,6 +37,8 @@ async fn server_dialing(listener: &TcpListener, remote_asn: u32) -> Result<TestS
             import: ImportPolicy::permissive(),
             role: None,
             require_role: false,
+            import_hook: None,
+            export_hook: None,
         }];
     })
     .await
@@ -194,6 +196,8 @@ async fn test_inbound_from_configured_address_uses_config() -> Result<()> {
             import: ImportPolicy::permissive(),
             role: None,
             require_role: false,
+            import_hook: None,
+            export_hook: None,
         }];
     })
     .await?;
@@ -245,6 +249,8 @@ async fn test_local_address_is_used_as_the_source() -> Result<()> {
             import: ImportPolicy::permissive(),
             role: None,
             require_role: false,
+            import_hook: None,
+            export_hook: None,
         }];
     })
     .await?;
@@ -265,6 +271,8 @@ async fn test_local_address_is_used_as_the_source() -> Result<()> {
             import: ImportPolicy::permissive(),
             role: None,
             require_role: false,
+            import_hook: None,
+            export_hook: None,
         }];
     })
     .await?;

crates/integration-tests/tests/import_policy.rs

@@ -51,6 +51,8 @@ async fn server_with(import: ImportPolicy) -> Result<(TestServer, TcpListener)>
             import,
             role: None,
             require_role: false,
+            import_hook: None,
+            export_hook: None,
         }];
     })
     .await?;

crates/integration-tests/tests/negotiation.rs

@@ -193,6 +193,8 @@ async fn test_no_family_in_common_refuses_the_session() -> Result<()> {
             import: ImportPolicy::permissive(),
             role: None,
             require_role: false,
+            import_hook: None,
+            export_hook: None,
         }];
         // Keep the dialer out of the way.
         c.connect_retry = Duration::from_secs(3600);

crates/integration-tests/tests/policy_script.rs

@@ -0,0 +1,361 @@
+//! The operator's own rules, as a script: what they do to routes on their
+//! way in, and to what a peer is told.
+
+use std::collections::BTreeMap;
+use std::net::Ipv4Addr;
+use std::time::Duration;
+
+use bgp_integration_tests::default_policy;
+use bgp_integration_tests::test_peer::TestPeer;
+use bgp_integration_tests::test_server::TestServer;
+use bgp_packet::message::{
+    AsPathAttribute, CommunitiesPathAttribute, LocalPrefPathAttribute, Message,
+    NextHopPathAttribute, OriginPathAttribute, PathAttribute, UpdateMessage,
+};
+use bgp_policy::Policy;
+use bgp_primitives::ip_prefix::IpPrefix;
+use bgp_rib::{LocalRoute, LocalRouteAttrs};
+use bgp_server::config::PeerConfig;
+use bgp_server::family::FamilySet;
+use bgp_server::policy::PeerRole;
+use eyre::Result;
+use tokio::net::TcpListener;
+
+const LOCAL_ASN: u32 = 1299;
+const PEER_ASN: u32 = 3333;
+const TIMEOUT: Duration = Duration::from_secs(5);
+
+const SCRIPT: &str = r#"
+    const BLACKHOLE = "65535:666";
+    const LEARNED_FROM = #{ customer: "1299:1000", peer: "1299:2000", provider: "1299:3000" };
+
+    fn from_peer(route, peer) {
+        if route.has_community(BLACKHOLE) { return false; }
+        // Nobody else's word for where we learned a route.
+        route.remove_community("1299:*");
+        route.add_community(LEARNED_FROM[peer.role]);
+        route.local_pref = if route.has_community("3333:80") { 80 } else { 200 };
+        true
+    }
+
+    fn by_prefix(route, peer) {
+        if route.within("194.0.0.0/8") { return false; }
+        if route.prefix == "193.0.4.0/24" { route.add_community("1299:4"); }
+        true
+    }
+
+    fn broken(route, peer) {
+        if route.prefix_len == 22 { return route.no_such_thing; }
+        true
+    }
+
+    fn to_peer(route, peer) {
+        if route.has_community("1299:9") { return false; }      // not for anyone else
+        if route.within("193.0.0.0/21") { route.prepend(2); }
+        route.add_community("1299:" + peer.asn % 1000);
+        route.med = 50;
+        true
+    }
+"#;
+
+fn prefix(s: &str) -> IpPrefix {
+    IpPrefix::try_from(s).unwrap()
+}
+
+/// A server with one configured peer, whose routes are given to the
+/// script's function `import`, and which is told what `export` lets by.
+async fn server_with(
+    import: Option<&str>,
+    export: Option<&str>,
+    originate: Vec<LocalRoute>,
+) -> Result<(TestServer, TcpListener)> {
+    let listener = TcpListener::bind((Ipv4Addr::LOCALHOST, 0)).await?;
+    let endpoint = listener.local_addr()?;
+    let policy = Policy::compile(SCRIPT)?;
+    let import_hook = import.map(|name| policy.hook(name)).transpose()?;
+    let export_hook = export.map(|name| policy.hook(name)).transpose()?;
+    let server = TestServer::start_with(LOCAL_ASN, |c| {
+        c.accept_unconfigured_peers = false;
+        c.originate = originate;
+        c.policy = Some(policy);
+        c.peers = vec![PeerConfig {
+            name: "r1".into(),
+            endpoint: endpoint.into(),
+            remote_asn: PEER_ASN,
+            hold_time_secs: 90,
+            local_address: None,
+            families: FamilySet::supported(),
+            add_path: None,
+            import: default_policy(),
+            role: Some(PeerRole::Peer),
+            require_role: false,
+            import_hook,
+            export_hook,
+        }];
+    })
+    .await?;
+    Ok((server, listener))
+}
+
+async fn established(server: &TestServer) -> Result<TestPeer> {
+    let mut peer = TestPeer::connect(server.addr).await?;
+    peer.handshake(PEER_ASN, Ipv4Addr::new(10, 0, 0, 2), 90)
+        .await?;
+    tokio::time::timeout(TIMEOUT, async {
+        while server.registry.list().is_empty() {
+            tokio::time::sleep(Duration::from_millis(5)).await;
+        }
+    })
+    .await?;
+    Ok(peer)
+}
+
+async fn announce(
+    peer: &mut TestPeer,
+    prefixes: &[&str],
+    communities: &[(u16, u16)],
+) -> Result<()> {
+    let mut path_attributes = vec![
+        PathAttribute::Origin(OriginPathAttribute::IGP),
+        AsPathAttribute::from_asns(vec![PEER_ASN]),
+        PathAttribute::NextHop(NextHopPathAttribute(Ipv4Addr::new(192, 0, 2, 1))),
+    ];
+    if !communities.is_empty() {
+        path_attributes.push(PathAttribute::Communities(CommunitiesPathAttribute(
+            communities.to_vec(),
+        )));
+    }
+    peer.send(&Message::Update(UpdateMessage {
+        withdrawn_routes: vec![],
+        path_attributes,
+        announced_routes: prefixes.iter().map(|p| prefix(p).into()).collect(),
+    }))
+    .await
+}
+
+/// Waits until the server has taken `taken` routes of the session's and
+/// refused `refused`.
+async fn dealt_with(server: &TestServer, taken: u64, refused: u64) -> Result<()> {
+    let counters = server.registry.list()[0].counters.clone();
+    tokio::time::timeout(TIMEOUT, async {
+        while (counters.routes(), counters.rejected()) != (taken, refused) {
+            tokio::time::sleep(Duration::from_millis(5)).await;
+        }
+    })
+    .await
+    .map_err(|_| {
+        eyre::eyre!(
+            "the server took {} routes and refused {}; expected {taken} and {refused}",
+            counters.routes(),
+            counters.rejected()
+        )
+    })
+}
+
+/// A route's communities and its LOCAL_PREF.
+type Marks = (Vec<(u16, u16)>, Option<u32>);
+
+/// The communities and LOCAL_PREF each prefix is in the table with.
+fn table(server: &TestServer) -> BTreeMap<String, Marks> {
+    server
+        .received_routes()
+        .iter()
+        .map(|route| {
+            let attributes = &route.attributes;
+            let communities = attributes
+                .iter()
+                .find_map(|attribute| match attribute {
+                    PathAttribute::Communities(communities) => Some(communities.0.clone()),
+                    _ => None,
+                })
+                .unwrap_or_default();
+            let local_pref = attributes.iter().find_map(|attribute| match attribute {
+                PathAttribute::LocalPref(LocalPrefPathAttribute(value)) => Some(*value),
+                _ => None,
+            });
+            (route.prefix.to_string(), (communities, local_pref))
+        })
+        .collect()
+}
+
+fn refusals(server: &TestServer) -> Vec<(&'static str, u64)> {
+    server.refusals.peers()[0]
+        .1
+        .by_reason()
+        .into_iter()
+        .map(|(reason, count)| (reason.as_str(), count))
+        .collect()
+}
+
+/// Routes are marked with where they were learned and given a
+/// preference, by what they carry and who sent them; one the function
+/// refuses is not taken.
+#[tokio::test]
+async fn test_routes_are_marked_preferred_and_refused_on_their_way_in() -> Result<()> {
+    let (server, _listener) = server_with(Some("from_peer"), None, vec![]).await?;
+    let mut peer = established(&server).await?;
+
+    // One UPDATE of two routes; one of a route with marks of its own, one
+    // of them forged; and one the function refuses.
+    announce(&mut peer, &["193.0.0.0/21", "193.0.8.0/22"], &[]).await?;
+    announce(&mut peer, &["194.0.1.0/24"], &[(3333, 80), (1299, 1000)]).await?;
+    announce(&mut peer, &["195.0.1.0/24"], &[(65535, 666)]).await?;
+    dealt_with(&server, 3, 1).await?;
+
+    assert_eq!(
+        table(&server),
+        BTreeMap::from([
+            ("193.0.0.0/21".to_string(), (vec![(1299, 2000)], Some(200))),
+            ("193.0.8.0/22".to_string(), (vec![(1299, 2000)], Some(200))),
+            (
+                "194.0.1.0/24".to_string(),
+                (vec![(3333, 80), (1299, 2000)], Some(80))
+            ),
+        ])
+    );
+    assert_eq!(refusals(&server), [("refused by policy function", 1)]);
+
+    // A route the function took is replaced by one it refuses: gone.
+    announce(&mut peer, &["193.0.8.0/22"], &[(65535, 666)]).await?;
+    dealt_with(&server, 2, 2).await?;
+    assert!(!table(&server).contains_key("193.0.8.0/22"));
+    Ok(())
+}
+
+/// A function that asks for the prefix is asked about each route of an
+/// UPDATE, and may answer differently for each.
+#[tokio::test]
+async fn test_a_function_may_treat_the_routes_of_one_update_differently() -> Result<()> {
+    let (server, _listener) = server_with(Some("by_prefix"), None, vec![]).await?;
+    let mut peer = established(&server).await?;
+    announce(
+        &mut peer,
+        &[
+            "193.0.0.0/21",
+            "194.0.1.0/24",
+            "193.0.4.0/24",
+            "194.0.2.0/24",
+            "195.0.0.0/24",
+        ],
+        &[(3333, 1)],
+    )
+    .await?;
+    dealt_with(&server, 3, 2).await?;
+    assert_eq!(
+        table(&server),
+        BTreeMap::from([
+            ("193.0.0.0/21".to_string(), (vec![(3333, 1)], None)),
+            (
+                "193.0.4.0/24".to_string(),
+                (vec![(3333, 1), (1299, 4)], None)
+            ),
+            ("195.0.0.0/24".to_string(), (vec![(3333, 1)], None)),
+        ])
+    );
+    assert_eq!(
+        server.rib.attrs().len(),
+        2,
+        "two routes share their attributes"
+    );
+    Ok(())
+}
+
+/// A function that fails on a route has refused it, and no other.
+#[tokio::test]
+async fn test_a_function_that_fails_refuses_the_route() -> Result<()> {
+    let (server, _listener) = server_with(Some("broken"), None, vec![]).await?;
+    let mut peer = established(&server).await?;
+    announce(
+        &mut peer,
+        &["193.0.0.0/21", "193.0.8.0/22", "195.0.0.0/24"],
+        &[],
+    )
+    .await?;
+    dealt_with(&server, 2, 1).await?;
+    assert_eq!(refusals(&server), [("policy function failed", 1)]);
+    assert!(!table(&server).contains_key("193.0.8.0/22"));
+    Ok(())
+}
+
+/// What a peer is told is what the function for it lets by, as the
+/// function left it.
+#[tokio::test]
+async fn test_what_a_peer_is_told_goes_through_its_function() -> Result<()> {
+    let route = |prefix_text: &str, communities: Vec<(u16, u16)>| {
+        LocalRoute::new(
+            prefix(prefix_text),
+            LocalRouteAttrs {
+                communities,
+                ..LocalRouteAttrs::default()
+            },
+        )
+    };
+    let originate = vec![
+        route("193.0.0.0/21", vec![(1299, 1)]),
+        route("194.0.0.0/22", vec![]),
+        route("195.0.0.0/22", vec![(1299, 9)]),
+    ];
+    let (server, _listener) = server_with(None, Some("to_peer"), originate).await?;
+    let mut peer = established(&server).await?;
+
+    let mut told = BTreeMap::new();
+    for _ in 0..2 {
+        let update = tokio::time::timeout(TIMEOUT, peer.recv_update()).await??;
+        let path = update
+            .path_attributes
+            .iter()
+            .find_map(|attribute| match attribute {
+                PathAttribute::AsPath(path) => Some(path.segments[0].path.clone()),
+                _ => None,
+            })
+            .unwrap();
+        let communities = update
+            .path_attributes
+            .iter()
+            .find_map(|attribute| match attribute {
+                PathAttribute::Communities(communities) => Some(communities.0.clone()),
+                _ => None,
+            })
+            .unwrap_or_default();
+        let med = update
+            .path_attributes
+            .iter()
+            .find_map(|attribute| match attribute {
+                PathAttribute::MultiExitDisc(med) => Some(med.0),
+                _ => None,
+            });
+        // In the order of their type codes, as any UPDATE of ours.
+        let codes: Vec<u8> = update
+            .path_attributes
+            .iter()
+            .map(PathAttribute::type_code)
+            .collect();
+        assert!(codes.is_sorted(), "{codes:?}");
+        // The peer is one: the route is still marked as only for customers.
+        assert!(codes.contains(&35), "{codes:?}");
+        told.insert(
+            update.announced_routes[0].prefix.to_string(),
+            (path, communities, med),
+        );
+    }
+    assert_eq!(
+        told,
+        BTreeMap::from([
+            (
+                "193.0.0.0/21".to_string(),
+                (vec![LOCAL_ASN; 3], vec![(1299, 1), (1299, 333)], Some(50))
+            ),
+            (
+                "194.0.0.0/22".to_string(),
+                (vec![LOCAL_ASN], vec![(1299, 333)], Some(50))
+            ),
+        ])
+    );
+    // And nothing of the third.
+    assert!(
+        tokio::time::timeout(Duration::from_millis(300), peer.recv_update())
+            .await
+            .is_err()
+    );
+    Ok(())
+}

crates/integration-tests/tests/roles.rs

@@ -58,6 +58,8 @@ async fn server_with(role: Option<PeerRole>, require_role: bool) -> Result<Setup
             import: default_policy(),
             role,
             require_role,
+            import_hook: None,
+            export_hook: None,
         }];
     })
     .await?;

crates/integration-tests/tests/rpki.rs

@@ -77,6 +77,8 @@ async fn server_with(
             import,
             role: None,
             require_role: false,
+            import_hook: None,
+            export_hook: None,
         }];
     })
     .await?;