Policy scripts: the operator's own rules for routes, in Rhai

A new crate, bgp-policy, runs functions of a script on routes: on their
way in, after the general checks of a peer's import and before the
checks that can change (RPKI, next hop reachability); and on their way
out to a peer. A function is given the route and the peer, may refuse
the route, and may change its communities, large communities,
LOCAL_PREF and MED, and on the way out prepend our AS:

    fn from_peer(route, peer) {
        if route.has_community("65535:666") { return false; }
        route.add_community(LEARNED_FROM[peer.role]);
        route.local_pref = PREFERENCE[peer.role];
        true
    }

Configuration: policy { path | source, import_function, export_function }
and, for a peer, import { function } and export { function }. The script
is compiled at startup; `bgp --check` reads it and says which function
is whose. A function that fails on a route has refused it. Scripts have
no files, network or clock, and are stopped after 200,000 steps.

Route selection now goes by a LOCAL_PREF on a route from an external
peer: the decoder never lets a peer's own through, so one that is there
was set by policy.

What it costs was measured before it was built on (benches/policy.rs):
a call that tests a few communities and changes two takes 1 to 3
microseconds, some two hundred times what Rust would. So a function is
asked once for the routes of an UPDATE, which share their attributes,
unless it asks for the prefix. One peer sending 1.23 M routes over a
session: 0.37 s without a function, 0.80 s with one that marks and
ranks every route, 3.4 s if it also looks at the prefix.

Export applies to what is announced, which today is what the daemon
originates; learned routes are not passed on yet.

Tested by unit tests of the crate, sessions end to end, a fuzz target on
the example script (testdata/policy.rhai), and a lab phase in which a
script changes which of FRR's and BIRD's route is chosen and what each
of them is told.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

bin/bgp/Cargo.toml

@@ -9,6 +9,7 @@ repository.workspace = true
 version.workspace = true
 
 [dependencies]
+bgp-policy.workspace = true
 bgp-server.workspace = true
 bgp-web.workspace    = true
 clap.workspace       = true

bin/bgp/README.md

@@ -42,8 +42,9 @@ connections, with RFC 4271 collision detection), announces its originated
 routes to every peer with itself as next hop, judges what peers send by each
 peer's import policy, keeps what passes, chooses a best route for each
 prefix, and on Linux can put those best routes in the kernel so that the
-machine forwards by them. It does not yet pass routes learned from one peer
-on to another, and has no export policy.
+machine forwards by them. The operator's own rules for what is taken, how
+it is marked and ranked, and what each peer is told are a script. It does
+not yet pass routes learned from one peer on to another.
 
 ## Who may connect
 
@@ -89,6 +90,112 @@ A refused route is dropped, along with the route it would have replaced.
 `bgp_routes_rejected_total` counts them by reason, and the status page shows
 how many routes each peer has and how many were refused.
 
+## Policy scripts
+
+The checks above say what no network should take from another. What this
+network wants beyond that (which routes it prefers, what it marks them
+with, what it tells which neighbor) is said in a script, in
+[Rhai](https://rhai.rs). A function is given a route and the peer it comes
+from or goes to, may change the route, and returns whether it is taken:
+
+```rust
+const BLACKHOLE = "65535:666";
+const LEARNED_FROM = #{ customer: "64601:1000", peer: "64601:2000", provider: "64601:3000" };
+const PREFERENCE = #{ customer: 300, peer: 200, provider: 100 };
+
+// Routes on their way in.
+fn from_peer(route, peer) {
+    if route.has_community(BLACKHOLE) { return false; }
+    route.remove_community("64601:*");              // nobody else's word for our marks
+    route.add_community(LEARNED_FROM[peer.role]);   // where we learned it
+    route.local_pref = PREFERENCE[peer.role];
+    true
+}
+
+// Routes on their way out.
+fn to_peer(route, peer) {
+    if route.has_community("64601:3000") && peer.role != "customer" { return false; }
+    if peer.name == "backup-transit" { route.prepend(2); }
+    route.remove_community("64601:*");
+    true
+}
+```
+
+```
+policy { path: "policy.rhai" }          # beside the configuration file
+peers {
+  name: "transit-a" ...
+  role: PROVIDER
+  import { function: "from_peer" }      # after every check of `import`
+  export { function: "to_peer" }
+}
+```
+
+`policy { import_function: ... export_function: ... }` names the functions
+for peers that name none. `testdata/policy.rhai` is a longer example.
+
+**What a function is given.**
+
+| | |
+|---|---|
+| `route.prefix`, `route.prefix_len`, `route.family` | `"203.0.113.0/24"`, `24`, `"ipv4"` |
+| `route.within("203.0.112.0/22")` | whether the prefix is that one or inside it |
+| `route.as_path`, `route.as_path_len` | `[64602, 3356, 15169]`, `3` |
+| `route.origin_as`, `route.neighbor_as`, `route.path_has(3356)` | the last and first AS of the path; whether an AS is on it |
+| `route.origin`, `route.next_hop` | `"igp"`, `"egp"` or `"incomplete"`; `"192.0.2.1"` |
+| `route.communities`, `route.large_communities` | `["3356:3", ...]`, `["64601:1:2", ...]` |
+| `route.has_community("3356:3")` | also `(3356, 3)`, and `"3356:*"` for any of that AS |
+| `route.add_community(..)`, `route.remove_community("3356:*")` | `"*:*"` removes all |
+| `route.has_large_community(..)`, `add_large_community(..)`, `remove_large_community(..)` | the same, as `"64601:1:2"` |
+| `route.local_pref`, `route.med` | the number, or `()` for none; set with `route.local_pref = 200`, unset with `= ()` |
+| `route.prepend(2)` | on the way out: our AS that many more times on the path |
+| `peer.name`, `peer.asn`, `peer.address`, `peer.ibgp`, `peer.local_asn` | |
+| `peer.role` | `"customer"`, `"provider"`, `"peer"`, `"route_server"`, `"route_server_client"`, or `""` |
+
+**How it fits in.**
+
+- On the way in, a function is asked after every check of the peer's
+  `import` has passed, and before the checks whose answer can change (the
+  RPKI's, and whether the next hop can be reached). A route it refuses is
+  counted as "refused by policy function". What it changes is what goes
+  into the table: a `local_pref` it sets is what route selection goes by.
+- On the way out it is asked about every route the peer would be sent. The
+  daemon announces the routes it originates; it does not yet pass on what
+  it learned from other peers. When it does, those routes will go through
+  the same function, with the communities they were given on the way in.
+  LOCAL_PREF does not leave the AS whatever a function sets.
+- The script is read when the daemon starts. One that does not parse, or a
+  function that is named and not there, stops it from starting. `bgp
+  --check config.textpb` reads the script and says which function is
+  whose. Changing the script takes a restart.
+- A function that fails on a route (it asks for something that is not
+  there, or does not end) has refused that route: failing is not a way
+  through. `bgp_policy_failures_total{function}` counts these, and the log
+  says why, at most once a minute.
+- A script can only compute. It has no files, no network and no clock, and
+  a function is stopped after 200,000 steps.
+
+**What it costs.** Rhai walks the script's syntax tree; a call like the
+ones above takes one to three microseconds, where the general checks take
+a fraction of one. So a function is not asked about every route. Routes
+that arrive in one UPDATE share their attributes, and a function that does
+not ask for the prefix (`prefix`, `prefix_len`, `within`) gives one answer
+for all of them. One that does is asked about each. With one peer sending
+a table of 1.23 million routes over a session (`cargo bench -p bgp-bench
+--bench policy`, on a laptop):
+
+| | Time until all are in the table |
+|---|---|
+| no function | 0.37 s |
+| a function that does nothing | 0.51 s |
+| one that marks and ranks every route | 0.80 s |
+| the same, and it looks at the prefix | 3.4 s |
+
+So: filter by prefix with `import { prefixes ... }` and the length limits,
+which cost next to nothing, and keep scripts to what routes carry. A
+function that has to look at the prefix still takes in a full table in a
+few seconds.
+
 ## Route origin validation (RPKI)
 
 With `rpki { caches { address: "..." port: 3323 } }` in the configuration,
@@ -369,6 +476,7 @@ Routinator 0.15.2. What it covers:
 | `limits` | A peer whose role does not fit gets no session; a peer over its prefix limit gets Cease / Maximum Number of Prefixes and is held down |
 | `hijacks` | FRR announces our own prefix, a route whose next hop is BIRD, and the exchange LAN's own prefix: none is taken |
 | `nexthops` | A better route from FRR whose next hop cannot be reached is held and BIRD's is chosen and installed; they swap when the next hop becomes a neighbor, and swap back when it stops being one |
+| `policy` | A policy script: routes marked with where they were learned and ranked by it, which changes which of FRR's and BIRD's route is chosen and installed; FRR told our route with a longer path and a mark, BIRD not told it |
 | `v6only` | The exchange LAN without IPv4 addresses: one IPv6 session with each router for both families, IPv4 routes with IPv6 next hops (RFC 8950) in BGP and in the kernel, IPv4 pings across |
 | `linklocal` | The exchange LAN with link-local addresses only, and the same ones on a second interface: sessions with FRR (by interface, with radvd beside the daemon) and BIRD at `fe80::…%ix0`, both families in BGP and in the kernel, pings across, the interface taken down and up |
 
@@ -416,6 +524,8 @@ For alerting, `/metricz` has:
 | `bgp_peer_max_prefixes{peer}` | The peer's limit, where it has one |
 | `bgp_prefix_limit_exceeded_total{peer}` | Sessions ended for going over the limit |
 | `bgp_connections_refused_total{reason}` | Connections closed without a session |
+| `bgp_policy_runs_total{direction}` | Times a policy function was asked about a route |
+| `bgp_policy_failures_total{function}` | Times one failed, which refuses the route |
 | `bgp_peer_up{peer}` | 1 while the session with a configured peer is Established, else 0 |
 | `bgp_build_info{version}` | Always 1; says which version is running |
 
@@ -454,6 +564,11 @@ groups:
       - alert: BgpHijackAttempt
         expr: increase(bgp_routes_rejected_total{reason=~"prefix is our own|next hop is not the peer"}[10m]) > 0
         labels: { severity: page }
+      # A function of the policy script is failing: every route it fails
+      # on is refused.
+      - alert: BgpPolicyFunctionFailing
+        expr: increase(bgp_policy_failures_total[10m]) > 0
+        labels: { severity: page }
       # The kernel is refusing routes, or its answers are being lost.
       - alert: BgpKernelErrors
         expr: increase(bgp_kernel_errors_total[10m]) > 0

bin/bgp/src/main.rs

@@ -1,4 +1,4 @@
-use std::path::PathBuf;
+use std::path::{Path, PathBuf};
 use std::sync::Arc;
 
 use bgp_server::config::Config;
@@ -39,7 +39,13 @@ async fn main() -> Result<()> {
         .wrap_err_with(|| format!("Failed to read {}", args.config.display()))?;
     let proto_config = protobuf::text_format::parse_from_str::<BgpConfig>(&config_string)
         .wrap_err("Failed to parse config file")?;
-    let config = Config::from_proto(proto_config).wrap_err("Invalid configuration")?;
+    // What the configuration names by a relative path is beside it.
+    let beside = args
+        .config
+        .parent()
+        .filter(|directory| !directory.as_os_str().is_empty())
+        .unwrap_or(Path::new("."));
+    let config = Config::from_proto_in(proto_config, beside).wrap_err("Invalid configuration")?;
     if args.check {
         println!(
             "{} is in order: AS{}, peers: {}, routes originated: {}",
@@ -48,6 +54,22 @@ async fn main() -> Result<()> {
             config.peers.len(),
             config.originate.len()
         );
+        if let Some(policy) = &config.policy {
+            println!("policy functions: {}", policy.functions().join(", "));
+            for peer in &config.peers {
+                let name = |hook: &Option<bgp_policy::Hook>| {
+                    hook.as_ref()
+                        .map_or("-", bgp_policy::Hook::name)
+                        .to_string()
+                };
+                println!(
+                    "  {}: import {}, export {}",
+                    peer.name,
+                    name(&peer.import_hook),
+                    name(&peer.export_hook)
+                );
+            }
+        }
         return Ok(());
     }