Carry IPv4 routes over IPv6 sessions with IPv6 next hops (RFC 8950)

For a link that has only IPv6 addresses.

- The Extended Next Hop capability is offered on every session over IPv6
  whose families include IPv4 unicast, and used if the peer offers it too.
  Nothing to configure.
- On such a session our IPv4 routes go out in MP_REACH_NLRI with the
  session's IPv6 address as next hop, and are withdrawn in
  MP_UNREACH_NLRI. Without the agreement they are not sent, as before.
- IPv4 routes are read with an IPv6 next hop, global or global with
  link-local.
- The kernel installer resolves an IPv4 route's IPv6 next hop like any
  other and installs it: by way of an IPv6 nexthop object, or with RTA_VIA
  where routes carry their own gateway.

Tested against a real kernel in both modes, with a link-local next hop
too, and in the lab against FRR and BIRD: the new `v6only` phase takes
the IPv4 addresses off the exchange LAN, runs one IPv6 session with each
router for both families, and pings IPv4 across.

Sessions between link-local addresses are not supported yet.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

bin/bgp/README.md

@@ -168,6 +168,26 @@ peers {
 }
 ```
 
+## IPv4 over a link that has only IPv6 addresses (RFC 8950)
+
+A session over IPv6 carries IPv4 routes too, with IPv6 next hops, if the
+peer agrees: the daemon offers the Extended Next Hop capability on every
+session over IPv6 whose address families include IPv4 unicast, and nothing
+has to be configured for it on this side. (On FRR it is `neighbor ...
+capability extended-nexthop`, on BIRD `extended next hop on` in the IPv4
+channel.) The peers table on the status page says "IPv4 with IPv6 next
+hops" for a session that agreed on it.
+
+Our IPv4 routes then go out with the session's own IPv6 address as next
+hop, and the peer's arrive with its IPv6 address, or with a global and a
+link-local one. In the kernel such a route is an IPv4 route by way of an
+IPv6 neighbor (`10.2.0.0/24 via inet6 2001:db8::2 dev eth0`, Linux 5.2 and
+later).
+
+The session's addresses have to be global ones (or unique local): a peer
+configured by a link-local address (`fe80::...`), which needs the
+interface named with it, is not supported yet.
+
 ## Forwarding: routes in the kernel (Linux)
 
 With `kernel {}` in the configuration the daemon puts the best route of every
@@ -272,13 +292,15 @@ Routinator 0.15.2. What it covers:
 | `restart` | The daemon stopped in good order (kernel emptied, peers told) and killed (routes kept, stale one cleared by the next start) |
 | `scale` | A table from BIRD (`SCALE_ROUTES`, a million for the Internet's size) into the table and the kernel; the daemon killed and restarted under it; the table withdrawn |
 | `limits` | A peer whose role does not fit gets no session; a peer over its prefix limit gets Cease / Maximum Number of Prefixes and is held down |
+| `v6only` | The exchange LAN without IPv4 addresses: one IPv6 session with each router for both families, IPv4 routes with IPv6 next hops (RFC 8950) in BGP and in the kernel, IPv4 pings across |
 
 `scripts/interop/real-rpki.sh` checks the RTR client against a Routinator
 that has the real RPKI, ASPAs included: the daemon has to end up with as
 many VRPs and ASPAs as the validator says it has.
 
 Not covered: routers other than FRR and BIRD; IPv6 routes over an IPv4
-session (the daemon does not send them); long runs.
+session (the daemon does not send them); sessions between link-local
+addresses; long runs.
 
 ## Finding a route
 

crates/fib/src/netlink.rs

@@ -84,6 +84,9 @@ mod route_attribute {
     pub const GATEWAY: u16 = 5;
     pub const PRIORITY: u16 = 6;
     pub const TABLE: u16 = 15;
+    /// A gateway of another family than the route's: the family, then
+    /// the address (`RTA_VIA`).
+    pub const VIA: u16 = 18;
     pub const NEXTHOP_ID: u16 = 30;
 }
 
@@ -380,6 +383,11 @@ fn route(fixed: &[u8], attributes_of_route: &[u8]) -> Option<Route> {
                 route.prefix = IpPrefix::new(version, value, destination_length).ok()?;
             }
             route_attribute::GATEWAY => route.gateway = Some(address(version, value)?),
+            route_attribute::VIA => {
+                let (family, gateway) = value.split_at_checked(2)?;
+                let family = version_of(u8::try_from(ne_u16(family)).ok()?)?;
+                route.gateway = Some(address(family, gateway)?);
+            }
             route_attribute::OUT_INTERFACE => route.interface = Some(ne_u32(value.get(..4)?)),
             route_attribute::PRIORITY => route.metric = ne_u32(value.get(..4)?),
             route_attribute::TABLE => route.table = ne_u32(value.get(..4)?),
@@ -546,7 +554,20 @@ pub fn route_request(
     }
     if let Some(gateway) = &route.gateway {
         let (bytes, length) = octets(gateway);
-        put_attribute(out, route_attribute::GATEWAY, &bytes[..length]);
+        let gateway_version = match gateway {
+            IpAddr::V4(_) => IpVersion::Ipv4,
+            IpAddr::V6(_) => IpVersion::Ipv6,
+        };
+        if gateway_version == version {
+            put_attribute(out, route_attribute::GATEWAY, &bytes[..length]);
+        } else {
+            // An IPv4 route by way of an IPv6 neighbor (Linux 5.2 and
+            // later).
+            let mut via = [0u8; 18];
+            via[..2].copy_from_slice(&u16::from(family_of(gateway_version)).to_ne_bytes());
+            via[2..2 + length].copy_from_slice(&bytes[..length]);
+            put_attribute(out, route_attribute::VIA, &via[..2 + length]);
+        }
     }
     if let Some(interface) = route.interface {
         put_attribute(
@@ -723,6 +744,37 @@ mod tests {
         );
     }
 
+    #[test]
+    fn an_ipv4_route_by_way_of_an_ipv6_gateway_reads_back() {
+        let route = Route {
+            prefix: prefix("203.0.113.0/24"),
+            table: TABLE_MAIN,
+            protocol: PROTOCOL_BGP,
+            metric: 20,
+            gateway: Some("2001:db8::2".parse().unwrap()),
+            interface: Some(3),
+            nexthop: None,
+            kind: KIND_UNICAST,
+        };
+        let mut wire = BytesMut::new();
+        route_request(&mut wire, 1, Change::Replace, false, &route);
+        // RTA_VIA after destination, table and priority: AF_INET6, then
+        // the address.
+        let via = &wire[16 + 12 + 3 * 8..][..24];
+        // Four of header, two of family, sixteen of address; padded to 24.
+        assert_eq!(via[..2], 22u16.to_ne_bytes());
+        assert_eq!(via[2..4], 18u16.to_ne_bytes());
+        assert_eq!(via[4..6], 10u16.to_ne_bytes());
+        assert_eq!(via[6..8], [0x20, 0x01]);
+        assert_eq!(
+            read_back(&route, Change::Replace).1,
+            Message::Route {
+                present: true,
+                route
+            }
+        );
+    }
+
     #[test]
     fn nexthop_objects_are_asked_for_named_and_read() {
         // The request: RTM_NEWNEXTHOP, REQUEST|ACK|ECHO|CREATE; nhmsg with

crates/fib/src/plan.rs

@@ -28,7 +28,7 @@ pub struct Target {
 pub enum Unresolved {
     /// The route has no next hop.
     Missing,
-    /// The next hop is of the other IP version than the prefix.
+    /// An IPv6 route with an IPv4 next hop.
     OtherFamily,
     /// The next hop is one of this machine's own addresses.
     Ourselves,
@@ -49,7 +49,7 @@ impl Unresolved {
     pub fn as_str(self) -> &'static str {
         match self {
             Self::Missing => "no next hop",
-            Self::OtherFamily => "next hop of the other IP version",
+            Self::OtherFamily => "IPv4 next hop for an IPv6 route",
             Self::Ourselves => "next hop is this machine",
             Self::NotConnected => "next hop not directly connected",
         }
@@ -77,7 +77,10 @@ pub fn resolve(
         NlriNextHop::Ipv4(_) => IpVersion::Ipv4,
         NlriNextHop::Ipv6(_) | NlriNextHop::Ipv6WithLl { .. } => IpVersion::Ipv6,
     };
-    if version != prefix.address_family() {
+    // An IPv4 route may have an IPv6 next hop: that is how routes go over
+    // a link with only IPv6 addresses (RFC 8950), and the kernel takes
+    // such a route. The other way about means nothing.
+    if version != prefix.address_family() && version == IpVersion::Ipv4 {
         return Err(Unresolved::OtherFamily);
     }
     let direct = |gateway: IpAddr| {
@@ -427,7 +430,6 @@ mod tests {
         for (hop, why) in [
             (Some(v4_hop("10.9.9.9")), Unresolved::NotConnected),
             (Some(v4_hop("192.0.2.1")), Unresolved::Ourselves),
-            (Some(v6_hop("2001:db8::2")), Unresolved::OtherFamily),
             (None, Unresolved::Missing),
         ] {
             assert_eq!(resolve(&interfaces, &v4, hop.as_ref(), peer), Err(why));
@@ -436,6 +438,24 @@ mod tests {
             resolve(&interfaces, &v6, Some(&v4_hop("192.0.2.2")), peer),
             Err(Unresolved::OtherFamily)
         );
+        // An IPv4 route by way of an IPv6 neighbor: a link with only IPv6
+        // addresses (RFC 8950).
+        assert_eq!(
+            resolve(&interfaces, &v4, Some(&v6_hop("2001:db8::2")), peer),
+            Ok(target("2001:db8::2", 2))
+        );
+        assert_eq!(
+            resolve(
+                &interfaces,
+                &v4,
+                Some(&NlriNextHop::Ipv6WithLl {
+                    global: "2001:db8:ffff::2".parse().unwrap(),
+                    link_local: "fe80::2".parse().unwrap(),
+                }),
+                Some("2001:db8::2".parse().unwrap()),
+            ),
+            Ok(target("fe80::2", 2))
+        );
     }
 
     #[test]

crates/fib/tests/kernel.rs

@@ -594,3 +594,69 @@ async fn routes_of_an_earlier_run_are_kept_while_peers_come_back() {
     fib.stop().await;
     task.await.unwrap().unwrap();
 }
+
+/// An IPv4 route whose next hop is an IPv6 neighbor: routes over a link
+/// that has only IPv6 addresses (RFC 8950). The kernel takes such a route,
+/// by way of a nexthop object or with its gateway spelled out.
+#[tokio::test(flavor = "multi_thread")]
+#[ignore = "changes the machine's routes; run scripts/kernel-tests.sh"]
+async fn an_ipv4_route_goes_by_way_of_an_ipv6_neighbor() {
+    for (n, nexthop_objects) in [(8, true), (9, false)] {
+        let mut machine = Machine::new(n);
+        machine.nexthop_objects = nexthop_objects;
+        // Only IPv6 on the link.
+        ip(&format!(
+            "addr del 10.{n}.0.1/24 dev {}",
+            machine.interface()
+        ));
+        let neighbor: IpAddr = format!("2001:db8:{n}::2").parse().unwrap();
+        let peer = machine
+            .rib
+            .add_peer(PeerInfo {
+                asn: 65002,
+                router_id: machine.neighbor(2),
+                address: neighbor,
+                ibgp: false,
+            })
+            .unwrap();
+        // Both families by way of the same neighbor.
+        let via = machine.via(neighbor, 0);
+        machine.rib.announce(
+            peer,
+            &via,
+            &[prefix("203.0.113.0/24"), prefix("2001:db8:100::/48")],
+        );
+        // And an IPv4 route whose next hop has a link-local address too,
+        // with a global one that is not on any link of ours.
+        let link_local = machine
+            .rib
+            .attrs()
+            .intern(
+                &[
+                    PathAttribute::Origin(OriginPathAttribute::IGP),
+                    AsPathAttribute::from_asns(vec![65002]),
+                ],
+                Some(&NlriNextHop::Ipv6WithLl {
+                    global: "2001:db8:ffff::2".parse().unwrap(),
+                    link_local: "fe80::2".parse().unwrap(),
+                }),
+            )
+            .unwrap();
+        machine
+            .rib
+            .announce(peer, &link_local, &[prefix("198.18.0.0/15")]);
+
+        let (fib, task) = machine.start();
+        let interface = machine.interface();
+        machine.kernel_becomes(&[
+            format!("203.0.113.0/24 via inet6 {neighbor} dev {interface} proto bgp metric 20"),
+            format!("198.18.0.0/15 via inet6 fe80::2 dev {interface} proto bgp metric 20"),
+            machine.shown("2001:db8:100::/48", neighbor),
+        ]);
+        assert_eq!(fib.status().errors, 0, "{:?}", fib.status().last_error);
+
+        fib.stop().await;
+        task.await.unwrap().unwrap();
+        machine.kernel_becomes(&[]);
+    }
+}

crates/integration-tests/tests/interop.rs

@@ -127,6 +127,7 @@ async fn test_ipv6_prefix_update_shape() -> Result<()> {
             four_octet_asn: true,
             ibgp: false,
             role: None,
+            extended_next_hop: false,
         },
     )
     .expect("ipv6 over ipv6");

crates/packet/src/message.rs

@@ -1182,11 +1182,13 @@ impl MpReachNlriPathAttribute {
 
         let (buf, next_hop_len) = be_u8(buf)?;
         let (buf, next_hop) = nom::bytes::take(usize::from(next_hop_len)).parse(buf)?;
-        // An IPv4 address for IPv4 routes; for IPv6, a global address,
-        // optionally followed by a link-local one (RFC 2545).
+        // For IPv6 routes, a global address, optionally followed by a
+        // link-local one (RFC 2545). For IPv4 routes an IPv4 address, or
+        // the same as for IPv6: routes over a link that has only IPv6
+        // addresses (RFC 8950). Which of the two it is shows in its length.
         let next_hop = NlriNextHop::from_bytes(next_hop)
             .filter(|next_hop| {
-                matches!(next_hop, NlriNextHop::Ipv4(_)) == (version == IpVersion::Ipv4)
+                version == IpVersion::Ipv4 || !matches!(next_hop, NlriNextHop::Ipv4(_))
             })
             .ok_or_else(|| {
                 malformed(format!(

crates/server/src/advertise.rs

@@ -233,6 +233,7 @@ impl Exporter {
             four_octet_asn: session.four_octet_asn,
             ibgp: session.ibgp,
             role: session.role,
+            extended_next_hop: session.extended_next_hop,
         };
         let Some(update) = build_announce(route, &towards) else {
             debug!(
@@ -255,7 +256,10 @@ impl Exporter {
         }
         debug!(peer = %self.session.peer, %prefix, "Withdrawing local route");
         self.session
-            .send(Message::Update(build_withdraw(&prefix)))
+            .send(Message::Update(build_withdraw(
+                &prefix,
+                self.session.extended_next_hop,
+            )))
             .await?;
         Ok(())
     }
@@ -276,13 +280,16 @@ pub struct Towards {
     pub ibgp: bool,
     /// What the peer is to us, if that is known.
     pub role: Option<PeerRole>,
+    /// Whether the session takes IPv4 routes with IPv6 next hops
+    /// (RFC 8950).
+    pub extended_next_hop: bool,
 }
 
 /// Builds the UPDATE announcing `route` on a session.
 ///
-/// Returns `None` when the prefix family does not match the next-hop family:
-/// carrying IPv4 NLRI over an IPv6 next hop (RFC 8950) or vice versa is not
-/// supported yet.
+/// Returns `None` when the route cannot be given the session's address as
+/// next hop: an IPv6 route on a session over IPv4, or an IPv4 route on a
+/// session over IPv6 that did not agree on RFC 8950.
 pub fn build_announce(route: &LocalRoute, towards: &Towards) -> Option<UpdateMessage> {
     let attrs = &route.attrs;
 
@@ -328,6 +335,16 @@ pub fn build_announce(route: &LocalRoute, towards: &Towards) -> Option<UpdateMes
                 prefixes: vec![route.prefix.into()],
             }))
         }
+        // An IPv4 route over a link that has only IPv6 addresses: in
+        // MP_REACH_NLRI like an IPv6 route, with the IPv6 next hop
+        // (RFC 8950 §4).
+        (IpVersion::Ipv4, IpAddr::V6(next_hop)) if towards.extended_next_hop => {
+            Some(PathAttribute::MpReachNlri(MpReachNlriPathAttribute {
+                family: AfiSafi::IPV4_UNICAST,
+                next_hop: NlriNextHop::Ipv6(next_hop),
+                prefixes: vec![route.prefix.into()],
+            }))
+        }
         _ => return None,
     };
     if let Some(med) = attrs.med {
@@ -378,9 +395,19 @@ pub fn build_announce(route: &LocalRoute, towards: &Towards) -> Option<UpdateMes
     })
 }
 
-/// Builds the UPDATE withdrawing `prefix`.
-pub fn build_withdraw(prefix: &IpPrefix) -> UpdateMessage {
+/// Builds the UPDATE withdrawing `prefix`. With `extended_next_hop` the
+/// session has IPv4 routes announced in MP_REACH_NLRI (RFC 8950), and they
+/// are withdrawn the way they were announced.
+pub fn build_withdraw(prefix: &IpPrefix, extended_next_hop: bool) -> UpdateMessage {
     match prefix.address_family() {
+        IpVersion::Ipv4 if extended_next_hop => UpdateMessage {
+            withdrawn_routes: vec![],
+            path_attributes: vec![PathAttribute::MpUnreachNlri(MpUnreachNlriPathAttribute {
+                family: AfiSafi::IPV4_UNICAST,
+                prefixes: vec![Nlri::from(*prefix)],
+            })],
+            announced_routes: vec![],
+        },
         IpVersion::Ipv4 => UpdateMessage {
             withdrawn_routes: vec![Nlri::from(*prefix)],
             path_attributes: vec![],
@@ -412,6 +439,69 @@ mod tests {
         IpPrefix::try_from(s).unwrap()
     }
 
+    /// RFC 8950: over a link that has only IPv6 addresses, an IPv4 route
+    /// goes in MP_REACH_NLRI with our IPv6 address as next hop, and is
+    /// withdrawn in MP_UNREACH_NLRI.
+    #[test]
+    fn an_ipv4_route_goes_over_an_ipv6_session_if_both_agreed() {
+        let route = LocalRoute::new(prefix("10.1.0.0/16"), LocalRouteAttrs::default());
+        let ours: Ipv6Addr = "2001:db8::1".parse().unwrap();
+        let over_ipv6 = towards(65000, IpAddr::V6(ours), true);
+        // Not agreed: the route has no next hop the peer could use.
+        assert!(build_announce(&route, &over_ipv6).is_none());
+
+        let agreed = Towards {
+            extended_next_hop: true,
+            ..over_ipv6
+        };
+        let update = build_announce(&route, &agreed).unwrap();
+        assert!(update.announced_routes.is_empty(), "not in the body");
+        assert!(
+            !update
+                .path_attributes
+                .iter()
+                .any(|attribute| matches!(attribute, PathAttribute::NextHop(_))),
+            "no NEXT_HOP attribute"
+        );
+        assert!(update.path_attributes.contains(&PathAttribute::MpReachNlri(
+            MpReachNlriPathAttribute {
+                family: AfiSafi::IPV4_UNICAST,
+                next_hop: NlriNextHop::Ipv6(ours),
+                prefixes: vec![prefix("10.1.0.0/16").into()],
+            }
+        )));
+        // What is written is what a peer reads.
+        let ctx = ParserContext {
+            four_octet_asn: true,
+            ibgp: false,
+            add_path: Vec::new(),
+        };
+        let mut wire = BytesMut::new();
+        Message::Update(update.clone())
+            .to_wire(&ctx, &mut wire)
+            .unwrap();
+        assert_eq!(
+            Message::decode(&ctx, &wire).unwrap(),
+            Message::Update(update)
+        );
+
+        let withdrawal = build_withdraw(&prefix("10.1.0.0/16"), true);
+        assert!(withdrawal.withdrawn_routes.is_empty());
+        assert_eq!(
+            withdrawal.path_attributes,
+            [PathAttribute::MpUnreachNlri(MpUnreachNlriPathAttribute {
+                family: AfiSafi::IPV4_UNICAST,
+                prefixes: vec![prefix("10.1.0.0/16").into()],
+            })]
+        );
+        // An IPv6 route is as it always was, agreed or not.
+        let v6 = LocalRoute::new(prefix("2001:db8:1::/48"), LocalRouteAttrs::default());
+        assert_eq!(
+            build_announce(&v6, &agreed),
+            build_announce(&v6, &over_ipv6)
+        );
+    }
+
     /// RFC 9234 §5, the egress procedure for routes we originate.
     #[test]
     fn a_route_is_marked_only_to_customer_on_its_way_down_or_across() {
@@ -456,6 +546,7 @@ mod tests {
             four_octet_asn,
             ibgp: false,
             role: None,
+            extended_next_hop: false,
         }
     }
 
@@ -599,11 +690,11 @@ mod tests {
 
     #[test]
     fn withdraw_by_family() {
-        let v4 = build_withdraw(&prefix("10.0.0.0/8"));
+        let v4 = build_withdraw(&prefix("10.0.0.0/8"), false);
         assert_eq!(v4.withdrawn_routes, vec![prefix("10.0.0.0/8")]);
         assert!(v4.path_attributes.is_empty());
 
-        let v6 = build_withdraw(&prefix("2001:db8::/32"));
+        let v6 = build_withdraw(&prefix("2001:db8::/32"), false);
         assert!(v6.withdrawn_routes.is_empty());
         assert!(
             matches!(&v6.path_attributes[..], [PathAttribute::MpUnreachNlri(m)] if m.prefixes == vec![prefix("2001:db8::/32")])
@@ -731,6 +822,7 @@ mod tests {
                 add_path_receive: FamilySet::default(),
                 add_path_send: FamilySet::default(),
                 role_confirmed: false,
+                extended_next_hop: false,
             };
             let rib_peer = self
                 .table

crates/server/src/connection.rs

@@ -401,6 +401,9 @@ impl Session {
             add_path: peer.add_path,
             role: peer.role,
             require_role: peer.require_role,
+            // Over IPv6 our address is no next hop for an IPv4 route as
+            // BGP first had it; this is how such routes go all the same.
+            extended_next_hop: peer_addr.is_ipv6(),
         });
         // The peer goes by its configured name, or by its address.
         let label = peer

crates/server/src/fsm.rs

@@ -16,7 +16,10 @@ use std::time::Duration;
 
 use bgp_packet::errors::{BgpError, DecodeError, OpenMsgSubcode};
 use bgp_packet::message::{AS_TRANS, BGP4_VERSION, Message, UpdateMessage};
-use bgp_packet::open::{AddPathEntry, AddPathSendRecv, Capability, OpenMessage, OpenOption};
+use bgp_packet::open::{
+    AddPathEntry, AddPathSendRecv, Capability, ExtendedNextHop, OpenMessage, OpenOption,
+};
+use bgp_primitives::constants::{Afi, AfiSafi, Safi};
 
 use crate::error::SessionError;
 use crate::family::FamilySet;
@@ -73,6 +76,9 @@ pub struct FsmConfig {
     /// Refuse a peer that does not say what it is (RFC 9234 §4.2, "strict
     /// mode"). Means nothing without `role`.
     pub require_role: bool,
+    /// Offer to exchange IPv4 routes with IPv6 next hops (RFC 8950): for a
+    /// session over a link that has only IPv6 addresses.
+    pub extended_next_hop: bool,
 }
 
 /// What the two OPENs settled.
@@ -101,6 +107,9 @@ pub struct Negotiated {
     /// Whether the peer said what it is to us, and it is what we have it
     /// configured as (RFC 9234).
     pub role_confirmed: bool,
+    /// Whether IPv4 routes go with IPv6 next hops on this session
+    /// (RFC 8950): both sides offered it, and the session carries IPv4.
+    pub extended_next_hop: bool,
 }
 
 /// A message from the peer, with what the decoder found out besides.
@@ -377,7 +386,16 @@ impl Fsm {
             }
         }
 
+        // RFC 8950 §4: in use only if both said so.
+        let extended_next_hop = self.config.extended_next_hop
+            && families.contains(AfiSafi::IPV4_UNICAST)
+            && open.capabilities().any(|capability| {
+                matches!(capability, Capability::ExtendedNextHop { entries }
+                    if entries.contains(&IPV4_WITH_IPV6_NEXT_HOP))
+            });
+
         let negotiated = Negotiated {
+            extended_next_hop,
             peer_asn,
             ibgp: peer_asn == self.config.local_asn,
             peer_identifier: open.identifier,
@@ -434,6 +452,14 @@ impl Fsm {
     }
 }
 
+/// IPv4 unicast routes with next hops that are IPv6 addresses, as the
+/// Extended Next Hop capability names it (RFC 8950 §3).
+const IPV4_WITH_IPV6_NEXT_HOP: ExtendedNextHop = ExtendedNextHop {
+    afi: Afi::IPV4,
+    safi: Safi::UNICAST,
+    nh_afi: Afi::IPV6,
+};
+
 /// Any message from the peer shows it is alive (RFC 4271 §4.4).
 fn heard_from_peer(negotiated: &Negotiated) -> Vec<Action> {
     hold_time(negotiated)
@@ -464,6 +490,11 @@ fn build_open(config: &FsmConfig) -> Message {
     if let Some(role) = config.role {
         capabilities.push(Capability::BgpRole { role: role.ours() });
     }
+    if config.extended_next_hop && config.families.contains(AfiSafi::IPV4_UNICAST) {
+        capabilities.push(Capability::ExtendedNextHop {
+            entries: vec![IPV4_WITH_IPV6_NEXT_HOP],
+        });
+    }
     if let Some(send_recv) = config.add_path {
         capabilities.push(Capability::AddPath {
             entries: config
@@ -531,6 +562,7 @@ mod tests {
             add_path: None,
             role: None,
             require_role: false,
+            extended_next_hop: false,
         }
     }
 
@@ -632,6 +664,7 @@ mod tests {
             add_path_receive: FamilySet::default(),
             add_path_send: FamilySet::default(),
             role_confirmed: false,
+            extended_next_hop: false,
         };
         assert!(matches!(
             &actions[..],
@@ -891,6 +924,51 @@ mod tests {
         }
     }
 
+    /// RFC 8950: IPv4 routes with IPv6 next hops are in use only if both
+    /// sides offer it, and we offer it only where asked to.
+    #[test]
+    fn ipv4_with_ipv6_next_hops_is_in_use_only_if_both_offer_it() {
+        let offer = Capability::ExtendedNextHop {
+            entries: vec![IPV4_WITH_IPV6_NEXT_HOP],
+        };
+        let outcome = |ours: bool, theirs: Vec<Capability>| {
+            let (mut fsm, first) = Fsm::new(FsmConfig {
+                extended_next_hop: ours,
+                ..config()
+            });
+            let Action::Send(open) = &first[0] else {
+                panic!("expected an OPEN first, got {first:?}");
+            };
+            assert_eq!(capabilities(open).contains(&&offer), ours);
+            let mut offered = vec![
+                Capability::multiprotocol(AfiSafi::IPV4_UNICAST),
+                Capability::multiprotocol(AfiSafi::IPV6_UNICAST),
+            ];
+            offered.extend(theirs);
+            let open = OpenMessage {
+                options: vec![OpenOption::Capabilities(offered)],
+                ..peer_open(90)
+            };
+            match &fsm.handle(received(Message::Open(open)))[..] {
+                [Action::Admit(negotiated), ..] => negotiated.extended_next_hop,
+                other => panic!("expected the session to go on, got {other:?}"),
+            }
+        };
+        assert!(outcome(true, vec![offer.clone()]));
+        assert!(!outcome(true, vec![]));
+        assert!(!outcome(false, vec![offer.clone()]));
+        // Offered for something else: IPv6 routes with IPv4 next hops,
+        // say.
+        let other = Capability::ExtendedNextHop {
+            entries: vec![ExtendedNextHop {
+                afi: Afi::IPV6,
+                safi: Safi::UNICAST,
+                nh_afi: Afi::IPV4,
+            }],
+        };
+        assert!(!outcome(true, vec![other]));
+    }
+
     /// RFC 9234 §4.1: our OPEN says what we are to the peer, which is the
     /// other half of what the peer is to us.
     #[test]

crates/server/src/session.rs

@@ -125,6 +125,9 @@ pub struct SessionHandle {
     pub role: Option<PeerRole>,
     /// Whether the peer said so itself in its OPEN (RFC 9234).
     pub role_confirmed: bool,
+    /// Whether IPv4 routes go with IPv6 next hops on this session
+    /// (RFC 8950).
+    pub extended_next_hop: bool,
     /// Whether the peer negotiated RFC 6793 four-octet ASNs. Determines how
     /// AS_PATH is encoded on the wire for this session.
     pub four_octet_asn: bool,
@@ -170,6 +173,7 @@ impl SessionHandle {
             ibgp: negotiated.ibgp,
             role: None,
             role_confirmed: false,
+            extended_next_hop: negotiated.extended_next_hop,
             four_octet_asn: negotiated.four_octet_asn,
             families: negotiated.families.clone(),
             add_path_receive: negotiated.add_path_receive.clone(),

crates/web/src/api.rs

@@ -64,6 +64,9 @@ pub struct PeerView {
     pub established_at_unix: Option<u64>,
     pub uptime_secs: Option<u64>,
     pub four_octet_asn: Option<bool>,
+    /// Whether IPv4 routes go with IPv6 next hops on the session
+    /// (RFC 8950).
+    pub extended_next_hop: Option<bool>,
     /// The address families negotiated with the peer, e.g. `IPv4 unicast`.
     pub address_families: Option<Vec<String>>,
     /// The families for which the peer sends us several routes for a
@@ -102,6 +105,7 @@ impl From<&SessionHandle> for PeerView {
             established_at_unix: Some(unix_secs(s.established_at)),
             uptime_secs: Some(s.established_at.elapsed().map(|d| d.as_secs()).unwrap_or(0)),
             four_octet_asn: Some(s.four_octet_asn),
+            extended_next_hop: Some(s.extended_next_hop),
             address_families: Some(s.families.iter().map(|f| f.to_string()).collect()),
             add_path_receive: Some(s.add_path_receive.iter().map(|f| f.to_string()).collect()),
             add_path_send: Some(s.add_path_send.iter().map(|f| f.to_string()).collect()),
@@ -135,6 +139,7 @@ impl PeerView {
             established_at_unix: None,
             uptime_secs: None,
             four_octet_asn: None,
+            extended_next_hop: None,
             address_families: None,
             add_path_receive: None,
             add_path_send: None,

crates/web/static/statusz.html

@@ -186,7 +186,7 @@ from <select id="lookup-peer"><option value="">any peer</option></select>
         <td><span class="${STATE_CLASS[p.state] || ''}">${esc(p.state)}</span></td><td>${up ? (p.direction === 'outbound' ? 'us' : 'peer') : DASH}</td>
         <td class="mono">${up ? esc(p.local_addr) : DASH}</td><td>${up ? fmtTime(p.established_at_unix) : DASH}</td>
         <td>${up ? age(p.established_at_unix) : DASH}</td><td>${up ? (p.four_octet_asn ? 'yes' : 'no') : DASH}</td>
-        <td>${up ? esc((p.address_families || []).join(', ')) + addPath(p) : DASH}</td>
+        <td>${up ? esc((p.address_families || []).join(', ')) + addPath(p) + (p.extended_next_hop ? ' &middot; IPv4 with IPv6 next hops' : '') : DASH}</td>
         <td>${up ? p.routes : DASH}</td>
         <td>${up ? (p.rejected_routes ? `<a class="bad" href="#refused">${p.rejected_routes}</a>` : 0) + (p.held_routes ? ` <span class="dim" title="Refused as RPKI or ASPA invalid and kept aside, to be taken if that changes">(${p.held_routes} held)</span>` : '') : DASH}</td><td>${detail}</td></tr>`;
     }).join(''));

docs/RFCs.md

@@ -120,7 +120,12 @@ https://datatracker.ietf.org/doc/html/rfc8950
 
 | Feature | Status |
 |---------|--------|
-| `ExtendedNextHop` capability wire format | 🟡 |
+| `ExtendedNextHop` capability offered on sessions over IPv6 that carry IPv4 unicast, and used if the peer offers it | ✅ |
+| IPv4 unicast routes sent and withdrawn in MP_REACH_NLRI / MP_UNREACH_NLRI with our IPv6 address as next hop | ✅ |
+| IPv4 unicast routes received with an IPv6 next hop (global, or global and link-local) | ✅ |
+| Such routes installed in the kernel (IPv4 route by way of an IPv6 neighbor) | ✅ |
+| Other families than IPv4 unicast (VPN-IPv4, labelled) | ❌ |
+| Sessions between link-local addresses | ❌ |
 
 ---
 

scripts/interop/lab.sh

@@ -29,7 +29,7 @@ cd "$(dirname "$0")/../.."
 bgp="${CARGO_TARGET_DIR:-target}/release/bgp"
 work=/tmp/ferrix-lab
 pathspace=ferrixlab
-all_phases="basics roles rpki refresh flaps restart scale limits"
+all_phases="basics roles rpki refresh flaps restart scale limits v6only"
 phases="${PHASES:-$all_phases}"
 scale_routes="${SCALE_ROUTES:-200000}"
 
@@ -170,6 +170,29 @@ http_endpoint { address: "127.0.0.1" port: 8080 }
 END
 }
 
+dut_config_v6only() {
+    cat > "$work/dut.textpb" <<END
+router_id: "10.9.0.1"
+local_asn: 64601
+listen_endpoints { address: "2001:db8:9::1" port: 179 }
+connect_retry_seconds: 1
+peers {
+  name: "frr" endpoint { address: "2001:db8:9::2" } remote_asn: 64602
+  role: PEER
+  import { accept_bogons: true max_prefixes: 1000 }
+}
+peers {
+  name: "bird" endpoint { address: "2001:db8:9::3" } remote_asn: 4200000003
+  role: PROVIDER
+  import { accept_bogons: true max_prefixes: 1000 }
+}
+originate { prefix: "10.1.0.0/24" }
+originate { prefix: "2001:db8:1::/48" }
+kernel { stale_seconds: 3 }
+http_endpoint { address: "127.0.0.1" port: 8080 }
+END
+}
+
 dut_up() {
     sudo ip netns exec lab-dut \
         setpriv --reuid "$(id -u)" --regid "$(id -g)" --init-groups \
@@ -197,7 +220,7 @@ dut_down() {
 
 # --- FRR ------------------------------------------------------------------------
 
-frr_up() {
+frr_config() {
     sudo mkdir -p "/etc/frr/$pathspace" "/var/run/frr/$pathspace"
     sudo tee "/etc/frr/$pathspace/frr.conf" > /dev/null <<END
 frr defaults traditional
@@ -229,6 +252,41 @@ router bgp 64602
  exit-address-family
 !
 END
+}
+
+# One session, over IPv6, for both families: IPv4 routes go with IPv6 next
+# hops (RFC 8950).
+frr_config_v6only() {
+    sudo tee "/etc/frr/$pathspace/frr.conf" > /dev/null <<END
+frr defaults traditional
+hostname frr
+log file $work/frr.log
+!
+router bgp 64602
+ bgp router-id 10.9.0.2
+ no bgp ebgp-requires-policy
+ no bgp network import-check
+ timers bgp 3 9
+ neighbor 2001:db8:9::1 remote-as 64601
+ neighbor 2001:db8:9::1 local-role peer
+ neighbor 2001:db8:9::1 timers connect 1
+ neighbor 2001:db8:9::1 capability extended-nexthop
+ !
+ address-family ipv4 unicast
+  network 10.2.0.0/24
+  network 10.2.5.0/24
+  neighbor 2001:db8:9::1 activate
+ exit-address-family
+ !
+ address-family ipv6 unicast
+  network 2001:db8:2::/48
+  neighbor 2001:db8:9::1 activate
+ exit-address-family
+!
+END
+}
+
+frr_up() {
     sudo touch "/etc/frr/$pathspace/vtysh.conf" "$work/frr.log"
     sudo chown -R frr:frr "/etc/frr/$pathspace" "/var/run/frr/$pathspace" "$work/frr.log"
     for daemon in zebra bgpd; do
@@ -294,6 +352,29 @@ protocol bgp dut6 from dut {
 END
 }
 
+bird_config_v6only() {
+    cat > "$work/bird.conf" <<END
+log "$work/bird.log" all;
+router id 10.9.0.3;
+protocol device {}
+protocol kernel kernel4 { ipv4 { export where source = RTS_BGP; }; }
+protocol kernel kernel6 { ipv6 { export where source = RTS_BGP; }; }
+protocol static static4 { ipv4; route 10.3.0.0/24 blackhole; }
+protocol static static6 { ipv6; route 2001:db8:3::/48 blackhole; }
+protocol bgp dut {
+  local as 4200000003;
+  neighbor 2001:db8:9::1 as 64601;
+  source address 2001:db8:9::3;
+  local role provider;
+  connect retry time 1;
+  hold time 9;
+  graceful restart off;
+  ipv4 { import all; export where source = RTS_STATIC; extended next hop on; };
+  ipv6 { import all; export where source = RTS_STATIC; };
+}
+END
+}
+
 bird_up() {
     sudo ip netns exec lab-bird bird -c "$work/bird.conf" -s "$work/bird.ctl" -P "$work/bird.pid"
 }
@@ -325,7 +406,7 @@ lab_up() {
     [ -x "$bgp" ] || fail "$bgp not found: cargo build --release -p bgp"
     network_up
     routinator_up
-    frr_up
+    frr_config; frr_up
     bird_config; bird_up
     dut_config; dut_up
 }
@@ -644,6 +725,63 @@ phase_limits() {
     ok "back to normal"
 }
 
+# The exchange LAN loses its IPv4 addresses. One session over IPv6 with
+# each router carries both families, and IPv4 routes have IPv6 next hops
+# (RFC 8950), in BGP and in the kernel. Run last: the lab stays that way.
+phase_v6only() {
+    dut_down TERM
+    frr_down TERM
+    sudo kill "$(sudo cat "$work/bird.pid")"; sleep 0.5; sudo rm -f "$work/bird.ctl"
+    local n=0 name
+    for name in dut frr bird; do
+        n=$((n + 1))
+        in_ns "$name" ip addr del "10.9.0.$n/24" dev ix0
+    done
+    frr_config_v6only; frr_up
+    bird_config_v6only; bird_up
+    dut_config_v6only; dut_up
+
+    wait_for "two sessions over IPv6" established 2
+    api /api/peers | jq -e 'all(.[]; .extended_next_hop and (.address_families | length == 2))' >/dev/null \
+        || fail "the sessions did not agree on IPv4 with IPv6 next hops: $(api /api/peers | jq -c '[.[] | {name, address_families, extended_next_hop}]')"
+    ok "both sessions carry IPv4 and IPv6, with IPv6 next hops for IPv4 (RFC 8950)"
+
+    # What they send us: IPv4 routes whose next hop is an IPv6 address,
+    # and the same in the kernel.
+    wait_for "FRR's IPv4 route" dut_has 10.2.0.0/24 frr
+    wait_for "BIRD's IPv4 route" dut_has 10.3.0.0/24 bird
+    api "/api/routes/received?prefix=10.2.0.0/24" | jq -e '.routes[0].summary.next_hop | startswith("2001:db8:9::2")' >/dev/null \
+        || fail "FRR's IPv4 route has the next hop $(api "/api/routes/received?prefix=10.2.0.0/24" | jq '.routes[0].summary.next_hop')"
+    wait_for "FRR's IPv4 route in the kernel" dut_kernel -4 "10.2.0.0/24 via inet6 2001:db8:9::2 dev ix0 metric 20"
+    wait_for "BIRD's IPv4 route in the kernel" dut_kernel -4 "10.3.0.0/24 via inet6 2001:db8:9::3 dev ix0 metric 20"
+    wait_for "FRR's IPv6 route in the kernel" dut_kernel -6 "2001:db8:2::/48 via 2001:db8:9::2 dev ix0 metric 20"
+    ok "their IPv4 routes arrive with IPv6 next hops and are in the kernel that way"
+
+    # What we send them.
+    wait_for "our IPv4 route at FRR" frr_has ipv4 10.1.0.0/24
+    vty -c "show bgp ipv4 unicast 10.1.0.0/24 json" | jq -e '.paths[0].nexthops | any(.ip == "2001:db8:9::1" and .afi == "ipv6")' >/dev/null \
+        || fail "FRR's copy of our IPv4 route has the next hops $(vty -c 'show bgp ipv4 unicast 10.1.0.0/24 json' | jq -c '.paths[0].nexthops')"
+    wait_for "our IPv4 route at BIRD" bird_has 10.1.0.0/24
+    bird_ctl show route for 10.1.0.0/24 all | grep -q "BGP.next_hop: 2001:db8:9::1" \
+        || fail "BIRD's copy of our IPv4 route: $(bird_ctl show route for 10.1.0.0/24 all)"
+    wait_for "our IPv6 route at FRR" frr_has ipv6 2001:db8:1::/48
+    ok "FRR and BIRD have our IPv4 route with our IPv6 address as next hop"
+
+    # IPv4 packets across a link that has no IPv4 addresses.
+    in_ns dut ping -q -c 2 -W 2 -I 10.1.0.1 10.2.0.1 >/dev/null || fail "no IPv4 ping to FRR's lan"
+    in_ns dut ping -q -c 2 -W 2 -I 10.1.0.1 10.3.0.1 >/dev/null || fail "no IPv4 ping to BIRD's lan"
+    in_ns frr ping -q -c 2 -W 2 -I 10.2.0.1 10.1.0.1 >/dev/null || fail "no IPv4 ping from FRR's lan"
+    in_ns dut ping -q -c 2 -W 2 -I 2001:db8:1::1 2001:db8:2::1 >/dev/null || fail "no IPv6 ping to FRR's lan"
+    ok "IPv4 packets cross the IPv6-only link, both ways"
+
+    # A withdrawal by FRR (in MP_UNREACH_NLRI) takes the route out.
+    frr_configure "router bgp 64602" "address-family ipv4 unicast" "no network 10.2.5.0/24"
+    wait_for "FRR's withdrawal" dut_lacks 10.2.5.0/24
+    wait_for "it out of the kernel" dut_kernel_lacks -4 10.2.5.0/24
+    dut_has 10.2.0.0/24 frr >/dev/null || fail "the withdrawal took the other route along"
+    ok "an IPv4 route withdrawn over the IPv6 session goes"
+}
+
 # --- main -----------------------------------------------------------------------
 
 case "${1:-test}" in