status: a session's TCP connection as the kernel sees it; a route in full

When a session advertised a full table to a route collector at two
thousand routes a second, nothing on the status page said whether the
daemon or the peer set that pace. The answer is in the kernel: the
peer's receive window, the queues, and how long the connection was held
up by the window or by the send buffer, which `ss -ti` shows.

This change asks the kernel the same way `ss` does, over the socket
diagnostics netlink family (`NETLINK_SOCK_DIAG`, `SOCK_DIAG_BY_FAMILY`
with the connection's addresses and ports), and reads `tcp_info` and
the congestion control name from the answer. No external program is
run and no unsafe code is needed: the request and the answer are byte
layouts of the kernel's ABI, read field by field, with fields an older
kernel does not send left at zero. A socket on a link-local address is
found by its interface, which the request names.

* `bgp_fib::sockdiag`: the request, the parser, `TcpConnection`, and
  `tcp_connection(local, peer)`; `Ok(None)` where there is no such
  connection or the system is not Linux.
* `GET /api/tcp?peer=ADDRESS` in the HTTP API: the connection of the
  session with that peer, with times in milliseconds, the two windows
  named by what they allow, and the held-up times also as shares of the
  busy time.
* The status page: a "tcp" link on each established session, which
  expands a row below it with the figures, re-asked on every refresh
  while open. The group rows' span is corrected for the "sent" column.

Also on the status page: the attributes of a received route that have
no column of their own were badges with their value in a tooltip, which
is hard to read. Clicking the attrs cell now expands a row under the
route with every attribute by its name (ORIGIN, AS_PATH, NEXT_HOP, MED,
LOCAL_PREF, COMMUNITIES, LARGE_COMMUNITIES, AGGREGATOR and the rest),
the path identifier, the selection rank and the RPKI and ASPA states;
the badges stay. Both expansions use the terms of `ss` and the RFCs
rather than sentences.

Documentation: the endpoint in the HTTP API reference, and the link and
the expansion in the monitoring guide.

Testing: unit tests of the request and of the parser on a reply built
field by field, and of a "no such socket" and a refused answer; a
Linux-only test that asks the kernel about a loopback connection of the
test's own and about one that does not exist; the lab's basics phase
checks the endpoint for the FRR session; the page was checked by hand
against the daemon in the Linux VM.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

crates/fib/src/lib.rs

@@ -43,6 +43,7 @@ mod kernel;
 pub mod netlink;
 pub mod plan;
 pub mod routes;
+pub mod sockdiag;
 pub mod topology;
 
 use std::net::IpAddr;

crates/fib/src/sockdiag.rs

@@ -0,0 +1,426 @@
+//! What the kernel knows about one of our TCP connections: the round-trip
+//! time, the windows, the queues and what has held the connection up,
+//! asked over the socket diagnostics netlink family (`NETLINK_SOCK_DIAG`,
+//! as `ss -ti` asks). For the status page, when someone wants to know
+//! why a session is slow.
+
+use std::io;
+use std::net::{IpAddr, SocketAddr};
+
+/// A TCP connection as the kernel sees it. Times are in microseconds,
+/// sizes in octets, rates in octets per second, unless said otherwise.
+/// Fields the running kernel does not report are zero.
+#[derive(Debug, Clone, Default, PartialEq, Eq)]
+pub struct TcpConnection {
+    /// The TCP state, by its number (1 is ESTABLISHED); see
+    /// [`TcpConnection::state_name`].
+    pub state: u8,
+    /// The congestion control algorithm, if the kernel named it.
+    pub congestion: Option<String>,
+    /// Octets received and not yet read by us.
+    pub recv_queue: u32,
+    /// Octets written by us and not yet acknowledged by the peer.
+    pub send_queue: u32,
+    pub rtt: u32,
+    pub rtt_variance: u32,
+    pub min_rtt: u32,
+    pub rto: u32,
+    pub mss: u32,
+    pub pmtu: u32,
+    /// The congestion window, in segments.
+    pub cwnd: u32,
+    /// The slow start threshold, in segments; `None` if there is none.
+    pub ssthresh: Option<u32>,
+    /// What the peer lets us send: its receive window.
+    pub send_window: u32,
+    /// What we let the peer send.
+    pub receive_window: u32,
+    pub receive_space: u32,
+    pub receive_ssthresh: u32,
+    pub send_window_scale: u8,
+    pub receive_window_scale: u8,
+    /// Segments sent and not acknowledged.
+    pub unacked: u32,
+    /// Segments retransmitted and still out.
+    pub retrans: u32,
+    pub lost: u32,
+    pub total_retrans: u32,
+    pub bytes_retrans: u64,
+    /// Octets in the socket's buffer that are not sent yet.
+    pub not_sent: u32,
+    pub pacing_rate: u64,
+    pub delivery_rate: u64,
+    pub bytes_sent: u64,
+    pub bytes_acked: u64,
+    pub bytes_received: u64,
+    pub segments_out: u32,
+    pub segments_in: u32,
+    /// How long the connection had something to send.
+    pub busy: u64,
+    /// Of that, how long it was held up by the peer's receive window.
+    pub limited_by_receive_window: u64,
+    /// And how long by our own send buffer being full.
+    pub limited_by_send_buffer: u64,
+    pub delivered: u32,
+    pub reordering: u32,
+    /// Milliseconds since data was last sent, received, and since the
+    /// last acknowledgement came.
+    pub last_data_sent_ms: u32,
+    pub last_data_received_ms: u32,
+    pub last_ack_received_ms: u32,
+}
+
+impl TcpConnection {
+    pub fn state_name(&self) -> &'static str {
+        match self.state {
+            1 => "established",
+            2 => "syn-sent",
+            3 => "syn-received",
+            4 => "fin-wait-1",
+            5 => "fin-wait-2",
+            6 => "time-wait",
+            7 => "close",
+            8 => "close-wait",
+            9 => "last-ack",
+            10 => "listen",
+            11 => "closing",
+            _ => "unknown",
+        }
+    }
+}
+
+const SOCK_DIAG_BY_FAMILY: u16 = 20;
+const NLMSG_ERROR: u16 = 2;
+const NLM_F_REQUEST: u16 = 1;
+const IPPROTO_TCP: u8 = 6;
+const AF_INET: u8 = 2;
+const AF_INET6: u8 = 10;
+const INET_DIAG_INFO: u16 = 2;
+const INET_DIAG_CONG: u16 = 4;
+/// `idiag_ext` bits asking for those two.
+const EXTENSIONS: u8 = (1 << (INET_DIAG_INFO - 1)) | (1 << (INET_DIAG_CONG - 1));
+const NO_COOKIE: u32 = u32::MAX;
+const HEADER_LEN: usize = 16;
+const REQUEST_LEN: usize = HEADER_LEN + 56;
+const DIAG_MSG_LEN: usize = 72;
+
+/// The request for the connection from `local` to `peer`, with `sequence`.
+pub fn request(local: SocketAddr, peer: SocketAddr, sequence: u32) -> Vec<u8> {
+    let family = match local {
+        SocketAddr::V4(_) => AF_INET,
+        SocketAddr::V6(_) => AF_INET6,
+    };
+    let mut buf = Vec::with_capacity(REQUEST_LEN);
+    buf.extend_from_slice(&(REQUEST_LEN as u32).to_ne_bytes());
+    buf.extend_from_slice(&SOCK_DIAG_BY_FAMILY.to_ne_bytes());
+    buf.extend_from_slice(&NLM_F_REQUEST.to_ne_bytes());
+    buf.extend_from_slice(&sequence.to_ne_bytes());
+    buf.extend_from_slice(&0u32.to_ne_bytes());
+    // inet_diag_req_v2
+    buf.extend_from_slice(&[family, IPPROTO_TCP, EXTENSIONS, 0]);
+    buf.extend_from_slice(&u32::MAX.to_ne_bytes()); // any state
+    // inet_diag_sockid: ports in network order, addresses as 16 octets.
+    buf.extend_from_slice(&local.port().to_be_bytes());
+    buf.extend_from_slice(&peer.port().to_be_bytes());
+    buf.extend_from_slice(&address_octets(local.ip()));
+    buf.extend_from_slice(&address_octets(peer.ip()));
+    // The interface, which a socket on a link-local address is bound to
+    // and is found by.
+    let interface = match local {
+        SocketAddr::V6(v6) => v6.scope_id(),
+        SocketAddr::V4(_) => 0,
+    };
+    buf.extend_from_slice(&interface.to_ne_bytes());
+    buf.extend_from_slice(&NO_COOKIE.to_ne_bytes());
+    buf.extend_from_slice(&NO_COOKIE.to_ne_bytes());
+    debug_assert_eq!(buf.len(), REQUEST_LEN);
+    buf
+}
+
+fn address_octets(address: IpAddr) -> [u8; 16] {
+    let mut octets = [0u8; 16];
+    match address {
+        IpAddr::V4(v4) => octets[..4].copy_from_slice(&v4.octets()),
+        IpAddr::V6(v6) => octets = v6.octets(),
+    }
+    octets
+}
+
+/// What the kernel answered.
+#[derive(Debug, PartialEq, Eq)]
+pub enum Answer {
+    Connection(Box<TcpConnection>),
+    /// No such connection.
+    None,
+    /// The kernel refused, with this error number.
+    Refused(i32),
+}
+
+/// Reads the kernel's answer to a [`request`].
+pub fn parse(buf: &[u8]) -> io::Result<Answer> {
+    let header = buf
+        .get(..HEADER_LEN)
+        .ok_or_else(|| io::Error::other("short netlink answer"))?;
+    let length = ne_u32(&header[0..4]) as usize;
+    let message_type = ne_u16(&header[4..6]);
+    let message = buf
+        .get(HEADER_LEN..length.min(buf.len()))
+        .ok_or_else(|| io::Error::other("short netlink answer"))?;
+    if message_type == NLMSG_ERROR {
+        let errno = -(message
+            .first_chunk::<4>()
+            .map_or(0, |e| i32::from_ne_bytes(*e)));
+        // ENOENT: no such socket.
+        return Ok(if errno == 2 {
+            Answer::None
+        } else {
+            Answer::Refused(errno)
+        });
+    }
+    if message_type != SOCK_DIAG_BY_FAMILY || message.len() < DIAG_MSG_LEN {
+        return Err(io::Error::other("not a socket diagnostic"));
+    }
+    let mut connection = TcpConnection {
+        state: message[1],
+        recv_queue: ne_u32(&message[56..60]),
+        send_queue: ne_u32(&message[60..64]),
+        ..TcpConnection::default()
+    };
+    let mut attributes = &message[DIAG_MSG_LEN..];
+    while attributes.len() >= 4 {
+        let length = usize::from(ne_u16(&attributes[0..2]));
+        let kind = ne_u16(&attributes[2..4]);
+        if length < 4 || length > attributes.len() {
+            break;
+        }
+        let value = &attributes[4..length];
+        match kind {
+            INET_DIAG_INFO => fill_from_tcp_info(&mut connection, value),
+            INET_DIAG_CONG => {
+                let name = value.split(|b| *b == 0).next().unwrap_or(&[]);
+                connection.congestion = Some(String::from_utf8_lossy(name).into_owned());
+            }
+            _ => {}
+        }
+        attributes = &attributes[(length + 3) & !3..];
+    }
+    Ok(Answer::Connection(Box::new(connection)))
+}
+
+/// Fills in what `tcp_info` (linux/tcp.h) has, as far as the kernel sent
+/// it: the structure has grown over the years, and a field that is not
+/// there is left at zero.
+fn fill_from_tcp_info(c: &mut TcpConnection, info: &[u8]) {
+    let u32_at = |at: usize| info.get(at..at + 4).map_or(0, ne_u32);
+    let u64_at = |at: usize| info.get(at..at + 8).map_or(0, ne_u64);
+    c.send_window_scale = info.get(6).map_or(0, |b| b & 0x0f);
+    c.receive_window_scale = info.get(6).map_or(0, |b| b >> 4);
+    c.rto = u32_at(8);
+    c.mss = u32_at(16);
+    c.unacked = u32_at(24);
+    c.lost = u32_at(32);
+    c.retrans = u32_at(36);
+    c.last_data_sent_ms = u32_at(44);
+    c.last_data_received_ms = u32_at(52);
+    c.last_ack_received_ms = u32_at(56);
+    c.pmtu = u32_at(60);
+    c.receive_ssthresh = u32_at(64);
+    c.rtt = u32_at(68);
+    c.rtt_variance = u32_at(72);
+    c.ssthresh = Some(u32_at(76)).filter(|t| *t < 0x7fff_ffff);
+    c.cwnd = u32_at(80);
+    c.reordering = u32_at(88);
+    c.receive_space = u32_at(96);
+    c.total_retrans = u32_at(100);
+    c.pacing_rate = u64_at(104);
+    c.bytes_acked = u64_at(120);
+    c.bytes_received = u64_at(128);
+    c.segments_out = u32_at(136);
+    c.segments_in = u32_at(140);
+    c.not_sent = u32_at(144);
+    c.min_rtt = u32_at(148);
+    c.delivery_rate = u64_at(160);
+    c.busy = u64_at(168);
+    c.limited_by_receive_window = u64_at(176);
+    c.limited_by_send_buffer = u64_at(184);
+    c.delivered = u32_at(192);
+    c.bytes_sent = u64_at(200);
+    c.bytes_retrans = u64_at(208);
+    c.send_window = u32_at(228);
+    c.receive_window = u32_at(232);
+}
+
+fn ne_u16(bytes: &[u8]) -> u16 {
+    u16::from_ne_bytes([bytes[0], bytes[1]])
+}
+
+fn ne_u32(bytes: &[u8]) -> u32 {
+    u32::from_ne_bytes([bytes[0], bytes[1], bytes[2], bytes[3]])
+}
+
+fn ne_u64(bytes: &[u8]) -> u64 {
+    let mut array = [0u8; 8];
+    array.copy_from_slice(bytes);
+    u64::from_ne_bytes(array)
+}
+
+/// Asks the kernel about the TCP connection from `local` to `peer`.
+/// `Ok(None)` if there is no such connection, or on an operating system
+/// that cannot be asked.
+#[cfg(target_os = "linux")]
+pub async fn tcp_connection(
+    local: SocketAddr,
+    peer: SocketAddr,
+) -> io::Result<Option<TcpConnection>> {
+    use netlink_sys::protocols::NETLINK_SOCK_DIAG;
+    use netlink_sys::{AsyncSocket, AsyncSocketExt, TokioSocket};
+
+    let mut socket = TokioSocket::new(NETLINK_SOCK_DIAG)?;
+    socket
+        .socket_mut()
+        .bind(&netlink_sys::SocketAddr::new(0, 0))?;
+    let sequence = 1;
+    socket.send(&request(local, peer, sequence)).await?;
+    let mut buf = bytes::BytesMut::with_capacity(4096);
+    tokio::time::timeout(std::time::Duration::from_secs(2), socket.recv(&mut buf))
+        .await
+        .map_err(|_| io::Error::new(io::ErrorKind::TimedOut, "the kernel did not answer"))??;
+    match parse(&buf)? {
+        Answer::Connection(connection) => Ok(Some(*connection)),
+        Answer::None => Ok(None),
+        Answer::Refused(errno) => Err(io::Error::from_raw_os_error(errno)),
+    }
+}
+
+#[cfg(not(target_os = "linux"))]
+pub async fn tcp_connection(
+    _local: SocketAddr,
+    _peer: SocketAddr,
+) -> io::Result<Option<TcpConnection>> {
+    Ok(None)
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    /// A reply as the kernel writes it, for an established connection
+    /// with a `tcp_info` of the full modern length.
+    fn reply() -> Vec<u8> {
+        let mut info = vec![0u8; 248];
+        info[0] = 1; // established
+        info[6] = 0x7a; // snd_wscale 10, rcv_wscale 7
+        let put32 = |info: &mut Vec<u8>, at: usize, v: u32| {
+            info[at..at + 4].copy_from_slice(&v.to_ne_bytes())
+        };
+        let put64 = |info: &mut Vec<u8>, at: usize, v: u64| {
+            info[at..at + 8].copy_from_slice(&v.to_ne_bytes())
+        };
+        put32(&mut info, 16, 1448); // mss
+        put32(&mut info, 24, 12); // unacked
+        put32(&mut info, 68, 2300); // rtt
+        put32(&mut info, 76, 0x7fff_ffff); // no ssthresh
+        put32(&mut info, 80, 42); // cwnd
+        put32(&mut info, 144, 65536); // notsent
+        put64(&mut info, 176, 900_000); // rwnd limited
+        put64(&mut info, 168, 1_000_000); // busy
+        put32(&mut info, 228, 131072); // snd_wnd
+        put32(&mut info, 232, 262144); // rcv_wnd
+        let cong = b"cubic\0";
+        let mut message = vec![0u8; DIAG_MSG_LEN];
+        message[1] = 1;
+        message[60..64].copy_from_slice(&80_000u32.to_ne_bytes()); // wqueue
+        // attributes: INFO, then CONG
+        let mut attrs = Vec::new();
+        attrs.extend_from_slice(&((4 + info.len()) as u16).to_ne_bytes());
+        attrs.extend_from_slice(&INET_DIAG_INFO.to_ne_bytes());
+        attrs.extend_from_slice(&info);
+        attrs.extend_from_slice(&((4 + cong.len()) as u16).to_ne_bytes());
+        attrs.extend_from_slice(&INET_DIAG_CONG.to_ne_bytes());
+        attrs.extend_from_slice(cong);
+        while attrs.len() % 4 != 0 {
+            attrs.push(0);
+        }
+        let length = HEADER_LEN + message.len() + attrs.len();
+        let mut buf = Vec::new();
+        buf.extend_from_slice(&(length as u32).to_ne_bytes());
+        buf.extend_from_slice(&SOCK_DIAG_BY_FAMILY.to_ne_bytes());
+        buf.extend_from_slice(&[0, 0, 1, 0, 0, 0, 0, 0, 0, 0]);
+        buf.extend_from_slice(&message);
+        buf.extend_from_slice(&attrs);
+        buf
+    }
+
+    #[test]
+    fn a_reply_is_read_field_by_field() {
+        let Answer::Connection(c) = parse(&reply()).unwrap() else {
+            panic!("a connection");
+        };
+        assert_eq!(c.state_name(), "established");
+        assert_eq!(c.congestion.as_deref(), Some("cubic"));
+        assert_eq!(c.send_queue, 80_000);
+        assert_eq!((c.send_window_scale, c.receive_window_scale), (10, 7));
+        assert_eq!((c.mss, c.unacked, c.rtt, c.cwnd), (1448, 12, 2300, 42));
+        assert_eq!(c.ssthresh, None);
+        assert_eq!(c.not_sent, 65536);
+        assert_eq!((c.busy, c.limited_by_receive_window), (1_000_000, 900_000));
+        assert_eq!((c.send_window, c.receive_window), (131072, 262144));
+    }
+
+    #[test]
+    fn no_such_connection_and_a_refusal_are_told_apart() {
+        let error = |errno: i32| {
+            let mut buf = vec![0u8; 20];
+            buf[0..4].copy_from_slice(&20u32.to_ne_bytes());
+            buf[4..6].copy_from_slice(&NLMSG_ERROR.to_ne_bytes());
+            buf[16..20].copy_from_slice(&(-errno).to_ne_bytes());
+            buf
+        };
+        assert_eq!(parse(&error(2)).unwrap(), Answer::None);
+        assert_eq!(parse(&error(13)).unwrap(), Answer::Refused(13));
+    }
+
+    #[test]
+    fn a_request_names_the_connection() {
+        let local: SocketAddr = "[2001:db8::1]:179".parse().unwrap();
+        let peer: SocketAddr = "[2001:db8::2]:40000".parse().unwrap();
+        let request = request(local, peer, 7);
+        assert_eq!(request.len(), REQUEST_LEN);
+        assert_eq!(request[16], AF_INET6);
+        assert_eq!(&request[24..26], &179u16.to_be_bytes());
+        assert_eq!(&request[26..28], &40000u16.to_be_bytes());
+        assert_eq!(
+            &request[28..44],
+            &"2001:db8::1"
+                .parse::<std::net::Ipv6Addr>()
+                .unwrap()
+                .octets()
+        );
+    }
+
+    #[cfg(target_os = "linux")]
+    #[tokio::test]
+    async fn the_kernel_tells_of_a_connection_of_ours() {
+        use tokio::io::AsyncWriteExt;
+        let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
+        let mut client = tokio::net::TcpStream::connect(listener.local_addr().unwrap())
+            .await
+            .unwrap();
+        let (_server, _) = listener.accept().await.unwrap();
+        client.write_all(&[0u8; 1000]).await.unwrap();
+        let connection = tcp_connection(client.local_addr().unwrap(), client.peer_addr().unwrap())
+            .await
+            .unwrap()
+            .expect("the kernel knows it");
+        assert_eq!(connection.state_name(), "established");
+        assert!(connection.mss > 0 && connection.cwnd > 0);
+        assert!(connection.bytes_sent >= 1000 || connection.bytes_acked >= 1000);
+        let none = tcp_connection(
+            "127.0.0.1:1".parse().unwrap(),
+            "127.0.0.1:2".parse().unwrap(),
+        )
+        .await
+        .unwrap();
+        assert!(none.is_none());
+    }
+}

crates/web/src/api.rs

@@ -1577,3 +1577,155 @@ mod tests {
         assert_eq!(other, expected);
     }
 }
+
+/// `GET /api/tcp?peer=ADDRESS:PORT`: what the kernel knows of the TCP
+/// connection of the session with that peer, as `/api/peers` names it.
+#[derive(Debug, Deserialize)]
+pub struct TcpQuery {
+    pub peer: String,
+}
+
+/// A session's TCP connection, as the kernel reports it (Linux only).
+/// Times in milliseconds, sizes in octets, rates in octets per second.
+#[derive(Debug, Serialize)]
+pub struct TcpView {
+    pub state: &'static str,
+    pub congestion: Option<String>,
+    pub rtt_ms: f64,
+    pub rtt_variance_ms: f64,
+    pub min_rtt_ms: f64,
+    pub rto_ms: f64,
+    pub mss: u32,
+    pub pmtu: u32,
+    pub cwnd_segments: u32,
+    pub ssthresh_segments: Option<u32>,
+    /// The peer's receive window: what it lets us send.
+    pub send_window: u32,
+    /// Our receive window: what we let the peer send.
+    pub receive_window: u32,
+    pub receive_space: u32,
+    pub send_window_scale: u8,
+    pub receive_window_scale: u8,
+    /// Octets written by us and not yet acknowledged, and of those the
+    /// octets not even sent yet.
+    pub send_queue: u32,
+    pub not_sent: u32,
+    pub recv_queue: u32,
+    pub unacked_segments: u32,
+    pub retrans_segments: u32,
+    pub lost_segments: u32,
+    pub total_retrans: u32,
+    pub bytes_retrans: u64,
+    pub pacing_rate: u64,
+    pub delivery_rate: u64,
+    pub bytes_sent: u64,
+    pub bytes_acked: u64,
+    pub bytes_received: u64,
+    pub segments_out: u32,
+    pub segments_in: u32,
+    /// How long the connection had something to send, and of that how
+    /// long it was held up by the peer's window and by our send buffer.
+    pub busy_ms: f64,
+    pub limited_by_receive_window_ms: f64,
+    pub limited_by_send_buffer_ms: f64,
+    /// The same two, as shares of the busy time, 0 to 1.
+    pub limited_by_receive_window_share: f64,
+    pub limited_by_send_buffer_share: f64,
+    pub last_data_sent_ms: u32,
+    pub last_data_received_ms: u32,
+    pub last_ack_received_ms: u32,
+}
+
+/// The answer to `/api/tcp`.
+#[derive(Debug, Serialize)]
+pub struct TcpAnswer {
+    /// Whether the kernel could be asked and knew the connection.
+    pub available: bool,
+    /// Why not, if not.
+    pub reason: Option<String>,
+    pub connection: Option<TcpView>,
+}
+
+pub async fn tcp(
+    State(state): State<AppState>,
+    Query(query): Query<TcpQuery>,
+) -> Result<Json<TcpAnswer>, QueryError> {
+    let session = state
+        .registry
+        .list()
+        .into_iter()
+        .find(|s| socket_text(s.peer.addr) == query.peer)
+        .ok_or_else(|| QueryError::Bad(format!("no session with {}", query.peer)))?;
+    let answer =
+        match bgp_fib::sockdiag::tcp_connection(session.local_addr, session.peer.addr).await {
+            Ok(Some(c)) => {
+                let ms = |us: u64| us as f64 / 1000.0;
+                let share = |part: u64| {
+                    if c.busy == 0 {
+                        0.0
+                    } else {
+                        part as f64 / c.busy as f64
+                    }
+                };
+                TcpAnswer {
+                    available: true,
+                    reason: None,
+                    connection: Some(TcpView {
+                        state: c.state_name(),
+                        congestion: c.congestion.clone(),
+                        rtt_ms: ms(c.rtt.into()),
+                        rtt_variance_ms: ms(c.rtt_variance.into()),
+                        min_rtt_ms: ms(c.min_rtt.into()),
+                        rto_ms: ms(c.rto.into()),
+                        mss: c.mss,
+                        pmtu: c.pmtu,
+                        cwnd_segments: c.cwnd,
+                        ssthresh_segments: c.ssthresh,
+                        send_window: c.send_window,
+                        receive_window: c.receive_window,
+                        receive_space: c.receive_space,
+                        send_window_scale: c.send_window_scale,
+                        receive_window_scale: c.receive_window_scale,
+                        send_queue: c.send_queue,
+                        not_sent: c.not_sent,
+                        recv_queue: c.recv_queue,
+                        unacked_segments: c.unacked,
+                        retrans_segments: c.retrans,
+                        lost_segments: c.lost,
+                        total_retrans: c.total_retrans,
+                        bytes_retrans: c.bytes_retrans,
+                        pacing_rate: c.pacing_rate,
+                        delivery_rate: c.delivery_rate,
+                        bytes_sent: c.bytes_sent,
+                        bytes_acked: c.bytes_acked,
+                        bytes_received: c.bytes_received,
+                        segments_out: c.segments_out,
+                        segments_in: c.segments_in,
+                        busy_ms: ms(c.busy),
+                        limited_by_receive_window_ms: ms(c.limited_by_receive_window),
+                        limited_by_send_buffer_ms: ms(c.limited_by_send_buffer),
+                        limited_by_receive_window_share: share(c.limited_by_receive_window),
+                        limited_by_send_buffer_share: share(c.limited_by_send_buffer),
+                        last_data_sent_ms: c.last_data_sent_ms,
+                        last_data_received_ms: c.last_data_received_ms,
+                        last_ack_received_ms: c.last_ack_received_ms,
+                    }),
+                }
+            }
+            Ok(None) => TcpAnswer {
+                available: false,
+                reason: Some(if cfg!(target_os = "linux") {
+                    "the kernel has no such connection".to_string()
+                } else {
+                    "only on Linux".to_string()
+                }),
+                connection: None,
+            },
+            Err(e) => TcpAnswer {
+                available: false,
+                reason: Some(format!("the kernel could not be asked: {e}")),
+                connection: None,
+            },
+        };
+    Ok(Json(answer))
+}

crates/web/src/lib.rs

@@ -144,6 +144,7 @@ pub fn router(state: AppState) -> Router {
         .route("/api/routes/rejected", get(api::rejected_routes))
         .route("/api/rpki", get(api::rpki))
         .route("/api/kernel", get(api::kernel))
+        .route("/api/tcp", get(api::tcp))
         .route("/api/events", get(events::stream))
         .route("/metricz", get(metrics::metricz))
         .with_state(state)

crates/web/static/statusz.html

@@ -68,6 +68,14 @@ table.data th { background: var(--head); font-weight: bold; position: sticky; to
 table.data > tbody > tr:hover > td { background: var(--hover); }
 table.data td.num, table.data th.num { text-align: right; font-variant-numeric: tabular-nums; white-space: nowrap; }
 table.data tr.group > td { background: var(--band); font-weight: bold; padding: 3px 6px; }
+table.data tr.tcp-row > td { background: var(--band); padding: 6px 10px 8px 28px; }
+table.data td.attrs { cursor: pointer; }
+table.data td.attrs:hover { background: var(--hover); }
+table.data tr.route-detail > td { background: var(--band); padding: 6px 10px 8px 40px; }
+.tcp-grid { display: grid; grid-template-columns: max-content 1fr; gap: 1px 16px; font-size: 12px; }
+.tcp-grid > div:nth-child(odd) { color: var(--fg-soft); white-space: nowrap; }
+a.tcp-link { color: var(--dim); font-size: 11.5px; text-decoration: none; border: 1px solid var(--line); border-radius: 3px; padding: 0 4px; margin-left: 4px; }
+a.tcp-link:hover, a.tcp-link.open { color: var(--link); border-color: var(--link); }
 table.data tr.best > td { background: var(--best-bg); }
 table.data tr.dim td { color: var(--dim); }
 .asn { cursor: help; }
@@ -241,7 +249,7 @@ from <select id="lookup-peer"><option value="">any peer</option></select>
 
   const $ = id => document.getElementById(id);
   const esc = s => String(s).replace(/[&<>"']/g, c => ({'&':'&amp;','<':'&lt;','>':'&gt;','"':'&quot;',"'":'&#39;'}[c]));
-  const state = { status: null, peers: [], local: [], received: [], receivedPage: {}, rejected: [], rpki: null, kernel: null };
+  const state = { status: null, peers: [], local: [], received: [], receivedPage: {}, rejected: [], rpki: null, kernel: null, tcpOpen: new Set(), routeOpen: new Set() };
   const DASH = '<span class="dim">–</span>';
   const fmtCount = n => Number(n).toLocaleString('en-US');
   // A glyph: a small badge with its meaning as the tooltip.
@@ -532,7 +540,7 @@ from <select id="lookup-peer"><option value="">any peer</option></select>
     const down = peers.filter(p => peerRank(p) === 0).length;
     const waiting = peers.filter(p => p.state === 'Passive').length;
     $('peers-count').textContent = `${up} of ${peers.length} established` + (down ? `, ${down} down` : '') + (waiting ? `, ${waiting} waiting` : '');
-    if (!peers.length) { setHTML(b, '<tr><td colspan="9" class="dim">No peers configured and none connected.</td></tr>'); return; }
+    if (!peers.length) { setHTML(b, '<tr><td colspan="10" class="dim">No peers configured and none connected.</td></tr>'); return; }
     const problemsOnly = $('peers-problems').checked;
     // The IPv4 and IPv6 sessions of one neighbour, named <stem>-v4 and
     // <stem>-v6, are one neighbour to the reader: listed together, IPv6
@@ -549,7 +557,7 @@ from <select id="lookup-peer"><option value="">any peer</option></select>
       const expected = up && p.expected_routes > 0
         ? ` <span class="${short ? 'amber' : 'dim'}" title="${short ? 'Well under' : 'About'} the ${fmtCount(p.expected_routes)} routes this peer is expected to send (expected_routes)">of ~${fmtCount(p.expected_routes)}</span>` : '';
       const classes = [peerRank(p) === 3 ? 'dim' : '', pair || ''].filter(Boolean).join(' ');
-      return `<tr class="${classes}" data-peer-row="${esc(p.name || p.peer_addr)}">
+      return `<tr class="${classes}" data-peer-row="${esc(p.name || p.peer_addr)}" data-peer-addr="${esc(p.peer_addr)}">
         <td class="name">${sinceGlyph(p)}${name}</td>
         <td class="mono">${esc(p.peer_addr)} <span class="dim">AS${asn(p.peer_asn)}</span> ${roleGlyph(p)}</td>
         <td><span class="${STATE_CLASS[p.state] || ''}">${esc(p.state)}</span></td>
@@ -559,7 +567,7 @@ from <select id="lookup-peer"><option value="">any peer</option></select>
         <td class="num">${up ? trendGlyph(p) + fmtCount(p.routes) + expected + stale : DASH}</td>
         <td class="num">${up ? fmtCount(p.advertised_routes) : DASH}</td>
         <td class="num">${rejected}${held}</td>
-        <td>${peerDetail(p)}</td></tr>`;
+        <td>${peerDetail(p)}${up ? tcpLink(p) : ''}</td></tr>`;
     };
     // Rows for a sorted list, with the v4/v6 pairs in it marked.
     const rows = list => list.map((p, i) => {
@@ -575,23 +583,93 @@ from <select id="lookup-peer"><option value="">any peer</option></select>
     // they are to us.
     const problems = peers.filter(p => peerRank(p) <= 1).sort(byName);
     if (problems.length) {
-      html += `<tr class="group"><td colspan="9"><span class="red">Needing attention</span> (${problems.length})</td></tr>` + rows(problems);
+      html += `<tr class="group"><td colspan="10"><span class="red">Needing attention</span> (${problems.length})</td></tr>` + rows(problems);
     }
     if (!problemsOnly) {
       for (const group of ROLE_ORDER) {
         const members = peers.filter(p => peerRank(p) >= 2 && peerGroup(p) === group).sort(byName);
         if (!members.length) continue;
-        html += `<tr class="group"><td colspan="9">${GROUP_TITLE[group]} (${members.length})</td></tr>` + rows(members);
+        html += `<tr class="group"><td colspan="10">${GROUP_TITLE[group]} (${members.length})</td></tr>` + rows(members);
       }
     } else if (!problems.length) {
-      html += '<tr><td colspan="9" class="dim">Nothing needs attention.</td></tr>';
+      html += '<tr><td colspan="10" class="dim">Nothing needs attention.</td></tr>';
     }
     setHTML(b, html);
+    renderTcpRows();
     const marked = peers.filter(changedSince).length;
     setHTML($('since-ctl'), marked ? `${glyph('●', 'Rows that changed while this tab was in the background', 'since')} <span class="dim">${marked} changed since you last looked ·</span> <a href="#" id="since-clear">clear</a>` : '');
     renderPeerChoices();
   }
 
+  // The TCP connection of a session, as the kernel sees it: a link on the
+  // row, and a row below it while it is open, asked for afresh on every
+  // refresh.
+  function tcpLink(p) {
+    const open = state.tcpOpen.has(p.peer_addr);
+    return `<a href="#" class="tcp-link${open ? ' open' : ''}" data-peer="${esc(p.peer_addr)}" title="TCP_INFO for this session's socket">tcp</a>`;
+  }
+  const fmtOctets = n => n >= 1048576 ? `${(n / 1048576).toFixed(1)} MiB` : n >= 1024 ? `${(n / 1024).toFixed(1)} KiB` : `${n} B`;
+  const fmtBits = n => { const b = n * 8; return b >= 1e9 ? `${(b / 1e9).toFixed(2)} Gbit/s` : b >= 1e6 ? `${(b / 1e6).toFixed(1)} Mbit/s` : `${(b / 1e3).toFixed(0)} kbit/s`; };
+  const fmtMs = n => n >= 1000 ? `${(n / 1000).toFixed(1)} s` : `${n.toFixed(n < 10 ? 2 : 0)} ms`;
+  const ago = ms => ms >= 60000 ? `${Math.round(ms / 60000)}m` : ms >= 1000 ? `${(ms / 1000).toFixed(1)}s` : `${ms}ms`;
+  function tcpDetail(a) {
+    if (!a.available) return `<span class="dim">${esc(a.reason || 'n/a')}</span>`;
+    const c = a.connection;
+    const pct = share => `${(share * 100).toFixed(0)}%`;
+    const held = share => share > 0.5 ? 'red' : share > 0.1 ? 'amber' : '';
+    const kv = (k, v) => `<span class="dim">${k}</span> ${v}`;
+    const rows = [
+      ['state', `${esc(c.state.toUpperCase())} ${c.congestion ? esc(c.congestion) : ''}`],
+      ['rtt', `${fmtMs(c.rtt_ms)} ${kv('minrtt', fmtMs(c.min_rtt_ms))} ${kv('rttvar', fmtMs(c.rtt_variance_ms))} ${kv('rto', fmtMs(c.rto_ms))}`],
+      ['cwnd', `${fmtCount(c.cwnd_segments)} ${kv('mss', `${c.mss}`)} ${kv('=', fmtOctets(c.cwnd_segments * c.mss))} ${kv('ssthresh', c.ssthresh_segments == null ? '∞' : fmtCount(c.ssthresh_segments))} ${kv('pmtu', `${c.pmtu}`)}`],
+      ['snd_wnd', `${fmtOctets(c.send_window)} ${kv('wscale', `${c.send_window_scale}`)}`],
+      ['rcv_wnd', `${fmtOctets(c.receive_window)} ${kv('rcv_space', fmtOctets(c.receive_space))} ${kv('wscale', `${c.receive_window_scale}`)}`],
+      ['send-q', `${fmtOctets(c.send_queue)} ${kv('notsent', fmtOctets(c.not_sent))} ${kv('unacked', fmtCount(c.unacked_segments))}`],
+      ['recv-q', fmtOctets(c.recv_queue)],
+      ['retrans', `${fmtCount(c.retrans_segments)} ${kv('lost', fmtCount(c.lost_segments))} ${kv('total', `${fmtCount(c.total_retrans)} / ${fmtOctets(c.bytes_retrans)}`)}`],
+      ['rate', `${kv('delivery', fmtBits(c.delivery_rate))} ${kv('pacing', fmtBits(c.pacing_rate))}`],
+      ['limited', `${kv('busy', fmtMs(c.busy_ms))} <span class="${held(c.limited_by_receive_window_share)}">${kv('rwnd', pct(c.limited_by_receive_window_share))}</span> <span class="${held(c.limited_by_send_buffer_share)}">${kv('sndbuf', pct(c.limited_by_send_buffer_share))}</span>`],
+      ['bytes', `${kv('sent', fmtOctets(c.bytes_sent))} ${kv('acked', fmtOctets(c.bytes_acked))} ${kv('received', fmtOctets(c.bytes_received))} ${kv('segs', `${fmtCount(c.segments_out)}↑ ${fmtCount(c.segments_in)}↓`)}`],
+      ['last', `${kv('snd', ago(c.last_data_sent_ms))} ${kv('rcv', ago(c.last_data_received_ms))} ${kv('ack', ago(c.last_ack_received_ms))}`],
+    ];
+    return `<div class="tcp-grid mono">${rows.map(([k, v]) => `<div>${k}</div><div>${v}</div>`).join('')}</div>`;
+  }
+  async function renderTcpRows() {
+    for (const addr of state.tcpOpen) {
+      const tr = document.querySelector(`#peers-body tr[data-peer-addr="${CSS.escape(addr)}"]`);
+      if (!tr) continue;
+      let row = tr.nextElementSibling;
+      if (!row || !row.classList.contains('tcp-row')) {
+        row = document.createElement('tr');
+        row.className = 'tcp-row';
+        row.innerHTML = '<td colspan="10"><span class="dim">asking the kernel…</span></td>';
+        tr.after(row);
+      }
+      try {
+        const answer = await getJSON(`/api/tcp?peer=${encodeURIComponent(addr)}`);
+        row.innerHTML = `<td colspan="10">${tcpDetail(answer)}</td>`;
+      } catch (e) {
+        row.innerHTML = `<td colspan="10"><span class="red">${esc(String(e))}</span></td>`;
+      }
+    }
+  }
+  document.addEventListener('click', e => {
+    const link = e.target.closest('a.tcp-link');
+    if (!link) return;
+    e.preventDefault();
+    const addr = link.dataset.peer;
+    if (state.tcpOpen.has(addr)) {
+      state.tcpOpen.delete(addr);
+      link.classList.remove('open');
+      const tr = link.closest('tr'), next = tr && tr.nextElementSibling;
+      if (next && next.classList.contains('tcp-row')) next.remove();
+    } else {
+      state.tcpOpen.add(addr);
+      link.classList.add('open');
+      renderTcpRows();
+    }
+  });
+
   // The strip: what needs attention, in one line, or that nothing does.
   function renderStrip() {
     const items = [];
@@ -748,18 +826,70 @@ from <select id="lookup-peer"><option value="">any peer</option></select>
     if (!state.received.length) {
       setHTML(b, `<tr><td colspan="12" class="dim">${looking ? 'No route matches.' : 'No routes received from peers.'}</td></tr>`); return;
     }
-    setHTML(b, state.received.map(r => { const s = r.summary; return `<tr class="${r.best ? 'best' : ''}">
+    setHTML(b, state.received.map(r => { const s = r.summary; return `<tr class="${r.best ? 'best' : ''}" data-route="${esc(routeKey(r))}">
       <td class="nowrap">${rankGlyph(r)}${r.stale ? glyph('stale', 'Kept from a session of the peer’s that ended (graceful restart), and not sent again since', 'amber') : ''}</td>
       <td class="mono">${esc(r.prefix)}</td>
       <td class="mono">${r.peer_name ? `<a href="#received" class="peer-link" data-peer="${esc(r.peer_name)}">${esc(r.peer_name)}</a> ` : ''}<span class="dim">${esc(r.peer_addr)} AS${asn(r.peer_asn)}${r.path_id ? ` path ${r.path_id}` : ''}</span></td>
       <td class="mono path">${clamp(asPath(s.as_path))}</td>
       <td class="mono">${s.next_hop ? esc(s.next_hop) : DASH}</td><td>${s.origin ? esc(s.origin) : DASH}</td>
       <td class="num">${s.med ?? DASH}</td><td class="num">${s.local_pref ?? DASH}</td>
-      <td>${clamp(commList(s.communities))}</td><td>${clamp(commList(s.large_communities, true))}</td><td>${otherList(s.other)}</td>
+      <td>${clamp(commList(s.communities))}</td><td>${clamp(commList(s.large_communities, true))}</td><td class="attrs" title="all attributes">${otherList(s.other)}</td>
       <td class="nowrap">${validity('RPKI', r.rpki)} ${validity('ASPA', r.aspa)}</td></tr>`; }).join(''));
     markTall(b);
+    renderRouteDetails();
   }
 
+  // A route, in full, under its row: every attribute by its name, as the
+  // RFCs write it, opened by a click on the attrs cell.
+  const routeKey = r => `${r.prefix}|${r.peer_addr}|${r.path_id}`;
+  const ATTR_NAMES = {
+    'atomic aggregate': 'ATOMIC_AGGREGATE', 'aggregator': 'AGGREGATOR', 'extended communities': 'EXTENDED_COMMUNITIES',
+    'only to customer': 'OTC', 'originator ID': 'ORIGINATOR_ID', 'cluster list': 'CLUSTER_LIST',
+  };
+  function routeDetail(r) {
+    const s = r.summary;
+    const rows = [
+      ['ORIGIN', s.origin ? esc(s.origin) : DASH],
+      ['AS_PATH', s.as_path.length ? s.as_path.map(asn).join(' ') : '<span class="dim">(empty)</span>'],
+      ['NEXT_HOP', s.next_hop ? esc(s.next_hop) : DASH],
+      ['MED', s.med ?? DASH],
+      ['LOCAL_PREF', s.local_pref ?? DASH],
+      ['COMMUNITIES', s.communities.length ? s.communities.map(esc).join(' ') : DASH],
+      ['LARGE_COMMUNITIES', s.large_communities.length ? s.large_communities.map(esc).join(' ') : DASH],
+      ...(s.other || []).map(a => [esc(ATTR_NAMES[a.name] || a.name), esc(a.value) || 'present']),
+    ];
+    if (r.path_id) rows.push(['PATH_ID', `${r.path_id}`]);
+    rows.push(['selection', `${r.best ? 'best' : `rank ${r.rank}`}${r.stale ? ', stale' : ''}`]);
+    rows.push(['rpki', r.rpki ? esc(r.rpki) : DASH], ['aspa', r.aspa ? esc(r.aspa) : DASH]);
+    return `<div class="tcp-grid mono">${rows.map(([k, v]) => `<div>${k}</div><div>${v}</div>`).join('')}</div>`;
+  }
+  function renderRouteDetails() {
+    for (const key of state.routeOpen) {
+      const r = state.received.find(r => routeKey(r) === key);
+      const tr = document.querySelector(`#received-body tr[data-route="${CSS.escape(key)}"]`);
+      if (!r || !tr) continue;
+      const next = tr.nextElementSibling;
+      if (next && next.classList.contains('route-detail')) continue;
+      const row = document.createElement('tr');
+      row.className = 'route-detail';
+      row.innerHTML = `<td colspan="12">${routeDetail(r)}</td>`;
+      tr.after(row);
+    }
+  }
+  document.addEventListener('click', e => {
+    const cell = e.target.closest('#received-body td.attrs');
+    if (!cell || e.target.closest('a')) return;
+    const tr = cell.closest('tr'), key = tr.dataset.route;
+    if (state.routeOpen.has(key)) {
+      state.routeOpen.delete(key);
+      const next = tr.nextElementSibling;
+      if (next && next.classList.contains('route-detail')) next.remove();
+    } else {
+      state.routeOpen.add(key);
+      renderRouteDetails();
+    }
+  });
+
   const RPKI_STATE_CLASS = { synchronised: 'green', synchronising: 'amber', connecting: 'red' };
   function renderRpki() {
     const r = state.rpki, b = $('rpki-body');