Rayhaan JaufeerallyandClaude Fable 5.1
41090966e4
The RIB handles swallowed every failure of their actor: an UPDATE applied to a stopped Adj-RIB-In vanished, and a snapshot of it came back empty. Sessions stayed up, peers' routes went nowhere, and the status page showed a healthy router with no routes. An actor stops only by panicking, so this is the state the server would be in after a bug. - Every operation on AdjRibIn and LocalRib returns Result<_, RibStopped>. LocalRibError is replaced by it. - The server waits on both actors, the advertiser and the dialers. If one stops it sends every peer Cease / Administrative Shutdown and returns an error, so the process exits and can be restarted. Before, the tasks' handles were held and never looked at. - A session that cannot store an UPDATE ends with the same Cease. - The HTTP API answers 503 with the reason for anything that needs a table it cannot read, including /metricz: a scrape that fails is noticed, last known sizes are not. Also: - The routes of a session are now removed even if its task is torn down without running its cleanup (a panic), instead of staying in the table as routes of a peer that is gone. - Table sizes for the status page and metrics are asked for as a count instead of copying the whole table to take its length. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
108 lines
3.6 KiB
Rust
108 lines
3.6 KiB
Rust
//! What the server does when a part of it that it cannot do without fails.
|
|
//!
|
|
//! A RIB's actor stops only by panicking. Before, nothing noticed: sessions
|
|
//! stayed up, routes sent by peers were dropped on the floor, and the status
|
|
//! page showed an empty table. Now the server tells its peers it is going
|
|
//! and stops with an error, for whatever supervises it to restart it.
|
|
|
|
use std::net::Ipv4Addr;
|
|
use std::time::Duration;
|
|
|
|
use bgp_integration_tests::test_peer::TestPeer;
|
|
use bgp_integration_tests::test_server::TestServer;
|
|
use bgp_packet::errors::{BgpError, CeaseSubcode};
|
|
use bgp_packet::message::Message;
|
|
use bgp_rib::RibStopped;
|
|
use bgp_server::error::ServerError;
|
|
use eyre::{Result, bail};
|
|
|
|
const LOCAL_ASN: u32 = 65000;
|
|
const PEER_ASN: u32 = 65001;
|
|
const TIMEOUT: Duration = Duration::from_secs(5);
|
|
|
|
async fn established(server: &TestServer) -> Result<TestPeer> {
|
|
let mut peer = TestPeer::connect(server.addr).await?;
|
|
peer.handshake(PEER_ASN, Ipv4Addr::new(10, 0, 0, 1), 90)
|
|
.await?;
|
|
tokio::time::timeout(TIMEOUT, async {
|
|
while server.registry.list().is_empty() {
|
|
tokio::time::sleep(Duration::from_millis(10)).await;
|
|
}
|
|
})
|
|
.await?;
|
|
Ok(peer)
|
|
}
|
|
|
|
/// Reads until the server's NOTIFICATION, skipping KEEPALIVEs.
|
|
async fn farewell(peer: &mut TestPeer) -> Result<BgpError> {
|
|
tokio::time::timeout(TIMEOUT, async {
|
|
loop {
|
|
match peer.recv().await? {
|
|
Message::KeepAlive => {}
|
|
Message::Notification(n) => return Ok(n.error),
|
|
other => bail!("expected NOTIFICATION, got {other:?}"),
|
|
}
|
|
}
|
|
})
|
|
.await?
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_losing_the_adj_rib_in_stops_the_server_and_tells_the_peers() -> Result<()> {
|
|
let mut server = TestServer::start(LOCAL_ASN).await?;
|
|
let (base, _http) = server.serve_http().await?;
|
|
let mut peer = established(&server).await?;
|
|
|
|
server.adj_rib_in.stop_for_testing();
|
|
|
|
// Nothing has used the table since; the server notices by itself.
|
|
let outcome = tokio::time::timeout(TIMEOUT, server.stopped()).await?;
|
|
assert!(
|
|
matches!(outcome, Err(ServerError::RibStopped(RibStopped { rib })) if rib == "Adj-RIB-In"),
|
|
"{outcome:?}"
|
|
);
|
|
assert_eq!(
|
|
farewell(&mut peer).await?,
|
|
BgpError::Cease(CeaseSubcode::AdminShutdown)
|
|
);
|
|
|
|
// The status interface says the table cannot be read, where it used to
|
|
// show an empty one.
|
|
let client = reqwest::Client::new();
|
|
for path in ["/api/status", "/api/routes/received", "/metricz"] {
|
|
let response = client.get(format!("{base}{path}")).send().await?;
|
|
assert_eq!(response.status(), 503, "{path}");
|
|
assert_eq!(
|
|
response.text().await?,
|
|
"the Adj-RIB-In has stopped",
|
|
"{path}"
|
|
);
|
|
}
|
|
// What does not need the table still answers.
|
|
let response = client
|
|
.get(format!("{base}/api/routes/local"))
|
|
.send()
|
|
.await?;
|
|
assert_eq!(response.status(), 200);
|
|
Ok(())
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn test_losing_the_local_rib_stops_the_server_and_tells_the_peers() -> Result<()> {
|
|
let mut server = TestServer::start(LOCAL_ASN).await?;
|
|
let mut peer = established(&server).await?;
|
|
|
|
server.local_rib.stop_for_testing();
|
|
|
|
let outcome = tokio::time::timeout(TIMEOUT, server.stopped()).await?;
|
|
assert!(
|
|
matches!(outcome, Err(ServerError::RibStopped(RibStopped { rib })) if rib == "local RIB"),
|
|
"{outcome:?}"
|
|
);
|
|
assert_eq!(
|
|
farewell(&mut peer).await?,
|
|
BgpError::Cease(CeaseSubcode::AdminShutdown)
|
|
);
|
|
Ok(())
|
|
}
|