A review of the session driver, the server loop, the dialer, the
advertiser and the HTTP API, with the fixes.
Bugs:
- A peer that stopped reading held its session, and our routes, for as
long as TCP kept retrying: a blocked write was awaited with no timer
running. Writes now give up after the Send Hold Time of RFC 9687.
- Routes we originate were sent to a peer in our own AS with our AS
number on the path and no LOCAL_PREF, which such a peer refuses. They
now go with an empty path and a LOCAL_PREF.
- Collision detection knew a peer by its BGP Identifier alone. A
neighbour with one session over IPv4 and one over IPv6 had the second
closed as a collision, and any host that could connect could displace
a peer's connection by claiming its identifier. A peer is now its
address and identifier together.
- The dialer reset its backoff whenever TCP connected, so a peer that
accepts the connection and refuses the session was dialled every few
seconds for ever. It resets only after a session was established.
- A failure on our side (routes that cannot be stored) dropped the
connection without a NOTIFICATION. It goes through the state machine
and the peer is sent Cease. A queued message that cannot be encoded
is logged and skipped instead of ending the session.
- The accept loop retried at once after an error, spinning when the
process was out of file descriptors.
- Attribute sets no route used any more were dropped only when a peer
went away. A task now does it every minute.
- The advertiser subscribed to session events inside its task, after the
dialers were started, and could miss the first session.
- Shutdown waited without limit on a session whose queue was full.
- /api/routes/received took any limit and built the reply on a runtime
thread. The limit is capped and the work is done off those threads.
- The default log level was TRACE, which logs every UPDATE.
- IPv6 listeners are IPv6-only: listening on 0.0.0.0 and [::] fails on
systems where the second would take IPv4 too, and IPv4 peers arriving
under mapped addresses matched no configured peer.
- A connection closed after a NOTIFICATION is kept for a moment so that
the peer can read it.
- The configuration accepted two peers with one name or one address,
and a router id of 0.0.0.0.
- An HTTP address that cannot be bound now stops the daemon at startup.
Simplifications:
- SessionConfig mixed what a server's sessions share with what is
configured for one peer, and was cloned and patched field by field in
two places. It is ServerContext, shared, and PeerSettings, made from a
PeerConfig in one place.
- The attributes of an UPDATE as they were on the wire travelled from
the reader to the table through a field of the session, around the
state machine. They go through it, with the message.
- run_session returns how the session ended and whether it had been
established; Ok no longer reads as "closed by peer".
- Session is not generic over its reader.
- BgpServerBuilder had nothing to build: BgpServer::bind takes the
configuration.
- One shutdown path, close_sessions, for the signal handler and for a
failed server.
- The advertiser's three copies of "send every route" are one.
- PeerId moved from the rib crate, which did not use it, to the server.
- Unused API removed or made private to the crate.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A review of the wire codec and the routing table, with the fixes.
Bugs in the codec:
- A NOTIFICATION with a code or subcode we have no name for (subcode 0,
"unspecific", is common) failed to decode and was answered as a bad
message. Every pair now reads, as BgpError::Other.
- Optional parameters in the extended form of RFC 9072 were read and
written with one-octet parameter lengths; they are two.
- An ORF capability listing more than one address family made the whole
OPEN malformed.
- Bytes after an OPEN's optional parameters were accepted.
- Extended communities were split into fields by rules that were wrong
for the common types. They are kept as their eight octets.
- Several length fields were computed in u16 and could wrap on encode.
Simplifications in the codec:
- Lengths are filled in after the value is written, for messages,
attributes, capabilities and the UPDATE's two fields. The parallel
wire_len functions, which had to be kept in step by hand, are gone.
Message::to_wire no longer encodes into a second buffer.
- Type codes for messages, attributes and capabilities are each defined
once. The strum discriminant enums and their hand-written conversions
are removed.
- One function splits an attribute off a buffer; the UPDATE decoder, the
plain attribute parser and the table's key reader all use it.
- The UPDATE decoder's 185-line loop is an AttributeReader with one
method that decides what becomes of an attribute. Duplicate detection
is a bitset, not a HashSet allocated per UPDATE.
- ParserContext::four_octet_asn is a bool. The "context not set" error
path, which treated a programming error as a malformed route, is gone.
- Codec functions take a context only if they read it, and return a
Result only if they can fail.
- MP_REACH_NLRI and MP_UNREACH_NLRI hold an AfiSafi.
- AddressFamilyId is IpVersion, which is what it was.
- Deserialize is no longer derived on codec types: nothing used it, and
it bypassed IpPrefix's invariants. IpPrefix::to_wire, which was
broken and unused, is removed.
Bugs in the table:
- The index of a row's best route was sixteen bits. A peer using
ADD-PATH could send more routes for a prefix than that counts, and
the index wrapped. It is 32 bits, at no cost in row size.
- Shards and attribute-store stripes were picked from the top bits of
the hash, which are the bits hashbrown keeps as the tag it compares
first. With 64 shards each shard's table had 2 of 128 tag values.
They are now picked from bits the table does not use.
- remove_peer held the peer list's lock for the whole scan of the
table, so no session could establish while a peer was removed.
- When more prefixes went at once than were left, the forced trim
dropped a fixed few entries, reset every reader, and re-ran the tidy
on every later write. It drops the older half.
- Readers were woken by every change, including a repeated announcement
and changes their journal does not record.
- Reader::poll with a budget of zero never made progress.
Simplifications in the table:
- The two journals of a shard are one Journals value written through
three calls; the five places that kept them in step by hand are gone.
Whether an entry is still live is decided in one place.
- The shortcuts of route selection moved to select.rs, next to the
invariant they depend on, which is now written down.
- announce and withdraw take routes with or without path identifiers;
the _paths variants are removed.
- The selection key is read off the wire form only; the second
implementation over parsed attributes is removed.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A peer can now send several routes for one prefix, told apart by a path
identifier, and each is a route of its own in the table.
Wire:
- The ADD-PATH capability lists several address families. It was read as
exactly one, so the OPEN of a router offering it for IPv4 and IPv6 was
malformed and the session never came up.
- Prefixes in UPDATEs are `Nlri`: a prefix and a path identifier. The
identifier is on the wire, in the body and in MP_REACH/MP_UNREACH, for
the families the session uses ADD-PATH for in that direction. Sending
and receiving are settled separately, so a session has one codec context
for each.
Negotiation:
- `add_path` on a peer in the configuration: off (the default), receive,
send or both, offered for every family offered to the peer.
- In use for a family and a direction when one side offered to send and
the other to receive. Shown on /statusz and in /api/peers.
Table:
- A route is known by its peer and path identifier together. Identifiers
are stored only in rows that have a non-zero one, so routes of peers
that send one route each cost nothing more.
- Equal routes from one peer are chosen between by the lower identifier.
- /api/routes/received gives each route's path_id.
Journal:
- A second journal in each shard records every change to any route, for
followers that want all the routes of a prefix and not only the best:
RibTable::reader_of_all_routes and Event::Routes. This is what sending
several routes to a peer will read. It costs nothing while nobody
follows it.
Sending: prefixes go out with a path identifier where that was agreed, but
there is still one route for each prefix to send: only originated routes
are announced.
Tested with the table and journal model tests extended to path identifiers
and to followers of all routes, the codec property tests and fuzz targets
extended to ADD-PATH sessions, and end to end in both directions and for
the case where only one side offers it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The server now keeps what it learns in table::RibTable. The Adj-RIB-In
actor is gone, and with it the queue every UPDATE from every session went
through.
- A session takes a number in the table when it is established and writes
its UPDATEs into the table itself. Its routes are removed when it ends,
on a blocking thread since that looks at every prefix; a session torn
down without cleaning up still has them removed.
- For every prefix the best route is chosen (RFC 4271 §9.1.2.2) as routes
come and go. Nothing acts on the choice yet beyond showing it.
- The UPDATE decoder hands over the bytes of the attributes it kept, so the
table finds the attribute set from them without writing the attributes
out again. On a session with two-octet AS numbers the bytes are not in
the table's form, and the parsed attributes are used.
- /api/routes/received marks the best route of each prefix and returns at
most `limit` routes (default 1000): the table can hold hundreds of
millions. /api/status has the totals, and the status page says how many
it is showing. The time a route was received is no longer kept or shown.
- The event stream reports changes of best route (best_changed,
best_withdrawn) in place of every route received. Each client follows
the table's journal with a reader of its own, from the moment it
connects: table::RibTable::reader_from_now.
- Metrics: bgp_rib_routes, bgp_rib_prefixes, bgp_rib_attribute_sets and
bgp_rib_journal_entries replace bgp_adj_rib_in_routes.
- A line is no longer logged at INFO for every UPDATE sent or received; a
full table is a million of them. They are at DEBUG.
- A table with no room for another peer ends the session with Cease / Out
of Resources.
Four peers sending a synthetic full table each over TCP, 4.9 million
routes: 0.80 s, against 4.76 s before. One peer: 0.45 s against 0.69 s.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Whatever acts on the best routes (export to peers, the forwarding table,
the status page) reads their changes from a per-shard journal through a
table::Reader. Nothing uses it yet.
- The journal records that a prefix's best route changed, not what to. A
reader that comes to an entry is given the route as it is then. A prefix
that changes a thousand times between reads is one event, and the
journal never holds more live entries than the table has prefixes.
- Each row keeps the sequence number of its latest change; an entry with
another number is dead and is dropped when the journal is tidied.
- Entries go once every reader has passed them. A reader that does not
read cannot hold them back for ever: past a limit the oldest half is
dropped, and that reader is given the whole table again (Reset, every
best route, Synced). A new reader starts the same way.
- With no reader nothing is recorded.
- Reader::changed waits for there to be something to read.
Tested against a mirror kept from the events: after any interleaving of
announcements, withdrawals, session flaps, reads with small budgets and
readers coming and going, with journals small enough to be tidied and
overrun constantly, a reader that has caught up holds exactly the table's
best routes. Also with writers on four threads and a reader on a fifth.
The property test found one fault (a pass that resumed part-way claimed
to have caught up) and catches three more put in on purpose.
On the AMS-IX dump, 12 threads: loading takes 2.29 s with a follower
reading throughout against 2.17 s with none, the journal peaking at
170,000 entries for 1.39 million prefixes. With a follower that does not
read, the journal ends at 1.74 million entries, and the follower then
catches up in 0.08 s.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
bgp_rib::table::RibTable holds every route from every peer and the best
one for each prefix. It is not used by the server yet; this adds it with
its tests and benchmarks.
- Organised by prefix. A row holds the prefix's routes from all peers in
one array; choosing again after a change reads that row and nothing else.
- A route is sixteen bytes: a pointer to its attribute set and one word
holding the peer and the first three steps of selection (LOCAL_PREF, AS
path length, ORIGIN) as a single number. That number alone settles most
changes; a tie goes on to MED, compared only within a neighbouring AS
and in a way that does not depend on arrival order, and the tie-breaks.
- Attribute sets are stored once, in wire form, with what selection needs
from them worked out when they are stored.
- Sharded by hash of the prefix, a lock per shard. A session applies its
own UPDATEs; there is no queue or owning thread in front of the table.
- Readers take one shard at a time for reading and look at rows in place.
Route selection is checked against a plain implementation with no
shortcuts by a property test over random announcements, withdrawals and
session flaps, for both the chosen route and the count of changes.
On a RIPE RIS dump from AMS-IX (148 peers, 39.3 million routes), against
the Adj-RIB-In actor, which does no selection:
actor table
memory 11.0 GiB 1.8 GiB (279 -> 48 bytes a route)
load, 1 thread 10.2 s 12.3 s
load, 12 threads (no gain) 2.1 s (19 million routes a second)
read every route 1.6 s, copying 8 GiB, table stopped
0.2 s, in place, table running
The benchmark crate counts memory only with --features count-memory: the
counters are shared between threads and distort the timings.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
crates/bench reads TABLE_DUMP_V2 dumps (RFC 6396) as published by RIPE RIS
and RouteViews, and `--bench real_table` runs the Adj-RIB-In on one. The
dumps are not checked in; crates/bench/data/README.md says how to fetch
one. Without a dump the benchmark says so and does nothing.
It also prints what in the table decides the cost of a RIB: routes per
prefix, distinct attribute sets, how many routes share a set, and how much
the sets are shared between peers.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A prefix was a Vec of its bytes: a heap allocation for every prefix parsed,
stored or copied, and a variable-length key to hash. It is now twenty
bytes, Copy, with private fields and one representation; `bytes()` gives
the wire form.
On the full-table benchmark this alone takes decoding from 9.3 to 10.7
million routes a second and inserting into the Adj-RIB-In from 2.9 to 3.6.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
crates/bench makes a synthetic full table (a million IPv4 and 230,000 IPv6
prefixes, announced by several peers in UPDATEs shaped like a router's)
and times the steps of taking it in: decoding, the Adj-RIB-In, snapshot,
peer removal, and the whole path over TCP. It also counts the memory the
table holds, and fills a plain packed hash map with the same routes as a
point of comparison.
Run with `cargo bench -p bgp-bench`.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A panic is a bug, and it should cost as little as it can. The server
already closed every session with a Cease and exited when a RIB or a
background task was lost. This fills in the rest:
- A panic inside a session is caught. The peer is sent Cease /
Administrative Reset, the session's routes are removed in the usual
order, and the dialer reconnects. Before, the peer was left with a TCP
connection closing under it and no reason.
- The tables every session shares (the session registry and the collision
table) no longer refuse to be locked after a panic. Lock poisoning would
have made each remaining session panic in turn the next time it touched
one, so one bug in one session took all of them down.
- The server's watcher reports a task that panicked with the panic's
message, and is now tested for both a panicking task and a stopped RIB.
- The daemon sends panics to its log, with their location, instead of
leaving them to the default report on stderr.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The RIB handles swallowed every failure of their actor: an UPDATE applied
to a stopped Adj-RIB-In vanished, and a snapshot of it came back empty.
Sessions stayed up, peers' routes went nowhere, and the status page showed
a healthy router with no routes. An actor stops only by panicking, so this
is the state the server would be in after a bug.
- Every operation on AdjRibIn and LocalRib returns Result<_, RibStopped>.
LocalRibError is replaced by it.
- The server waits on both actors, the advertiser and the dialers. If one
stops it sends every peer Cease / Administrative Shutdown and returns an
error, so the process exits and can be restarted. Before, the tasks'
handles were held and never looked at.
- A session that cannot store an UPDATE ends with the same Cease.
- The HTTP API answers 503 with the reason for anything that needs a table
it cannot read, including /metricz: a scrape that fails is noticed, last
known sizes are not.
Also:
- The routes of a session are now removed even if its task is torn down
without running its cleanup (a panic), instead of staying in the table
as routes of a peer that is gone.
- Table sizes for the status page and metrics are asked for as a count
instead of copying the whole table to take its length.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Any malformed UPDATE used to reset the session, withdrawing every route
learned over it. One bad attribute, typically originated ASes away and
passed on by a peer that did not look at it, could take down a session
carrying a full table.
UpdateMessage::decode now applies RFC 7606 and returns the UPDATE as it is
to be acted on, plus a report of what was wrong:
- Treat-as-withdraw: a malformed or misflagged attribute, a missing
mandatory one, or an attribute that overruns the attribute list. The
routes the UPDATE announced (in the body and in MP_REACH_NLRI) are
withdrawn.
- Attribute discard: malformed ATOMIC_AGGREGATE, AGGREGATOR, AS4_PATH and
AS4_AGGREGATOR; an attribute sent twice (the first is kept); LOCAL_PREF,
ORIGINATOR_ID and CLUSTER_LIST from an external peer; AS4_PATH between
four-octet speakers.
- Session reset remains where the prefixes are in doubt: the lengths that
frame the message, an unreadable prefix, a malformed or repeated
MP_REACH_NLRI or MP_UNREACH_NLRI, an unrecognised well-known attribute.
Each now sends its own subcode instead of always Malformed Attribute
List.
The session logs each handled error and counts it in
bgp_update_errors_total{handling}. The parser context says whether the
peer is internal (replacing a field nothing read).
Stricter than before, because the checks did not exist:
- An announcement must carry ORIGIN and AS_PATH, NEXT_HOP for routes in the
body, and LOCAL_PREF from an internal peer.
- An AS_PATH segment must be an AS_SET or AS_SEQUENCE and not empty.
- An attribute value must be used up entirely by its parser; a NEXT_HOP of
six octets, say, was accepted before.
Bugs found on the way: we sent ATOMIC_AGGREGATE flagged optional and
AGGREGATOR flagged non-transitive. The flags now come from one table used
for both writing and checking.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Property tests (run by `cargo test`):
- crates/packet/tests/properties.rs: every kind of message, generated,
must be written, be as long as wire_len says, and read back equal; and
noise, noise behind a valid header, and valid messages with bytes damaged
must never do more than return an error. Anything the decoder accepts
must be showable and survive being written and read again.
- IpPrefix: text round trip, noise as text, `contains` against a bitwise
model.
- The session's framing: a stream of messages yields the same messages
however it is cut into reads.
Fuzz targets (fuzz/, cargo-fuzz on nightly, a minute each in CI): raw
messages, message bodies behind a valid header, prefix text, the
configuration file.
Bugs found, each with a named regression test:
- A prefix longer than its address family (e.g. an IPv4 NLRI of length 99)
was accepted, and showing it panicked. The session logs every route, so
one UPDATE from a peer ended the session task. Such a prefix is now
malformed, and showing a prefix cannot fail.
- Bits past a prefix's length were kept, so the same prefix sent with stray
bits was a different key in the RIB. Prefixes now have one form.
- An opaque extended community was read as seven octets instead of eight,
putting every community after it out of step.
- AGGREGATOR could be neither read from nor written to a peer with
two-octet AS numbers.
- The Outbound Route Filter and Long-Lived Graceful Restart capabilities
were written differently from how they are read.
- A capability value or unknown attribute too long for its length field
was written with the length wrapped around; it is now an error, or for
the attribute the two-octet length.
Also: ROUTE-REFRESH carries its AFI and SAFI instead of being modelled as
an empty message, and one with a body of the wrong length is refused.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
AFI and SAFI were closed enums, so a family without a variant could not be
represented at all, and the set of negotiated families was a struct with
one bool per family. Supporting another family meant touching both.
- `Afi`, `Safi` and `AfiSafi` in bgp-primitives are newtypes over the wire
numbers, with constants and names for the common families. Capabilities
and MP_REACH/MP_UNREACH use them, so any family a peer names is read as
what it is. The special case that turned a capability for an unlisted
family into an unknown capability is gone.
- `FamilySet` is a set of `AfiSafi`. What we can exchange routes for is the
explicit `family::SUPPORTED` list; only those are offered in our OPEN.
- `AddressFamilyId` stays as the IP version of a prefix.
- NLRI is still a list of IP prefixes. MP_REACH/MP_UNREACH for a family
whose NLRI is something else (labelled, VPN, EVPN, ...) is refused
rather than misread as prefixes, which is what happened before for the
SAFIs the old enum listed.
Bug found on the way: the reserved octet in MP_REACH_NLRI was skipped only
for IPv6, so an IPv4 MP_REACH_NLRI was parsed with an extra 0.0.0.0/0
route that the peer never sent.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The OPEN used to offer IPv4 and IPv6 unicast unconditionally, and what the
peer offered was ignored apart from its four-octet AS number.
Address families (RFC 4760):
- A peer's `address_families` in the configuration says which to offer it;
all of them by default.
- A session carries the families both OPENs offered. A peer that sends no
Multiprotocol capability is a plain RFC 4271 speaker: IPv4 unicast.
- Routes of a family that was not negotiated are not sent, and are ignored
when received; the rest of the UPDATE is applied and the session stays up.
- A session with no family in common is refused with Unsupported
Capability, naming the capabilities we needed (RFC 5492).
- The negotiated families are shown on /statusz and in /api/peers.
Bugs found on the way:
- A capability naming an address family we have no constant for (EVPN,
VPNv4, ...) failed the whole OPEN, so such a peer could never establish.
It is now kept as an unknown capability and ignored.
- An UPDATE could not be encoded for a peer without four-octet AS support,
which ended the session on the first route. AS_PATH is now written two
octets wide with AS_TRANS where needed, and AS4_PATH carries the real
numbers (RFC 6793).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A session was one function that walked OpenSent, OpenConfirm and
Established with the socket, the timers, collision detection, the registry
and the RIB all in the same loop. There was no state or event type, and no
way to test a transition without a TCP connection.
It is now two pieces:
- fsm.rs is the state machine of RFC 4271 section 8 for a connection. It
takes an Event and returns the Actions to carry out. It owns no socket,
timer or task. What the OPENs negotiated lives in the states that have
it, so there is nothing to unwrap. Thirteen tests cover the handshake,
hold-time negotiation, refused OPENs, messages out of turn, timer expiry,
collisions and shutdown by feeding it events.
- connection.rs is the driver: it turns the socket, the timers and the
outbound queue into events and carries the actions out.
Behaviour changes:
- A message that arrived in pieces was lost if a timer fired between the
pieces: read_exact was used inside select!, which discards what it has
read when another branch wins. Messages are now framed from a buffer
that survives cancellation.
- A NOTIFICATION from the peer during the handshake is no longer answered
with an FSM error; RFC 4271 says to close without replying.
- A session the peer ended is reported as 'peer sent NOTIFICATION: ...'
with its error, where the dialer used to record 'session closed by peer'.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The library crates returned eyre reports, so a failure was a string and the
session code could not tell a truncated UPDATE from a bad header. Whatever
went wrong, the peer got an FSM error or nothing.
Each crate now has its own error types:
- bgp-primitives: PrefixError, UnknownAfi and UnknownSafi, and EncodeError
for every to_wire.
- bgp-packet: Message::decode returns a DecodeError naming the NOTIFICATION
code, subcode and data for the failure (header errors per section 6.1,
OPEN per 6.2, UPDATE per 6.3). InvalidValue and CommunityError replace
the remaining strings. from_wire is now a thin wrapper around decode.
- bgp-server: SessionError, ServerError and ConfigError. Every abnormal end
of a session goes through one function that sends the NOTIFICATION the
error maps to. The message length is checked against the 4096-octet
maximum before the body is read.
- bgp-web: serve returns io::Error.
eyre remains in the binary and in tests.
Bugs found and fixed on the way:
- An OPEN with no optional parameters, which is valid, failed to parse.
- An unsupported BGP version was reported as an unknown optional parameter.
It now gets Unsupported Version Number with the version we speak as data.
- A SAFI above 255 was truncated to its low octet and could match a known
one.
- Prefix text with a length beyond the address size (10.0.0.0/99) was
accepted.
- Message::to_wire truncated the body length to 16 bits before checking it.
- A peer hold time above 65535 seconds in the configuration wrapped.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Each received prefix kept its own copy of the UPDATE's whole attribute list,
and for multiprotocol routes that list included MP_REACH_NLRI with every
other prefix from the same UPDATE: n prefixes stored n copies of n prefixes.
An UPDATE is now turned into a ReceivedUpdate once. Its prefixes share one
attribute list, which leaves out MP_REACH_NLRI and MP_UNREACH_NLRI; the next
hop from MP_REACH_NLRI is kept on the entry. The RIB takes one command per
UPDATE instead of one per prefix, and its table is keyed by peer so that
dropping a peer's routes no longer scans every route.
The status API shows the multiprotocol next hop as an address instead of
its Debug form.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Cargo.toml has declared Apache-2.0 without the licence being in the tree.
CI checks formatting, clippy and the tests on every push, with warnings as
errors. The workflow lives in .github/workflows, which Gitea Actions also
reads.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Lints are declared once under [workspace.lints] and every crate opts in:
clippy's default set, plus dbg!, todo! and unimplemented!, and unsafe code
is forbidden. Every crate now inherits its version, licence, authors and
repository from the workspace, so they no longer disagree (four were at
0.0.0).
The poptrie module is unfinished and unused; it says so and expects the
dead-code lint instead of warning on every build. Its construction method
is renamed from new, which took &mut self, to build.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The capability parser had a todo!() for capability code 8 (RFC 8277), and
the encoder two more. A peer advertising it took the session task down.
It is now parsed into its (AFI, SAFI, count) entries and round-trips.
Also carries clippy's mechanical fixes for this file.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Default settings. No change in behaviour; the tree was last formatted
before the lecture work and had drifted to a mix of line widths.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The repository was renamed. Also drops workspace.package.name, which Cargo
ignores (a package name cannot be inherited) and warned about on every build.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The preview launcher pointed at the removed slides and editor state does
not belong in the tree. The Dockerfile and README no longer refer to the
lab.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The 'Sending UPDATE' and 'Received UPDATE' lines counted only the IPv4
fields of the message body, so every IPv6 UPDATE logged announced=0.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Routes were announced and withdrawn through an HTTP form, each tagged with a
free-text owner and checked against an allow-list and a per-owner limit.
That was the lab's interface, not a router's.
Routes now come from 'originate' entries in the configuration file and are
in the local RIB before any session starts. The local RIB holds one route
per prefix with no notion of owners or policy. local_route_policy is gone
from the configuration.
The HTTP API keeps its GET endpoints (status, config, peers, originated and
received routes, events, metrics). The announce, withdraw, wire-bytes and
well-known-community endpoints are removed, and the status page loses the
announce form, the community presets and the byte decoder. Community text
parsing moves from the web crate to bgp-packet so the configuration can use
it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The slides, handout, lab router configs, RIPE objects and load test were
for the Network Security 2026 lecture and are kept on the netsec-2026 tag.
The ping monitor sniffed ICMPv6 echo requests so students could watch their
prefix being used. It has no place in a BGP speaker: it needs CAP_NET_RAW,
adds the only unsafe code in the tree and a libc dependency.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The fill-in fields put the label on its own line with dotted lines to write
on. handout.tex takes \studentnum and \studenthex; build-class.py compiles
one copy per student with the prefix filled in, checks each is two pages
and joins them for double-sided printing.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
As presented. The palette is light on request of the reviewer; code blocks
and byte dumps keep a dark ground so the byte colours still work. The OPEN
walk lights the hovered bytes as an inverted block and its table cells are
dark text on a tint of their group colour.
build.py inlines reveal.js, the stylesheets, scripts and Markdown sections
into slides/dist/hands-on-bgp.html for uploading.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
slides/dist/ (the single-file bundle) and docs/lab/handout/class.pdf (one
sheet per student) are build products.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
One double-sided A4 sheet with 1in margins, still in the LNCS look.
1. Who announces the address you are using: find your current address and
its prefix and origin AS, then whois it and your lab prefix.
2. Announce your prefix, read the bytes, find it from outside, ping it.
3. Announce with communities: only the SwissIX route servers, then only
SWITCH, with reachability checkboxes, one discussion question, clean up.
Sections 1 and 2 fill page one; section 3 has page two. Dropped the ETH
lookup, the long registry comparison, NO_EXPORT, the byte-decoding task,
the reference appendix and most of the bibliography.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
docs/lab/handout/handout.tex, typeset with Springer's LNCS class (title
block, abstract and keywords, figure, booktabs tables, RFC references), six
A4 pages so it prints double-sided on three sheets. The built PDF is
committed alongside for printing.
Three sections students work through with the slides:
1. Who holds which addresses: whois their own /48 and ethz.ch's prefix in
the RIPE database, then compare with what is actually routed.
2. Announcing the prefix on the lab page, reading the UPDATE bytes, finding
it from outside, and reaching it.
3. Steering with communities until the prefix is reachable only from the
university network (AS559 via the SwissIX route servers), with
reachability checkboxes for laptop and phone at each step.
Plus a one-page reference appendix: IPv6 notation and hex, the UPDATE
layout, path attributes, glossary.
The lab host, the lab address space and the prefix pattern are macros at
the top of the file. Expected answers were checked against the RIPE
database and RIPEstat (ETH: 2001:67c:10ec::/48, ASSIGNED PI, origin AS559;
lab: 2a0d:d741::/40, AGGREGATED-BY-LIR, route6 origin AS210036).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- Removed every fragment and stepped code highlight: a slide now shows
everything at once. Two code blocks keep a static highlight on one line.
- Dropped "What stands between you and an outage".
- The UPDATE walked through in section 3 is now 2a0d:d740::/48 as AS210036
announces it to the SwissIX route servers: next hop 2001:7f8:24::c9
(pr01's port on the peering LAN), AS path 210036, MED 0 and 210036:10:2,
matching what pr01's export filter puts on routes to peers. Encoded with
the packet crate and parsed back to the identical message; the slide
generator asserted every quoted number, including the next hop, against
the bytes. Adds a MULTI_EXIT_DISC slide to the walkthrough.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- Title is now "Hands-on BGP"; the AS-path diagram is gone from the cover.
- Sections: IPv6 addresses and prefixes; where AS numbers and prefixes
come from (IANA, RIRs, the RIPE database, IRR vs RPKI); BGP and the
bytes of an UPDATE; IP transit vs Internet exchanges (the peering LAN as
an Ethernet segment, route servers); the lab (ferrix, pr01.rue, SwissIX,
NL-ix, France-IX, DE-CIX, transit).
- The UPDATE walked through in section 3 is a real 94-byte capture from
ferrix for 2a0d:d741:1::/48. Vertical slides light up one part at a
time with its decode; the generator asserted every quoted number
against the capture. Session counts come from the tracked bird.conf,
the route6 object from the RPSL file, SwissIX's LAN from PeeringDB.
- New components: focusable bytes, decode table, prefix bits, and SVG
diagrams drawn in the theme's colours.
- Opening index.html straight from disk used to give a title and empty
panels, because browsers block the module and the section files there.
The page now explains that, and also reports a failed reveal.js load.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Structured like Matt Godbolt's newer talks (hrt-theme.css in
what-happens-beneath-accu-2026): reveal.js with no stock theme but a custom
dark one, every slide on a rounded panel, IBM Plex Sans and Mono, monokai
code with glowing stepped lines, 16:9 with fade transitions, and the
content in one Markdown file per section.
Made our own: the palette is a handful of variables (with ?accent= presets
for comparing), and his ring-and-thread corner motif became a chain of
nodes and links, an AS path, which also opens the title slide. A wordmark
sits where his decks carry the employer logo.
Also: a small `bird` syntax highlighter for router config on slides, a
bytes component in the four groups of the /statusz bytes view, columns,
boxes, and a `bare` slide class. reveal.js 6 loads from a vendored copy
when slides/vendor-reveal.sh has been run (works offline, ignored by git)
and from the CDN otherwise. Section files carry a starter outline for the
lab that doubles as a tour of the slide types.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- docs/lab/prod/pr01/bird.conf is now the config generated for per-session
export control (every eBGP session passes its neighbour AS and location
to the shared export function; customer sessions go through the same
gate), with the ferrix stanza in its same-host form (::1, multihop).
- docs/lab/ripe/lab-objects.rpsl: the inet6num and the 128 route6 objects
for the student /48s. Public registry data, no credentials.
- Ignore Python bytecode caches.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
tools/loadtest.py releases N simulated students through a barrier so they
announce at the same instant, then verifies behaviour rather than only
status codes: the table holds every route under the right owner, each
route became an UPDATE per established session, the event stream lost no
event, concurrent takeover attempts are all refused, withdrawal leaves
nothing behind, and the daemon's own counters show the matching number of
UPDATEs sent for announcements and withdrawals. Standard library only;
supports basic auth and repeated rounds.
Safe by default against a production router: every route carries
210036:0:0 (announce nowhere), leftovers from earlier runs are removed,
prefixes held by real owners are skipped, and everything is withdrawn even
if the run fails.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Preparing to run ferrix next to BIRD on the production router.
- build: the config proto is now parsed in pure Rust, so building needs no
`protoc` binary. It previously worked only where one happened to be
installed; the Dockerfile in this repo could not have built.
- peers gain `local_address`, the source IP for connections we open (and
therefore the next hop we advertise). When both speakers share a host
the OS would otherwise pick an arbitrary local address and the peer,
which identifies us by address, would refuse the session. Address-family
mismatches are rejected at config load; an address the host lacks shows
up as the peer's backoff reason.
- docs/lab/prod/on-router: static musl cross-build instructions, config,
hardened systemd unit (unprivileged, SIGTERM gives BIRD a Cease),
loopback addresses for networkd, the BIRD stanza for a same-host
multihop session, and reverse-proxy examples with TLS and a password,
since the API has no authentication.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Against a production BIRD router, restarting ferrix was followed by
several refused connection attempts before the session came back.
- The daemon had no signal handling: Ctrl-C killed the process and the
peer only ever saw a dead TCP connection. On SIGINT or SIGTERM it now
sends Cease / Administrative Shutdown (RFC 4486) to every established
peer, waits up to two seconds for the sessions to close, and stops the
dialers from reconnecting meanwhile. A NOTIFICATION queued for a session
ends that session once sent.
- ROUTE-REFRESH was parsed and ignored. A peer sends it when its import
policy changes (e.g. after `birdc configure`), expecting our routes
again so the new policy can be applied. The session now reports the
request and the advertiser re-sends every local route to that peer. The
request also appears in the event stream and the status page log.
Tests: a refresh request in real wire format re-sends all routes and keeps
the session up; shutdown delivers the Cease to every peer and closes the
sessions. Verified with the real binary: SIGINT, peer logs the Cease.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The production router has the same neighbour AS at several exchanges, so
per-AS control alone cannot express "only at SwissIX". Every session now
has two coordinates, neighbour AS and location, and the scheme gains two
functions:
210036:0:<as> / 210036:2:<location> never announce (wins)
210036:1:<as> / 210036:3:<location> announce
210036:0:0 announce nowhere
nothing announce everywhere
docs/lab/prod/bird-ferrix.conf documents the functions, the per-session
call (export where peers_out(<as>, <location>)) and the gate that sessions
exporting everything need as well.
Status page: location presets, and AS210036 communities are decoded from
their structure so any value reads well in the bytes hover.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
BIRD side (docs/lab/prod/bird-ferrix.conf): the ferrix import filter now
admits 210036:0:* and 210036:1:*, and lab_export_allowed(peer_as) decides
per eBGP session whether a lab route is announced, with the same shape and
precedence as the SwissIX route-server communities:
1. 210036:0:<peer-as> never 2. 210036:1:<peer-as> announce
3. 210036:0:0 nobody 4. default everyone
The per-session export block applies only to routes learned from ferrix,
strips the private ASN and removes the action communities before sending.
Status page: the large-community picker is grouped into "AS210036" and
"SwissIX route servers" presets, including "announce to the SwissIX route
servers" and "send to SWITCH (AS559)"; the bytes hover names both families.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- The status page has a collapsible "Configuration" section showing the
config file as read at startup, with its path. Served by the new
GET /api/config; RouterInfo::with_config carries the text.
- Removed the AS-path prepends field and the SwissIX prepend presets
(42476:101..103) from the form to keep the exercise focused. The API
still accepts `prepend`.
- Docs follow: the handout loses its prepending exercise, the lab README
its prepend row, and the BIRD config notes that the 42476 namespace is
still admitted for hand-typed values.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- Hovering (or focusing) a byte segment opens a popup that decodes it:
header fields, the two length fields, and each path attribute with its
flags spelled out and its value in readable form. AS_PATH shows segment
type, ASNs, path length and ASN width; COMMUNITIES shows well-known
names; LARGE_COMMUNITIES shows the SwissIX meaning of known values;
MP_REACH/UNREACH show address family, next hop and prefixes; NLRI and
withdrawn routes list prefixes. Decoding happens on hover from the bytes
themselves. /api/routes/local/wire gains four_octet_asn so the AS_PATH
width is known.
- The page is now a snapshot by default. ?refresh=<seconds> (1..3600)
turns on reloading of all tables at that interval and connects the
event log; the header offers 2s / 5s / 30s, "turn off" and "reload now".
Events only feed the log, so the tables never change faster than asked.
The reader's own announce / withdraw still shows its effect at once.
- Removed the "tables refresh on every event" footer text.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Every poll and event rebuilt each table from scratch, and an open bytes
view collapsed to "loading..." and was refetched every five seconds.
- Tables are rewritten only when their markup changes. Clock-dependent
text (ages, retry countdowns, uptime) is rendered as placeholders and
ticked in place once a second, so unchanged data means no DOM writes.
- The bytes view is cached per route and set of established sessions, and
the previous bytes stay visible while new ones load.
Measured with a MutationObserver: zero table rebuilds across 13 s of
polling with a bytes view open; one announcement redraws only the local
and received tables once each, with no "loading..." flash.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Modelled on Vitess's servenv status page and golang.org/x/net/trace, both
written by Google engineers to mirror the internal pages: plain sans-serif
HTML, default links and native form controls, dense bordered tables, and a
full-width centred light blue (#eef) heading bar per section. Header shows
"Status for ferrix_bgp" with start time, uptime, version, router ID and AS
on the left and host, "View:" links and live-update state on the right.
The dark theme is dropped on purpose.
Behaviour is unchanged (live updates, owner-gated withdraw, per-session
UPDATE bytes, looped-route highlight, peers polling). Additions:
- a SwissIX large-community picker next to the field; presets with a peer
AS ask for the number
- event log lines include prepend and MED
- /api/status gains hostname and started_at_unix (RouterInfo::new)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Preparing to peer with a production BIRD router turned up two problems.
- packet: most Capability::from_wire arms returned the per-capability
buffer (always empty by then) instead of the bytes following the
capability. The list parser then saw an empty remainder and silently
dropped every later capability. Any OPEN listing extended message, role,
graceful restart, add-path, enhanced route refresh, LLGR, extended next
hop or an unknown capability before the 4-octet ASN capability lost the
ASN, so the peer appeared as AS_TRANS (23456) and a configured
remote_asn was refused with Bad Peer AS. BIRD and FRR both do this.
Single-capability round-trip tests could not catch it; new tests parse
every capability with another following it, and a full BIRD capability
list.
- server: advertise MultiProtocol IPv6 unicast alongside IPv4. Without it
a peer will not accept the IPv6 routes we originate via MP_REACH_NLRI.
- Interop test: byte-for-byte BIRD 2 OPEN is accepted with the right ASN,
and an IPv6 End-of-RIB neither drops the session nor creates a route.
- docs/lab/prod: BIRD config for an announce-only ferrix neighbour
(strict /48 import filter, community scrubbing, unreachable next hop,
import limit, export none, passive) and the matching ferrix config.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
When both ends dial each other, exactly one TCP connection must survive
and both ends must pick the same one. Previously ferrix relied on the
other router to sort this out and could briefly hold two sessions.
- collision: a shared map keyed by the peer's BGP Identifier holds the one
connection currently preferred for that peer plus a kill signal. Each
connection calls admit() on receiving the peer's OPEN. The connection
initiated by the higher BGP Identifier wins; the loser gets Cease /
Connection Collision Resolution, whether it is the new connection or an
existing one in OpenConfirm running in another task.
- A collision with an Established session closes the new connection (RFC
default). The optional CollisionDetectEstablishedState is available as
collision_detect_established_state: a restarted peer's new connection
then replaces the stale session at once instead of being refused until
the hold timer expires. Enabled in the lab config.
- Two connections opened by the same side: the newer wins.
- OPEN validation rejects a peer using our own BGP Identifier or 0.0.0.0
with Bad BGP Identifier; collision resolution needs distinct identifiers.
- OpenConfirm no longer times out instantly when the negotiated hold time
is zero.
- Sessions record which side opened them; shown as "opened by" on the
status page and as `direction` in /api/peers. New counter
bgp_connection_collisions_total{closed}.
- Tests: unit tests for the rule (including that both ends agree), and
integration tests for simultaneous open in both identifier orders and
both arrival orders, Established default, restart replacement with route
flush, distinct identifiers, slot release, and Bad BGP Identifier.
Existing fan-out test now gives its two peers distinct identifiers.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Phase 5 of the teaching demo plus two fixes found while exercising the
dialer on the status page.
- Adj-RIB-In: routes learned from a peer are removed when its session ends
(RFC 4271 §8.2.2), emitting a Withdrawn event per prefix before the
session Down event. Previously stale routes from dead sessions stayed in
the received table.
- statusz: poll the peers table every 5 s so dial state (Connecting /
Backoff and its retry countdown) stays current; those changes carry no
event.
- docs/lab: docker-compose topology with ferrix (AS65000) and two FRR
routers. r2 accepts nothing directly from ferrix so student prefixes
only reach it via r1, then come back with AS path 65002 65001 65000. r1
applies community-driven policy: NO_EXPORT stops forwarding, 65001:100
prepends, BLACKHOLE drops. Dockerfile for the daemon.
- docs/lab/HANDOUT.md: student exercises walking through announcement,
reading the bytes, prepending, NO_EXPORT, action communities,
blackholing and withdrawal.
The container files have not been executed (no runtime available here);
docs/lab/README.md lists what is most likely to need adjusting.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Phase 4 of the teaching demo: the daemon now brings up its own sessions.
- New dialer task per configured peer: connect with a timeout, run the
same FSM as the passive side, back off exponentially (from
connect_retry_seconds, capped at two minutes) and retry forever. Stays
idle while a session with the same remote IP and ASN already exists.
- The peer's OPEN must carry the configured ASN, otherwise the session is
refused with a Bad Peer AS notification. Inbound connections from a
configured peer's address get the same check plus the configured name
and hold time.
- SessionRegistry tracks configured peers and their dial state
(Idle / Connecting / Backoff with last error and retry time).
- /api/peers merges configured peers with live sessions so the page shows
peers that are down, why, and when the next attempt is; unconfigured
inbound sessions are listed after them.
- fake_peer gains --listen so it can stand in for a router the daemon
dials.
- Integration tests: dial and establish, reconnect after the peer closes,
wrong ASN refused, inbound from a configured address checked and named.
Not implemented: RFC 4271 §6.8 collision detection; a real router on the
other end resolves the rare race by closing one connection.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Phase 3 of the teaching demo.
- crates/web serves an embedded, dependency-free status page at /statusz
(/ redirects there). Sections: router header with live indicator, peers,
announce form (owner remembered locally, well-known community picker,
prepend / MED / large communities), local routes with per-session UPDATE
bytes segmented into header / lengths / attributes / NLRI, received
routes with rows highlighted when a locally announced prefix has come
back, and a live event log. Tables refresh on every SSE event. Light
and dark themes.
- fake_peer example in crates/integration-tests: connects, announces,
prints received UPDATEs and with --echo re-announces everything back
with its ASN prepended (dropping NO_EXPORT), so one daemon plus one
process demonstrates the full loop without lab routers.
- README for the binary documents the endpoints and the demo setup.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Phase 2 of the teaching demo: everything the status page will need is now
reachable over HTTP.
- New crate crates/web (axum). JSON endpoints for status, peers, local
routes (announce / withdraw), received routes, the exact UPDATE bytes a
local route produces per session, and well-known community names.
Communities are accepted as ASN:VALUE or by name (NO_EXPORT, BLACKHOLE,
...); large communities as A:B:C. Policy errors map to 403/409/422.
- GET /api/events is a server-sent event stream merging session up/down,
local route and Adj-RIB-In changes, with lag notifications.
- GET /metricz serves Prometheus metrics; the server crate now counts
messages sent/received by type and sessions established.
- The runnable main moves from crates/server to bin/bgp so it can depend
on both server and web without a cycle. `cargo run -- config.textpb`.
- Config gains http_endpoint; unset disables the API.
- Integration tests drive the API against a live server with a real BGP
peer, including verifying the wire view matches the bytes the peer got.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Phase 1 of the BGP teaching demo: the daemon can now originate routes and
advertise them to every established peer.
- rib: add LocalRib actor holding locally originated routes with owner,
origin, AS-path prepend, MED, standard and large communities. Enforces a
LocalRibPolicy (prefix allowlist, per-owner limit, owner match on
replace/withdraw) and emits LocalRibEvents.
- server: add SessionRegistry with per-session outbound queues; sessions
register on Established and deregister via a drop guard. Emits
SessionEvent::{Up,Down}.
- server: advertiser task fans LocalRib changes out to all sessions and
replays the table to new sessions. UPDATE builders handle IPv4 (legacy
NLRI) and IPv6 (MP_REACH/MP_UNREACH).
- fsm: merge the timer and timer-less established loops into one select!
with an outbound branch; reset the hold timer on any received message
per RFC 4271 §8.2.2 (previously only on KEEPALIVE).
- config: new local_route_policy proto message.
- packet: fix Communitites typo, expose CommunitiesPathAttribute inner vec.
- primitives: IpPrefix::contains.
- integration tests: announce/withdraw/replay/fan-out/policy coverage.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- Implement passive-side BGP FSM (OpenSent → OpenConfirm → Established)
with hold timer, keepalive timer, and NOTIFICATION on expiry
- Add bgp-server crate: config validation, TCP listener accept loop
(BgpServerBuilder → BgpServer), connection dispatch
- Add bgp-rib crate: Adj-RIB-In actor owning route state with event
broadcast for streaming consumers and snapshot for introspection
- Wire Adj-RIB-In into FSM: announce/withdraw on UPDATE for both legacy
IPv4 NLRI fields and MP_REACH/MP_UNREACH path attributes
- Add bgp-integration-tests crate with 15 end-to-end tests covering
session establishment, timers, UPDATE handling, and RIB introspection
- Fix require_empty! macro bug in open.rs (checked wrong buffer)
- Fix FourByteAsn capability parser returning consumed buffer as remainder
- Fix tracing subscriber not being installed in TracingArgs::install
- Expand docs/RFCs.md with full implementation status across 15 RFCs
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
