rayhaan/ferrix
0
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

merge into: rayhaan/ferrix:netsec_2026
rayhaan/ferrix:main rayhaan/ferrix:speaker-cleanup rayhaan/ferrix:netsec_2026
rayhaan/ferrix:v0.2.0
...
pull from: rayhaan/ferrix:speaker-cleanup
rayhaan/ferrix:main rayhaan/ferrix:speaker-cleanup rayhaan/ferrix:netsec_2026
rayhaan/ferrix:v0.2.0
27 Commits
netsec_2026 ... speaker-cleanup
Author SHA1 Message Date
Rayhaan JaufeerallyandClaude Fable 5.1 9caaf5018e Review pass on the server: bugs fixed, session setup simplified
ci / check (push) Canceled after 0s
Details
ci / fuzz (push) Canceled after 0s
Details 
A review of the session driver, the server loop, the dialer, the
advertiser and the HTTP API, with the fixes.

Bugs:

- A peer that stopped reading held its session, and our routes, for as
  long as TCP kept retrying: a blocked write was awaited with no timer
  running. Writes now give up after the Send Hold Time of RFC 9687.
- Routes we originate were sent to a peer in our own AS with our AS
  number on the path and no LOCAL_PREF, which such a peer refuses. They
  now go with an empty path and a LOCAL_PREF.
- Collision detection knew a peer by its BGP Identifier alone. A
  neighbour with one session over IPv4 and one over IPv6 had the second
  closed as a collision, and any host that could connect could displace
  a peer's connection by claiming its identifier. A peer is now its
  address and identifier together.
- The dialer reset its backoff whenever TCP connected, so a peer that
  accepts the connection and refuses the session was dialled every few
  seconds for ever. It resets only after a session was established.
- A failure on our side (routes that cannot be stored) dropped the
  connection without a NOTIFICATION. It goes through the state machine
  and the peer is sent Cease. A queued message that cannot be encoded
  is logged and skipped instead of ending the session.
- The accept loop retried at once after an error, spinning when the
  process was out of file descriptors.
- Attribute sets no route used any more were dropped only when a peer
  went away. A task now does it every minute.
- The advertiser subscribed to session events inside its task, after the
  dialers were started, and could miss the first session.
- Shutdown waited without limit on a session whose queue was full.
- /api/routes/received took any limit and built the reply on a runtime
  thread. The limit is capped and the work is done off those threads.
- The default log level was TRACE, which logs every UPDATE.
- IPv6 listeners are IPv6-only: listening on 0.0.0.0 and [::] fails on
  systems where the second would take IPv4 too, and IPv4 peers arriving
  under mapped addresses matched no configured peer.
- A connection closed after a NOTIFICATION is kept for a moment so that
  the peer can read it.
- The configuration accepted two peers with one name or one address,
  and a router id of 0.0.0.0.
- An HTTP address that cannot be bound now stops the daemon at startup.

Simplifications:

- SessionConfig mixed what a server's sessions share with what is
  configured for one peer, and was cloned and patched field by field in
  two places. It is ServerContext, shared, and PeerSettings, made from a
  PeerConfig in one place.
- The attributes of an UPDATE as they were on the wire travelled from
  the reader to the table through a field of the session, around the
  state machine. They go through it, with the message.
- run_session returns how the session ended and whether it had been
  established; Ok no longer reads as "closed by peer".
- Session is not generic over its reader.
- BgpServerBuilder had nothing to build: BgpServer::bind takes the
  configuration.
- One shutdown path, close_sessions, for the signal handler and for a
  failed server.
- The advertiser's three copies of "send every route" are one.
- PeerId moved from the rib crate, which did not use it, to the server.
- Unused API removed or made private to the crate.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-02 10:46:09 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 a8aba52562 Review pass on the codec and the table: bugs fixed, encoders simplified 
A review of the wire codec and the routing table, with the fixes.

Bugs in the codec:

- A NOTIFICATION with a code or subcode we have no name for (subcode 0,
  "unspecific", is common) failed to decode and was answered as a bad
  message. Every pair now reads, as BgpError::Other.
- Optional parameters in the extended form of RFC 9072 were read and
  written with one-octet parameter lengths; they are two.
- An ORF capability listing more than one address family made the whole
  OPEN malformed.
- Bytes after an OPEN's optional parameters were accepted.
- Extended communities were split into fields by rules that were wrong
  for the common types. They are kept as their eight octets.
- Several length fields were computed in u16 and could wrap on encode.

Simplifications in the codec:

- Lengths are filled in after the value is written, for messages,
  attributes, capabilities and the UPDATE's two fields. The parallel
  wire_len functions, which had to be kept in step by hand, are gone.
  Message::to_wire no longer encodes into a second buffer.
- Type codes for messages, attributes and capabilities are each defined
  once. The strum discriminant enums and their hand-written conversions
  are removed.
- One function splits an attribute off a buffer; the UPDATE decoder, the
  plain attribute parser and the table's key reader all use it.
- The UPDATE decoder's 185-line loop is an AttributeReader with one
  method that decides what becomes of an attribute. Duplicate detection
  is a bitset, not a HashSet allocated per UPDATE.
- ParserContext::four_octet_asn is a bool. The "context not set" error
  path, which treated a programming error as a malformed route, is gone.
- Codec functions take a context only if they read it, and return a
  Result only if they can fail.
- MP_REACH_NLRI and MP_UNREACH_NLRI hold an AfiSafi.
- AddressFamilyId is IpVersion, which is what it was.
- Deserialize is no longer derived on codec types: nothing used it, and
  it bypassed IpPrefix's invariants. IpPrefix::to_wire, which was
  broken and unused, is removed.

Bugs in the table:

- The index of a row's best route was sixteen bits. A peer using
  ADD-PATH could send more routes for a prefix than that counts, and
  the index wrapped. It is 32 bits, at no cost in row size.
- Shards and attribute-store stripes were picked from the top bits of
  the hash, which are the bits hashbrown keeps as the tag it compares
  first. With 64 shards each shard's table had 2 of 128 tag values.
  They are now picked from bits the table does not use.
- remove_peer held the peer list's lock for the whole scan of the
  table, so no session could establish while a peer was removed.
- When more prefixes went at once than were left, the forced trim
  dropped a fixed few entries, reset every reader, and re-ran the tidy
  on every later write. It drops the older half.
- Readers were woken by every change, including a repeated announcement
  and changes their journal does not record.
- Reader::poll with a budget of zero never made progress.

Simplifications in the table:

- The two journals of a shard are one Journals value written through
  three calls; the five places that kept them in step by hand are gone.
  Whether an entry is still live is decided in one place.
- The shortcuts of route selection moved to select.rs, next to the
  invariant they depend on, which is now written down.
- announce and withdraw take routes with or without path identifiers;
  the _paths variants are removed.
- The selection key is read off the wire form only; the second
  implementation over parsed attributes is removed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-02 10:32:27 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 d49bab58a5 Add ADD-PATH (RFC 7911): several routes for a prefix from one peer 
A peer can now send several routes for one prefix, told apart by a path
identifier, and each is a route of its own in the table.

Wire:
- The ADD-PATH capability lists several address families. It was read as
  exactly one, so the OPEN of a router offering it for IPv4 and IPv6 was
  malformed and the session never came up.
- Prefixes in UPDATEs are `Nlri`: a prefix and a path identifier. The
  identifier is on the wire, in the body and in MP_REACH/MP_UNREACH, for
  the families the session uses ADD-PATH for in that direction. Sending
  and receiving are settled separately, so a session has one codec context
  for each.

Negotiation:
- `add_path` on a peer in the configuration: off (the default), receive,
  send or both, offered for every family offered to the peer.
- In use for a family and a direction when one side offered to send and
  the other to receive. Shown on /statusz and in /api/peers.

Table:
- A route is known by its peer and path identifier together. Identifiers
  are stored only in rows that have a non-zero one, so routes of peers
  that send one route each cost nothing more.
- Equal routes from one peer are chosen between by the lower identifier.
- /api/routes/received gives each route's path_id.

Journal:
- A second journal in each shard records every change to any route, for
  followers that want all the routes of a prefix and not only the best:
  RibTable::reader_of_all_routes and Event::Routes. This is what sending
  several routes to a peer will read. It costs nothing while nobody
  follows it.

Sending: prefixes go out with a path identifier where that was agreed, but
there is still one route for each prefix to send: only originated routes
are announced.

Tested with the table and journal model tests extended to path identifiers
and to followers of all routes, the codec property tests and fuzz targets
extended to ADD-PATH sessions, and end to end in both directions and for
the case where only one side offers it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-02 00:57:15 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 31aa3a4563 Use the sharded table in the server: sessions choose best routes 
The server now keeps what it learns in table::RibTable. The Adj-RIB-In
actor is gone, and with it the queue every UPDATE from every session went
through.

- A session takes a number in the table when it is established and writes
  its UPDATEs into the table itself. Its routes are removed when it ends,
  on a blocking thread since that looks at every prefix; a session torn
  down without cleaning up still has them removed.
- For every prefix the best route is chosen (RFC 4271 §9.1.2.2) as routes
  come and go. Nothing acts on the choice yet beyond showing it.
- The UPDATE decoder hands over the bytes of the attributes it kept, so the
  table finds the attribute set from them without writing the attributes
  out again. On a session with two-octet AS numbers the bytes are not in
  the table's form, and the parsed attributes are used.
- /api/routes/received marks the best route of each prefix and returns at
  most `limit` routes (default 1000): the table can hold hundreds of
  millions. /api/status has the totals, and the status page says how many
  it is showing. The time a route was received is no longer kept or shown.
- The event stream reports changes of best route (best_changed,
  best_withdrawn) in place of every route received. Each client follows
  the table's journal with a reader of its own, from the moment it
  connects: table::RibTable::reader_from_now.
- Metrics: bgp_rib_routes, bgp_rib_prefixes, bgp_rib_attribute_sets and
  bgp_rib_journal_entries replace bgp_adj_rib_in_routes.
- A line is no longer logged at INFO for every UPDATE sent or received; a
  full table is a million of them. They are at DEBUG.
- A table with no room for another peer ends the session with Cease / Out
  of Resources.

Four peers sending a synthetic full table each over TCP, 4.9 million
routes: 0.80 s, against 4.76 s before. One peer: 0.45 s against 0.69 s.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-02 00:31:28 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 3b20576971 Add the journal: following the table's best routes at one's own pace 
Whatever acts on the best routes (export to peers, the forwarding table,
the status page) reads their changes from a per-shard journal through a
table::Reader. Nothing uses it yet.

- The journal records that a prefix's best route changed, not what to. A
  reader that comes to an entry is given the route as it is then. A prefix
  that changes a thousand times between reads is one event, and the
  journal never holds more live entries than the table has prefixes.
- Each row keeps the sequence number of its latest change; an entry with
  another number is dead and is dropped when the journal is tidied.
- Entries go once every reader has passed them. A reader that does not
  read cannot hold them back for ever: past a limit the oldest half is
  dropped, and that reader is given the whole table again (Reset, every
  best route, Synced). A new reader starts the same way.
- With no reader nothing is recorded.
- Reader::changed waits for there to be something to read.

Tested against a mirror kept from the events: after any interleaving of
announcements, withdrawals, session flaps, reads with small budgets and
readers coming and going, with journals small enough to be tidied and
overrun constantly, a reader that has caught up holds exactly the table's
best routes. Also with writers on four threads and a reader on a fifth.
The property test found one fault (a pass that resumed part-way claimed
to have caught up) and catches three more put in on purpose.

On the AMS-IX dump, 12 threads: loading takes 2.29 s with a follower
reading throughout against 2.17 s with none, the journal peaking at
170,000 entries for 1.39 million prefixes. With a follower that does not
read, the journal ends at 1.74 million entries, and the follower then
catches up in 0.08 s.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-02 00:14:20 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 eab891567b Add the sharded routing table: Adj-RIBs-In and Loc-RIB in one 
bgp_rib::table::RibTable holds every route from every peer and the best
one for each prefix. It is not used by the server yet; this adds it with
its tests and benchmarks.

- Organised by prefix. A row holds the prefix's routes from all peers in
  one array; choosing again after a change reads that row and nothing else.
- A route is sixteen bytes: a pointer to its attribute set and one word
  holding the peer and the first three steps of selection (LOCAL_PREF, AS
  path length, ORIGIN) as a single number. That number alone settles most
  changes; a tie goes on to MED, compared only within a neighbouring AS
  and in a way that does not depend on arrival order, and the tie-breaks.
- Attribute sets are stored once, in wire form, with what selection needs
  from them worked out when they are stored.
- Sharded by hash of the prefix, a lock per shard. A session applies its
  own UPDATEs; there is no queue or owning thread in front of the table.
- Readers take one shard at a time for reading and look at rows in place.

Route selection is checked against a plain implementation with no
shortcuts by a property test over random announcements, withdrawals and
session flaps, for both the chosen route and the count of changes.

On a RIPE RIS dump from AMS-IX (148 peers, 39.3 million routes), against
the Adj-RIB-In actor, which does no selection:

                          actor       table
  memory                  11.0 GiB    1.8 GiB   (279 -> 48 bytes a route)
  load, 1 thread          10.2 s      12.3 s
  load, 12 threads        (no gain)   2.1 s     (19 million routes a second)
  read every route        1.6 s, copying 8 GiB, table stopped
                                      0.2 s, in place, table running

The benchmark crate counts memory only with --features count-memory: the
counters are shared between threads and distort the timings.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-02 00:01:50 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 c1cf15c792 Benchmark on a real table: read MRT dumps from route collectors 
crates/bench reads TABLE_DUMP_V2 dumps (RFC 6396) as published by RIPE RIS
and RouteViews, and `--bench real_table` runs the Adj-RIB-In on one. The
dumps are not checked in; crates/bench/data/README.md says how to fetch
one. Without a dump the benchmark says so and does nothing.

It also prints what in the table decides the cost of a RIB: routes per
prefix, distinct attribute sets, how many routes share a set, and how much
the sets are shared between peers.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 23:43:22 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 7e488e31aa Make IpPrefix a fixed-size value with no allocation behind it 
A prefix was a Vec of its bytes: a heap allocation for every prefix parsed,
stored or copied, and a variable-length key to hash. It is now twenty
bytes, Copy, with private fields and one representation; `bytes()` gives
the wire form.

On the full-table benchmark this alone takes decoding from 9.3 to 10.7
million routes a second and inserting into the Adj-RIB-In from 2.9 to 3.6.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 23:38:39 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 64df419b3a Add a benchmark of taking in the Internet routing table 
crates/bench makes a synthetic full table (a million IPv4 and 230,000 IPv6
prefixes, announced by several peers in UPDATEs shaped like a router's)
and times the steps of taking it in: decoding, the Adj-RIB-In, snapshot,
peer removal, and the whole path over TCP. It also counts the memory the
table holds, and fills a plain packed hash map with the same routes as a
point of comparison.

Run with `cargo bench -p bgp-bench`.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 23:30:29 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 a772322d9e Unwind panics safely: say goodbye to the peer, keep the other sessions 
A panic is a bug, and it should cost as little as it can. The server
already closed every session with a Cease and exited when a RIB or a
background task was lost. This fills in the rest:

- A panic inside a session is caught. The peer is sent Cease /
  Administrative Reset, the session's routes are removed in the usual
  order, and the dialer reconnects. Before, the peer was left with a TCP
  connection closing under it and no reason.
- The tables every session shares (the session registry and the collision
  table) no longer refuse to be locked after a panic. Lock poisoning would
  have made each remaining session panic in turn the next time it touched
  one, so one bug in one session took all of them down.
- The server's watcher reports a task that panicked with the panic's
  message, and is now tested for both a panicking task and a stopped RIB.
- The daemon sends panics to its log, with their location, instead of
  leaving them to the default report on stderr.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 23:24:00 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 41090966e4 Stop treating a dead RIB as an empty one 
The RIB handles swallowed every failure of their actor: an UPDATE applied
to a stopped Adj-RIB-In vanished, and a snapshot of it came back empty.
Sessions stayed up, peers' routes went nowhere, and the status page showed
a healthy router with no routes. An actor stops only by panicking, so this
is the state the server would be in after a bug.

- Every operation on AdjRibIn and LocalRib returns Result<_, RibStopped>.
  LocalRibError is replaced by it.
- The server waits on both actors, the advertiser and the dialers. If one
  stops it sends every peer Cease / Administrative Shutdown and returns an
  error, so the process exits and can be restarted. Before, the tasks'
  handles were held and never looked at.
- A session that cannot store an UPDATE ends with the same Cease.
- The HTTP API answers 503 with the reason for anything that needs a table
  it cannot read, including /metricz: a scrape that fails is noticed, last
  known sizes are not.

Also:
- The routes of a session are now removed even if its task is torn down
  without running its cleanup (a panic), instead of staying in the table
  as routes of a peer that is gone.
- Table sizes for the status page and metrics are asked for as a count
  instead of copying the whole table to take its length.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 23:18:59 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 6197a7ad74 Handle errors in UPDATE messages as RFC 7606 says, not by resetting 
Any malformed UPDATE used to reset the session, withdrawing every route
learned over it. One bad attribute, typically originated ASes away and
passed on by a peer that did not look at it, could take down a session
carrying a full table.

UpdateMessage::decode now applies RFC 7606 and returns the UPDATE as it is
to be acted on, plus a report of what was wrong:

- Treat-as-withdraw: a malformed or misflagged attribute, a missing
  mandatory one, or an attribute that overruns the attribute list. The
  routes the UPDATE announced (in the body and in MP_REACH_NLRI) are
  withdrawn.
- Attribute discard: malformed ATOMIC_AGGREGATE, AGGREGATOR, AS4_PATH and
  AS4_AGGREGATOR; an attribute sent twice (the first is kept); LOCAL_PREF,
  ORIGINATOR_ID and CLUSTER_LIST from an external peer; AS4_PATH between
  four-octet speakers.
- Session reset remains where the prefixes are in doubt: the lengths that
  frame the message, an unreadable prefix, a malformed or repeated
  MP_REACH_NLRI or MP_UNREACH_NLRI, an unrecognised well-known attribute.
  Each now sends its own subcode instead of always Malformed Attribute
  List.

The session logs each handled error and counts it in
bgp_update_errors_total{handling}. The parser context says whether the
peer is internal (replacing a field nothing read).

Stricter than before, because the checks did not exist:
- An announcement must carry ORIGIN and AS_PATH, NEXT_HOP for routes in the
  body, and LOCAL_PREF from an internal peer.
- An AS_PATH segment must be an AS_SET or AS_SEQUENCE and not empty.
- An attribute value must be used up entirely by its parser; a NEXT_HOP of
  six octets, say, was accepted before.

Bugs found on the way: we sent ATOMIC_AGGREGATE flagged optional and
AGGREGATOR flagged non-transitive. The flags now come from one table used
for both writing and checking.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 23:12:33 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 39632e3c38 Property tests and fuzz targets for the parsers, and the bugs they found 
Property tests (run by `cargo test`):
- crates/packet/tests/properties.rs: every kind of message, generated,
  must be written, be as long as wire_len says, and read back equal; and
  noise, noise behind a valid header, and valid messages with bytes damaged
  must never do more than return an error. Anything the decoder accepts
  must be showable and survive being written and read again.
- IpPrefix: text round trip, noise as text, `contains` against a bitwise
  model.
- The session's framing: a stream of messages yields the same messages
  however it is cut into reads.

Fuzz targets (fuzz/, cargo-fuzz on nightly, a minute each in CI): raw
messages, message bodies behind a valid header, prefix text, the
configuration file.

Bugs found, each with a named regression test:
- A prefix longer than its address family (e.g. an IPv4 NLRI of length 99)
  was accepted, and showing it panicked. The session logs every route, so
  one UPDATE from a peer ended the session task. Such a prefix is now
  malformed, and showing a prefix cannot fail.
- Bits past a prefix's length were kept, so the same prefix sent with stray
  bits was a different key in the RIB. Prefixes now have one form.
- An opaque extended community was read as seven octets instead of eight,
  putting every community after it out of step.
- AGGREGATOR could be neither read from nor written to a peer with
  two-octet AS numbers.
- The Outbound Route Filter and Long-Lived Graceful Restart capabilities
  were written differently from how they are read.
- A capability value or unknown attribute too long for its length field
  was written with the length wrapped around; it is now an error, or for
  the attribute the two-octet length.

Also: ROUTE-REFRESH carries its AFI and SAFI instead of being modelled as
an empty message, and one with a body of the wrong length is refused.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 22:54:37 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 b0762874e0 Make address families open values; keep "supported" a separate list 
AFI and SAFI were closed enums, so a family without a variant could not be
represented at all, and the set of negotiated families was a struct with
one bool per family. Supporting another family meant touching both.

- `Afi`, `Safi` and `AfiSafi` in bgp-primitives are newtypes over the wire
  numbers, with constants and names for the common families. Capabilities
  and MP_REACH/MP_UNREACH use them, so any family a peer names is read as
  what it is. The special case that turned a capability for an unlisted
  family into an unknown capability is gone.
- `FamilySet` is a set of `AfiSafi`. What we can exchange routes for is the
  explicit `family::SUPPORTED` list; only those are offered in our OPEN.
- `AddressFamilyId` stays as the IP version of a prefix.
- NLRI is still a list of IP prefixes. MP_REACH/MP_UNREACH for a family
  whose NLRI is something else (labelled, VPN, EVPN, ...) is refused
  rather than misread as prefixes, which is what happened before for the
  SAFIs the old enum listed.

Bug found on the way: the reserved octet in MP_REACH_NLRI was skipped only
for IPv6, so an IPv4 MP_REACH_NLRI was parsed with an extra 0.0.0.0/0
route that the peer never sent.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 22:38:40 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 22d0099a46 Negotiate address families and AS number width instead of assuming them 
The OPEN used to offer IPv4 and IPv6 unicast unconditionally, and what the
peer offered was ignored apart from its four-octet AS number.

Address families (RFC 4760):
- A peer's `address_families` in the configuration says which to offer it;
  all of them by default.
- A session carries the families both OPENs offered. A peer that sends no
  Multiprotocol capability is a plain RFC 4271 speaker: IPv4 unicast.
- Routes of a family that was not negotiated are not sent, and are ignored
  when received; the rest of the UPDATE is applied and the session stays up.
- A session with no family in common is refused with Unsupported
  Capability, naming the capabilities we needed (RFC 5492).
- The negotiated families are shown on /statusz and in /api/peers.

Bugs found on the way:
- A capability naming an address family we have no constant for (EVPN,
  VPNv4, ...) failed the whole OPEN, so such a peer could never establish.
  It is now kept as an unknown capability and ignored.
- An UPDATE could not be encoded for a peer without four-octet AS support,
  which ended the session on the first route. AS_PATH is now written two
  octets wide with AS_TRANS where needed, and AS4_PATH carries the real
  numbers (RFC 6793).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 22:22:54 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 a4a180e948 Make the session state machine explicit and free of I/O 
A session was one function that walked OpenSent, OpenConfirm and
Established with the socket, the timers, collision detection, the registry
and the RIB all in the same loop. There was no state or event type, and no
way to test a transition without a TCP connection.

It is now two pieces:

- fsm.rs is the state machine of RFC 4271 section 8 for a connection. It
  takes an Event and returns the Actions to carry out. It owns no socket,
  timer or task. What the OPENs negotiated lives in the states that have
  it, so there is nothing to unwrap. Thirteen tests cover the handshake,
  hold-time negotiation, refused OPENs, messages out of turn, timer expiry,
  collisions and shutdown by feeding it events.
- connection.rs is the driver: it turns the socket, the timers and the
  outbound queue into events and carries the actions out.

Behaviour changes:

- A message that arrived in pieces was lost if a timer fired between the
  pieces: read_exact was used inside select!, which discards what it has
  read when another branch wins. Messages are now framed from a buffer
  that survives cancellation.
- A NOTIFICATION from the peer during the handshake is no longer answered
  with an FSM error; RFC 4271 says to close without replying.
- A session the peer ended is reported as 'peer sent NOTIFICATION: ...'
  with its error, where the dialer used to record 'session closed by peer'.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 21:59:55 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 16a4e3b640 Typed errors in the library crates; send the NOTIFICATION RFC 4271 assigns 
The library crates returned eyre reports, so a failure was a string and the
session code could not tell a truncated UPDATE from a bad header. Whatever
went wrong, the peer got an FSM error or nothing.

Each crate now has its own error types:

- bgp-primitives: PrefixError, UnknownAfi and UnknownSafi, and EncodeError
  for every to_wire.
- bgp-packet: Message::decode returns a DecodeError naming the NOTIFICATION
  code, subcode and data for the failure (header errors per section 6.1,
  OPEN per 6.2, UPDATE per 6.3). InvalidValue and CommunityError replace
  the remaining strings. from_wire is now a thin wrapper around decode.
- bgp-server: SessionError, ServerError and ConfigError. Every abnormal end
  of a session goes through one function that sends the NOTIFICATION the
  error maps to. The message length is checked against the 4096-octet
  maximum before the body is read.
- bgp-web: serve returns io::Error.

eyre remains in the binary and in tests.

Bugs found and fixed on the way:

- An OPEN with no optional parameters, which is valid, failed to parse.
- An unsupported BGP version was reported as an unknown optional parameter.
  It now gets Unsupported Version Number with the version we speak as data.
- A SAFI above 255 was truncated to its low octet and could match a known
  one.
- Prefix text with a length beyond the address size (10.0.0.0/99) was
  accepted.
- Message::to_wire truncated the body length to 16 bits before checking it.
- A peer hold time above 65535 seconds in the configuration wrapped.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 21:18:35 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 666471e38d Share path attributes between Adj-RIB-In entries 
Each received prefix kept its own copy of the UPDATE's whole attribute list,
and for multiprotocol routes that list included MP_REACH_NLRI with every
other prefix from the same UPDATE: n prefixes stored n copies of n prefixes.

An UPDATE is now turned into a ReceivedUpdate once. Its prefixes share one
attribute list, which leaves out MP_REACH_NLRI and MP_UNREACH_NLRI; the next
hop from MP_REACH_NLRI is kept on the entry. The RIB takes one command per
UPDATE instead of one per prefix, and its table is keyed by peer so that
dropping a peer's routes no longer scans every route.

The status API shows the multiprotocol next hop as an address instead of
its Debug form.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 21:08:14 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 2de0a14712 Add the Apache 2.0 licence text and CI 
Cargo.toml has declared Apache-2.0 without the licence being in the tree.

CI checks formatting, clippy and the tests on every push, with warnings as
errors. The workflow lives in .github/workflows, which Gitea Actions also
reads.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 21:01:26 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 b7c7514d3e Fix all clippy warnings and enforce lints across the workspace 
Lints are declared once under [workspace.lints] and every crate opts in:
clippy's default set, plus dbg!, todo! and unimplemented!, and unsafe code
is forbidden. Every crate now inherits its version, licence, authors and
repository from the workspace, so they no longer disagree (four were at
0.0.0).

The poptrie module is unfinished and unused; it says so and expects the
dead-code lint instead of warning on every build. Its construction method
is renamed from new, which took &mut self, to build.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 21:01:26 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 126368852a Parse the Multiple Labels capability instead of panicking 
The capability parser had a todo!() for capability code 8 (RFC 8277), and
the encoder two more. A peer advertising it took the session task down.
It is now parsed into its (AFI, SAFI, count) entries and round-trips.

Also carries clippy's mechanical fixes for this file.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 21:01:26 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 2ab6c1c38b Format the workspace with rustfmt 
Default settings. No change in behaviour; the tree was last formatted
before the lecture work and had drifted to a mix of line widths.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 20:58:12 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 d8e8924b4e Point the repository URL at ferrix 
The repository was renamed. Also drops workspace.package.name, which Cargo
ignores (a package name cannot be inherited) and warned about on every build.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 20:53:26 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 b7e5084f83 Tidy the remaining top-level files after the lecture 
The preview launcher pointed at the removed slides and editor state does
not belong in the tree. The Dockerfile and README no longer refer to the
lab.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 20:51:32 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 22b9eeacdc Count multiprotocol prefixes in the UPDATE log lines 
The 'Sending UPDATE' and 'Received UPDATE' lines counted only the IPv4
fields of the message body, so every IPv6 UPDATE logged announced=0.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 20:51:15 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 b800e3a322 Originate routes from the configuration; make the HTTP API read-only 
Routes were announced and withdrawn through an HTTP form, each tagged with a
free-text owner and checked against an allow-list and a per-owner limit.
That was the lab's interface, not a router's.

Routes now come from 'originate' entries in the configuration file and are
in the local RIB before any session starts. The local RIB holds one route
per prefix with no notion of owners or policy. local_route_policy is gone
from the configuration.

The HTTP API keeps its GET endpoints (status, config, peers, originated and
received routes, events, metrics). The announce, withdraw, wire-bytes and
well-known-community endpoints are removed, and the status page loses the
announce form, the community presets and the byte decoder. Community text
parsing moves from the web crate to bgp-packet so the configuration can use
it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 20:51:15 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 c37cfd029a Remove the lecture material and the ping monitor 
The slides, handout, lab router configs, RIPE objects and load test were
for the Network Security 2026 lecture and are kept on the netsec-2026 tag.

The ping monitor sniffed ICMPv6 echo requests so students could watch their
prefix being used. It has no place in a BGP speaker: it needs CAP_NET_RAW,
adds the only unsafe code in the tree and a libc dependency.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 20:45:24 +02:00
177 changed files with 17193 additions and 11668 deletions
Expand all files Collapse all files

No files matched your search

.claude/launch.json
-11
View File
@@ -1,11 +0,0 @@
{
"version": "0.0.1",
"configurations": [
{
"name": "slides",
"runtimeExecutable": "python3",
"runtimeArgs": ["slides/serve.py", "8765"],
"port": 8765
}
]
}
.github/workflows/ci.yml
+50
View File
@@ -0,0 +1,50 @@
# Runs on every push and pull request. Gitea Actions reads this file too
# (it falls back to .github/workflows), so one definition serves both.
name: ci
on:
push:
pull_request:
env:
CARGO_TERM_COLOR: always
# Warnings fail the build here, and only here: locally they stay warnings
# so that work in progress compiles.
RUSTFLAGS: -D warnings
jobs:
check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
with:
components: rustfmt, clippy
- name: Format
run: cargo fmt --all -- --check
- name: Lint
run: cargo clippy --workspace --all-targets -- -D warnings
- name: Test
run: cargo test --workspace
# A short run of each fuzz target: enough to notice a parser change that
# breaks on input the seeds are close to. Long runs are done by hand; see
# fuzz/README.md.
fuzz:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@nightly
- name: Install cargo-fuzz
run: cargo install cargo-fuzz --locked
- name: Fuzz
working-directory: fuzz
# cargo-fuzz sets its own flags; the workspace's -D warnings would
# replace them.
env:
RUSTFLAGS: ""
run: |
for target in message message_body prefix_text config; do
mkdir -p "corpus/$target"
cargo fuzz run "$target" "corpus/$target" "seeds/$target" -- -max_total_time=60
done
.gitignore
+1 -2
View File
@@ -1,5 +1,4 @@
/target
__pycache__/
slides/reveal.js/
slides/dist/
.DS_Store
.claude/
Cargo.lock
Generated
+252 -14
View File
@@ -2,6 +2,12 @@
# It is not intended for manual editing.
version = 4
[[package]]
name = "adler2"
version = "2.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "320119579fcad9c21884f5c4861d16174d0e06250625266f50fe6898340abefa"
[[package]]
name = "aho-corasick"
version = "1.1.4"
@@ -11,6 +17,12 @@ dependencies = [
"memchr",
]
[[package]]
name = "allocator-api2"
version = "0.2.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "683d7910e743518b0e34f1186f92494becacb047c7b6bf616c96772180fef923"
[[package]]
name = "anstream"
version = "0.6.21"
@@ -156,9 +168,26 @@ dependencies = [
"tracing",
]
[[package]]
name = "bgp-bench"
version = "0.0.2"
dependencies = [
"bgp-integration-tests",
"bgp-packet",
"bgp-primitives",
"bgp-rib",
"bgp-server",
"bytes",
"flate2",
"stats_alloc",
"tokio",
"tracing",
"tracing-subscriber",
]
[[package]]
name = "bgp-integration-tests"
version = "0.0.0"
version = "0.0.2"
dependencies = [
"bgp-packet",
"bgp-primitives",
@@ -182,6 +211,7 @@ dependencies = [
"bytes",
"eyre",
"nom",
"proptest",
"serde",
"serde_repr",
"strum",
@@ -194,8 +224,8 @@ version = "0.0.2"
dependencies = [
"bitfield",
"bytes",
"eyre",
"nom",
"proptest",
"serde",
"serde_repr",
"strum",
@@ -204,10 +234,14 @@ dependencies = [
[[package]]
name = "bgp-rib"
version = "0.0.0"
version = "0.0.2"
dependencies = [
"bgp-packet",
"bgp-primitives",
"bytes",
"foldhash",
"hashbrown 0.17.1",
"proptest",
"serde",
"thiserror 2.0.12",
"tokio",
@@ -216,19 +250,19 @@ dependencies = [
[[package]]
name = "bgp-server"
version = "0.0.0"
version = "0.0.2"
dependencies = [
"bgp-packet",
"bgp-primitives",
"bgp-rib",
"bytes",
"clap",
"eyre",
"futures",
"libc",
"prometheus",
"proptest",
"protobuf",
"protobuf-codegen",
"socket2",
"thiserror 2.0.12",
"tokio",
"tracing",
@@ -237,25 +271,36 @@ dependencies = [
[[package]]
name = "bgp-web"
version = "0.0.0"
version = "0.0.2"
dependencies = [
"axum",
"bgp-packet",
"bgp-primitives",
"bgp-rib",
"bgp-server",
"bytes",
"eyre",
"futures",
"prometheus",
"reqwest",
"serde",
"serde_json",
"tokio",
"tokio-stream",
"tracing",
]
[[package]]
name = "bit-set"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "08807e080ed7f9d5433fa9b275196cfc35414f66a0c79d864dc51a0d825231a3"
dependencies = [
"bit-vec",
]
[[package]]
name = "bit-vec"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5e764a1d40d510daf35e07be9eb06e75770908c27d411ee6c92109c9840eaaf7"
[[package]]
name = "bitfield"
version = "0.19.0"
@@ -278,9 +323,9 @@ dependencies = [
[[package]]
name = "bitflags"
version = "2.9.0"
version = "2.13.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5c8214115b7bf84099f1309324e63141d4c5d7cc26862f97a0a857dbefe165bd"
checksum = "3ded4057c258ba199e2d26386d3af3780957ecaee6c4ef4041c6b4b8b97c0b06"
[[package]]
name = "bumpalo"
@@ -346,6 +391,15 @@ version = "1.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b05b61dc5112cbb17e4b6cd61790d9845d13888356391624cbe7e41efeac1e75"
[[package]]
name = "crc32fast"
version = "1.5.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "01a7799fd6b852db0e61728dde9a204c423b44d689dbd432522543614b490e78"
dependencies = [
"cfg-if",
]
[[package]]
name = "displaydoc"
version = "0.2.7"
@@ -395,12 +449,29 @@ version = "2.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "37909eebbb50d72f9059c3b6d82c0463f2ff062c9e95845c43a6c9c0355411be"
[[package]]
name = "flate2"
version = "1.1.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6e634e2e0ebac1ee034020da1ca582e17ffe4e0f5e985823721e168928136dcb"
dependencies = [
"crc32fast",
"miniz_oxide",
"zlib-rs",
]
[[package]]
name = "fnv"
version = "1.0.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3f9eec918d3f24069decb9af1554cad7c880e2da24a9afd88aca000531ab82c1"
[[package]]
name = "foldhash"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "77ce24cb58228fbb8aa041425bb1050850ac19177686ea6e0f41a70416f56fdb"
[[package]]
name = "form_urlencoded"
version = "1.2.2"
@@ -516,6 +587,17 @@ version = "0.16.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "841d1cc9bed7f9236f321df977030373f4a4163ae1a7dbfe1a51a2c1a51d9100"
[[package]]
name = "hashbrown"
version = "0.17.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed5909b6e89a2db4456e54cd5f673791d7eca6732202bbf2a9cc504fe2f9b84a"
dependencies = [
"allocator-api2",
"equivalent",
"foldhash",
]
[[package]]
name = "heck"
version = "0.5.0"
@@ -737,7 +819,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0ad4bb2b565bca0645f4d68c5c9af97fba094e9791da685bf83cb5f3ce74acf2"
dependencies = [
"equivalent",
"hashbrown",
"hashbrown 0.16.1",
]
[[package]]
@@ -842,6 +924,16 @@ version = "0.3.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6877bb514081ee2a7ff5ef9de3281f14a4dd4bceac4c09388074a6b5df8a139a"
[[package]]
name = "miniz_oxide"
version = "0.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b63fbc4a50860e98e7b2aa7804ded1db5cbc3aff9193adaff57a6931bf7c4b4c"
dependencies = [
"adler2",
"simd-adler32",
]
[[package]]
name = "mio"
version = "1.2.3"
@@ -871,6 +963,15 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "num-traits"
version = "0.2.19"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "071dfc062690e90b734c0b2273ce72ad0ffa95f0c74596bc250dcfd960262841"
dependencies = [
"autocfg",
]
[[package]]
name = "once_cell"
version = "1.21.1"
@@ -927,6 +1028,15 @@ dependencies = [
"zerovec",
]
[[package]]
name = "ppv-lite86"
version = "0.2.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "85eae3c4ed2f50dcfe72643da4befc30deadb458a9b590d720cde2f2b1e97da9"
dependencies = [
"zerocopy",
]
[[package]]
name = "proc-macro2"
version = "1.0.94"
@@ -951,6 +1061,25 @@ dependencies = [
"thiserror 2.0.12",
]
[[package]]
name = "proptest"
version = "1.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4b45fcc2344c680f5025fe57779faef368840d0bd1f42f216291f0dc4ace4744"
dependencies = [
"bit-set",
"bit-vec",
"bitflags",
"num-traits",
"rand",
"rand_chacha",
"rand_xorshift",
"regex-syntax",
"rusty-fork",
"tempfile",
"unarray",
]
[[package]]
name = "protobuf"
version = "3.7.2"
@@ -1002,6 +1131,12 @@ dependencies = [
"thiserror 1.0.69",
]
[[package]]
name = "quick-error"
version = "1.2.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a1d01941d82fa2ab50be1e79e6714289dd7cde78eba4c074bc5a4374f650dfe0"
[[package]]
name = "quote"
version = "1.0.47"
@@ -1017,6 +1152,44 @@ version = "5.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "69cdb34c158ceb288df11e18b4bd39de994f6657d83847bdffdbd7f346754b0f"
[[package]]
name = "rand"
version = "0.9.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b9ef1d0d795eb7d84685bca4f72f3649f064e6641543d3a8c415898726a57b41"
dependencies = [
"rand_chacha",
"rand_core",
]
[[package]]
name = "rand_chacha"
version = "0.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d3022b5f1df60f26e1ffddd6c66e8aa15de382ae63b3a0c1bfc0e4d3e3f325cb"
dependencies = [
"ppv-lite86",
"rand_core",
]
[[package]]
name = "rand_core"
version = "0.9.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "76afc826de14238e6e8c374ddcc1fa19e374fd8dd986b0d2af0d02377261d83c"
dependencies = [
"getrandom",
]
[[package]]
name = "rand_xorshift"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "513962919efc330f829edb2535844d1b912b0fbe2ca165d613e4e8788bb05a5a"
dependencies = [
"rand_core",
]
[[package]]
name = "redox_syscall"
version = "0.5.10"
@@ -1118,6 +1291,18 @@ version = "1.0.20"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "eded382c5f5f786b989652c49544c4877d9f015cc22e145a5ea8ea66c2921cd2"
[[package]]
name = "rusty-fork"
version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cc6bf79ff24e648f6da1f8d1f011e9cac26491b619e6b9280f2b47f1774e6ee2"
dependencies = [
"fnv",
"quick-error",
"tempfile",
"wait-timeout",
]
[[package]]
name = "ryu"
version = "1.0.23"
@@ -1225,6 +1410,12 @@ dependencies = [
"libc",
]
[[package]]
name = "simd-adler32"
version = "0.3.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3a219298ac11a56ea9a6d2120044824d6f01aeb034955e7af7bc16858527deea"
[[package]]
name = "slab"
version = "0.4.12"
@@ -1253,6 +1444,12 @@ version = "1.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6ce2be8dc25455e1f91df71bfa12ad37d7af1092ae736f3a6cd0e37bc7810596"
[[package]]
name = "stats_alloc"
version = "0.1.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5c0e04424e733e69714ca1bbb9204c1a57f09f5493439520f9f68c132ad25eec"
[[package]]
name = "strsim"
version = "0.11.1"
@@ -1562,6 +1759,12 @@ version = "0.2.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e421abadd41a4225275504ea4d6566923418b7f05506fbc9c0fe86ba7396114b"
[[package]]
name = "unarray"
version = "0.1.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "eaea85b334db583fe3274d12b4cd1880032beab409c0d774be044d4480ab9a94"
[[package]]
name = "unicode-ident"
version = "1.0.18"
@@ -1598,6 +1801,15 @@ version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ba73ea9cf16a25df0c8caa16c51acb937d5712a8429db78a3ee29d5dcacd3a65"
[[package]]
name = "wait-timeout"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "09ac3b126d3914f9849036f826e054cbabdc8519970b8998ddaf3b5bd3c65f11"
dependencies = [
"libc",
]
[[package]]
name = "want"
version = "0.3.1"
@@ -1822,6 +2034,26 @@ dependencies = [
"synstructure",
]
[[package]]
name = "zerocopy"
version = "0.8.59"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6df92bf3d9227be3d53173901ddbffac2babc27ae50f397776ffd6dc33f800cb"
dependencies = [
"zerocopy-derive",
]
[[package]]
name = "zerocopy-derive"
version = "0.8.59"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ac4f328cf2f05d084e496c3e9c3f33ed0a183656a16e1fcec4d464d8373aec82"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.100",
]
[[package]]
name = "zerofrom"
version = "0.1.8"
@@ -1876,6 +2108,12 @@ dependencies = [
"syn 3.0.5",
]
[[package]]
name = "zlib-rs"
version = "0.6.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b268e58e7c693d7c271f93ffc4ba3b380412554231c85bf61ca7af91042a4112"
[[package]]
name = "zmij"
version = "1.0.23"
Cargo.toml
+19 -3
View File
@@ -1,6 +1,8 @@
[workspace]
default-members = ["bin/bgp"]
members = ["bin/*", "crates/*"]
# Fuzz targets need a nightly toolchain; see fuzz/README.md.
exclude = ["fuzz"]
resolver = "3"
[workspace.package]
@@ -9,10 +11,20 @@ description = "A Border Gateway Protocol implementation"
edition = "2024"
homepage = "https://rayhaan.ch"
license = "Apache-2.0"
name = "bgp"
repository = "https://git.rayhaan.net/rayhaan/bgp"
repository = "https://git.rayhaan.net/rayhaan/ferrix"
version = "0.0.2"
# Every crate opts in with `[lints] workspace = true`. CI turns warnings into
# errors, so a lint added here is enforced.
[workspace.lints.rust]
unsafe_code = "forbid"
[workspace.lints.clippy]
all = { level = "warn", priority = -1 }
dbg_macro = "warn"
todo = "warn"
unimplemented = "warn"
[workspace.dependencies]
# --- Our crates ---
bgp-packet = { path = "crates/packet" }
@@ -21,6 +33,7 @@ bgp-rib = { path = "crates/rib" }
bgp-server = { path = "crates/server" }
bgp-web = { path = "crates/web" }
bgp-integration-tests = { path = "crates/integration-tests" }
bgp-bench = { path = "crates/bench" }
# --- General ---
bitfield = "0.19.0"
@@ -28,6 +41,7 @@ bytes = "1.10.1"
eyre = "0.6.12"
nom = "8.0.0"
serde = { version = "1.0.219", features = ["derive"] }
socket2 = "0.6"
serde_json = "1.0.140"
serde_repr = "0.1.20"
thiserror = "2.0.12"
@@ -36,11 +50,13 @@ strum = { version = "0.27.1", features = ["derive"] }
tracing = "0.1.44"
tracing-subscriber = { version = "0.3.22", features = ["env-filter"] }
prometheus = "0.14.0"
proptest = "1.7"
protobuf = { package = "protobuf", version = "3.7.2" }
protobuf-codegen = { package = "protobuf-codegen", version = "3.7.2" }
clap = { version = "4.5.53", features = ["derive"] }
foldhash = "0.2"
futures = "0.3"
libc = "0.2"
hashbrown = "0.17"
axum = "0.8.9"
tokio-stream = { version = "0.1.19", features = ["sync"] }
reqwest = { version = "0.13.5", default-features = false, features = ["json"] }
Dockerfile
+2 -1
View File
@@ -1,4 +1,5 @@
# Builds the ferrix_bgp daemon for the lab. See docs/lab/README.md.
# Builds the daemon into a small image. Mount a configuration file at
# /etc/ferrix/server.textpb, or pass another path as the first argument.
FROM rust:1.92-bookworm AS build
WORKDIR /src
COPY . .
LICENSE
+176
View File
@@ -0,0 +1,176 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
README.md
+33 -5
View File
@@ -1,8 +1,8 @@
# BGP
The `bgp` crate is an implementation of the Border Gateway Protocol, which is used to exchange routing and control plane information between Autonomous Systms.
The `bgp` crate is an implementation of the Border Gateway Protocol, which is used to exchange routing and control plane information between Autonomous Systems.
The aim of this project is to provide a modern, high performance, secure and programatic and customizable software package that can be useful to network operators, researchers, and people interested in Internet.
The aim of this project is to provide a modern, high performance, secure, programmatic and customizable software package that can be useful to network operators, researchers, and people interested in Internet.
## Current status.
@@ -22,8 +22,36 @@ We aim for the 1.0 release to be functional and stable to operate peering sessio
- ☐ Notification message
- ✅ Implement scaffolding required to start and configure a server
- ✅ Implement connection logic to establish sessions with peers (passive and active)
- ✅ Locally originated routes with communities, announced to all peers
- ✅ HTTP API, live status page and Prometheus metrics
- ✅ Routes originated from the configuration, with communities, announced to all peers
- ✅ Read-only HTTP API, status page and Prometheus metrics
- ☐ Implement an efficient data structure for storing prefixes (e.g. Poptrie)
- ☐ Filtering for peer RIB in / out
- ☐ Main / auxillary RIBs for exchanging routes between peers
- ☐ Main / auxiliary RIBs for exchanging routes between peers
## Development
CI runs these three on every push, and fails on any warning:
```
cargo fmt --all -- --check
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace
```
Lints are configured once, under `[workspace.lints]` in the root `Cargo.toml`.
`unsafe` code is forbidden throughout.
`cargo test` includes property tests: generated messages must survive being
written and read back, and no input, however damaged, may make a parser
panic. They take about a second; for a longer hunt:
```
PROPTEST_CASES=1000000 cargo test --release -p bgp-packet --test properties
```
Coverage-guided fuzz targets for the same parsers are in `fuzz/`; see
`fuzz/README.md`. CI runs each of them for a minute.
## License
Apache License 2.0; see `LICENSE`.
bin/bgp/Cargo.toml
+10 -5
View File
@@ -1,10 +1,12 @@
[package]
description = "A Border Gateway Protocol implementation"
edition.workspace = true
name = "bgp"
description = "A Border Gateway Protocol implementation"
authors.workspace = true
edition.workspace = true
homepage.workspace = true
license.workspace = true
name = "bgp"
version.workspace = true
license.workspace = true
repository.workspace = true
version.workspace = true
[dependencies]
bgp-server.workspace = true
@@ -14,3 +16,6 @@ eyre.workspace = true
protobuf.workspace = true
tokio.workspace = true
tracing.workspace = true
[lints]
workspace = true
bin/bgp/README.md
+21 -27
View File
@@ -1,54 +1,48 @@
# BGP
This crate provides the `bgp` binary which runs the Border Gateway Protocol
daemon together with its HTTP API.
This crate provides the `bgp` binary, which runs the Border Gateway Protocol
daemon together with its read-only HTTP status interface.
```
cargo run -- path/to/config.textpb
```
The configuration format is defined in `crates/server/proto/config.proto`;
`testdata/server.textpb` is a worked example. When `http_endpoint` is set the
binary also serves:
`testdata/server.textpb` is a worked example. The daemon is configured
entirely through that file: peers, and the routes it originates
(`originate { prefix: ... }`). There is no way to change either at runtime.
When `http_endpoint` is set the binary also serves:
| Path | Purpose |
|---|---|
| `GET /statusz` | Status page: peers, announce form, local and received routes, live event log |
| `GET /statusz` | Status page: configuration, peers and session state, originated and received routes, live event log |
| `GET /api/status` | Router ID, ASN, uptime and table sizes |
| `GET /api/peers` | Established sessions |
| `GET /api/routes/local` | Locally originated routes |
| `POST /api/routes/local` | Announce a route (JSON body, see below) |
| `DELETE /api/routes/local` | Withdraw a route (JSON body: `prefix`, optional `owner`) |
| `GET /api/routes/local/wire?prefix=…` | The exact UPDATE bytes sent to each peer |
| `GET /api/config` | The configuration file as read at startup |
| `GET /api/peers` | Configured peers and established sessions |
| `GET /api/routes/local` | Originated routes |
| `GET /api/routes/received` | Adj-RIB-In contents |
| `GET /api/communities/well-known` | Names accepted in place of numeric communities |
| `GET /api/events` | Server-sent events for session and route changes |
| `GET /metricz` | Prometheus metrics |
Announce body:
Everything under it is read-only and unauthenticated. Bind it to loopback or
put it behind a reverse proxy.
```json
{
"prefix": "10.100.1.0/24",
"owner": "alice",
"origin": "IGP",
"prepend": 2,
"med": 100,
"communities": ["65000:100", "NO_EXPORT"],
"large_communities": ["65000:1:2"]
}
```
## What it does today
Only `prefix` and `owner` are required. Routes are subject to the
`local_route_policy` in the configuration.
It establishes sessions (both dialling configured peers and accepting
connections, with RFC 4271 collision detection), announces its originated
routes to every peer with itself as next hop, and records what peers send in
an Adj-RIB-In. It does not select best paths, re-advertise received routes or
program a forwarding table, and it has no import or export policy.
## Trying it without a router
`fake_peer` is a minimal BGP speaker that connects to the daemon, announces a
prefix, prints every UPDATE it receives and, with `--echo`, re-announces each
received prefix back with its own ASN prepended (dropping NO_EXPORT routes).
That makes locally announced prefixes show up in the received table as if
they had travelled through a neighbouring AS:
That makes originated prefixes show up in the received table as if they had
travelled through a neighbouring AS:
```
cargo run -- testdata/server.textpb
bin/bgp/src/main.rs
+35 -18
View File
@@ -1,15 +1,13 @@
use std::sync::Arc;
use std::time::Duration;
use bgp_server::config::Config;
use bgp_server::ping::PingMonitor;
use bgp_server::proto::config::BgpConfig;
use bgp_server::server::BgpServerBuilder;
use bgp_server::server::{BgpServer, close_sessions};
use bgp_server::tracing::TracingArgs;
use bgp_web::{AppState, RouterInfo};
use clap::Parser;
use eyre::{Context, Result};
use tracing::{error, info, warn};
use tracing::{error, info};
#[derive(Parser, Debug)]
struct ServerArgs {
@@ -24,6 +22,7 @@ struct ServerArgs {
async fn main() -> Result<()> {
let args = ServerArgs::parse();
args.tracing.install()?;
log_panics();
let config_string =
std::fs::read_to_string(&args.config).wrap_err("Failed to open config file")?;
@@ -31,14 +30,14 @@ async fn main() -> Result<()> {
.wrap_err("Failed to parse config file")?;
let config = Config::from_proto(proto_config).wrap_err("Invalid configuration")?;
let server = BgpServerBuilder::new(config).bind().await?;
let pings = match &server.config().ping_monitor {
Some(prefixes) => PingMonitor::start(prefixes.clone(), server.local_rib.clone()),
None => PingMonitor::disabled(),
};
let server = BgpServer::bind(config).await?;
if let Some(http_endpoint) = server.config().http_endpoint {
// Bound here, so that an address that cannot be bound stops the
// daemon from starting, as it does for the BGP listeners.
let listener = tokio::net::TcpListener::bind(http_endpoint)
.await
.wrap_err_with(|| format!("Failed to bind HTTP listener on {http_endpoint}"))?;
let state = AppState {
info: Arc::new(
RouterInfo::new(
@@ -50,11 +49,10 @@ async fn main() -> Result<()> {
),
registry: server.registry.clone(),
local_rib: server.local_rib.clone(),
adj_rib_in: server.adj_rib_in.clone(),
pings,
rib: Arc::clone(&server.rib),
};
tokio::spawn(async move {
if let Err(e) = bgp_web::serve(state, http_endpoint).await {
if let Err(e) = bgp_web::serve_on(state, listener).await {
error!(err = %e, "HTTP server exited");
}
});
@@ -65,18 +63,37 @@ async fn main() -> Result<()> {
// makes them slower to take us back.
let registry = server.registry.clone();
tokio::select! {
result = server.run() => result,
result = server.run() => Ok(result?),
signal = shutdown_signal() => {
info!(signal, "Shutting down, closing BGP sessions");
let left = registry.shutdown(Duration::from_secs(2)).await;
if left > 0 {
warn!(sessions = left, "Some sessions did not close in time");
}
close_sessions(&registry).await;
Ok(())
}
}
}
/// Sends panics to the log, with where they happened. A panic is a bug, and
/// the default report goes to stderr in a format of its own, apart from
/// everything else the daemon says. The task that panicked still unwinds;
/// the server deals with what it leaves behind.
fn log_panics() {
std::panic::set_hook(Box::new(|info| {
let location = info
.location()
.map(|l| format!("{}:{}", l.file(), l.line()))
.unwrap_or_else(|| "an unknown location".into());
let message = info
.payload()
.downcast_ref::<&str>()
.map(|s| s.to_string())
.or_else(|| info.payload().downcast_ref::<String>().cloned())
.unwrap_or_else(|| "no message".into());
// Captured only if RUST_BACKTRACE is set.
let backtrace = std::backtrace::Backtrace::capture();
error!(%location, %message, %backtrace, "Panic: this is a bug");
}));
}
/// Resolves with the name of the first termination signal received.
async fn shutdown_signal() -> &'static str {
#[cfg(unix)]
crates/bench/Cargo.toml
+42
View File
@@ -0,0 +1,42 @@
[package]
name = "bgp-bench"
description = "Benchmarks at the scale of the Internet routing table"
authors.workspace = true
edition.workspace = true
homepage.workspace = true
license.workspace = true
repository.workspace = true
version.workspace = true
publish = false
[dependencies]
bgp-packet.workspace = true
bgp-primitives.workspace = true
bytes.workspace = true
flate2 = "1"
stats_alloc = { version = "0.1.10", optional = true }
[dev-dependencies]
bgp-integration-tests.workspace = true
bgp-rib.workspace = true
bgp-server.workspace = true
tokio.workspace = true
tracing.workspace = true
tracing-subscriber.workspace = true
[features]
# Counts every allocation, to report how much memory a table holds. The
# counters are shared by all threads and slow them down, so timings are
# taken without this and memory is measured in a run of its own.
count-memory = ["dep:stats_alloc"]
[[bench]]
name = "full_table"
harness = false
[[bench]]
name = "real_table"
harness = false
[lints]
workspace = true
crates/bench/benches/full_table.rs
+215
View File
@@ -0,0 +1,215 @@
//! What it costs to take in the Internet routing table, from the bytes on
//! the wire to a chosen best route.
//!
//! Run with `cargo bench -p bgp-bench --bench full_table`. Set
//! `BENCH_PEERS` (default 4) and `BENCH_SCALE_DOWN` (default 1; 10 makes the
//! table ten times smaller) to change the size. The table is synthetic (see
//! the crate documentation); `real_table` runs on a real one.
//!
//! This is a stopwatch around whole operations at full size, not a
//! statistical micro-benchmark: the questions are "how long until a full
//! table is in" and "how much memory does it take", and their answers are
//! seconds and gigabytes.
use std::hint::black_box;
use std::net::Ipv4Addr;
use std::sync::atomic::{AtomicUsize, Ordering};
use std::time::{Duration, Instant};
use bgp_bench::memory::{Region, describe};
use bgp_bench::{Prefixes, Shape, announcements, encoded, wire_context};
use bgp_integration_tests::test_peer::TestPeer;
use bgp_integration_tests::test_server::TestServer;
use bgp_packet::message::{Message, PathAttribute};
use bgp_primitives::ip_prefix::IpPrefix;
use bgp_rib::table::{PeerIndex, PeerInfo, RibTable};
use bytes::BytesMut;
fn env(name: &str, default: usize) -> usize {
std::env::var(name)
.ok()
.and_then(|v| v.parse().ok())
.unwrap_or(default)
}
fn mib(bytes: usize) -> f64 {
bytes as f64 / (1024.0 * 1024.0)
}
fn per_second(count: usize, elapsed: Duration) -> String {
let rate = count as f64 / elapsed.as_secs_f64();
if rate >= 1e6 {
format!("{:.2} M/s", rate / 1e6)
} else {
format!("{:.0} k/s", rate / 1e3)
}
}
fn row(what: &str, elapsed: Duration, detail: String) {
println!("{what:<44} {:>9.3} s {detail}", elapsed.as_secs_f64());
}
fn main() {
let peers = env("BENCH_PEERS", 4) as u32;
let shape = Shape::INTERNET.scaled_down(env("BENCH_SCALE_DOWN", 1));
let threads = std::thread::available_parallelism().map_or(1, |n| n.get());
println!(
"== Table: {} IPv4 + {} IPv6 prefixes, from each of {peers} peers",
shape.ipv4_prefixes, shape.ipv6_prefixes
);
println!(
"(IpPrefix is {} bytes, PathAttribute {})",
size_of::<IpPrefix>(),
size_of::<PathAttribute>()
);
let prefixes = Prefixes::generate(shape);
let wire: Vec<Vec<BytesMut>> = (1..=peers)
.map(|peer| encoded(&announcements(&prefixes, shape, peer)))
.collect();
let updates: usize = wire.iter().map(Vec::len).sum();
let routes = shape.prefixes() * peers as usize;
let wire_bytes: usize = wire.iter().flatten().map(BytesMut::len).sum();
println!(
"{updates} UPDATE messages, {:.1} prefixes each, {:.1} MiB on the wire, {routes} routes",
routes as f64 / updates as f64,
mib(wire_bytes),
);
println!("\n== Reading the messages (one thread)");
let ctx = wire_context();
let started = Instant::now();
for message in wire.iter().flatten() {
black_box(Message::decode(&ctx, message).expect("what we wrote is well formed"));
}
let elapsed = started.elapsed();
row(
"decode UPDATEs",
elapsed,
format!(
"{} routes, {} messages, {:.0} MiB/s",
per_second(routes, elapsed),
per_second(updates, elapsed),
mib(wire_bytes) / elapsed.as_secs_f64()
),
);
println!("\n== From bytes to best routes, without the network");
for threads in [1, threads] {
into_table(&wire, routes, threads);
}
println!("\n== End to end: {peers} peers send the table over TCP at once");
let runtime = tokio::runtime::Builder::new_multi_thread()
.enable_all()
.build()
.expect("a runtime");
runtime.block_on(async {
let server = TestServer::start(65000).await.expect("a server");
let mut senders = Vec::new();
for (index, messages) in wire.iter().enumerate() {
let peer = index as u32 + 1;
let mut session = TestPeer::connect(server.addr).await.expect("a connection");
session
.handshake(64512 + peer, Ipv4Addr::new(10, 0, 0, peer as u8), 90)
.await
.expect("a session");
let stream: Vec<u8> = messages.iter().flat_map(|m| m.iter().copied()).collect();
senders.push((session, stream));
}
let started = Instant::now();
let sending: Vec<_> = senders
.into_iter()
.map(|(mut session, stream)| {
tokio::spawn(async move {
session.send_raw(&stream).await.expect("the server reads");
// Held open until the table is in.
session
})
})
.collect();
while server.rib.len() < routes {
tokio::time::sleep(Duration::from_millis(2)).await;
}
let elapsed = started.elapsed();
row(
"sessions, state machine, table",
elapsed,
format!(
"{} routes; {} prefixes with a best route",
per_second(routes, elapsed),
server.rib.prefix_count()
),
);
for session in sending {
drop(session.await);
}
});
}
/// What a session does with each message, without the session: decode it,
/// find its attribute set, put its routes in the table, choose the best.
/// `threads` threads each take whole peers.
fn into_table(wire: &[Vec<BytesMut>], routes: usize, threads: usize) {
let region = Region::begin();
let table = RibTable::new(threads * 16);
let peers: Vec<PeerIndex> = (0..wire.len())
.map(|peer| {
table
.add_peer(PeerInfo {
asn: 64512 + peer as u32 + 1,
router_id: Ipv4Addr::from(peer as u32 + 1),
address: Ipv4Addr::new(192, 0, 2, peer as u8 + 1).into(),
ibgp: false,
})
.expect("room for the peer")
})
.collect();
let ctx = wire_context();
let next = AtomicUsize::new(0);
let started = Instant::now();
std::thread::scope(|scope| {
for _ in 0..threads {
scope.spawn(|| {
loop {
let at = next.fetch_add(1, Ordering::Relaxed);
let Some(messages) = wire.get(at) else {
break;
};
for message in messages {
let decoded =
Message::decode_with_report(&ctx, message).expect("well formed");
let Message::Update(update) = decoded.message else {
continue;
};
let wire = decoded.attributes_wire.as_deref().expect("kept");
if !update.announced_routes.is_empty() {
let attrs = table.attrs().intern_received(wire, None).expect("valid");
table.announce(peers[at], &attrs, &update.announced_routes);
}
for attribute in &update.path_attributes {
if let PathAttribute::MpReachNlri(mp) = attribute {
let attrs = table
.attrs()
.intern_received(wire, Some(&mp.next_hop))
.expect("valid");
table.announce(peers[at], &attrs, &mp.prefixes);
}
}
}
}
});
}
});
let elapsed = started.elapsed();
assert_eq!(table.len(), routes);
row(
&format!("decode, intern, insert, choose: {threads} thread(s)"),
elapsed,
format!(
"{} routes; {}",
per_second(routes, elapsed),
describe(region.held(), routes, "route")
),
);
}
crates/bench/benches/real_table.rs
+431
View File
@@ -0,0 +1,431 @@
//! The same questions as `full_table`, asked of a real table dump: every
//! route that the peers of a RIPE RIS route collector had announced to it.
//!
//! Needs a dump in `crates/bench/data/` (see the README there) or at the
//! path in `BENCH_MRT`. Without one it says so and does nothing, so that
//! `cargo bench` works on a fresh checkout. `BENCH_PEERS` limits the run to
//! the peers with the most routes (default: all of them).
use std::collections::HashSet;
use std::net::Ipv4Addr;
use std::path::PathBuf;
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering};
use std::time::{Duration, Instant};
use bgp_bench::memory::Region;
use bgp_bench::mrt::{Dump, parse_attributes};
use bgp_packet::message::{NlriNextHop, PathAttribute};
use bgp_primitives::ip_prefix::IpPrefix;
use bgp_rib::table::{Event, PeerIndex, PeerInfo, RibTable};
/// Bytes allocated and not freed since `region` began; zero when memory
/// is not being counted (see `bgp_bench::memory`).
fn retained(region: &Region) -> i64 {
region.held().unwrap_or(0)
}
fn mib(bytes: i64) -> f64 {
bytes as f64 / (1024.0 * 1024.0)
}
fn per_second(count: usize, elapsed: Duration) -> String {
format!("{:.2} M/s", count as f64 / elapsed.as_secs_f64() / 1e6)
}
fn row(what: &str, elapsed: Duration, detail: String) {
println!("{what:<44} {:>9.3} s {detail}", elapsed.as_secs_f64());
}
fn dump_path() -> Option<PathBuf> {
if let Ok(path) = std::env::var("BENCH_MRT") {
return Some(path.into());
}
let data = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("data");
let mut dumps: Vec<PathBuf> = std::fs::read_dir(data)
.ok()?
.filter_map(|entry| Some(entry.ok()?.path()))
.filter(|path| path.extension().is_some_and(|e| e == "gz"))
.collect();
dumps.sort();
dumps.pop()
}
type Attributes = (Arc<[PathAttribute]>, Option<NlriNextHop>);
fn main() {
if Region::begin().held().is_none() {
println!(
"Memory is not counted in this run: sizes print as 0. For sizes (and slower\ntimings) add --features count-memory.\n"
);
}
let Some(path) = dump_path() else {
println!("No table dump found; see crates/bench/data/README.md. Nothing to do.");
return;
};
println!("== Reading {}", path.display());
let started = Instant::now();
let dump = Dump::read(&path).expect("a readable dump");
println!(
"{} peers, {} routes, {} distinct attribute sets, read in {:.0} s",
dump.peers.len(),
dump.route_count(),
dump.attribute_sets.len(),
started.elapsed().as_secs_f64()
);
let limit = std::env::var("BENCH_PEERS")
.ok()
.and_then(|v| v.parse().ok())
.unwrap_or(dump.peers.len());
let peers = dump.largest_peers(limit);
let peers: Vec<usize> = peers
.into_iter()
.filter(|&peer| !dump.routes[peer].is_empty())
.collect();
describe(&dump, &peers);
// Parsed once, outside the timings: reading messages is measured by
// `full_table`, and here the attributes arrive already parsed.
let region = Region::begin();
let parsed: Vec<Option<Attributes>> = dump
.attribute_sets
.iter()
.map(|wire| parse_attributes(wire).map(|(a, next_hop)| (Arc::from(a), next_hop)))
.collect();
let parsed_bytes = retained(&region);
println!(
"\nattribute sets, parsed (Arc<[PathAttribute]>) {:.0} MiB = {:.0} bytes a set",
mib(parsed_bytes),
parsed_bytes as f64 / parsed.len() as f64
);
let unparsed = parsed.iter().filter(|a| a.is_none()).count();
if unparsed > 0 {
println!("{unparsed} attribute sets could not be parsed; their routes are left out");
}
let threads = std::thread::available_parallelism().map_or(1, |n| n.get());
sharded_table(&dump, &peers, &parsed, 1);
sharded_table(&dump, &peers, &parsed, threads);
followed_table(&dump, &peers, &parsed, threads);
}
/// What the table looks like, as far as it decides what a RIB costs.
fn describe(dump: &Dump, peers: &[usize]) {
let routes: usize = peers.iter().map(|&p| dump.routes[p].len()).sum();
let full = peers
.iter()
.filter(|&&p| dump.routes[p].len() > 800_000)
.count();
let prefixes: HashSet<IpPrefix> = peers
.iter()
.flat_map(|&p| dump.routes[p].iter().map(|(prefix, _)| *prefix))
.collect();
let mut sets_used = HashSet::new();
let mut sets_per_peer = 0usize;
for &peer in peers {
let own: HashSet<u32> = dump.routes[peer].iter().map(|(_, set)| *set).collect();
sets_per_peer += own.len();
sets_used.extend(own);
}
let attribute_bytes: usize = sets_used
.iter()
.map(|&set| dump.attribute_sets[set as usize].len())
.sum();
println!("\n== The table ({} peers used)", peers.len());
println!("routes {routes:>12}");
println!("peers with more than 800,000 routes {full:>12}");
println!(
"distinct prefixes {:>12}",
prefixes.len()
);
println!(
"routes per prefix {:>12.1}",
routes as f64 / prefixes.len() as f64
);
println!(
"distinct attribute sets {:>12}",
sets_used.len()
);
println!(
" if each peer's were kept apart {sets_per_peer:>12} ({:.2}x: sets shared between peers)",
sets_per_peer as f64 / sets_used.len() as f64
);
println!(
"routes per attribute set {:>12.1}",
routes as f64 / sets_used.len() as f64
);
println!(
"attribute set on the wire, mean {:>12.0} bytes",
attribute_bytes as f64 / sets_used.len() as f64
);
}
/// The sharded table, filled by `threads` threads. Each takes whole peers,
/// as sessions would, and for every UPDATE interns the attributes and then
/// announces the prefixes: everything between a decoded message and a
/// chosen best route.
fn sharded_table(dump: &Dump, peers: &[usize], parsed: &[Option<Attributes>], threads: usize) {
println!("\n== The sharded table, {threads} thread(s)");
let updates = grouped(dump, peers, parsed);
let routes: usize = updates
.iter()
.flatten()
.map(|(_, prefixes)| prefixes.len())
.sum();
let region = Region::begin();
let table = Arc::new(RibTable::new(threads * 16));
let indices = add_peers(&table, dump, peers);
let started = Instant::now();
let best_changes = load(&table, dump, &updates, &indices, threads);
let elapsed = started.elapsed();
assert_eq!(table.len(), routes);
let held = retained(&region);
row(
"intern attributes, insert, choose best",
elapsed,
format!(
"{} routes; {:.0} MiB = {:.0} bytes a route, attribute sets included",
per_second(routes, elapsed),
mib(held),
held as f64 / routes as f64
),
);
println!(
" {} prefixes, {} attribute sets, best route changed {} times",
table.prefix_count(),
table.attrs().len(),
best_changes
);
let started = Instant::now();
let mut best = 0usize;
table.for_each_best(|prefix, _, _| {
best += usize::from(std::hint::black_box(prefix).length() > 0)
});
row(
"read every best route",
started.elapsed(),
format!("{best} of them, in place, table not stopped"),
);
// Reading has to look at each route for the timing to mean anything.
let started = Instant::now();
let mut all = 0usize;
let mut hops = 0u64;
table.for_each_route(|route| {
all += 1;
hops += u64::from(route.attrs.key().as_path_len);
});
row(
"read every route and its attributes",
started.elapsed(),
format!(
"{all} of them, mean AS path {:.1}",
hops as f64 / all as f64
),
);
let started = Instant::now();
let outcome = table.remove_peer(indices[0]);
let elapsed = started.elapsed();
row(
"remove one peer (session down)",
elapsed,
format!(
"{} routes; best route changed for {} prefixes",
per_second(outcome.removed, elapsed),
outcome.best_changed
),
);
let started = Instant::now();
let swept = table.attrs().sweep();
row(
"drop attribute sets no longer used",
started.elapsed(),
format!("{swept} dropped"),
);
}
/// A peer's routes grouped as UPDATEs: the attribute set and the prefixes
/// announced with it.
type Updates = Vec<Vec<(u32, Vec<IpPrefix>)>>;
fn grouped(dump: &Dump, peers: &[usize], parsed: &[Option<Attributes>]) -> Updates {
peers
.iter()
.map(|&peer| {
let mut routes = dump.routes[peer].clone();
routes.sort_by_key(|(_, set)| *set);
routes
.chunk_by(|a, b| a.1 == b.1)
.filter(|group| parsed[group[0].1 as usize].is_some())
.map(|group| {
(
group[0].1,
group.iter().map(|(prefix, _)| *prefix).collect(),
)
})
.collect()
})
.collect()
}
fn add_peers(table: &RibTable, dump: &Dump, peers: &[usize]) -> Vec<PeerIndex> {
peers
.iter()
.map(|&peer| {
table
.add_peer(PeerInfo {
asn: dump.peers[peer].asn,
router_id: Ipv4Addr::from(peer as u32 + 1),
address: dump.peers[peer].address,
ibgp: false,
})
.expect("room for the peer")
})
.collect()
}
/// Announces everything in `updates` on `threads` threads, each taking
/// whole peers. Returns how many times a best route changed.
fn load(
table: &RibTable,
dump: &Dump,
updates: &Updates,
indices: &[PeerIndex],
threads: usize,
) -> usize {
let next = AtomicUsize::new(0);
let best_changes = AtomicUsize::new(0);
std::thread::scope(|scope| {
for _ in 0..threads {
scope.spawn(|| {
let mut changes = 0;
loop {
let at = next.fetch_add(1, Ordering::Relaxed);
let Some(updates) = updates.get(at) else {
break;
};
for (set, prefixes) in updates {
// From the bytes, as a dump has them and as a
// session could hand them over without parsing.
let Some(attrs) = table
.attrs()
.intern_wire(&dump.attribute_sets[*set as usize])
else {
continue;
};
changes += table.announce(indices[at], &attrs, prefixes).best_changed;
}
}
best_changes.fetch_add(changes, Ordering::Relaxed);
});
}
});
best_changes.load(Ordering::Relaxed)
}
/// The same load with someone following the best routes through the
/// journal: once reading as fast as it can while the table fills, once not
/// reading at all until the table is full.
fn followed_table(dump: &Dump, peers: &[usize], parsed: &[Option<Attributes>], threads: usize) {
println!("\n== The sharded table with a follower, {threads} thread(s)");
let updates = grouped(dump, peers, parsed);
let routes: usize = updates
.iter()
.flatten()
.map(|(_, prefixes)| prefixes.len())
.sum();
let count = |counts: &mut (usize, usize, usize), event: Event<'_>| match event {
Event::Best { attrs, .. } => {
// Look at the route, as a follower would.
counts.0 += usize::from(attrs.key().as_path_len > 0);
}
Event::Withdrawn { .. } => counts.1 += 1,
Event::Reset | Event::Synced => counts.2 += 1,
// Not given to a follower of the best routes.
Event::Routes { .. } => {}
};
// Reading while the table fills.
let table = Arc::new(RibTable::new(threads * 16));
let indices = add_peers(&table, dump, peers);
let done = AtomicBool::new(false);
let started = Instant::now();
let (best_changes, counts, longest, after_load) = std::thread::scope(|scope| {
let follower = scope.spawn(|| {
let mut reader = table.reader();
let mut counts = (0, 0, 0);
let mut longest = 0;
while !done.load(Ordering::Acquire) {
reader.poll(4096, |event| count(&mut counts, event));
longest = longest.max(table.journal_len());
}
let finishing = Instant::now();
while !reader
.poll(4096, |event| count(&mut counts, event))
.caught_up
{}
(counts, longest, finishing.elapsed())
});
let best_changes = load(&table, dump, &updates, &indices, threads);
done.store(true, Ordering::Release);
let (counts, longest, after_load) = follower.join().expect("the follower runs");
(best_changes, counts, longest, after_load)
});
let elapsed = started.elapsed();
assert_eq!(table.len(), routes);
row(
"load, follower reading throughout",
elapsed,
format!(
"{} routes; caught up {:.3} s after the last route",
per_second(routes, elapsed),
after_load.as_secs_f64()
),
);
println!(
" best route changed {best_changes} times; the follower was given {} events for it; \n journal at most {longest} entries for {} prefixes",
counts.0 + counts.1,
table.prefix_count()
);
drop(table);
// Not reading: the journal has to stay small on its own account.
let table = Arc::new(RibTable::new(threads * 16));
let indices = add_peers(&table, dump, peers);
let mut reader = table.reader();
let mut counts = (0, 0, 0);
reader.poll(usize::MAX, |event| count(&mut counts, event));
let started = Instant::now();
load(&table, dump, &updates, &indices, threads);
let elapsed = started.elapsed();
row(
"load, follower not reading",
elapsed,
format!(
"{} routes; journal holds {} entries for {} prefixes",
per_second(routes, elapsed),
table.journal_len(),
table.prefix_count()
),
);
let started = Instant::now();
let mut counts = (0, 0, 0);
while !reader
.poll(4096, |event| count(&mut counts, event))
.caught_up
{}
let elapsed = started.elapsed();
row(
"the follower then catches up",
elapsed,
format!(
"{} events, {} of them a reset",
counts.0 + counts.1,
counts.2
),
);
}
crates/bench/data/.gitignore
+4
View File
@@ -0,0 +1,4 @@
# Table dumps downloaded for benchmarks; see README.md.
*
!.gitignore
!README.md
crates/bench/data/README.md
+18
View File
@@ -0,0 +1,18 @@
# Table dumps for the benchmarks
`cargo bench -p bgp-bench --bench real_table` runs on a real routing table:
a dump of every route that the peers of a route collector had announced to
it. The dumps are large and are not checked in. Fetch one into this
directory:
```bash
curl -O https://data.ris.ripe.net/rrc03/2026.09/bview.20260915.0000.gz
```
That is RIPE RIS collector `rrc03`, which peers at AMS-IX: about 340 MB,
3.4 GB uncompressed. Any `bview` file from <https://data.ris.ripe.net/> or
a RouteViews RIB works; the most recent `.gz` here is used, or the one named
by `BENCH_MRT`.
The benchmark needs memory in proportion to the dump. `BENCH_PEERS=8`
limits it to the eight peers with the most routes.
crates/bench/src/lib.rs
+285
View File
@@ -0,0 +1,285 @@
//! A synthetic copy of the Internet routing table, as the UPDATE messages a
//! peer would send to announce it.
//!
//! It is made up, not captured, so that the benchmarks need no data files
//! and give the same input every run. It is shaped like the real thing where
//! that matters for cost:
//!
//! - about a million IPv4 prefixes, most of them /24s, and about a quarter
//! of a million IPv6 prefixes, most of them /48s;
//! - prefixes that share their attributes are announced together, a handful
//! to an UPDATE, as routers do;
//! - AS paths a few hops long, and communities on most routes;
//! - several peers announce the same prefixes, each with attributes of its
//! own.
pub mod memory;
pub mod mrt;
use std::collections::HashSet;
use std::net::{Ipv4Addr, Ipv6Addr};
use bgp_packet::message::{
AsPathAttribute, AsPathSegment, CommunitiesPathAttribute, LargeCommunitiesPathAttribute,
LargeCommunity, Message, MpReachNlriPathAttribute, MultiExitDiscPathAttribute,
NextHopPathAttribute, NlriNextHop, OriginPathAttribute, PathAttribute, UpdateMessage,
};
use bgp_primitives::constants::{AfiSafi, IpVersion};
use bgp_primitives::ip_prefix::{IpPrefix, Nlri};
use bgp_primitives::parser::ParserContext;
use bytes::BytesMut;
/// How big a table to make.
#[derive(Debug, Clone, Copy)]
pub struct Shape {
pub ipv4_prefixes: usize,
pub ipv6_prefixes: usize,
/// How many prefixes share one set of attributes, on average.
pub prefixes_per_attribute_set: usize,
}
impl Shape {
/// Roughly the default-free zone in 2026.
pub const INTERNET: Self = Self {
ipv4_prefixes: 1_000_000,
ipv6_prefixes: 230_000,
prefixes_per_attribute_set: 6,
};
pub fn prefixes(&self) -> usize {
self.ipv4_prefixes + self.ipv6_prefixes
}
/// The same shape, `factor` times smaller.
pub fn scaled_down(&self, factor: usize) -> Self {
Self {
ipv4_prefixes: self.ipv4_prefixes / factor,
ipv6_prefixes: self.ipv6_prefixes / factor,
..*self
}
}
}
/// A small deterministic generator (SplitMix64); no need for a dependency.
pub struct Rng(u64);
impl Rng {
pub fn new(seed: u64) -> Self {
Self(seed)
}
pub fn next_u64(&mut self) -> u64 {
self.0 = self.0.wrapping_add(0x9e37_79b9_7f4a_7c15);
let mut z = self.0;
z = (z ^ (z >> 30)).wrapping_mul(0xbf58_476d_1ce4_e5b9);
z = (z ^ (z >> 27)).wrapping_mul(0x94d0_49bb_1331_11eb);
z ^ (z >> 31)
}
/// A number below `bound`.
pub fn below(&mut self, bound: u64) -> u64 {
self.next_u64() % bound
}
/// An entry of `weighted`, picked with probability proportional to its
/// weight.
fn pick<T: Copy>(&mut self, weighted: &[(T, u64)]) -> T {
let total: u64 = weighted.iter().map(|(_, w)| w).sum();
let mut at = self.below(total);
for (value, weight) in weighted {
if at < *weight {
return *value;
}
at -= weight;
}
weighted[weighted.len() - 1].0
}
}
/// The prefixes of the table. Every peer announces these same prefixes.
pub struct Prefixes {
pub ipv4: Vec<IpPrefix>,
pub ipv6: Vec<IpPrefix>,
}
impl Prefixes {
pub fn generate(shape: Shape) -> Self {
let mut rng = Rng::new(0x1234_5678);
// Shares of prefix lengths, close to what the table looks like.
let v4_lengths: [(u8, u64); 9] = [
(24, 60),
(23, 10),
(22, 11),
(21, 5),
(20, 4),
(19, 3),
(18, 2),
(16, 4),
(15, 1),
];
let v6_lengths: [(u8, u64); 7] = [
(48, 48),
(32, 14),
(40, 10),
(44, 10),
(36, 6),
(29, 6),
(47, 6),
];
let mut seen = HashSet::new();
let mut ipv4 = Vec::with_capacity(shape.ipv4_prefixes);
while ipv4.len() < shape.ipv4_prefixes {
let length = rng.pick(&v4_lengths);
// Unicast space, 1.0.0.0 to 223.255.255.255.
let address = 0x0100_0000 + (rng.next_u64() as u32) % 0xdf00_0000;
let prefix = IpPrefix::new(IpVersion::Ipv4, address.to_be_bytes(), length)
.expect("four bytes cover any IPv4 prefix");
if seen.insert(prefix) {
ipv4.push(prefix);
}
}
let mut ipv6 = Vec::with_capacity(shape.ipv6_prefixes);
while ipv6.len() < shape.ipv6_prefixes {
let length = rng.pick(&v6_lengths);
// Global unicast, 2000::/3.
let address = (0x2000_u128 << 112) | (u128::from(rng.next_u64()) << 61 >> 3);
let prefix = IpPrefix::new(IpVersion::Ipv6, address.to_be_bytes(), length)
.expect("sixteen bytes cover any IPv6 prefix");
if seen.insert(prefix) {
ipv6.push(prefix);
}
}
Self { ipv4, ipv6 }
}
}
/// The UPDATE messages with which the peer numbered `peer` announces the
/// whole table. The peer's AS is `64512 + peer`.
pub fn announcements(prefixes: &Prefixes, shape: Shape, peer: u32) -> Vec<UpdateMessage> {
let mut rng = Rng::new(0x9999 + u64::from(peer));
let peer_asn = 64512 + peer;
let attributes = |rng: &mut Rng| {
let hops = 2 + rng.below(4) as usize;
let mut path = vec![peer_asn];
path.extend((0..hops).map(|_| 1 + rng.below(400_000) as u32));
let mut attributes = vec![
PathAttribute::Origin(OriginPathAttribute::IGP),
PathAttribute::AsPath(AsPathAttribute {
segments: vec![AsPathSegment {
ordered: true,
path,
}],
}),
];
let mut later = Vec::new();
if rng.below(4) == 0 {
later.push(PathAttribute::MultiExitDisc(MultiExitDiscPathAttribute(
rng.below(1000) as u32,
)));
}
let communities = rng.below(7) as usize;
if communities > 0 {
later.push(PathAttribute::Communities(CommunitiesPathAttribute(
(0..communities)
.map(|_| (rng.below(65000) as u16, rng.below(1000) as u16))
.collect(),
)));
}
let large = if rng.below(5) == 0 {
Some(PathAttribute::LargeCommunities(
LargeCommunitiesPathAttribute {
communities: vec![LargeCommunity {
global_admin: peer_asn,
data_1: rng.below(100) as u32,
data_2: rng.below(100) as u32,
}],
},
))
} else {
None
};
(std::mem::take(&mut attributes), later, large)
};
let mut updates = Vec::new();
// Prefixes that share attributes travel together. A set's prefixes are a
// run of the (randomly ordered) list, a few long.
let mut grouped = |list: &[IpPrefix], rng: &mut Rng, ipv6: bool| {
let mut rest = list;
while !rest.is_empty() {
let size = (1 + rng.below(2 * shape.prefixes_per_attribute_set as u64 - 1) as usize)
.min(rest.len());
let (group, after) = rest.split_at(size);
rest = after;
let (mut path_attributes, later, large) = attributes(rng);
let update = if ipv6 {
// In order of type code: MED and COMMUNITIES come before
// MP_REACH_NLRI.
path_attributes.extend(later);
path_attributes.push(PathAttribute::MpReachNlri(MpReachNlriPathAttribute {
family: AfiSafi::IPV6_UNICAST,
next_hop: NlriNextHop::Ipv6(Ipv6Addr::new(
0x2001,
0xdb8,
0,
0,
0,
0,
0,
peer as u16,
)),
prefixes: Nlri::plain(group.iter().copied()),
}));
path_attributes.extend(large);
UpdateMessage {
withdrawn_routes: vec![],
path_attributes,
announced_routes: vec![],
}
} else {
path_attributes.push(PathAttribute::NextHop(NextHopPathAttribute(Ipv4Addr::new(
192, 0, 2, peer as u8,
))));
path_attributes.extend(later);
path_attributes.extend(large);
UpdateMessage {
withdrawn_routes: vec![],
path_attributes,
announced_routes: Nlri::plain(group.iter().copied()),
}
};
updates.push(update);
}
};
grouped(&prefixes.ipv4, &mut rng, false);
grouped(&prefixes.ipv6, &mut rng, true);
updates
}
/// `updates` as they are on the wire, on a session with four-octet AS
/// numbers.
pub fn encoded(updates: &[UpdateMessage]) -> Vec<BytesMut> {
let ctx = wire_context();
updates
.iter()
.map(|update| {
let mut wire = BytesMut::new();
Message::Update(update.clone())
.to_wire(&ctx, &mut wire)
.expect("a few prefixes fit one message");
wire
})
.collect()
}
/// The context of the sessions the benchmarks pretend to have: an external
/// peer with four-octet AS numbers.
pub fn wire_context() -> ParserContext {
ParserContext {
four_octet_asn: true,
ibgp: false,
add_path: Vec::new(),
}
}
crates/bench/src/memory.rs
+60
View File
@@ -0,0 +1,60 @@
//! Counting the memory a benchmark holds.
//!
//! With the `count-memory` feature every allocation is counted, and
//! [`Region::held`] says how many bytes were allocated and not freed since
//! the region began. Without it nothing is counted and the answer is
//! `None`: the counters are shared by all threads, and counting slows down
//! exactly the code whose speed is being measured.
#[cfg(feature = "count-memory")]
mod counting {
use std::alloc::System;
use stats_alloc::{INSTRUMENTED_SYSTEM, StatsAlloc};
#[global_allocator]
static GLOBAL: &StatsAlloc<System> = &INSTRUMENTED_SYSTEM;
pub struct Region(stats_alloc::Region<'static, System>);
impl Region {
pub fn begin() -> Self {
Self(stats_alloc::Region::new(GLOBAL))
}
pub fn held(&self) -> Option<i64> {
let stats = self.0.change();
Some(stats.bytes_allocated as i64 - stats.bytes_deallocated as i64)
}
}
}
#[cfg(not(feature = "count-memory"))]
mod counting {
pub struct Region;
impl Region {
pub fn begin() -> Self {
Self
}
pub fn held(&self) -> Option<i64> {
None
}
}
}
pub use counting::Region;
/// "`bytes` MiB = n bytes a `unit`", or a note that memory is not being
/// counted.
pub fn describe(held: Option<i64>, count: usize, unit: &str) -> String {
match held {
Some(bytes) => format!(
"{:.0} MiB = {:.0} bytes a {unit}",
bytes as f64 / (1024.0 * 1024.0),
bytes as f64 / count as f64
),
None => "memory not counted (run with --features count-memory)".into(),
}
}
crates/bench/src/mrt.rs
+237
View File
@@ -0,0 +1,237 @@
//! Reads a table dump in MRT format (RFC 6396, TABLE_DUMP_V2), as published
//! by RIPE RIS and RouteViews: the routes that every peer of a route
//! collector had announced to it at one moment.
//!
//! Only what the benchmarks need is read: the IPv4 and IPv6 unicast tables.
use std::collections::HashMap;
use std::fs::File;
use std::io::{self, BufReader, Read};
use std::net::{IpAddr, Ipv4Addr, Ipv6Addr};
use std::path::Path;
use bgp_packet::message::{NlriNextHop, PathAttribute};
use bgp_primitives::constants::IpVersion;
use bgp_primitives::ip_prefix::IpPrefix;
use flate2::read::MultiGzDecoder;
use crate::wire_context;
const TABLE_DUMP_V2: u16 = 13;
const PEER_INDEX_TABLE: u16 = 1;
const RIB_IPV4_UNICAST: u16 = 2;
const RIB_IPV6_UNICAST: u16 = 4;
const MP_REACH_NLRI: u8 = 14;
/// A peer of the collector.
#[derive(Debug, Clone)]
pub struct DumpPeer {
pub address: IpAddr,
pub asn: u32,
}
/// The contents of a table dump.
pub struct Dump {
/// When the dump was taken, in seconds since the Unix epoch.
pub timestamp: u32,
pub peers: Vec<DumpPeer>,
/// Every distinct list of path attributes in the dump, as it is on the
/// wire. Routes refer to these by index.
pub attribute_sets: Vec<Box<[u8]>>,
/// For each peer (by index into `peers`), its routes: a prefix and the
/// index of its attributes.
pub routes: Vec<Vec<(IpPrefix, u32)>>,
}
impl Dump {
/// Reads the dump at `path`, gzip-compressed as it is published.
pub fn read(path: &Path) -> io::Result<Self> {
let mut input = BufReader::with_capacity(
1 << 20,
MultiGzDecoder::new(BufReader::new(File::open(path)?)),
);
let mut dump = Dump {
timestamp: 0,
peers: Vec::new(),
attribute_sets: Vec::new(),
routes: Vec::new(),
};
let mut interned: HashMap<Box<[u8]>, u32> = HashMap::new();
let mut body = Vec::new();
loop {
let mut header = [0u8; 12];
match input.read_exact(&mut header) {
Ok(()) => {}
Err(e) if e.kind() == io::ErrorKind::UnexpectedEof => break,
Err(e) => return Err(e),
}
let timestamp = u32::from_be_bytes([header[0], header[1], header[2], header[3]]);
let kind = u16::from_be_bytes([header[4], header[5]]);
let subtype = u16::from_be_bytes([header[6], header[7]]);
let length = u32::from_be_bytes([header[8], header[9], header[10], header[11]]);
body.resize(length as usize, 0);
input.read_exact(&mut body)?;
if kind != TABLE_DUMP_V2 {
continue;
}
let mut record = Cursor(&body);
let malformed = || io::Error::new(io::ErrorKind::InvalidData, "truncated MRT record");
match subtype {
PEER_INDEX_TABLE => {
dump.timestamp = timestamp;
dump.peers = peer_index_table(&mut record).ok_or_else(malformed)?;
dump.routes = vec![Vec::new(); dump.peers.len()];
}
RIB_IPV4_UNICAST | RIB_IPV6_UNICAST => {
let version = if subtype == RIB_IPV4_UNICAST {
IpVersion::Ipv4
} else {
IpVersion::Ipv6
};
rib_entries(&mut record, version, &mut dump, &mut interned)
.ok_or_else(malformed)?;
}
_ => {}
}
}
Ok(dump)
}
pub fn route_count(&self) -> usize {
self.routes.iter().map(Vec::len).sum()
}
/// The peers with the most routes, most first: the ones that send a
/// full table.
pub fn largest_peers(&self, count: usize) -> Vec<usize> {
let mut peers: Vec<usize> = (0..self.peers.len()).collect();
peers.sort_by_key(|&peer| std::cmp::Reverse(self.routes[peer].len()));
peers.truncate(count);
peers
}
}
struct Cursor<'a>(&'a [u8]);
impl<'a> Cursor<'a> {
fn take(&mut self, n: usize) -> Option<&'a [u8]> {
if self.0.len() < n {
return None;
}
let (taken, rest) = self.0.split_at(n);
self.0 = rest;
Some(taken)
}
fn u8(&mut self) -> Option<u8> {
self.take(1).map(|b| b[0])
}
fn u16(&mut self) -> Option<u16> {
self.take(2).map(|b| u16::from_be_bytes([b[0], b[1]]))
}
fn u32(&mut self) -> Option<u32> {
self.take(4)
.map(|b| u32::from_be_bytes([b[0], b[1], b[2], b[3]]))
}
}
fn peer_index_table(record: &mut Cursor<'_>) -> Option<Vec<DumpPeer>> {
record.u32()?; // collector's BGP identifier
let name = record.u16()?;
record.take(name.into())?;
let count = record.u16()?;
let mut peers = Vec::with_capacity(count.into());
for _ in 0..count {
let kind = record.u8()?;
record.u32()?; // peer's BGP identifier
let address = if kind & 1 == 1 {
let bytes: [u8; 16] = record.take(16)?.try_into().ok()?;
IpAddr::from(Ipv6Addr::from(bytes))
} else {
let bytes: [u8; 4] = record.take(4)?.try_into().ok()?;
IpAddr::from(Ipv4Addr::from(bytes))
};
let asn = if kind & 2 == 2 {
record.u32()?
} else {
record.u16()?.into()
};
peers.push(DumpPeer { address, asn });
}
Some(peers)
}
fn rib_entries(
record: &mut Cursor<'_>,
version: IpVersion,
dump: &mut Dump,
interned: &mut HashMap<Box<[u8]>, u32>,
) -> Option<()> {
record.u32()?; // sequence number
let length = record.u8()?;
let bytes = record.take(usize::from(length.div_ceil(8)))?;
// A prefix the collector should not have had is not ours to fix.
let prefix = IpPrefix::new(version, bytes, length).ok();
let count = record.u16()?;
for _ in 0..count {
let peer = usize::from(record.u16()?);
record.u32()?; // when the route was received
let attributes_len = record.u16()?;
let attributes = record.take(attributes_len.into())?;
let (Some(prefix), Some(routes)) = (prefix, dump.routes.get_mut(peer)) else {
continue;
};
let id = match interned.get(attributes) {
Some(id) => *id,
None => {
let id = u32::try_from(dump.attribute_sets.len()).ok()?;
dump.attribute_sets.push(attributes.into());
interned.insert(attributes.into(), id);
id
}
};
routes.push((prefix, id));
}
Some(())
}
/// The path attributes of a route in a dump, parsed.
///
/// In a dump the AS numbers are always four octets wide, and
/// MP_REACH_NLRI holds only the next hop (RFC 6396 §4.3.4): the prefix is in
/// the record. The next hop is returned beside the other attributes, as the
/// RIB keeps it. `None` if the attributes cannot be parsed.
pub fn parse_attributes(mut wire: &[u8]) -> Option<(Vec<PathAttribute>, Option<NlriNextHop>)> {
let ctx = wire_context();
let mut attributes = Vec::new();
let mut next_hop = None;
while !wire.is_empty() {
if wire.get(1) == Some(&MP_REACH_NLRI) {
let extended = wire[0] & 0x10 != 0;
let (length, header) = if extended {
(
usize::from(u16::from_be_bytes([*wire.get(2)?, *wire.get(3)?])),
4,
)
} else {
(usize::from(*wire.get(2)?), 3)
};
let value = wire.get(header..header + length)?;
let address = |bytes: &[u8]| Some(Ipv6Addr::from(<[u8; 16]>::try_from(bytes).ok()?));
next_hop = match value {
[4, a, b, c, d] => Some(NlriNextHop::Ipv4(Ipv4Addr::new(*a, *b, *c, *d))),
[16, rest @ ..] => Some(NlriNextHop::Ipv6(address(rest)?)),
[32, rest @ ..] if rest.len() == 32 => Some(NlriNextHop::Ipv6WithLl {
global: address(&rest[..16])?,
link_local: address(&rest[16..])?,
}),
_ => return None,
};
wire = &wire[header + length..];
continue;
}
let (rest, attribute) = PathAttribute::from_wire(&ctx, wire).ok()?;
attributes.push(attribute);
wire = rest;
}
Some((attributes, next_hop))
}
crates/integration-tests/Cargo.toml
+8
View File
@@ -1,6 +1,11 @@
[package]
name = "bgp-integration-tests"
authors.workspace = true
edition.workspace = true
homepage.workspace = true
license.workspace = true
repository.workspace = true
version.workspace = true
publish = false
[dependencies]
@@ -15,3 +20,6 @@ eyre.workspace = true
reqwest.workspace = true
serde_json.workspace = true
tokio.workspace = true
[lints]
workspace = true
crates/integration-tests/examples/fake_peer.rs
+40 -14
View File
@@ -1,13 +1,14 @@
//! A minimal stand-in for a lab router: connects to a ferrix_bgp instance,
//! A minimal stand-in for a neighbouring router: connects to the daemon,
//! completes the handshake, optionally announces some prefixes, and prints
//! every UPDATE it receives. Useful for demos when no real router is around.
//! every UPDATE it receives. Useful for trying things out when no real
//! router is around.
//!
//! cargo run -p bgp-integration-tests --example fake_peer -- \
//! --server 127.0.0.1:1790 --asn 65001 --announce 203.0.113.0/24
//!
//! With `--echo`, every prefix received is re-announced back to the server
//! with this peer's ASN prepended, which lets a single ferrix instance show
//! its own routes "coming back" in the received table.
//! with this peer's ASN prepended, which lets a single daemon show its own
//! routes "coming back" in the received table.
//!
//! With `--listen ADDR` it instead waits for the server to dial it (one
//! connection at a time), which exercises the server's active side.
@@ -27,7 +28,7 @@ use tokio::net::TcpListener;
#[derive(Parser, Debug)]
struct Args {
/// Address of the ferrix_bgp listener to connect to.
/// Address of the daemon's BGP listener to connect to.
#[arg(long, default_value = "127.0.0.1:1790", conflicts_with = "listen")]
server: String,
/// Instead of connecting, listen here and accept connections from the
@@ -77,12 +78,17 @@ async fn main() -> Result<()> {
}
async fn run(args: &Args, mut peer: TestPeer, addr: std::net::SocketAddr) -> Result<()> {
let open = peer.handshake(args.asn, args.router_id, args.hold_time).await?;
println!("established with {} (AS{} router-id {})", addr, open.asn, open.identifier);
let open = peer
.handshake(args.asn, args.router_id, args.hold_time)
.await?;
println!(
"established with {} (AS{} router-id {})",
addr, open.asn, open.identifier
);
for p in &args.announce {
let prefix = IpPrefix::try_from(p.as_str())?;
let update = announce(prefix.clone(), vec![args.asn], vec![(args.asn as u16, 1)]);
let update = announce(prefix, vec![args.asn], vec![(args.asn as u16, 1)]);
peer.send(&Message::Update(update)).await?;
println!("announced {prefix}");
}
@@ -122,9 +128,15 @@ fn announce(prefix: IpPrefix, as_path: Vec<u32>, communities: Vec<(u16, u16)>) -
PathAttribute::NextHop(NextHopPathAttribute(Ipv4Addr::new(10, 0, 0, 2))),
];
if !communities.is_empty() {
path_attributes.push(PathAttribute::Communities(CommunitiesPathAttribute(communities)));
path_attributes.push(PathAttribute::Communities(CommunitiesPathAttribute(
communities,
)));
}
UpdateMessage {
withdrawn_routes: vec![],
path_attributes,
announced_routes: vec![prefix.into()],
}
UpdateMessage { withdrawn_routes: vec![], path_attributes, announced_routes: vec![prefix] }
}
async fn echo_back(peer: &mut TestPeer, u: &UpdateMessage, asn: u32) -> Result<()> {
@@ -132,7 +144,9 @@ async fn echo_back(peer: &mut TestPeer, u: &UpdateMessage, asn: u32) -> Result<(
let mut communities = vec![];
for attr in &u.path_attributes {
match attr {
PathAttribute::AsPath(p) => as_path.extend(p.segments.iter().flat_map(|s| s.path.iter().copied())),
PathAttribute::AsPath(p) => {
as_path.extend(p.segments.iter().flat_map(|s| s.path.iter().copied()))
}
PathAttribute::Communities(c) => communities = c.0.clone(),
_ => {}
}
@@ -141,8 +155,14 @@ async fn echo_back(peer: &mut TestPeer, u: &UpdateMessage, asn: u32) -> Result<(
println!(" not echoing: NO_EXPORT");
return Ok(());
}
for prefix in &u.announced_routes {
peer.send(&Message::Update(announce(prefix.clone(), as_path.clone(), communities.clone()))).await?;
for nlri in &u.announced_routes {
let prefix = nlri.prefix;
peer.send(&Message::Update(announce(
prefix,
as_path.clone(),
communities.clone(),
)))
.await?;
println!(" echoed {prefix} back with path {as_path:?}");
}
if !u.withdrawn_routes.is_empty() {
@@ -152,7 +172,13 @@ async fn echo_back(peer: &mut TestPeer, u: &UpdateMessage, asn: u32) -> Result<(
announced_routes: vec![],
}))
.await?;
println!(" echoed withdraw of {:?}", u.withdrawn_routes.iter().map(|p| p.to_string()).collect::<Vec<_>>());
println!(
" echoed withdraw of {:?}",
u.withdrawn_routes
.iter()
.map(|p| p.to_string())
.collect::<Vec<_>>()
);
}
Ok(())
}
crates/integration-tests/src/test_peer.rs
+86 -18
View File
@@ -2,7 +2,7 @@ use std::net::{Ipv4Addr, SocketAddr};
use bgp_packet::message::{BGP_HEADER_LEN, Message, UpdateMessage};
use bgp_packet::open::{Capability, OpenMessage, OpenOption};
use bgp_primitives::constants::{AddressFamilyId, SubsequentAfi};
use bgp_primitives::constants::{Afi, AfiSafi, Safi};
use bgp_primitives::parser::ParserContext;
use bytes::BytesMut;
use eyre::{Result, bail, eyre};
@@ -14,6 +14,14 @@ use tokio::net::TcpStream;
pub struct TestPeer {
stream: TcpStream,
pub ctx: ParserContext,
/// The address families for which this peer sends path identifiers
/// with its prefixes (ADD-PATH, RFC 7911). What it expects to be sent
/// is `ctx.add_path`. A test sets both to match the capabilities it
/// exchanged.
pub send_add_path: Vec<AfiSafi>,
/// Whether the OPEN we sent offered four-octet AS numbers. They are in
/// use only if the server's did too.
offered_four_octet_asn: bool,
}
impl TestPeer {
@@ -25,13 +33,22 @@ impl TestPeer {
/// Wraps an already-connected stream, e.g. one accepted from a server
/// that dialled us.
pub fn from_stream(stream: TcpStream) -> Self {
Self { stream, ctx: ParserContext::default() }
Self {
stream,
ctx: ParserContext::default(),
send_add_path: Vec::new(),
offered_four_octet_asn: true,
}
}
/// Send a BGP message.
pub async fn send(&mut self, msg: &Message) -> Result<()> {
let send_ctx = ParserContext {
add_path: self.send_add_path.clone(),
..self.ctx.clone()
};
let mut buf = BytesMut::new();
msg.to_wire(&self.ctx, &mut buf)?;
msg.to_wire(&send_ctx, &mut buf)?;
self.stream.write_all(&buf).await?;
Ok(())
}
@@ -48,11 +65,11 @@ impl TestPeer {
let mut buf = vec![0u8; total_len];
buf[..BGP_HEADER_LEN as usize].copy_from_slice(&header);
self.stream.read_exact(&mut buf[BGP_HEADER_LEN as usize..]).await?;
self.stream
.read_exact(&mut buf[BGP_HEADER_LEN as usize..])
.await?;
let (_, msg) = Message::from_wire(&self.ctx, &buf)
.map_err(|e| eyre!("Failed to parse BGP message: {e:?}"))?;
Ok(msg)
Message::decode(&self.ctx, &buf).map_err(|e| eyre!("Failed to parse BGP message: {e}"))
}
/// Read messages until an UPDATE arrives, skipping KEEPALIVEs. Any other
@@ -76,34 +93,67 @@ impl TestPeer {
matches!(opt, OpenOption::Capabilities(caps)
if caps.iter().any(|c| matches!(c, Capability::FourByteAsn { .. })))
});
self.ctx.four_octet_asn = Some(four_byte);
self.ctx.four_octet_asn = four_byte && self.offered_four_octet_asn;
Ok(o)
}
other => bail!("Expected OPEN, got {:?}", other),
}
}
/// Send an OPEN message with the given parameters and standard capabilities.
/// Send an OPEN message with the given parameters and the capabilities
/// of a typical dual-stack speaker: four-octet AS numbers, IPv4 and IPv6
/// unicast, route refresh.
pub async fn send_open(
&mut self,
local_asn: u32,
router_id: Ipv4Addr,
hold_time: u16,
) -> Result<()> {
let asn_field = if local_asn > u16::MAX as u32 { 23456 } else { local_asn as u16 };
let capabilities = vec![
Capability::FourByteAsn { asn: local_asn },
Capability::MultiProtocol {
afi: Afi::IPV4,
safi: Safi::UNICAST,
},
Capability::MultiProtocol {
afi: Afi::IPV6,
safi: Safi::UNICAST,
},
Capability::RouteRefresh,
];
self.send_open_with(local_asn, router_id, hold_time, capabilities)
.await
}
/// Send an OPEN message carrying exactly `capabilities`.
pub async fn send_open_with(
&mut self,
local_asn: u32,
router_id: Ipv4Addr,
hold_time: u16,
capabilities: Vec<Capability>,
) -> Result<()> {
self.offered_four_octet_asn = capabilities
.iter()
.any(|c| matches!(c, Capability::FourByteAsn { .. }));
// If the server's OPEN has been read already, what was settled
// then still depends on what is offered now.
self.ctx.four_octet_asn &= self.offered_four_octet_asn;
let asn_field = if local_asn > u16::MAX as u32 {
23456
} else {
local_asn as u16
};
let open = Message::Open(OpenMessage {
version: 4,
asn: asn_field,
hold_time,
identifier: router_id,
options: vec![OpenOption::Capabilities(vec![
Capability::FourByteAsn { asn: local_asn },
Capability::MultiProtocol {
afi: AddressFamilyId::Ipv4,
safi: SubsequentAfi::Unicast,
},
Capability::RouteRefresh,
])],
options: if capabilities.is_empty() {
vec![]
} else {
vec![OpenOption::Capabilities(capabilities)]
},
});
self.send(&open).await
}
@@ -117,6 +167,24 @@ impl TestPeer {
hold_time: u16,
) -> Result<OpenMessage> {
self.send_open(local_asn, router_id, hold_time).await?;
self.finish_handshake().await
}
/// As [`TestPeer::handshake`], with an OPEN carrying exactly
/// `capabilities`.
pub async fn handshake_with(
&mut self,
local_asn: u32,
router_id: Ipv4Addr,
hold_time: u16,
capabilities: Vec<Capability>,
) -> Result<OpenMessage> {
self.send_open_with(local_asn, router_id, hold_time, capabilities)
.await?;
self.finish_handshake().await
}
async fn finish_handshake(&mut self) -> Result<OpenMessage> {
let server_open = self.recv_open().await?;
// Expect server KEEPALIVE
match self.recv().await? {
crates/integration-tests/src/test_server.rs
+120 -36
View File
@@ -1,10 +1,13 @@
use std::net::{IpAddr, Ipv4Addr, SocketAddr};
use std::sync::Arc;
use bgp_rib::{AdjRibIn, LocalRib, LocalRibPolicy};
use bgp_packet::message::{NlriNextHop, PathAttribute};
use bgp_primitives::ip_prefix::IpPrefix;
use bgp_rib::LocalRib;
use bgp_rib::table::{Event, PeerIndex, Reader, RibTable};
use bgp_server::config::Config;
use bgp_server::ping::PingMonitor;
use bgp_server::server::BgpServerBuilder;
use bgp_server::error::ServerError;
use bgp_server::server::BgpServer;
use bgp_server::session::SessionRegistry;
use bgp_web::{AppState, RouterInfo};
use eyre::Result;
@@ -14,18 +17,15 @@ use tokio::task::JoinHandle;
/// A BGP server started on an ephemeral port for testing.
/// Aborts the server task when dropped.
pub struct TestServer {
handle: JoinHandle<Result<()>>,
handle: JoinHandle<Result<(), ServerError>>,
/// The single IPv4 loopback address the server is bound to.
pub addr: SocketAddr,
/// Handle to the server's Adj-RIB-In for test introspection.
pub adj_rib_in: AdjRibIn,
/// The server's routing table.
pub rib: Arc<RibTable>,
/// Handle to the server's locally originated routes.
pub local_rib: LocalRib,
/// Live view of the server's established sessions.
pub registry: SessionRegistry,
/// Ping monitor without a socket, watching 2001:db8::/32; tests feed it
/// through `record`.
pub pings: PingMonitor,
pub local_asn: u32,
}
@@ -37,20 +37,7 @@ impl TestServer {
/// Start a server with an explicit hold time (useful for timer tests).
pub async fn start_with_hold_time(local_asn: u32, hold_time_secs: u16) -> Result<Self> {
Self::start_inner(local_asn, hold_time_secs, LocalRibPolicy::default()).await
}
/// Start a server with a policy restricting locally originated routes.
pub async fn start_with_policy(local_asn: u32, policy: LocalRibPolicy) -> Result<Self> {
Self::start_inner(local_asn, 90, policy).await
}
async fn start_inner(local_asn: u32, hold_time_secs: u16, policy: LocalRibPolicy) -> Result<Self> {
Self::start_with(local_asn, |c| {
c.hold_time_secs = hold_time_secs;
c.local_route_policy = policy;
})
.await
Self::start_with(local_asn, |c| c.hold_time_secs = hold_time_secs).await
}
/// Start a server after letting `customize` edit the default test config
@@ -64,33 +51,131 @@ impl TestServer {
peers: vec![],
connect_retry: std::time::Duration::from_secs(5),
collision_detect_established_state: false,
local_route_policy: LocalRibPolicy::default(),
originate: vec![],
http_endpoint: None,
ping_monitor: None,
};
customize(&mut config);
let bound = BgpServerBuilder::new(config).bind().await?;
let bound = BgpServer::bind(config).await?;
let addr = bound
.local_addrs()
.into_iter()
.next()
.expect("at least one listener");
let adj_rib_in = bound.adj_rib_in.clone();
let rib = Arc::clone(&bound.rib);
let local_rib = bound.local_rib.clone();
let registry = bound.registry.clone();
let pings = PingMonitor::without_socket(
vec![IpPrefix::try_from("2001:db8::/32").expect("valid prefix")],
Some(local_rib.clone()),
);
let handle = tokio::spawn(bound.run());
Ok(Self { handle, addr, adj_rib_in, local_rib, registry, pings, local_asn })
Ok(Self {
handle,
addr,
rib,
local_rib,
registry,
local_asn,
})
}
}
/// A route in the server's table, as a test wants to look at it.
#[derive(Debug, Clone)]
pub struct ReceivedRoute {
pub prefix: IpPrefix,
pub peer: PeerIndex,
/// The peer's identifier for the route; zero without ADD-PATH.
pub path_id: u32,
/// The route's attributes, without MP_REACH_NLRI.
pub attributes: Vec<PathAttribute>,
/// The next hop, if the route came in MP_REACH_NLRI.
pub next_hop: Option<NlriNextHop>,
/// Whether it is the best route for its prefix.
pub best: bool,
}
/// A change of a prefix's best route.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum RouteChange {
/// The prefix has a best route, or another one than before.
Announced { prefix: IpPrefix, peer: PeerIndex },
/// The prefix has no route any more.
Withdrawn { prefix: IpPrefix },
}
/// The changes of best route in a server's table from the moment this was
/// made, read from the table's journal.
pub struct RouteChanges {
reader: Reader,
pending: std::collections::VecDeque<RouteChange>,
}
impl RouteChanges {
/// The next change, waiting for one if need be. Changes to one prefix
/// that happen before they are read arrive as one.
pub async fn recv(&mut self) -> Result<RouteChange> {
loop {
if let Some(change) = self.pending.pop_front() {
return Ok(change);
}
self.reader.changed().await;
self.read();
}
}
fn read(&mut self) {
let pending = &mut self.pending;
let mut read = |event: Event<'_>| match event {
Event::Best { prefix, peer, .. } => {
pending.push_back(RouteChange::Announced { prefix, peer })
}
Event::Withdrawn { prefix } => pending.push_back(RouteChange::Withdrawn { prefix }),
// Not given to a follower of the best routes.
Event::Routes { .. } => {}
Event::Reset | Event::Synced => {}
};
while !self.reader.poll(1024, &mut read).caught_up {}
}
}
impl TestServer {
/// Every route in the server's table.
pub fn received_routes(&self) -> Vec<ReceivedRoute> {
let mut routes = Vec::new();
self.rib.for_each_route(|route| {
let (attributes, next_hop) = route.attrs.attributes().expect("stored attributes parse");
routes.push(ReceivedRoute {
prefix: route.prefix,
peer: route.peer,
path_id: route.path_id,
attributes,
next_hop,
best: route.best,
});
});
routes
}
/// Follows the changes of best route from now on.
pub fn route_changes(&self) -> RouteChanges {
let mut changes = RouteChanges {
reader: self.rib.reader(),
pending: Default::default(),
};
// A new reader is first given the table as it is. That is not a
// change.
changes.read();
changes.pending.clear();
changes
}
/// Waits for the server to stop by itself and returns why. A healthy
/// server never does.
pub async fn stopped(&mut self) -> Result<(), ServerError> {
(&mut self.handle)
.await
.expect("the server task does not panic")
}
/// Starts the HTTP API on an ephemeral loopback port and returns its
/// base URL (e.g. `http://127.0.0.1:12345`). The API task is aborted
/// when the returned guard drops.
@@ -104,8 +189,7 @@ impl TestServer {
),
registry: self.registry.clone(),
local_rib: self.local_rib.clone(),
adj_rib_in: self.adj_rib_in.clone(),
pings: self.pings.clone(),
rib: Arc::clone(&self.rib),
};
let handle = tokio::spawn(bgp_web::serve_on(state, listener));
Ok((base_url, HttpGuard { handle }))
@@ -114,7 +198,7 @@ impl TestServer {
/// Aborts the HTTP API task when dropped.
pub struct HttpGuard {
handle: JoinHandle<Result<()>>,
handle: JoinHandle<std::io::Result<()>>,
}
impl Drop for HttpGuard {
crates/integration-tests/tests/add_path.rs
+288
View File
@@ -0,0 +1,288 @@
//! End-to-end tests of ADD-PATH (RFC 7911): a peer sending several routes
//! for one prefix, told apart by path identifiers.
use std::net::{Ipv4Addr, SocketAddr};
use std::time::Duration;
use bgp_integration_tests::test_peer::TestPeer;
use bgp_integration_tests::test_server::TestServer;
use bgp_packet::message::{
AsPathAttribute, Message, NextHopPathAttribute, OriginPathAttribute, PathAttribute,
UpdateMessage,
};
use bgp_packet::open::{AddPathEntry, AddPathSendRecv, Capability, OpenMessage};
use bgp_primitives::constants::AfiSafi;
use bgp_primitives::ip_prefix::{IpPrefix, Nlri};
use bgp_rib::{LocalRoute, LocalRouteAttrs};
use bgp_server::config::PeerConfig;
use bgp_server::family::{FamilySet, SUPPORTED};
use eyre::Result;
use serde_json::Value;
const LOCAL_ASN: u32 = 65000;
const PEER_ASN: u32 = 65001;
const TIMEOUT: Duration = Duration::from_secs(5);
fn prefix(s: &str) -> IpPrefix {
IpPrefix::try_from(s).unwrap()
}
/// A server on which connections from this machine are those of a
/// configured peer, with ADD-PATH offered to it as `add_path` says.
async fn server_offering(
add_path: Option<AddPathSendRecv>,
originate: &[&str],
) -> Result<TestServer> {
let originate: Vec<LocalRoute> = originate
.iter()
.map(|p| LocalRoute::new(prefix(p), LocalRouteAttrs::default()))
.collect();
TestServer::start_with(LOCAL_ASN, |c| {
c.peers = vec![PeerConfig {
name: "route-server-client".into(),
endpoint: SocketAddr::new(Ipv4Addr::LOCALHOST.into(), 1),
remote_asn: PEER_ASN,
hold_time_secs: 90,
local_address: None,
families: FamilySet::supported(),
add_path,
}];
c.originate = originate;
// Keep the dialer out of the way.
c.connect_retry = Duration::from_secs(3600);
})
.await
}
/// A session from a peer whose OPEN offers ADD-PATH as `offer` says, for
/// IPv4 unicast. Returns the peer and the server's OPEN.
async fn peer_offering(
server: &TestServer,
offer: Option<AddPathSendRecv>,
) -> Result<(TestPeer, OpenMessage)> {
let mut capabilities = vec![Capability::FourByteAsn { asn: PEER_ASN }];
capabilities.extend(SUPPORTED.into_iter().map(Capability::multiprotocol));
if let Some(send_recv) = offer {
capabilities.push(Capability::AddPath {
entries: vec![AddPathEntry {
afi: AfiSafi::IPV4_UNICAST.afi,
safi: AfiSafi::IPV4_UNICAST.safi,
send_recv,
}],
});
}
let mut peer = TestPeer::connect(server.addr).await?;
let open = peer
.handshake_with(PEER_ASN, Ipv4Addr::new(10, 0, 0, 1), 90, capabilities)
.await?;
tokio::time::timeout(TIMEOUT, async {
while server.registry.list().is_empty() {
tokio::time::sleep(Duration::from_millis(5)).await;
}
})
.await?;
Ok((peer, open))
}
fn add_path_offered(open: &OpenMessage) -> Vec<(AfiSafi, AddPathSendRecv)> {
open.capabilities()
.filter_map(|capability| match capability {
Capability::AddPath { entries } => Some(entries),
_ => None,
})
.flatten()
.map(|entry| (entry.family(), entry.send_recv))
.collect()
}
fn announcement(nlri: Nlri, as_path: &[u32]) -> Message {
Message::Update(UpdateMessage {
withdrawn_routes: vec![],
path_attributes: vec![
PathAttribute::Origin(OriginPathAttribute::IGP),
AsPathAttribute::from_asns(as_path.to_vec()),
PathAttribute::NextHop(NextHopPathAttribute(Ipv4Addr::new(192, 0, 2, 1))),
],
announced_routes: vec![nlri],
})
}
async fn wait_for_routes(server: &TestServer, routes: usize) -> Result<()> {
tokio::time::timeout(TIMEOUT, async {
while server.rib.len() != routes {
tokio::time::sleep(Duration::from_millis(5)).await;
}
})
.await?;
Ok(())
}
/// The routes the server holds, as (path identifier, AS path, best).
fn held(server: &TestServer) -> Vec<(u32, Vec<u32>, bool)> {
let mut held: Vec<(u32, Vec<u32>, bool)> = server
.received_routes()
.into_iter()
.map(|route| {
let path = route
.attributes
.iter()
.find_map(|attribute| match attribute {
PathAttribute::AsPath(path) => Some(path.segments[0].path.clone()),
_ => None,
})
.unwrap_or_default();
(route.path_id, path, route.best)
})
.collect();
held.sort();
held
}
/// A peer that sends several routes for one prefix: each is kept, the best
/// of them is chosen, and each is withdrawn on its own.
#[tokio::test]
async fn test_several_routes_for_a_prefix_from_one_peer() -> Result<()> {
let server = server_offering(Some(AddPathSendRecv::Receive), &[]).await?;
let (base, _http) = server.serve_http().await?;
let (mut peer, open) = peer_offering(&server, Some(AddPathSendRecv::Send)).await?;
// The server offered to receive, for every family it offers.
assert_eq!(
add_path_offered(&open),
SUPPORTED.map(|family| (family, AddPathSendRecv::Receive))
);
// In use for the one family the peer sends for, and in that direction.
let session = &server.registry.list()[0];
assert_eq!(
session.add_path_receive,
[AfiSafi::IPV4_UNICAST].into_iter().collect()
);
assert!(session.add_path_send.is_empty());
peer.send_add_path = vec![AfiSafi::IPV4_UNICAST];
let p = prefix("203.0.113.0/24");
let path = |path_id| Nlri { prefix: p, path_id };
peer.send(&announcement(path(1), &[PEER_ASN, 64999, 64998]))
.await?;
peer.send(&announcement(path(2), &[PEER_ASN, 64998]))
.await?;
wait_for_routes(&server, 2).await?;
assert_eq!(
held(&server),
[
(1, vec![PEER_ASN, 64999, 64998], false),
(2, vec![PEER_ASN, 64998], true),
]
);
assert_eq!(server.rib.prefix_count(), 1);
// The API tells them apart too.
let routes: Value = reqwest::get(format!("{base}/api/routes/received"))
.await?
.json()
.await?;
let mut ids: Vec<u64> = routes
.as_array()
.unwrap()
.iter()
.map(|route| route["path_id"].as_u64().unwrap())
.collect();
ids.sort_unstable();
assert_eq!(ids, [1, 2]);
let peers: Value = reqwest::get(format!("{base}/api/peers"))
.await?
.json()
.await?;
assert_eq!(
peers[0]["add_path_receive"],
serde_json::json!(["IPv4 unicast"])
);
// Announcing a path again replaces that path only.
peer.send(&announcement(path(1), &[PEER_ASN])).await?;
tokio::time::timeout(TIMEOUT, async {
while held(&server)[0].1.len() != 1 {
tokio::time::sleep(Duration::from_millis(5)).await;
}
})
.await?;
assert_eq!(
held(&server),
[(1, vec![PEER_ASN], true), (2, vec![PEER_ASN, 64998], false),]
);
// And withdrawing one leaves the other.
peer.send(&Message::Update(UpdateMessage {
withdrawn_routes: vec![path(1)],
path_attributes: vec![],
announced_routes: vec![],
}))
.await?;
wait_for_routes(&server, 1).await?;
assert_eq!(held(&server), [(2, vec![PEER_ASN, 64998], true)]);
// When the session ends, all of its routes go.
peer.send(&announcement(path(3), &[PEER_ASN, 1, 2, 3]))
.await?;
wait_for_routes(&server, 2).await?;
drop(peer);
wait_for_routes(&server, 0).await?;
Ok(())
}
/// ADD-PATH changes how prefixes are written. It must be in use only when
/// both sides said so; a peer that offers it to a server that does not is
/// spoken to, and listened to, in plain BGP.
#[tokio::test]
async fn test_add_path_is_not_used_unless_both_sides_offer_it() -> Result<()> {
let server = server_offering(None, &["10.100.0.0/16"]).await?;
let (mut peer, open) = peer_offering(&server, Some(AddPathSendRecv::Both)).await?;
assert!(add_path_offered(&open).is_empty());
let session = &server.registry.list()[0];
assert!(session.add_path_receive.is_empty() && session.add_path_send.is_empty());
// Plain prefixes both ways.
let update = tokio::time::timeout(TIMEOUT, peer.recv_update()).await??;
assert_eq!(update.announced_routes, vec![prefix("10.100.0.0/16")]);
peer.send(&announcement(prefix("203.0.113.0/24").into(), &[PEER_ASN]))
.await?;
wait_for_routes(&server, 1).await?;
assert_eq!(held(&server), [(0, vec![PEER_ASN], true)]);
// And the other way round: offered by the server, not by the peer.
let server = server_offering(Some(AddPathSendRecv::Both), &["10.100.0.0/16"]).await?;
let (mut peer, open) = peer_offering(&server, None).await?;
assert_eq!(add_path_offered(&open).len(), SUPPORTED.len());
let update = tokio::time::timeout(TIMEOUT, peer.recv_update()).await??;
assert_eq!(update.announced_routes, vec![prefix("10.100.0.0/16")]);
Ok(())
}
/// Where the peer offered to receive and the server to send, the server's
/// prefixes carry a path identifier, and a peer that reads them so
/// understands them.
#[tokio::test]
async fn test_prefixes_are_sent_with_path_identifiers_when_agreed() -> Result<()> {
let server = server_offering(Some(AddPathSendRecv::Both), &["10.100.0.0/16"]).await?;
let (mut peer, _) = peer_offering(&server, Some(AddPathSendRecv::Receive)).await?;
let session = &server.registry.list()[0];
assert_eq!(
session.add_path_send,
[AfiSafi::IPV4_UNICAST].into_iter().collect()
);
assert!(session.add_path_receive.is_empty());
// Read as agreed: with identifiers.
peer.ctx.add_path = vec![AfiSafi::IPV4_UNICAST];
let update = tokio::time::timeout(TIMEOUT, peer.recv_update()).await??;
assert_eq!(update.announced_routes.len(), 1);
assert_eq!(update.announced_routes[0].prefix, prefix("10.100.0.0/16"));
// The peer did not offer to send, so the server reads its prefixes
// plain.
peer.send(&announcement(prefix("203.0.113.0/24").into(), &[PEER_ASN]))
.await?;
wait_for_routes(&server, 1).await?;
assert_eq!(held(&server), [(0, vec![PEER_ASN], true)]);
Ok(())
}
crates/integration-tests/tests/announce.rs
+180 -60
View File
@@ -5,13 +5,14 @@ use std::net::Ipv4Addr;
use std::time::Duration;
use bgp_integration_tests::test_peer::TestPeer;
use bgp_integration_tests::test_server::TestServer;
use bgp_integration_tests::test_server::{RouteChange, TestServer};
use bgp_packet::message::{
AsPathAttribute, CommunitiesPathAttribute, LargeCommunitiesPathAttribute, LargeCommunity,
NextHopPathAttribute, OriginPathAttribute, PathAttribute, UpdateMessage,
LocalPrefPathAttribute, NextHopPathAttribute, OriginPathAttribute, PathAttribute,
UpdateMessage,
};
use bgp_primitives::ip_prefix::IpPrefix;
use bgp_rib::{LocalRibError, LocalRibPolicy, LocalRoute, LocalRouteAttrs};
use bgp_rib::{LocalRoute, LocalRouteAttrs};
use bgp_server::session::SessionEvent;
use eyre::Result;
@@ -24,7 +25,7 @@ fn prefix(s: &str) -> IpPrefix {
}
fn route(p: &str, attrs: LocalRouteAttrs) -> LocalRoute {
LocalRoute::new(prefix(p), "alice", attrs)
LocalRoute::new(prefix(p), attrs)
}
async fn established_peer(server: &TestServer) -> Result<TestPeer> {
@@ -67,7 +68,9 @@ async fn test_registry_tracks_session_lifecycle() -> Result<()> {
let peer = established_peer(&server).await?;
let up = tokio::time::timeout(TIMEOUT, events.recv()).await??;
let SessionEvent::Up(handle) = up else { panic!("expected Up, got {up:?}") };
let SessionEvent::Up(handle) = up else {
panic!("expected Up, got {up:?}")
};
assert_eq!(handle.peer.asn, PEER_ASN);
assert_eq!(handle.local_addr, server.addr);
assert!(handle.four_octet_asn);
@@ -76,7 +79,10 @@ async fn test_registry_tracks_session_lifecycle() -> Result<()> {
drop(peer);
let down = tokio::time::timeout(TIMEOUT, events.recv()).await??;
assert!(matches!(down, SessionEvent::Down(p) if p == handle.peer), "got {down:?}");
assert!(
matches!(down, SessionEvent::Down(p) if p == handle.peer),
"got {down:?}"
);
assert!(server.registry.list().is_empty());
Ok(())
}
@@ -102,7 +108,11 @@ async fn test_announce_after_session_up() -> Result<()> {
prepend: 0,
med: None,
communities: vec![(65000, 100), (65535, 65281)],
large_communities: vec![LargeCommunity { global_admin: 65000, data_1: 7, data_2: 8 }],
large_communities: vec![LargeCommunity {
global_admin: 65000,
data_1: 7,
data_2: 8,
}],
},
))
.await?;
@@ -116,9 +126,16 @@ async fn test_announce_after_session_up() -> Result<()> {
PathAttribute::Origin(OriginPathAttribute::IGP),
AsPathAttribute::from_asns(vec![LOCAL_ASN]),
PathAttribute::NextHop(NextHopPathAttribute(Ipv4Addr::LOCALHOST)),
PathAttribute::Communities(CommunitiesPathAttribute(vec![(65000, 100), (65535, 65281)])),
PathAttribute::Communities(CommunitiesPathAttribute(vec![
(65000, 100),
(65535, 65281)
])),
PathAttribute::LargeCommunities(LargeCommunitiesPathAttribute {
communities: vec![LargeCommunity { global_admin: 65000, data_1: 7, data_2: 8 }],
communities: vec![LargeCommunity {
global_admin: 65000,
data_1: 7,
data_2: 8
}],
}),
]
);
@@ -130,8 +147,14 @@ async fn test_announce_after_session_up() -> Result<()> {
#[tokio::test]
async fn test_routes_replayed_on_session_up() -> Result<()> {
let server = TestServer::start(LOCAL_ASN).await?;
server.local_rib.announce(route("10.100.1.0/24", LocalRouteAttrs::default())).await?;
server.local_rib.announce(route("10.100.2.0/24", LocalRouteAttrs::default())).await?;
server
.local_rib
.announce(route("10.100.1.0/24", LocalRouteAttrs::default()))
.await?;
server
.local_rib
.announce(route("10.100.2.0/24", LocalRouteAttrs::default()))
.await?;
let mut peer = established_peer(&server).await?;
@@ -153,7 +176,10 @@ async fn test_announce_reaches_all_sessions() -> Result<()> {
let mut b = established_peer_with_id(&server, Ipv4Addr::new(10, 0, 0, 2)).await?;
wait_for_sessions(&server, 2).await?;
server.local_rib.announce(route("10.100.1.0/24", LocalRouteAttrs::default())).await?;
server
.local_rib
.announce(route("10.100.1.0/24", LocalRouteAttrs::default()))
.await?;
for peer in [&mut a, &mut b] {
let update = next_update(peer).await?;
@@ -168,10 +194,13 @@ async fn test_withdraw_sends_withdrawn_route() -> Result<()> {
let mut peer = established_peer(&server).await?;
wait_for_sessions(&server, 1).await?;
server.local_rib.announce(route("10.100.1.0/24", LocalRouteAttrs::default())).await?;
server
.local_rib
.announce(route("10.100.1.0/24", LocalRouteAttrs::default()))
.await?;
let _ = next_update(&mut peer).await?;
server.local_rib.withdraw(prefix("10.100.1.0/24"), Some("alice")).await?;
server.local_rib.withdraw(prefix("10.100.1.0/24")).await?;
let update = next_update(&mut peer).await?;
assert_eq!(update.withdrawn_routes, vec![prefix("10.100.1.0/24")]);
assert!(update.announced_routes.is_empty());
@@ -186,24 +215,39 @@ async fn test_reannounce_with_prepend_and_med() -> Result<()> {
let mut peer = established_peer(&server).await?;
wait_for_sessions(&server, 1).await?;
server.local_rib.announce(route("10.100.1.0/24", LocalRouteAttrs::default())).await?;
server
.local_rib
.announce(route("10.100.1.0/24", LocalRouteAttrs::default()))
.await?;
let first = next_update(&mut peer).await?;
assert_eq!(first.path_attributes[1], AsPathAttribute::from_asns(vec![LOCAL_ASN]));
assert_eq!(
first.path_attributes[1],
AsPathAttribute::from_asns(vec![LOCAL_ASN])
);
server
.local_rib
.announce(route(
"10.100.1.0/24",
LocalRouteAttrs { prepend: 3, med: Some(42), ..LocalRouteAttrs::default() },
LocalRouteAttrs {
prepend: 3,
med: Some(42),
..LocalRouteAttrs::default()
},
))
.await?;
let second = next_update(&mut peer).await?;
assert_eq!(second.announced_routes, vec![prefix("10.100.1.0/24")]);
assert_eq!(second.path_attributes[1], AsPathAttribute::from_asns(vec![LOCAL_ASN; 4]));
assert!(second
.path_attributes
.iter()
.any(|a| matches!(a, PathAttribute::MultiExitDisc(m) if m.0 == 42)));
assert_eq!(
second.path_attributes[1],
AsPathAttribute::from_asns(vec![LOCAL_ASN; 4])
);
assert!(
second
.path_attributes
.iter()
.any(|a| matches!(a, PathAttribute::MultiExitDisc(m) if m.0 == 42))
);
Ok(())
}
@@ -212,10 +256,14 @@ async fn test_reannounce_with_prepend_and_med() -> Result<()> {
async fn test_keepalives_continue_alongside_updates() -> Result<()> {
let server = TestServer::start_with_hold_time(LOCAL_ASN, 3).await?;
let mut peer = TestPeer::connect(server.addr).await?;
peer.handshake(PEER_ASN, Ipv4Addr::new(10, 0, 0, 1), 3).await?;
peer.handshake(PEER_ASN, Ipv4Addr::new(10, 0, 0, 1), 3)
.await?;
wait_for_sessions(&server, 1).await?;
server.local_rib.announce(route("10.100.1.0/24", LocalRouteAttrs::default())).await?;
server
.local_rib
.announce(route("10.100.1.0/24", LocalRouteAttrs::default()))
.await?;
let mut saw_update = false;
let mut saw_keepalive = false;
@@ -238,39 +286,44 @@ async fn test_keepalives_continue_alongside_updates() -> Result<()> {
#[tokio::test]
async fn test_received_routes_removed_when_session_closes() -> Result<()> {
let server = TestServer::start(LOCAL_ASN).await?;
let mut rib_events = server.adj_rib_in.subscribe();
let mut rib_events = server.route_changes();
let mut session_events = server.registry.subscribe();
let mut peer = established_peer(&server).await?;
wait_for_sessions(&server, 1).await?;
peer.send(&bgp_packet::message::Message::Update(bgp_packet::message::UpdateMessage {
withdrawn_routes: vec![],
path_attributes: vec![
PathAttribute::Origin(OriginPathAttribute::IGP),
AsPathAttribute::from_asns(vec![PEER_ASN]),
PathAttribute::NextHop(NextHopPathAttribute(Ipv4Addr::new(10, 0, 0, 1))),
],
announced_routes: vec![prefix("203.0.113.0/24"), prefix("198.51.100.0/24")],
}))
peer.send(&bgp_packet::message::Message::Update(
bgp_packet::message::UpdateMessage {
withdrawn_routes: vec![],
path_attributes: vec![
PathAttribute::Origin(OriginPathAttribute::IGP),
AsPathAttribute::from_asns(vec![PEER_ASN]),
PathAttribute::NextHop(NextHopPathAttribute(Ipv4Addr::new(10, 0, 0, 1))),
],
announced_routes: vec![
prefix("203.0.113.0/24").into(),
prefix("198.51.100.0/24").into(),
],
},
))
.await?;
for _ in 0..2 {
let ev = tokio::time::timeout(TIMEOUT, rib_events.recv()).await??;
assert!(matches!(ev, bgp_rib::RibEvent::Announced { .. }), "{ev:?}");
assert!(matches!(ev, RouteChange::Announced { .. }), "{ev:?}");
}
assert_eq!(server.adj_rib_in.snapshot().await.len(), 2);
assert_eq!(server.received_routes().len(), 2);
drop(peer);
let mut withdrawn = vec![];
for _ in 0..2 {
match tokio::time::timeout(TIMEOUT, rib_events.recv()).await?? {
bgp_rib::RibEvent::Withdrawn { prefix, .. } => withdrawn.push(prefix.to_string()),
RouteChange::Withdrawn { prefix, .. } => withdrawn.push(prefix.to_string()),
other => panic!("expected Withdrawn, got {other:?}"),
}
}
withdrawn.sort();
assert_eq!(withdrawn, vec!["198.51.100.0/24", "203.0.113.0/24"]);
assert!(server.adj_rib_in.snapshot().await.is_empty());
assert!(server.received_routes().is_empty());
// Withdrawals precede the Down event.
let _up = tokio::time::timeout(TIMEOUT, session_events.recv()).await??;
@@ -300,8 +353,14 @@ async fn test_route_refresh_resends_local_routes() -> Result<()> {
let mut peer = established_peer(&server).await?;
wait_for_sessions(&server, 1).await?;
server.local_rib.announce(route("10.100.1.0/24", LocalRouteAttrs::default())).await?;
server.local_rib.announce(route("10.100.2.0/24", LocalRouteAttrs::default())).await?;
server
.local_rib
.announce(route("10.100.1.0/24", LocalRouteAttrs::default()))
.await?;
server
.local_rib
.announce(route("10.100.2.0/24", LocalRouteAttrs::default()))
.await?;
let _ = next_update(&mut peer).await?;
let _ = next_update(&mut peer).await?;
@@ -314,7 +373,11 @@ async fn test_route_refresh_resends_local_routes() -> Result<()> {
.collect::<Vec<_>>();
resent.sort();
assert_eq!(resent, vec!["10.100.1.0/24", "10.100.2.0/24"]);
assert_eq!(server.registry.list().len(), 1, "session must survive a route refresh");
assert_eq!(
server.registry.list().len(),
1,
"session must survive a route refresh"
);
Ok(())
}
@@ -345,36 +408,93 @@ async fn test_shutdown_sends_cease_to_every_peer() -> Result<()> {
}
})
.await??;
assert!(matches!(notif.error, BgpError::Cease(CeaseSubcode::AdminShutdown)), "{notif:?}");
assert!(
matches!(notif.error, BgpError::Cease(CeaseSubcode::AdminShutdown)),
"{notif:?}"
);
assert!(peer.is_closed().await);
}
Ok(())
}
// ---------------------------------------------------------------------------
// Policy
// Routes from the configuration
// ---------------------------------------------------------------------------
/// Routes listed under `originate` are in the table before any peer
/// connects, and each session is sent them when it comes up.
#[tokio::test]
async fn test_policy_rejects_prefix_outside_allowlist() -> Result<()> {
let server = TestServer::start_with_policy(
LOCAL_ASN,
LocalRibPolicy { allowed_prefixes: vec![prefix("10.100.0.0/16")], max_routes_per_owner: 2 },
)
.await?;
let mut peer = established_peer(&server).await?;
wait_for_sessions(&server, 1).await?;
async fn test_configured_routes_are_announced_to_every_new_session() -> Result<()> {
let attrs = LocalRouteAttrs {
med: Some(7),
communities: vec![(65000, 100)],
..Default::default()
};
let configured = vec![
route("10.100.1.0/24", attrs),
route("10.100.2.0/24", LocalRouteAttrs::default()),
];
let server = TestServer::start_with(LOCAL_ASN, |c| c.originate = configured).await?;
assert_eq!(server.local_rib.snapshot().await.unwrap().len(), 2);
let err = server
for router_id in [Ipv4Addr::new(10, 0, 0, 1), Ipv4Addr::new(10, 0, 0, 2)] {
let mut peer = established_peer_with_id(&server, router_id).await?;
let mut seen = [next_update(&mut peer).await?, next_update(&mut peer).await?];
seen.sort_by_key(|u| u.announced_routes[0].to_string());
assert_eq!(seen[0].announced_routes, vec![prefix("10.100.1.0/24")]);
assert!(
seen[0]
.path_attributes
.contains(&PathAttribute::Communities(CommunitiesPathAttribute(vec![
(65000, 100)
])))
);
assert_eq!(seen[1].announced_routes, vec![prefix("10.100.2.0/24")]);
}
Ok(())
}
/// A peer in our own AS is sent a route we originate with an empty AS path
/// and a LOCAL_PREF, whatever prepending is configured: our AS number goes
/// on the path only as the route leaves the AS. It used to be sent as to
/// an external peer, which an internal one refuses.
#[tokio::test]
async fn test_a_peer_in_our_own_as_gets_an_empty_path_and_a_local_pref() -> Result<()> {
let server = TestServer::start(LOCAL_ASN).await?;
let mut peer = TestPeer::connect(server.addr).await?;
peer.handshake(LOCAL_ASN, Ipv4Addr::new(10, 0, 0, 1), 90)
.await?;
// The peer reads as a router in the server's AS does: one outside it
// would drop the LOCAL_PREF.
peer.ctx.ibgp = true;
wait_for_sessions(&server, 1).await?;
assert!(server.registry.list()[0].ibgp);
server
.local_rib
.announce(route("192.0.2.0/24", LocalRouteAttrs::default()))
.await
.unwrap_err();
assert_eq!(err, LocalRibError::NotAllowed(prefix("192.0.2.0/24")));
// An allowed prefix still goes through on the same session.
server.local_rib.announce(route("10.100.9.0/24", LocalRouteAttrs::default())).await?;
.announce(route(
"10.100.1.0/24",
LocalRouteAttrs {
prepend: 2,
..LocalRouteAttrs::default()
},
))
.await?;
let update = next_update(&mut peer).await?;
assert_eq!(update.announced_routes, vec![prefix("10.100.9.0/24")]);
assert_eq!(update.announced_routes, vec![prefix("10.100.1.0/24")]);
assert!(
update
.path_attributes
.contains(&PathAttribute::AsPath(AsPathAttribute { segments: vec![] })),
"{:?}",
update.path_attributes
);
assert!(
update
.path_attributes
.contains(&PathAttribute::LocalPref(LocalPrefPathAttribute(100))),
"{:?}",
update.path_attributes
);
Ok(())
}
crates/integration-tests/tests/basic_connection.rs
+225 -54
View File
@@ -2,17 +2,41 @@ use std::net::Ipv4Addr;
use std::time::Duration;
use bgp_integration_tests::test_peer::TestPeer;
use bgp_integration_tests::test_server::RouteChange;
use bgp_integration_tests::test_server::TestServer;
use bgp_packet::errors::{BgpError, CeaseSubcode};
use bgp_packet::errors::{BgpError, CeaseSubcode, FsmSubcode, MsgHeaderSubcode, UpdateMsgSubcode};
use bgp_packet::message::{
Message, MpReachNlriPathAttribute, MpUnreachNlriPathAttribute, NlriNextHop,
NotificationMessage, PathAttribute, UpdateMessage,
AsPathAttribute, Message, MpReachNlriPathAttribute, MpUnreachNlriPathAttribute,
NextHopPathAttribute, NlriNextHop, NotificationMessage, OriginPathAttribute, PathAttribute,
UpdateMessage,
};
use bgp_primitives::constants::{AddressFamilyId, SubsequentAfi};
use bgp_primitives::constants::{AfiSafi, IpVersion};
use bgp_primitives::ip_prefix::IpPrefix;
use bgp_rib::RibEvent;
use eyre::Result;
const TIMEOUT: Duration = Duration::from_secs(5);
/// The attributes every announcement of IPv4 routes must carry. An UPDATE
/// without them has its routes withdrawn instead (RFC 7606).
fn mandatory_attributes() -> Vec<PathAttribute> {
let mut attributes = with_origin_and_as_path(PathAttribute::NextHop(NextHopPathAttribute(
Ipv4Addr::new(192, 0, 2, 1),
)));
// In order of type code: ORIGIN, AS_PATH, NEXT_HOP.
attributes.rotate_left(1);
attributes
}
/// `first` followed by the attributes every announcement must carry.
/// Routes in MP_REACH_NLRI have their next hop there.
fn with_origin_and_as_path(first: PathAttribute) -> Vec<PathAttribute> {
vec![
first,
PathAttribute::Origin(OriginPathAttribute::IGP),
AsPathAttribute::from_asns(vec![65001]),
]
}
// ---------------------------------------------------------------------------
// Connectivity
// ---------------------------------------------------------------------------
@@ -74,6 +98,147 @@ async fn test_bad_marker_does_not_crash_server() -> Result<()> {
Ok(())
}
/// Reads until a NOTIFICATION arrives, skipping KEEPALIVEs.
async fn expect_notification(peer: &mut TestPeer) -> Result<NotificationMessage> {
tokio::time::timeout(TIMEOUT, async {
loop {
match peer.recv().await? {
Message::Notification(n) => return Ok(n),
Message::KeepAlive => {}
other => eyre::bail!("expected NOTIFICATION, got {other:?}"),
}
}
})
.await?
}
/// A header with the given length and type, followed by `body`.
fn framed(length: u16, msg_type: u8, body: &[u8]) -> Vec<u8> {
let mut buf = vec![0xFF; 16];
buf.extend_from_slice(&length.to_be_bytes());
buf.push(msg_type);
buf.extend_from_slice(body);
buf
}
/// What the peer sent decides the NOTIFICATION it gets back (RFC 4271 §6),
/// in every state, and the session ends.
#[tokio::test]
async fn test_malformed_messages_are_answered_with_the_matching_notification() -> Result<()> {
let server = TestServer::start(65000).await?;
let header = |subcode| BgpError::MsgHeader(subcode);
let mut bad_marker = framed(19, 4, &[]);
bad_marker[0] = 0;
let cases: Vec<(&str, Vec<u8>, BgpError, Vec<u8>)> = vec![
(
"length beyond the maximum",
framed(5000, 2, &[]),
header(MsgHeaderSubcode::BadMessageLength),
5000u16.to_be_bytes().to_vec(),
),
(
"length below a header",
framed(5, 4, &[]),
header(MsgHeaderSubcode::BadMessageLength),
vec![0, 5],
),
(
"unknown message type",
framed(19, 9, &[]),
header(MsgHeaderSubcode::BadMessageType),
vec![9],
),
(
"marker not all ones",
bad_marker,
header(MsgHeaderSubcode::ConnNotSynchronized),
vec![],
),
(
"UPDATE with attributes running past its end",
framed(23, 2, &[0, 0, 0, 40]),
BgpError::UpdateMsg(UpdateMsgSubcode::MalformedAttrs),
vec![],
),
];
for (i, (what, bytes, error, data)) in cases.into_iter().enumerate() {
// Established: a distinct router ID each time, so that a session
// still closing does not collide with the next one.
let mut peer = TestPeer::connect(server.addr).await?;
peer.handshake(65001, Ipv4Addr::new(10, 0, 1, i as u8 + 1), 90)
.await?;
peer.send_raw(&bytes).await?;
let notification = expect_notification(&mut peer).await?;
assert_eq!(
(notification.error, notification.data),
(error.clone(), data.clone()),
"{what}, Established"
);
// OpenSent: the same bytes instead of an OPEN.
let mut peer = TestPeer::connect(server.addr).await?;
let _ = peer.recv_open().await?;
peer.send_raw(&bytes).await?;
let notification = expect_notification(&mut peer).await?;
assert_eq!(
(notification.error, notification.data),
(error, data),
"{what}, OpenSent"
);
}
// A well-formed message at the wrong time is an FSM error, not a
// message error.
let mut peer = TestPeer::connect(server.addr).await?;
let _ = peer.recv_open().await?;
peer.send(&Message::KeepAlive).await?;
let notification = expect_notification(&mut peer).await?;
assert_eq!(
notification.error,
BgpError::FsmError(FsmSubcode::UnexpectedOpenSent)
);
Ok(())
}
/// A message may arrive in pieces, with the server's own timers firing in
/// between. The server once threw away the piece it had already read when
/// that happened, so the message was silently lost (and the stream was only
/// back in step if the remainder happened to be thrown away as well).
#[tokio::test]
async fn test_message_split_across_a_keepalive_tick_is_still_one_message() -> Result<()> {
// Hold time 3 s: the server sends a KEEPALIVE every second.
let server = TestServer::start_with_hold_time(65000, 3).await?;
let mut peer = TestPeer::connect(server.addr).await?;
peer.handshake(65001, Ipv4Addr::new(10, 0, 0, 1), 3).await?;
let prefix = IpPrefix::new(IpVersion::Ipv4, vec![10], 8)?;
let mut update = bytes::BytesMut::new();
Message::Update(UpdateMessage {
withdrawn_routes: vec![],
path_attributes: mandatory_attributes(),
announced_routes: vec![prefix.into()],
})
.to_wire(&peer.ctx, &mut update)?;
// The first half, a pause long enough for the server's keepalive timer
// to fire, then the rest.
peer.send_raw(&update[..10]).await?;
tokio::time::sleep(Duration::from_millis(1300)).await;
peer.send_raw(&update[10..]).await?;
tokio::time::timeout(TIMEOUT, async {
while !server.received_routes().iter().any(|e| e.prefix == prefix) {
tokio::time::sleep(Duration::from_millis(20)).await;
}
})
.await
.map_err(|_| eyre::eyre!("the split UPDATE never reached the RIB"))?;
assert_eq!(server.registry.list().len(), 1, "session was dropped");
Ok(())
}
// ---------------------------------------------------------------------------
// OpenConfirm → Established
// ---------------------------------------------------------------------------
@@ -173,9 +338,9 @@ async fn test_server_accepts_update() -> Result<()> {
let update = Message::Update(UpdateMessage {
withdrawn_routes: vec![],
path_attributes: vec![],
path_attributes: mandatory_attributes(),
// 10.0.0.0/8 — one byte of prefix for a /8.
announced_routes: vec![IpPrefix::new(AddressFamilyId::Ipv4, vec![10], 8)?],
announced_routes: vec![IpPrefix::new(IpVersion::Ipv4, vec![10], 8)?.into()],
});
peer.send(&update).await?;
@@ -193,15 +358,16 @@ async fn test_server_accepts_ipv6_update() -> Result<()> {
.await?;
// 2001:db8::/32
let prefix = IpPrefix::new(AddressFamilyId::Ipv6, vec![0x20, 0x01, 0x0d, 0xb8], 32)?;
let prefix = IpPrefix::new(IpVersion::Ipv6, vec![0x20, 0x01, 0x0d, 0xb8], 32)?;
let update = Message::Update(UpdateMessage {
withdrawn_routes: vec![],
path_attributes: vec![PathAttribute::MpReachNlri(MpReachNlriPathAttribute {
afi: AddressFamilyId::Ipv6,
safi: SubsequentAfi::Unicast,
next_hop: NlriNextHop::Ipv6("2001:db8::1".parse().unwrap()),
prefixes: vec![prefix],
})],
path_attributes: with_origin_and_as_path(PathAttribute::MpReachNlri(
MpReachNlriPathAttribute {
family: AfiSafi::IPV6_UNICAST,
next_hop: NlriNextHop::Ipv6("2001:db8::1".parse().unwrap()),
prefixes: vec![prefix.into()],
},
)),
announced_routes: vec![],
});
peer.send(&update).await?;
@@ -219,26 +385,27 @@ async fn test_server_accepts_ipv6_update() -> Result<()> {
#[tokio::test]
async fn test_rib_records_ipv4_announce() -> Result<()> {
let server = TestServer::start(65000).await?;
let mut events = server.adj_rib_in.subscribe();
let mut events = server.route_changes();
let mut peer = TestPeer::connect(server.addr).await?;
peer.handshake(65001, Ipv4Addr::new(10, 0, 0, 1), 90).await?;
peer.handshake(65001, Ipv4Addr::new(10, 0, 0, 1), 90)
.await?;
let prefix = IpPrefix::new(AddressFamilyId::Ipv4, vec![10], 8)?;
let prefix = IpPrefix::new(IpVersion::Ipv4, vec![10], 8)?;
peer.send(&Message::Update(UpdateMessage {
withdrawn_routes: vec![],
path_attributes: vec![],
announced_routes: vec![prefix.clone()],
path_attributes: mandatory_attributes(),
announced_routes: vec![prefix.into()],
}))
.await?;
let event = tokio::time::timeout(Duration::from_secs(1), events.recv()).await??;
assert!(
matches!(&event, RibEvent::Announced { prefix: p, .. } if p == &prefix),
matches!(&event, RouteChange::Announced { prefix: p, .. } if p == &prefix),
"Expected Announced for 10.0.0.0/8, got {event:?}",
);
let snapshot = server.adj_rib_in.snapshot().await;
let snapshot = server.received_routes();
assert!(
snapshot.iter().any(|e| e.prefix == prefix),
"10.0.0.0/8 missing from RIB snapshot",
@@ -250,25 +417,26 @@ async fn test_rib_records_ipv4_announce() -> Result<()> {
#[tokio::test]
async fn test_rib_records_ipv4_withdraw() -> Result<()> {
let server = TestServer::start(65000).await?;
let mut events = server.adj_rib_in.subscribe();
let mut events = server.route_changes();
let mut peer = TestPeer::connect(server.addr).await?;
peer.handshake(65001, Ipv4Addr::new(10, 0, 0, 1), 90).await?;
peer.handshake(65001, Ipv4Addr::new(10, 0, 0, 1), 90)
.await?;
let prefix = IpPrefix::new(AddressFamilyId::Ipv4, vec![10], 8)?;
let prefix = IpPrefix::new(IpVersion::Ipv4, vec![10], 8)?;
// Announce first.
peer.send(&Message::Update(UpdateMessage {
withdrawn_routes: vec![],
path_attributes: vec![],
announced_routes: vec![prefix.clone()],
path_attributes: mandatory_attributes(),
announced_routes: vec![prefix.into()],
}))
.await?;
tokio::time::timeout(Duration::from_secs(1), events.recv()).await??;
// Now withdraw.
peer.send(&Message::Update(UpdateMessage {
withdrawn_routes: vec![prefix.clone()],
withdrawn_routes: vec![prefix.into()],
path_attributes: vec![],
announced_routes: vec![],
}))
@@ -276,11 +444,11 @@ async fn test_rib_records_ipv4_withdraw() -> Result<()> {
let event = tokio::time::timeout(Duration::from_secs(1), events.recv()).await??;
assert!(
matches!(&event, RibEvent::Withdrawn { prefix: p, .. } if p == &prefix),
matches!(&event, RouteChange::Withdrawn { prefix: p, .. } if p == &prefix),
"Expected Withdrawn for 10.0.0.0/8, got {event:?}",
);
let snapshot = server.adj_rib_in.snapshot().await;
let snapshot = server.received_routes();
assert!(
!snapshot.iter().any(|e| e.prefix == prefix),
"10.0.0.0/8 should have been removed from RIB snapshot",
@@ -292,31 +460,33 @@ async fn test_rib_records_ipv4_withdraw() -> Result<()> {
#[tokio::test]
async fn test_rib_records_ipv6_announce() -> Result<()> {
let server = TestServer::start(65000).await?;
let mut events = server.adj_rib_in.subscribe();
let mut events = server.route_changes();
let mut peer = TestPeer::connect(server.addr).await?;
peer.handshake(65001, Ipv4Addr::new(10, 0, 0, 1), 90).await?;
peer.handshake(65001, Ipv4Addr::new(10, 0, 0, 1), 90)
.await?;
let prefix = IpPrefix::new(AddressFamilyId::Ipv6, vec![0x20, 0x01, 0x0d, 0xb8], 32)?;
let prefix = IpPrefix::new(IpVersion::Ipv6, vec![0x20, 0x01, 0x0d, 0xb8], 32)?;
peer.send(&Message::Update(UpdateMessage {
withdrawn_routes: vec![],
path_attributes: vec![PathAttribute::MpReachNlri(MpReachNlriPathAttribute {
afi: AddressFamilyId::Ipv6,
safi: SubsequentAfi::Unicast,
next_hop: NlriNextHop::Ipv6("2001:db8::1".parse().unwrap()),
prefixes: vec![prefix.clone()],
})],
path_attributes: with_origin_and_as_path(PathAttribute::MpReachNlri(
MpReachNlriPathAttribute {
family: AfiSafi::IPV6_UNICAST,
next_hop: NlriNextHop::Ipv6("2001:db8::1".parse().unwrap()),
prefixes: vec![prefix.into()],
},
)),
announced_routes: vec![],
}))
.await?;
let event = tokio::time::timeout(Duration::from_secs(1), events.recv()).await??;
assert!(
matches!(&event, RibEvent::Announced { prefix: p, .. } if p == &prefix),
matches!(&event, RouteChange::Announced { prefix: p, .. } if p == &prefix),
"Expected Announced for 2001:db8::/32, got {event:?}",
);
let snapshot = server.adj_rib_in.snapshot().await;
let snapshot = server.received_routes();
assert!(
snapshot.iter().any(|e| e.prefix == prefix),
"2001:db8::/32 missing from RIB snapshot",
@@ -328,22 +498,24 @@ async fn test_rib_records_ipv6_announce() -> Result<()> {
#[tokio::test]
async fn test_rib_records_ipv6_withdraw() -> Result<()> {
let server = TestServer::start(65000).await?;
let mut events = server.adj_rib_in.subscribe();
let mut events = server.route_changes();
let mut peer = TestPeer::connect(server.addr).await?;
peer.handshake(65001, Ipv4Addr::new(10, 0, 0, 1), 90).await?;
peer.handshake(65001, Ipv4Addr::new(10, 0, 0, 1), 90)
.await?;
let prefix = IpPrefix::new(AddressFamilyId::Ipv6, vec![0x20, 0x01, 0x0d, 0xb8], 32)?;
let prefix = IpPrefix::new(IpVersion::Ipv6, vec![0x20, 0x01, 0x0d, 0xb8], 32)?;
// Announce first.
peer.send(&Message::Update(UpdateMessage {
withdrawn_routes: vec![],
path_attributes: vec![PathAttribute::MpReachNlri(MpReachNlriPathAttribute {
afi: AddressFamilyId::Ipv6,
safi: SubsequentAfi::Unicast,
next_hop: NlriNextHop::Ipv6("2001:db8::1".parse().unwrap()),
prefixes: vec![prefix.clone()],
})],
path_attributes: with_origin_and_as_path(PathAttribute::MpReachNlri(
MpReachNlriPathAttribute {
family: AfiSafi::IPV6_UNICAST,
next_hop: NlriNextHop::Ipv6("2001:db8::1".parse().unwrap()),
prefixes: vec![prefix.into()],
},
)),
announced_routes: vec![],
}))
.await?;
@@ -353,9 +525,8 @@ async fn test_rib_records_ipv6_withdraw() -> Result<()> {
peer.send(&Message::Update(UpdateMessage {
withdrawn_routes: vec![],
path_attributes: vec![PathAttribute::MpUnreachNlri(MpUnreachNlriPathAttribute {
afi: AddressFamilyId::Ipv6,
safi: SubsequentAfi::Unicast,
prefixes: vec![prefix.clone()],
family: AfiSafi::IPV6_UNICAST,
prefixes: vec![prefix.into()],
})],
announced_routes: vec![],
}))
@@ -363,11 +534,11 @@ async fn test_rib_records_ipv6_withdraw() -> Result<()> {
let event = tokio::time::timeout(Duration::from_secs(1), events.recv()).await??;
assert!(
matches!(&event, RibEvent::Withdrawn { prefix: p, .. } if p == &prefix),
matches!(&event, RouteChange::Withdrawn { prefix: p, .. } if p == &prefix),
"Expected Withdrawn for 2001:db8::/32, got {event:?}",
);
let snapshot = server.adj_rib_in.snapshot().await;
let snapshot = server.received_routes();
assert!(
!snapshot.iter().any(|e| e.prefix == prefix),
"2001:db8::/32 should have been removed from RIB snapshot",
crates/integration-tests/tests/best_route.rs
+239
View File
@@ -0,0 +1,239 @@
//! End-to-end tests of route selection: routes for one prefix from several
//! peers, which of them the server chooses, and how that is shown.
use std::net::Ipv4Addr;
use std::time::Duration;
use bgp_integration_tests::test_peer::TestPeer;
use bgp_integration_tests::test_server::{RouteChange, TestServer};
use bgp_packet::message::{
AsPathAttribute, Message, NextHopPathAttribute, OriginPathAttribute, PathAttribute,
UpdateMessage,
};
use bgp_primitives::ip_prefix::{IpPrefix, Nlri};
use eyre::Result;
use serde_json::Value;
const LOCAL_ASN: u32 = 65000;
const TIMEOUT: Duration = Duration::from_secs(5);
fn prefix(s: &str) -> IpPrefix {
IpPrefix::try_from(s).unwrap()
}
/// A session from the peer numbered `number`, in AS 65000 + `number`.
async fn peer(server: &TestServer, number: u8) -> Result<TestPeer> {
let mut peer = TestPeer::connect(server.addr).await?;
peer.handshake(
LOCAL_ASN + u32::from(number),
Ipv4Addr::new(10, 0, 0, number),
90,
)
.await?;
Ok(peer)
}
fn announcement(prefixes: &[IpPrefix], as_path: &[u32]) -> Message {
Message::Update(UpdateMessage {
withdrawn_routes: vec![],
path_attributes: vec![
PathAttribute::Origin(OriginPathAttribute::IGP),
AsPathAttribute::from_asns(as_path.to_vec()),
PathAttribute::NextHop(NextHopPathAttribute(Ipv4Addr::new(192, 0, 2, 1))),
],
announced_routes: Nlri::plain(prefixes.iter().copied()),
})
}
fn withdrawal(prefixes: &[IpPrefix]) -> Message {
Message::Update(UpdateMessage {
withdrawn_routes: Nlri::plain(prefixes.iter().copied()),
path_attributes: vec![],
announced_routes: vec![],
})
}
/// Waits until the table holds `routes` routes.
async fn wait_for_routes(server: &TestServer, routes: usize) -> Result<()> {
tokio::time::timeout(TIMEOUT, async {
while server.rib.len() != routes {
tokio::time::sleep(Duration::from_millis(5)).await;
}
})
.await?;
Ok(())
}
/// The AS path of the route the server holds as best for `prefix`.
fn best_path(server: &TestServer, prefix: &IpPrefix) -> Option<Vec<u32>> {
let best = server
.received_routes()
.into_iter()
.find(|route| route.prefix == *prefix && route.best)?;
best.attributes
.iter()
.find_map(|attribute| match attribute {
PathAttribute::AsPath(path) => Some(path.segments[0].path.clone()),
_ => None,
})
}
async fn get_json(client: &reqwest::Client, url: &str) -> Result<Value> {
Ok(client
.get(url)
.send()
.await?
.error_for_status()?
.json()
.await?)
}
/// Two peers offer the same prefix. The route with the shorter AS path is
/// chosen; when it goes, the other is.
#[tokio::test]
async fn test_the_best_route_is_chosen_and_replaced_when_it_goes() -> Result<()> {
let server = TestServer::start(LOCAL_ASN).await?;
let (base, _http) = server.serve_http().await?;
let client = reqwest::Client::new();
let mut changes = server.route_changes();
let p = prefix("203.0.113.0/24");
let mut far = peer(&server, 1).await?;
far.send(&announcement(&[p], &[65001, 64999, 64998]))
.await?;
wait_for_routes(&server, 1).await?;
assert_eq!(best_path(&server, &p), Some(vec![65001, 64999, 64998]));
let mut near = peer(&server, 2).await?;
near.send(&announcement(&[p], &[65002, 64998])).await?;
wait_for_routes(&server, 2).await?;
assert_eq!(best_path(&server, &p), Some(vec![65002, 64998]));
// Both routes are kept and shown; one is marked.
let status = get_json(&client, &format!("{base}/api/status")).await?;
assert_eq!(status["received_routes"], 2);
assert_eq!(status["received_prefixes"], 1);
let routes = get_json(&client, &format!("{base}/api/routes/received")).await?;
let best: Vec<&Value> = routes
.as_array()
.unwrap()
.iter()
.filter(|route| route["best"] == true)
.collect();
assert_eq!(routes.as_array().unwrap().len(), 2);
assert_eq!(best.len(), 1);
assert_eq!(best[0]["peer_asn"], 65002);
// The session with the best route ends: the other route takes over.
drop(near);
wait_for_routes(&server, 1).await?;
assert_eq!(best_path(&server, &p), Some(vec![65001, 64999, 64998]));
far.send(&withdrawal(&[p])).await?;
wait_for_routes(&server, 0).await?;
assert_eq!(best_path(&server, &p), None);
// A follower of the best routes is told of each change: the first
// route, the better one, back to the first, none.
let mut seen = Vec::new();
while seen.last() != Some(&RouteChange::Withdrawn { prefix: p }) {
seen.push(tokio::time::timeout(TIMEOUT, changes.recv()).await??);
}
assert!(
seen.iter()
.all(|change| matches!(change, RouteChange::Announced { prefix, .. } | RouteChange::Withdrawn { prefix } if *prefix == p)),
"{seen:?}"
);
Ok(())
}
/// The event stream reports changes of best route, and only those.
#[tokio::test]
async fn test_changes_of_best_route_are_on_the_event_stream() -> Result<()> {
let server = TestServer::start(LOCAL_ASN).await?;
let (base, _http) = server.serve_http().await?;
let p = prefix("203.0.113.0/24");
// A route that is there before the client connects is not an event.
let mut first = peer(&server, 1).await?;
first
.send(&announcement(&[prefix("198.51.100.0/24")], &[65001]))
.await?;
wait_for_routes(&server, 1).await?;
let mut stream = reqwest::Client::new()
.get(format!("{base}/api/events"))
.send()
.await?
.error_for_status()?;
first.send(&announcement(&[p], &[65001, 64998])).await?;
wait_for_routes(&server, 2).await?;
first.send(&withdrawal(&[p])).await?;
wait_for_routes(&server, 1).await?;
let mut text = String::new();
let mut events: Vec<Value> = Vec::new();
tokio::time::timeout(TIMEOUT, async {
while !events.iter().any(|event| event["type"] == "best_withdrawn") {
let chunk = stream
.chunk()
.await?
.ok_or_else(|| eyre::eyre!("stream ended"))?;
text.push_str(std::str::from_utf8(&chunk)?);
events = text
.lines()
.filter_map(|line| line.strip_prefix("data:"))
.filter_map(|data| serde_json::from_str(data.trim()).ok())
.collect();
}
Ok::<_, eyre::Error>(())
})
.await??;
let about_routes: Vec<&Value> = events
.iter()
.filter(|event| {
event["type"]
.as_str()
.is_some_and(|t| t.starts_with("best_"))
})
.collect();
// The announcement and the withdrawal may have been read as one change
// or as two; either way nothing is said of the route that was there
// from the start, and the last word is that the prefix is gone.
assert!(about_routes.len() <= 2, "{about_routes:?}");
for event in &about_routes {
let named = event["prefix"]
.as_str()
.or(event["route"]["prefix"].as_str());
assert_eq!(named, Some("203.0.113.0/24"), "{event}");
}
assert_eq!(about_routes.last().unwrap()["type"], "best_withdrawn");
Ok(())
}
/// The table can be far too large to send whole: the API gives a part of
/// it, and says how much there is.
#[tokio::test]
async fn test_received_routes_are_returned_up_to_a_limit() -> Result<()> {
let server = TestServer::start(LOCAL_ASN).await?;
let (base, _http) = server.serve_http().await?;
let client = reqwest::Client::new();
let prefixes: Vec<IpPrefix> = (0..40).map(|n| prefix(&format!("10.{n}.0.0/16"))).collect();
let mut peer = peer(&server, 1).await?;
peer.send(&announcement(&prefixes, &[65001])).await?;
wait_for_routes(&server, 40).await?;
let count = |routes: Value| routes.as_array().unwrap().len();
let url = format!("{base}/api/routes/received");
assert_eq!(count(get_json(&client, &url).await?), 40);
assert_eq!(
count(get_json(&client, &format!("{url}?limit=7")).await?),
7
);
let status = get_json(&client, &format!("{base}/api/status")).await?;
assert_eq!(status["received_routes"], 40);
Ok(())
}
crates/integration-tests/tests/collision.rs
+39 -11
View File
@@ -17,6 +17,7 @@ use bgp_primitives::ip_prefix::IpPrefix;
use bgp_rib::{LocalRoute, LocalRouteAttrs};
use bgp_server::collision::Direction;
use bgp_server::config::PeerConfig;
use bgp_server::family::FamilySet;
use eyre::Result;
use tokio::net::TcpListener;
@@ -33,7 +34,15 @@ async fn server_dialing(listener: &TcpListener) -> Result<TestServer> {
let endpoint: SocketAddr = listener.local_addr()?;
TestServer::start_with(LOCAL_ASN, |c| {
c.connect_retry = Duration::from_millis(200);
c.peers = vec![PeerConfig { name: "lab".into(), endpoint, remote_asn: PEER_ASN, hold_time_secs: 90, local_address: None }];
c.peers = vec![PeerConfig {
name: "r1".into(),
endpoint,
remote_asn: PEER_ASN,
hold_time_secs: 90,
local_address: None,
families: FamilySet::supported(),
add_path: None,
}];
})
.await
}
@@ -70,7 +79,10 @@ async fn expect_notification(peer: &mut TestPeer) -> Result<NotificationMessage>
}
fn is_collision_cease(n: &NotificationMessage) -> bool {
matches!(n.error, BgpError::Cease(CeaseSubcode::ConnCollisionResolution))
matches!(
n.error,
BgpError::Cease(CeaseSubcode::ConnCollisionResolution)
)
}
async fn wait_for_sessions(server: &TestServer, n: usize) -> Result<()> {
@@ -107,19 +119,25 @@ async fn test_simultaneous_open_peer_higher_keeps_inbound() -> Result<()> {
wait_for_sessions(&server, 1).await?;
let session = server.registry.list().remove(0);
assert_eq!(session.direction, Direction::Inbound);
assert_eq!(session.name.as_deref(), Some("lab"));
assert_eq!(session.name.as_deref(), Some("r1"));
// The survivor is fully functional.
server
.local_rib
.announce(LocalRoute::new(IpPrefix::try_from("10.100.1.0/24")?, "alice", LocalRouteAttrs::default()))
.announce(LocalRoute::new(
IpPrefix::try_from("10.100.1.0/24")?,
LocalRouteAttrs::default(),
))
.await?;
let update = tokio::time::timeout(TIMEOUT, inbound.recv_update()).await??;
assert_eq!(update.announced_routes[0].to_string(), "10.100.1.0/24");
// The dialer notices the surviving session and does not open another.
let extra = tokio::time::timeout(Duration::from_millis(700), listener.accept()).await;
assert!(extra.is_err(), "dialer opened a second connection despite an established session");
assert!(
extra.is_err(),
"dialer opened a second connection despite an established session"
);
Ok(())
}
@@ -190,7 +208,10 @@ async fn test_established_session_wins_by_default() -> Result<()> {
assert_eq!(server.registry.list().len(), 1);
server
.local_rib
.announce(LocalRoute::new(IpPrefix::try_from("10.100.1.0/24")?, "alice", LocalRouteAttrs::default()))
.announce(LocalRoute::new(
IpPrefix::try_from("10.100.1.0/24")?,
LocalRouteAttrs::default(),
))
.await?;
let update = tokio::time::timeout(TIMEOUT, first.recv_update()).await??;
assert_eq!(update.announced_routes[0].to_string(), "10.100.1.0/24");
@@ -202,7 +223,8 @@ async fn test_established_session_wins_by_default() -> Result<()> {
/// old session are flushed.
#[tokio::test]
async fn test_restarted_peer_replaces_stale_session_when_enabled() -> Result<()> {
let server = TestServer::start_with(LOCAL_ASN, |c| c.collision_detect_established_state = true).await?;
let server =
TestServer::start_with(LOCAL_ASN, |c| c.collision_detect_established_state = true).await?;
let mut stale = TestPeer::connect(server.addr).await?;
stale.handshake(PEER_ASN, HIGH_ID, 90).await?;
@@ -215,11 +237,11 @@ async fn test_restarted_peer_replaces_stale_session_when_enabled() -> Result<()>
AsPathAttribute::from_asns(vec![PEER_ASN]),
PathAttribute::NextHop(NextHopPathAttribute(Ipv4Addr::new(10, 0, 0, 9))),
],
announced_routes: vec![IpPrefix::try_from("203.0.113.0/24")?],
announced_routes: vec![IpPrefix::try_from("203.0.113.0/24")?.into()],
}))
.await?;
tokio::time::timeout(TIMEOUT, async {
while server.adj_rib_in.snapshot().await.is_empty() {
while server.received_routes().is_empty() {
tokio::time::sleep(Duration::from_millis(10)).await;
}
})
@@ -243,7 +265,10 @@ async fn test_restarted_peer_replaces_stale_session_when_enabled() -> Result<()>
}
})
.await?;
assert!(server.adj_rib_in.snapshot().await.is_empty(), "stale routes were not flushed");
assert!(
server.received_routes().is_empty(),
"stale routes were not flushed"
);
Ok(())
}
@@ -282,6 +307,9 @@ async fn test_peer_with_our_identifier_is_refused() -> Result<()> {
let mut peer = TestPeer::connect(server.addr).await?;
open_only(&mut peer, Ipv4Addr::new(127, 0, 0, 1)).await?;
let notif = expect_notification(&mut peer).await?;
assert!(matches!(notif.error, BgpError::OpenMsg(OpenMsgSubcode::BadBgpId)), "{notif:?}");
assert!(
matches!(notif.error, BgpError::OpenMsg(OpenMsgSubcode::BadBgpId)),
"{notif:?}"
);
Ok(())
}
crates/integration-tests/tests/dialer.rs
+115 -20
View File
@@ -11,6 +11,7 @@ use bgp_packet::message::Message;
use bgp_primitives::ip_prefix::IpPrefix;
use bgp_rib::{LocalRoute, LocalRouteAttrs};
use bgp_server::config::PeerConfig;
use bgp_server::family::FamilySet;
use bgp_server::session::DialState;
use eyre::Result;
use tokio::net::TcpListener;
@@ -19,12 +20,20 @@ const LOCAL_ASN: u32 = 65000;
const PEER_ASN: u32 = 65001;
const TIMEOUT: Duration = Duration::from_secs(5);
/// Starts a server configured to dial `listener`'s address as peer "lab".
/// Starts a server configured to dial `listener`'s address as peer "r1".
async fn server_dialing(listener: &TcpListener, remote_asn: u32) -> Result<TestServer> {
let endpoint: SocketAddr = listener.local_addr()?;
TestServer::start_with(LOCAL_ASN, |c| {
c.connect_retry = Duration::from_millis(200);
c.peers = vec![PeerConfig { name: "lab".into(), endpoint, remote_asn, hold_time_secs: 90, local_address: None }];
c.peers = vec![PeerConfig {
name: "r1".into(),
endpoint,
remote_asn,
hold_time_secs: 90,
local_address: None,
families: FamilySet::supported(),
add_path: None,
}];
})
.await
}
@@ -42,7 +51,9 @@ async fn test_server_dials_configured_peer_and_establishes() -> Result<()> {
let mut peer = accept(&listener).await?;
// Both sides send OPEN on connect; the test peer's handshake order works
// whichever side opened TCP.
let open = peer.handshake(PEER_ASN, Ipv4Addr::new(10, 0, 0, 2), 90).await?;
let open = peer
.handshake(PEER_ASN, Ipv4Addr::new(10, 0, 0, 2), 90)
.await?;
assert_eq!(open.asn, LOCAL_ASN as u16);
let mut events = server.registry.subscribe();
@@ -56,19 +67,30 @@ async fn test_server_dials_configured_peer_and_establishes() -> Result<()> {
}
})
.await?;
assert_eq!(listed.name.as_deref(), Some("lab"));
assert_eq!(listed.name.as_deref(), Some("r1"));
assert_eq!(listed.peer.asn, PEER_ASN);
assert_eq!(listed.peer.addr, listener.local_addr()?);
// Configured view reflects it; the dialer is not in backoff.
let (cfg, state) = server.registry.configured_peers().into_iter().next().unwrap();
assert_eq!(cfg.name, "lab");
assert!(matches!(state, DialState::Connecting | DialState::Idle), "{state:?}");
let (cfg, state) = server
.registry
.configured_peers()
.into_iter()
.next()
.unwrap();
assert_eq!(cfg.name, "r1");
assert!(
matches!(state, DialState::Connecting | DialState::Idle),
"{state:?}"
);
// Local routes flow over the dialled session too.
server
.local_rib
.announce(LocalRoute::new(IpPrefix::try_from("10.100.1.0/24")?, "alice", LocalRouteAttrs::default()))
.announce(LocalRoute::new(
IpPrefix::try_from("10.100.1.0/24")?,
LocalRouteAttrs::default(),
))
.await?;
let update = tokio::time::timeout(TIMEOUT, peer.recv_update()).await??;
assert_eq!(update.announced_routes[0].to_string(), "10.100.1.0/24");
@@ -81,7 +103,9 @@ async fn test_server_reconnects_after_peer_closes() -> Result<()> {
let server = server_dialing(&listener, PEER_ASN).await?;
let mut first = accept(&listener).await?;
first.handshake(PEER_ASN, Ipv4Addr::new(10, 0, 0, 2), 90).await?;
first
.handshake(PEER_ASN, Ipv4Addr::new(10, 0, 0, 2), 90)
.await?;
tokio::time::timeout(TIMEOUT, async {
while server.registry.list().is_empty() {
tokio::time::sleep(Duration::from_millis(10)).await;
@@ -97,11 +121,18 @@ async fn test_server_reconnects_after_peer_closes() -> Result<()> {
}
})
.await?;
let (_, state) = server.registry.configured_peers().into_iter().next().unwrap();
let (_, state) = server
.registry
.configured_peers()
.into_iter()
.next()
.unwrap();
assert!(matches!(state, DialState::Backoff { .. }), "{state:?}");
let mut second = accept(&listener).await?;
second.handshake(PEER_ASN, Ipv4Addr::new(10, 0, 0, 2), 90).await?;
second
.handshake(PEER_ASN, Ipv4Addr::new(10, 0, 0, 2), 90)
.await?;
tokio::time::timeout(TIMEOUT, async {
while server.registry.list().is_empty() {
tokio::time::sleep(Duration::from_millis(10)).await;
@@ -117,7 +148,8 @@ async fn test_wrong_asn_is_refused_with_bad_peer_as() -> Result<()> {
let _server = server_dialing(&listener, PEER_ASN).await?;
let mut peer = accept(&listener).await?;
peer.send_open(65099, Ipv4Addr::new(10, 0, 0, 2), 90).await?;
peer.send_open(65099, Ipv4Addr::new(10, 0, 0, 2), 90)
.await?;
let _ = peer.recv_open().await?;
let notif = tokio::time::timeout(TIMEOUT, async {
loop {
@@ -129,7 +161,10 @@ async fn test_wrong_asn_is_refused_with_bad_peer_as() -> Result<()> {
}
})
.await??;
assert!(matches!(notif.error, BgpError::OpenMsg(OpenMsgSubcode::BadPeerAs)), "{notif:?}");
assert!(
matches!(notif.error, BgpError::OpenMsg(OpenMsgSubcode::BadPeerAs)),
"{notif:?}"
);
// The dialer retries rather than giving up.
let _again = accept(&listener).await?;
@@ -145,11 +180,13 @@ async fn test_inbound_from_configured_address_uses_config() -> Result<()> {
let server = TestServer::start_with(LOCAL_ASN, |c| {
c.connect_retry = Duration::from_secs(60);
c.peers = vec![PeerConfig {
name: "lab".into(),
name: "r1".into(),
endpoint: SocketAddr::new(Ipv4Addr::LOCALHOST.into(), 1),
remote_asn: PEER_ASN,
hold_time_secs: 90,
local_address: None,
families: FamilySet::supported(),
add_path: None,
}];
})
.await?;
@@ -159,11 +196,15 @@ async fn test_inbound_from_configured_address_uses_config() -> Result<()> {
bad.send_open(65099, Ipv4Addr::new(10, 0, 0, 2), 90).await?;
let _ = bad.recv_open().await?;
let msg = tokio::time::timeout(TIMEOUT, bad.recv()).await??;
assert!(matches!(&msg, Message::Notification(n) if matches!(n.error, BgpError::OpenMsg(OpenMsgSubcode::BadPeerAs))), "{msg:?}");
assert!(
matches!(&msg, Message::Notification(n) if matches!(n.error, BgpError::OpenMsg(OpenMsgSubcode::BadPeerAs))),
"{msg:?}"
);
// Right ASN inbound → established, named.
let mut good = TestPeer::connect(server.addr).await?;
good.handshake(PEER_ASN, Ipv4Addr::new(10, 0, 0, 2), 90).await?;
good.handshake(PEER_ASN, Ipv4Addr::new(10, 0, 0, 2), 90)
.await?;
let listed = tokio::time::timeout(TIMEOUT, async {
loop {
if let Some(s) = server.registry.list().first().cloned() {
@@ -173,7 +214,7 @@ async fn test_inbound_from_configured_address_uses_config() -> Result<()> {
}
})
.await?;
assert_eq!(listed.name.as_deref(), Some("lab"));
assert_eq!(listed.name.as_deref(), Some("r1"));
Ok(())
}
@@ -187,11 +228,13 @@ async fn test_local_address_is_used_as_the_source() -> Result<()> {
let _server = TestServer::start_with(LOCAL_ASN, |c| {
c.connect_retry = Duration::from_millis(200);
c.peers = vec![PeerConfig {
name: "lab".into(),
name: "r1".into(),
endpoint,
remote_asn: PEER_ASN,
hold_time_secs: 90,
local_address: Some(Ipv4Addr::LOCALHOST.into()),
families: FamilySet::supported(),
add_path: None,
}];
})
.await?;
@@ -202,17 +245,21 @@ async fn test_local_address_is_used_as_the_source() -> Result<()> {
let server = TestServer::start_with(LOCAL_ASN, |c| {
c.connect_retry = Duration::from_millis(200);
c.peers = vec![PeerConfig {
name: "lab".into(),
name: "r1".into(),
endpoint,
remote_asn: PEER_ASN,
hold_time_secs: 90,
local_address: Some(Ipv4Addr::new(192, 0, 2, 1).into()),
families: FamilySet::supported(),
add_path: None,
}];
})
.await?;
let reason = tokio::time::timeout(TIMEOUT, async {
loop {
if let Some((_, DialState::Backoff { last_error, .. })) = server.registry.configured_peers().into_iter().next() {
if let Some((_, DialState::Backoff { last_error, .. })) =
server.registry.configured_peers().into_iter().next()
{
return last_error;
}
tokio::time::sleep(Duration::from_millis(20)).await;
@@ -222,3 +269,51 @@ async fn test_local_address_is_used_as_the_source() -> Result<()> {
assert!(reason.contains("local_address 192.0.2.1"), "{reason}");
Ok(())
}
/// A peer that takes the connection and then refuses the session, here
/// because it is not the AS we are configured to expect, will refuse it
/// again. It is dialled ever more slowly, like a peer that cannot be
/// reached: connecting used to count as success and reset the wait, so
/// such a peer was dialled every few seconds for ever.
#[tokio::test]
async fn test_a_peer_that_refuses_the_session_is_dialled_ever_more_slowly() -> Result<()> {
let listener = TcpListener::bind((Ipv4Addr::LOCALHOST, 0)).await?;
let server = server_dialing(&listener, PEER_ASN).await?;
let mut waits = Vec::new();
for _ in 0..3 {
let mut peer = accept(&listener).await?;
peer.send_open(65099, Ipv4Addr::new(10, 0, 0, 2), 90)
.await?;
let _ = peer.recv_open().await?;
let refusal = tokio::time::timeout(TIMEOUT, peer.recv()).await??;
assert!(
matches!(&refusal, Message::Notification(n) if n.error == BgpError::OpenMsg(OpenMsgSubcode::BadPeerAs)),
"{refusal:?}"
);
drop(peer);
// How long the dialer says it will wait before trying again.
let wait = tokio::time::timeout(TIMEOUT, async {
loop {
if let Some((_, DialState::Backoff { until, last_error })) =
server.registry.configured_peers().into_iter().next()
{
assert!(last_error.contains("AS65099"), "{last_error}");
return until
.duration_since(std::time::SystemTime::now())
.unwrap_or_default();
}
tokio::time::sleep(Duration::from_millis(5)).await;
}
})
.await?;
waits.push(wait);
}
// 200 ms, then 400, then 800, less what passed before each was read.
assert!(
waits[2] > Duration::from_millis(500),
"the waits were {waits:?}"
);
Ok(())
}
crates/integration-tests/tests/failure.rs
+87
View File
@@ -0,0 +1,87 @@
//! What the server does when a part of it that it cannot do without fails.
//!
//! The local RIB's actor stops only by panicking. Before, nothing noticed:
//! sessions stayed up, nothing was announced any more, and the status page
//! showed an empty table. Now the server tells its peers it is going and
//! stops with an error, for whatever supervises it to restart it.
use std::net::Ipv4Addr;
use std::time::Duration;
use bgp_integration_tests::test_peer::TestPeer;
use bgp_integration_tests::test_server::TestServer;
use bgp_packet::errors::{BgpError, CeaseSubcode};
use bgp_packet::message::Message;
use bgp_rib::RibStopped;
use bgp_server::error::ServerError;
use eyre::{Result, bail};
const LOCAL_ASN: u32 = 65000;
const PEER_ASN: u32 = 65001;
const TIMEOUT: Duration = Duration::from_secs(5);
async fn established(server: &TestServer) -> Result<TestPeer> {
let mut peer = TestPeer::connect(server.addr).await?;
peer.handshake(PEER_ASN, Ipv4Addr::new(10, 0, 0, 1), 90)
.await?;
tokio::time::timeout(TIMEOUT, async {
while server.registry.list().is_empty() {
tokio::time::sleep(Duration::from_millis(10)).await;
}
})
.await?;
Ok(peer)
}
/// Reads until the server's NOTIFICATION, skipping KEEPALIVEs.
async fn farewell(peer: &mut TestPeer) -> Result<BgpError> {
tokio::time::timeout(TIMEOUT, async {
loop {
match peer.recv().await? {
Message::KeepAlive => {}
Message::Notification(n) => return Ok(n.error),
other => bail!("expected NOTIFICATION, got {other:?}"),
}
}
})
.await?
}
#[tokio::test]
async fn test_losing_the_local_rib_stops_the_server_and_tells_the_peers() -> Result<()> {
let mut server = TestServer::start(LOCAL_ASN).await?;
let (base, _http) = server.serve_http().await?;
let mut peer = established(&server).await?;
server.local_rib.stop_for_testing();
let outcome = tokio::time::timeout(TIMEOUT, server.stopped()).await?;
assert!(
matches!(outcome, Err(ServerError::RibStopped(RibStopped { rib })) if rib == "local RIB"),
"{outcome:?}"
);
assert_eq!(
farewell(&mut peer).await?,
BgpError::Cease(CeaseSubcode::AdminShutdown)
);
// The status interface says the table cannot be read, where it used to
// show an empty one.
let client = reqwest::Client::new();
for path in ["/api/status", "/api/routes/local", "/metricz"] {
let response = client.get(format!("{base}{path}")).send().await?;
assert_eq!(response.status(), 503, "{path}");
assert_eq!(
response.text().await?,
"the local RIB has stopped",
"{path}"
);
}
// What does not need that table still answers.
let response = client
.get(format!("{base}/api/routes/received"))
.send()
.await?;
assert_eq!(response.status(), 200);
Ok(())
}
crates/integration-tests/tests/http_api.rs
+121 -144
View File
@@ -6,12 +6,11 @@ use std::time::Duration;
use bgp_integration_tests::test_peer::TestPeer;
use bgp_integration_tests::test_server::TestServer;
use bgp_packet::message::{
AsPathAttribute, CommunitiesPathAttribute, Message, NextHopPathAttribute,
AsPathAttribute, CommunitiesPathAttribute, LargeCommunity, Message, NextHopPathAttribute,
OriginPathAttribute, PathAttribute, UpdateMessage,
};
use bgp_primitives::ip_prefix::IpPrefix;
use bgp_primitives::parser::ParserContext;
use bgp_rib::LocalRibPolicy;
use bgp_rib::{LocalRoute, LocalRouteAttrs};
use eyre::Result;
use serde_json::{Value, json};
@@ -25,7 +24,8 @@ fn prefix(s: &str) -> IpPrefix {
async fn established_peer(server: &TestServer) -> Result<TestPeer> {
let mut peer = TestPeer::connect(server.addr).await?;
peer.handshake(PEER_ASN, Ipv4Addr::new(10, 0, 0, 1), 90).await?;
peer.handshake(PEER_ASN, Ipv4Addr::new(10, 0, 0, 1), 90)
.await?;
tokio::time::timeout(TIMEOUT, async {
while server.registry.list().is_empty() {
tokio::time::sleep(Duration::from_millis(10)).await;
@@ -36,7 +36,13 @@ async fn established_peer(server: &TestServer) -> Result<TestPeer> {
}
async fn get_json(client: &reqwest::Client, url: &str) -> Result<Value> {
Ok(client.get(url).send().await?.error_for_status()?.json().await?)
Ok(client
.get(url)
.send()
.await?
.error_for_status()?
.json()
.await?)
}
#[tokio::test]
@@ -66,138 +72,78 @@ async fn test_status_and_peers() -> Result<()> {
assert_eq!(peers[0]["peer_asn"], PEER_ASN);
assert_eq!(peers[0]["local_addr"], server.addr.to_string());
assert_eq!(peers[0]["four_octet_asn"], true);
assert_eq!(
peers[0]["address_families"],
serde_json::json!(["IPv4 unicast", "IPv6 unicast"])
);
Ok(())
}
/// The full loop: POST an announcement, the BGP peer receives the UPDATE,
/// the local table lists it, the wire view matches the bytes, DELETE
/// withdraws it.
/// Originated routes are listed with their attributes as text.
#[tokio::test]
async fn test_announce_over_http_reaches_peer() -> Result<()> {
let server = TestServer::start(LOCAL_ASN).await?;
let (base, _http) = server.serve_http().await?;
let client = reqwest::Client::new();
let mut peer = established_peer(&server).await?;
let resp = client
.post(format!("{base}/api/routes/local"))
.json(&json!({
"prefix": "10.100.1.0/24",
"owner": "alice",
"prepend": 1,
"med": 10,
"communities": ["65000:100", "NO_EXPORT"],
"large_communities": ["65000:1:2"],
}))
.send()
.await?;
assert_eq!(resp.status(), 200, "{}", resp.text().await?);
let body: Value = resp.json().await?;
assert_eq!(body["prefix"], "10.100.1.0/24");
assert_eq!(body["origin"], "IGP");
assert_eq!(body["communities"], json!(["65000:100", "65535:65281"]));
assert_eq!(body["large_communities"], json!(["65000:1:2"]));
// The peer sees the UPDATE.
let update: UpdateMessage = tokio::time::timeout(TIMEOUT, peer.recv_update()).await??;
assert_eq!(update.announced_routes, vec![prefix("10.100.1.0/24")]);
assert_eq!(update.path_attributes[1], AsPathAttribute::from_asns(vec![LOCAL_ASN, LOCAL_ASN]));
assert_eq!(update.path_attributes[2], PathAttribute::NextHop(NextHopPathAttribute(Ipv4Addr::LOCALHOST)));
assert!(update.path_attributes.contains(&PathAttribute::Communities(CommunitiesPathAttribute(vec![
(65000, 100),
(65535, 65281)
]))));
// Local table lists it.
let local = get_json(&client, &format!("{base}/api/routes/local")).await?;
assert_eq!(local.as_array().unwrap().len(), 1);
assert_eq!(local[0]["owner"], "alice");
// Wire view reproduces exactly what went over the socket.
let wire = get_json(&client, &format!("{base}/api/routes/local/wire?prefix=10.100.1.0/24")).await?;
assert_eq!(wire.as_array().unwrap().len(), 1);
assert_eq!(wire[0]["peer_asn"], PEER_ASN);
assert_eq!(wire[0]["four_octet_asn"], true);
assert_eq!(wire[0]["summary"]["as_path"], json!([LOCAL_ASN, LOCAL_ASN]));
let hex = wire[0]["hex"].as_str().unwrap();
let ctx = ParserContext { four_octet_asn: peer.ctx.four_octet_asn, address_family: None };
let mut expected = bytes::BytesMut::new();
Message::Update(update).to_wire(&ctx, &mut expected)?;
let expected_hex: String = expected.iter().map(|b| format!("{b:02x}")).collect();
assert_eq!(hex, expected_hex);
assert!(hex.starts_with(&"ff".repeat(16)), "marker missing");
// Withdraw.
let resp = client
.delete(format!("{base}/api/routes/local"))
.json(&json!({ "prefix": "10.100.1.0/24", "owner": "alice" }))
.send()
.await?;
assert_eq!(resp.status(), 200);
let withdrawn: UpdateMessage = tokio::time::timeout(TIMEOUT, peer.recv_update()).await??;
assert_eq!(withdrawn.withdrawn_routes, vec![prefix("10.100.1.0/24")]);
let local = get_json(&client, &format!("{base}/api/routes/local")).await?;
assert!(local.as_array().unwrap().is_empty());
Ok(())
}
#[tokio::test]
async fn test_validation_and_policy_status_codes() -> Result<()> {
let server = TestServer::start_with_policy(
LOCAL_ASN,
LocalRibPolicy { allowed_prefixes: vec![prefix("10.100.0.0/16")], max_routes_per_owner: 1 },
)
async fn test_originated_routes_are_listed() -> Result<()> {
let attrs = LocalRouteAttrs {
prepend: 1,
med: Some(10),
communities: vec![(65000, 100), (65535, 65281)],
large_communities: vec![LargeCommunity {
global_admin: 65000,
data_1: 1,
data_2: 2,
}],
..Default::default()
};
let server = TestServer::start_with(LOCAL_ASN, |c| {
c.originate = vec![LocalRoute::new(prefix("10.100.1.0/24"), attrs)];
})
.await?;
let (base, _http) = server.serve_http().await?;
let client = reqwest::Client::new();
let url = format!("{base}/api/routes/local");
let post = |body: Value| client.post(&url).json(&body).send();
let local = get_json(&client, &format!("{base}/api/routes/local")).await?;
assert_eq!(local.as_array().unwrap().len(), 1);
assert_eq!(local[0]["prefix"], "10.100.1.0/24");
assert_eq!(local[0]["origin"], "IGP");
assert_eq!(local[0]["prepend"], 1);
assert_eq!(local[0]["med"], 10);
assert_eq!(local[0]["communities"], json!(["65000:100", "65535:65281"]));
assert_eq!(local[0]["large_communities"], json!(["65000:1:2"]));
assert_eq!(
get_json(&client, &format!("{base}/api/status")).await?["local_routes"],
1
);
Ok(())
}
// Malformed input → 400 with an error message.
let resp = post(json!({ "prefix": "not-a-prefix", "owner": "alice" })).await?;
assert_eq!(resp.status(), 400);
assert!(resp.json::<Value>().await?["error"].as_str().unwrap().contains("invalid prefix"));
/// The API only reports. Nothing reachable over HTTP changes what the
/// router announces.
#[tokio::test]
async fn test_api_is_read_only() -> Result<()> {
let server = TestServer::start(LOCAL_ASN).await?;
let (base, _http) = server.serve_http().await?;
let client = reqwest::Client::new();
let resp = post(json!({ "prefix": "10.100.1.0/24", "owner": "" })).await?;
assert_eq!(resp.status(), 400);
let resp = post(json!({ "prefix": "10.100.1.0/24", "owner": "alice", "communities": ["bogus"] })).await?;
assert_eq!(resp.status(), 400);
let resp = post(json!({ "prefix": "10.100.1.0/24", "owner": "alice", "origin": "weird" })).await?;
assert_eq!(resp.status(), 400);
// Outside allowlist → 403.
let resp = post(json!({ "prefix": "192.0.2.0/24", "owner": "alice" })).await?;
assert_eq!(resp.status(), 403);
// First allowed route is fine.
assert_eq!(post(json!({ "prefix": "10.100.1.0/24", "owner": "alice" })).await?.status(), 200);
// Per-owner limit → 422.
let resp = post(json!({ "prefix": "10.100.2.0/24", "owner": "alice" })).await?;
assert_eq!(resp.status(), 422);
// Someone else's prefix → 409, for both replace and withdraw.
let resp = post(json!({ "prefix": "10.100.1.0/24", "owner": "bob" })).await?;
assert_eq!(resp.status(), 409);
let resp = client
.delete(&url)
.json(&json!({ "prefix": "10.100.1.0/24", "owner": "bob" }))
.send()
.await?;
assert_eq!(resp.status(), 409);
// Withdrawing an unknown prefix is a 200 with null.
let resp = client.delete(&url).json(&json!({ "prefix": "10.100.7.0/24" })).send().await?;
assert_eq!(resp.status(), 200);
assert_eq!(resp.json::<Value>().await?, Value::Null);
// Wire view for an unannounced prefix → 404.
let resp = client.get(format!("{base}/api/routes/local/wire?prefix=10.100.7.0/24")).send().await?;
assert_eq!(resp.status(), 404);
let body = json!({ "prefix": "10.100.1.0/24" });
for path in [
"/api/routes/local",
"/api/routes/received",
"/api/peers",
"/api/status",
] {
let post = client
.post(format!("{base}{path}"))
.json(&body)
.send()
.await?;
assert_eq!(post.status(), 405, "POST {path}");
let delete = client
.delete(format!("{base}{path}"))
.json(&body)
.send()
.await?;
assert_eq!(delete.status(), 405, "DELETE {path}");
}
assert!(server.local_rib.snapshot().await.unwrap().is_empty());
Ok(())
}
@@ -217,7 +163,7 @@ async fn test_received_routes_summary() -> Result<()> {
PathAttribute::NextHop(NextHopPathAttribute(Ipv4Addr::new(10, 0, 0, 1))),
PathAttribute::Communities(CommunitiesPathAttribute(vec![(65001, 7)])),
],
announced_routes: vec![prefix("203.0.113.0/24")],
announced_routes: vec![prefix("203.0.113.0/24").into()],
}))
.await?;
@@ -246,13 +192,21 @@ async fn test_event_stream() -> Result<()> {
let (base, _http) = server.serve_http().await?;
let client = reqwest::Client::new();
let mut resp = client.get(format!("{base}/api/events")).send().await?.error_for_status()?;
assert!(resp.headers()["content-type"].to_str()?.starts_with("text/event-stream"));
let mut resp = client
.get(format!("{base}/api/events"))
.send()
.await?
.error_for_status()?;
assert!(
resp.headers()["content-type"]
.to_str()?
.starts_with("text/event-stream")
);
let _peer = established_peer(&server).await?;
server
.local_rib
.announce(bgp_rib::LocalRoute::new(prefix("10.100.1.0/24"), "alice", Default::default()))
.announce(LocalRoute::new(prefix("10.100.1.0/24"), Default::default()))
.await?;
// Collect SSE frames until we have seen both events.
@@ -261,13 +215,18 @@ async fn test_event_stream() -> Result<()> {
let mut seen_announce = false;
tokio::time::timeout(TIMEOUT, async {
while !(seen_up && seen_announce) {
let chunk = resp.chunk().await?.ok_or_else(|| eyre::eyre!("stream ended"))?;
let chunk = resp
.chunk()
.await?
.ok_or_else(|| eyre::eyre!("stream ended"))?;
buf.push_str(std::str::from_utf8(&chunk)?);
for line in buf.lines().filter_map(|l| l.strip_prefix("data:")) {
let ev: Value = serde_json::from_str(line.trim())?;
match ev["type"].as_str() {
Some("session_up") if ev["peer"]["peer_asn"] == PEER_ASN => seen_up = true,
Some("local_announced") if ev["route"]["prefix"] == "10.100.1.0/24" => seen_announce = true,
Some("local_announced") if ev["route"]["prefix"] == "10.100.1.0/24" => {
seen_announce = true
}
_ => {}
}
}
@@ -286,13 +245,19 @@ async fn test_statusz_page_is_served() -> Result<()> {
let resp = client.get(format!("{base}/statusz")).send().await?;
assert_eq!(resp.status(), 200);
assert!(resp.headers()["content-type"].to_str()?.starts_with("text/html"));
assert!(
resp.headers()["content-type"]
.to_str()?
.starts_with("text/html")
);
let body = resp.text().await?;
assert!(body.contains("<title>Status for ferrix_bgp</title>"));
assert!(body.contains("/api/events"));
// The root redirects to the page.
let no_redirect = reqwest::Client::builder().redirect(reqwest::redirect::Policy::none()).build()?;
let no_redirect = reqwest::Client::builder()
.redirect(reqwest::redirect::Policy::none())
.build()?;
let resp = no_redirect.get(format!("{base}/")).send().await?;
assert_eq!(resp.status(), 307);
assert_eq!(resp.headers()["location"], "/statusz");
@@ -300,13 +265,19 @@ async fn test_statusz_page_is_served() -> Result<()> {
}
#[tokio::test]
async fn test_metricz_and_well_known() -> Result<()> {
async fn test_metricz() -> Result<()> {
let server = TestServer::start(LOCAL_ASN).await?;
let (base, _http) = server.serve_http().await?;
let client = reqwest::Client::new();
let _peer = established_peer(&server).await?;
let text = client.get(format!("{base}/metricz")).send().await?.error_for_status()?.text().await?;
let text = client
.get(format!("{base}/metricz"))
.send()
.await?
.error_for_status()?
.text()
.await?;
// Metrics are process-global, so other tests in this binary contribute
// too; assert on presence and a lower bound rather than exact values.
let metric = |name: &str| -> Option<i64> {
@@ -314,13 +285,19 @@ async fn test_metricz_and_well_known() -> Result<()> {
.find(|l| l.starts_with(name) && l[name.len()..].starts_with(' '))
.and_then(|l| l.rsplit(' ').next()?.parse().ok())
};
assert!(metric("bgp_sessions_established").unwrap_or(0) >= 1, "{text}");
assert!(metric("bgp_messages_sent_total{type=\"open\"}").unwrap_or(0) >= 1, "{text}");
assert!(metric("bgp_messages_received_total{type=\"keepalive\"}").unwrap_or(0) >= 1, "{text}");
assert!(
metric("bgp_sessions_established").unwrap_or(0) >= 1,
"{text}"
);
assert!(
metric("bgp_messages_sent_total{type=\"open\"}").unwrap_or(0) >= 1,
"{text}"
);
assert!(
metric("bgp_messages_received_total{type=\"keepalive\"}").unwrap_or(0) >= 1,
"{text}"
);
assert!(metric("bgp_local_routes").is_some(), "{text}");
assert!(metric("bgp_adj_rib_in_routes").is_some(), "{text}");
let wk = get_json(&client, &format!("{base}/api/communities/well-known")).await?;
assert!(wk.as_array().unwrap().iter().any(|c| c["name"] == "NO_EXPORT" && c["value"] == "65535:65281"));
assert!(metric("bgp_rib_routes").is_some(), "{text}");
Ok(())
}
crates/integration-tests/tests/interop.rs
+42 -13
View File
@@ -7,7 +7,7 @@ use bgp_integration_tests::test_peer::TestPeer;
use bgp_integration_tests::test_server::TestServer;
use bgp_packet::message::{Message, PathAttribute};
use bgp_packet::open::{Capability, OpenOption};
use bgp_primitives::constants::{AddressFamilyId, SubsequentAfi};
use bgp_primitives::constants::{Afi, AfiSafi, Safi};
use bgp_primitives::ip_prefix::IpPrefix;
use bgp_rib::{LocalRoute, LocalRouteAttrs};
use eyre::Result;
@@ -61,9 +61,9 @@ async fn test_server_advertises_ipv4_and_ipv6_unicast() -> Result<()> {
OpenOption::Capabilities(c) => c.iter(),
})
.collect();
for afi in [AddressFamilyId::Ipv4, AddressFamilyId::Ipv6] {
for afi in [Afi::IPV4, Afi::IPV6] {
assert!(
caps.iter().any(|c| matches!(c, Capability::MultiProtocol { afi: a, safi: SubsequentAfi::Unicast } if *a == afi)),
caps.iter().any(|c| matches!(c, Capability::MultiProtocol { afi: a, safi: Safi::UNICAST } if *a == afi)),
"missing MP capability for {afi:?}: {caps:?}"
);
}
@@ -90,30 +90,59 @@ async fn test_bird_open_and_end_of_rib_are_accepted() -> Result<()> {
})
.await?;
let session = server.registry.list().remove(0);
assert_eq!(session.peer.asn, 210036, "4-octet ASN must come from the capability, not AS_TRANS");
assert_eq!(
session.peer.asn, 210036,
"4-octet ASN must come from the capability, not AS_TRANS"
);
assert!(session.four_octet_asn);
// End-of-RIB must neither break the session nor create a route.
peer.send_raw(IPV6_END_OF_RIB).await?;
peer.send(&Message::KeepAlive).await?;
tokio::time::sleep(Duration::from_millis(200)).await;
assert_eq!(server.registry.list().len(), 1, "session dropped after End-of-RIB");
assert!(server.adj_rib_in.snapshot().await.is_empty());
assert_eq!(
server.registry.list().len(),
1,
"session dropped after End-of-RIB"
);
assert!(server.received_routes().is_empty());
Ok(())
}
/// Over an IPv4 test transport an IPv6 route cannot be sent (no IPv6 next
/// hop), so this only checks the builder the advertiser uses: the UPDATE for
/// a lab /48 carries MP_REACH_NLRI with an IPv6 next hop and no NEXT_HOP.
/// an IPv6 prefix carries MP_REACH_NLRI with an IPv6 next hop and no NEXT_HOP.
#[tokio::test]
async fn test_ipv6_lab_prefix_update_shape() -> Result<()> {
let route = LocalRoute::new(IpPrefix::try_from("2a0d:d741:1::/48")?, "student1", LocalRouteAttrs::default());
async fn test_ipv6_prefix_update_shape() -> Result<()> {
let route = LocalRoute::new(
IpPrefix::try_from("2001:db8:1::/48")?,
LocalRouteAttrs::default(),
);
let nh: std::net::Ipv6Addr = "fd00:179::2".parse()?;
let update = bgp_server::advertise::build_announce(&route, 4200000001, nh.into()).expect("ipv6 over ipv6");
let update = bgp_server::advertise::build_announce(
&route,
&bgp_server::advertise::Towards {
local_asn: 4200000001,
next_hop: nh.into(),
four_octet_asn: true,
ibgp: false,
},
)
.expect("ipv6 over ipv6");
assert!(update.announced_routes.is_empty());
assert!(!update.path_attributes.iter().any(|a| matches!(a, PathAttribute::NextHop(_))));
assert!(update.path_attributes.iter().any(|a| matches!(a, PathAttribute::MpReachNlri(m)
if m.afi == AddressFamilyId::Ipv6 && m.prefixes[0].to_string() == "2a0d:d741:1::/48")));
assert!(
!update
.path_attributes
.iter()
.any(|a| matches!(a, PathAttribute::NextHop(_)))
);
assert!(
update
.path_attributes
.iter()
.any(|a| matches!(a, PathAttribute::MpReachNlri(m)
if m.family == AfiSafi::IPV6_UNICAST && m.prefixes[0].to_string() == "2001:db8:1::/48"))
);
let _ = Ipv4Addr::LOCALHOST;
Ok(())
}
crates/integration-tests/tests/negotiation.rs
+321
View File
@@ -0,0 +1,321 @@
//! End-to-end tests for capability negotiation: what the two OPENs agree on
//! decides which routes are sent and accepted, and how they are encoded.
use std::net::{Ipv4Addr, Ipv6Addr, SocketAddr};
use std::time::Duration;
use bgp_integration_tests::test_peer::TestPeer;
use bgp_integration_tests::test_server::TestServer;
use bgp_packet::errors::{BgpError, OpenMsgSubcode};
use bgp_packet::message::{
AsPathAttribute, Message, MpReachNlriPathAttribute, NextHopPathAttribute, NlriNextHop,
OriginPathAttribute, PathAttribute, UpdateMessage,
};
use bgp_packet::open::Capability;
use bgp_primitives::constants::{Afi, AfiSafi, Safi};
use bgp_primitives::ip_prefix::IpPrefix;
use bgp_rib::{LocalRoute, LocalRouteAttrs};
use bgp_server::config::PeerConfig;
use bgp_server::family::{FamilySet, SUPPORTED};
use eyre::{Result, bail};
const LOCAL_ASN: u32 = 65000;
const PEER_ASN: u32 = 65001;
const PEER_ID: Ipv4Addr = Ipv4Addr::new(10, 0, 0, 1);
const TIMEOUT: Duration = Duration::from_secs(5);
fn prefix(s: &str) -> IpPrefix {
IpPrefix::try_from(s).unwrap()
}
fn only(family: AfiSafi) -> FamilySet {
[family].into_iter().collect()
}
fn four_octet() -> Capability {
Capability::FourByteAsn { asn: PEER_ASN }
}
async fn wait_for_session(server: &TestServer) -> Result<bgp_server::session::SessionHandle> {
Ok(tokio::time::timeout(TIMEOUT, async {
loop {
if let Some(session) = server.registry.list().into_iter().next() {
return session;
}
tokio::time::sleep(Duration::from_millis(10)).await;
}
})
.await?)
}
/// An UPDATE announcing one IPv4 route in the body and one IPv6 route in
/// MP_REACH_NLRI.
fn dual_stack_update() -> UpdateMessage {
UpdateMessage {
withdrawn_routes: vec![],
path_attributes: vec![
PathAttribute::Origin(OriginPathAttribute::IGP),
AsPathAttribute::from_asns(vec![PEER_ASN]),
PathAttribute::NextHop(NextHopPathAttribute(Ipv4Addr::new(192, 0, 2, 1))),
PathAttribute::MpReachNlri(MpReachNlriPathAttribute {
family: AfiSafi::IPV6_UNICAST,
next_hop: NlriNextHop::Ipv6(Ipv6Addr::new(0x2001, 0xdb8, 0, 0, 0, 0, 0, 1)),
prefixes: vec![prefix("2001:db8:1::/48").into()],
}),
],
announced_routes: vec![prefix("192.0.2.0/24").into()],
}
}
/// The session carries the families both OPENs offered, and the server's
/// OPEN offers every family by default.
#[tokio::test]
async fn test_families_are_the_ones_both_sides_offered() -> Result<()> {
let server = TestServer::start(LOCAL_ASN).await?;
let mut peer = TestPeer::connect(server.addr).await?;
let open = peer
.handshake_with(
PEER_ASN,
PEER_ID,
90,
vec![
four_octet(),
Capability::multiprotocol(AfiSafi::IPV4_UNICAST),
],
)
.await?;
for family in SUPPORTED {
assert!(
open.capabilities()
.any(|c| *c == Capability::multiprotocol(family)),
"the server's OPEN offers {family}"
);
}
let session = wait_for_session(&server).await?;
assert_eq!(session.families, only(AfiSafi::IPV4_UNICAST));
Ok(())
}
/// A route of a family the peer did not offer is not accepted from it. The
/// rest of the UPDATE is, and the session stays up.
#[tokio::test]
async fn test_routes_of_a_family_not_negotiated_are_not_accepted() -> Result<()> {
let server = TestServer::start(LOCAL_ASN).await?;
let mut peer = TestPeer::connect(server.addr).await?;
peer.handshake_with(
PEER_ASN,
PEER_ID,
90,
vec![
four_octet(),
Capability::multiprotocol(AfiSafi::IPV4_UNICAST),
],
)
.await?;
wait_for_session(&server).await?;
peer.send(&Message::Update(dual_stack_update())).await?;
let routes = tokio::time::timeout(TIMEOUT, async {
loop {
let routes = server.received_routes();
if !routes.is_empty() {
return routes;
}
tokio::time::sleep(Duration::from_millis(10)).await;
}
})
.await?;
// One UPDATE is applied as a whole, so the IPv6 route is not on its way.
assert_eq!(routes.len(), 1);
assert_eq!(routes[0].prefix, prefix("192.0.2.0/24"));
assert_eq!(server.registry.list().len(), 1, "the session survives");
Ok(())
}
/// A route of a family the peer did not offer is not sent to it.
#[tokio::test]
async fn test_routes_of_a_family_not_negotiated_are_not_sent() -> Result<()> {
let server = TestServer::start_with(LOCAL_ASN, |c| {
c.originate = vec![LocalRoute::new(
prefix("10.100.0.0/16"),
LocalRouteAttrs::default(),
)];
})
.await?;
// This peer wants IPv6 only; the server has an IPv4 route.
let mut v6_only = TestPeer::connect(server.addr).await?;
v6_only
.handshake_with(
PEER_ASN,
PEER_ID,
90,
vec![
four_octet(),
Capability::multiprotocol(AfiSafi::IPV6_UNICAST),
],
)
.await?;
let session = wait_for_session(&server).await?;
assert_eq!(session.families, only(AfiSafi::IPV6_UNICAST));
// A second peer that takes IPv4 is sent the route. The advertiser
// handles sessions in the order they came up, so by the time this one
// has its route the first has been dealt with.
let mut dual = TestPeer::connect(server.addr).await?;
dual.handshake(PEER_ASN, Ipv4Addr::new(10, 0, 0, 2), 90)
.await?;
let update = tokio::time::timeout(TIMEOUT, dual.recv_update()).await??;
assert_eq!(update.announced_routes, vec![prefix("10.100.0.0/16")]);
let nothing = tokio::time::timeout(Duration::from_millis(200), v6_only.recv_update()).await;
assert!(nothing.is_err(), "got {nothing:?}");
Ok(())
}
/// A configured peer is offered only the families configured for it, and a
/// peer that shares none of them is refused with the capability we needed.
#[tokio::test]
async fn test_no_family_in_common_refuses_the_session() -> Result<()> {
let server = TestServer::start_with(LOCAL_ASN, |c| {
// Inbound connections from this address inherit the peer's settings.
c.peers = vec![PeerConfig {
name: "v6-only".into(),
endpoint: SocketAddr::new(Ipv4Addr::LOCALHOST.into(), 1),
remote_asn: PEER_ASN,
hold_time_secs: 90,
local_address: None,
families: only(AfiSafi::IPV6_UNICAST),
add_path: None,
}];
// Keep the dialer out of the way.
c.connect_retry = Duration::from_secs(3600);
})
.await?;
let mut peer = TestPeer::connect(server.addr).await?;
peer.send_open_with(
PEER_ASN,
PEER_ID,
90,
vec![
four_octet(),
Capability::multiprotocol(AfiSafi::IPV4_UNICAST),
],
)
.await?;
let open = peer.recv_open().await?;
let offered: Vec<_> = open
.capabilities()
.filter(|c| matches!(c, Capability::MultiProtocol { .. }))
.collect();
assert_eq!(offered, [&Capability::multiprotocol(AfiSafi::IPV6_UNICAST)]);
match tokio::time::timeout(TIMEOUT, peer.recv()).await?? {
Message::Notification(n) => {
assert_eq!(
n.error,
BgpError::OpenMsg(OpenMsgSubcode::UnsupportedCapability)
);
// The Multiprotocol capability for IPv6 unicast.
assert_eq!(n.data, [1, 4, 0, 2, 0, 1]);
}
other => bail!("expected NOTIFICATION, got {other:?}"),
}
assert!(server.registry.list().is_empty());
Ok(())
}
/// Real routers offer families we do not support. That must not stop
/// the families we do share from working.
#[tokio::test]
async fn test_a_peer_offering_an_unsupported_family_is_still_accepted() -> Result<()> {
let server = TestServer::start(LOCAL_ASN).await?;
let mut peer = TestPeer::connect(server.addr).await?;
peer.handshake_with(
PEER_ASN,
PEER_ID,
90,
vec![
four_octet(),
Capability::multiprotocol(AfiSafi::new(Afi::L2VPN, Safi::EVPN)),
Capability::multiprotocol(AfiSafi::IPV4_UNICAST),
],
)
.await?;
let session = wait_for_session(&server).await?;
assert_eq!(session.families, only(AfiSafi::IPV4_UNICAST));
assert!(
session.four_octet_asn,
"capabilities after it are still read"
);
Ok(())
}
/// A peer that offers no capabilities at all is a plain RFC 4271 speaker:
/// IPv4 unicast, two-octet AS numbers. Our four-octet AS number reaches it
/// as AS_TRANS in AS_PATH and in full in AS4_PATH (RFC 6793 §4.2.2).
#[tokio::test]
async fn test_a_peer_without_capabilities_gets_two_octet_as_paths() -> Result<()> {
const WIDE_ASN: u32 = 4_200_000_001;
let server = TestServer::start_with(WIDE_ASN, |c| {
c.originate = vec![LocalRoute::new(
prefix("10.100.0.0/16"),
LocalRouteAttrs::default(),
)];
})
.await?;
let mut peer = TestPeer::connect(server.addr).await?;
peer.handshake_with(PEER_ASN, PEER_ID, 90, vec![]).await?;
let session = wait_for_session(&server).await?;
assert!(!session.four_octet_asn);
assert_eq!(session.families, only(AfiSafi::IPV4_UNICAST));
// The test peer now decodes AS_PATH two octets wide, as such a speaker
// would; an UPDATE encoded four octets wide would not parse.
let update = tokio::time::timeout(TIMEOUT, peer.recv_update()).await??;
assert_eq!(update.announced_routes, vec![prefix("10.100.0.0/16")]);
assert!(
update
.path_attributes
.contains(&AsPathAttribute::from_asns(vec![23456])),
"{update:?}"
);
let as4_path = update.path_attributes.iter().find_map(|a| match a {
PathAttribute::As4Path(path) => Some(path.segments[0].path.clone()),
_ => None,
});
assert_eq!(as4_path, Some(vec![WIDE_ASN]));
// And its own two-octet paths are read correctly.
peer.send(&Message::Update(UpdateMessage {
withdrawn_routes: vec![],
path_attributes: vec![
PathAttribute::Origin(OriginPathAttribute::IGP),
AsPathAttribute::from_asns(vec![PEER_ASN, 64512]),
PathAttribute::NextHop(NextHopPathAttribute(Ipv4Addr::new(192, 0, 2, 1))),
],
announced_routes: vec![prefix("192.0.2.0/24").into()],
}))
.await?;
let routes = tokio::time::timeout(TIMEOUT, async {
loop {
let routes = server.received_routes();
if !routes.is_empty() {
return routes;
}
tokio::time::sleep(Duration::from_millis(10)).await;
}
})
.await?;
assert!(
routes[0]
.attributes
.contains(&AsPathAttribute::from_asns(vec![PEER_ASN, 64512]))
);
Ok(())
}
crates/integration-tests/tests/pings.rs
-109
View File
@@ -1,109 +0,0 @@
//! The ping monitor as seen through the HTTP API. The socket is left out
//! (it needs privileges); observations are fed in directly.
use std::time::Duration;
use bgp_integration_tests::test_server::TestServer;
use bgp_primitives::ip_prefix::IpPrefix;
use bgp_rib::LocalRoute;
use bgp_server::ping::PingObservation;
use eyre::Result;
use serde_json::Value;
const TIMEOUT: Duration = Duration::from_secs(5);
fn ping(src: &str, dst: &str, hop_limit: u8, seq: u16) -> PingObservation {
PingObservation {
src: src.parse().unwrap(),
dst: dst.parse().unwrap(),
interface: Some("swissix0".into()),
hop_limit: Some(hop_limit),
ident: 1,
seq,
bytes: 64,
packet: [&[128, 0, 0xab, 0xcd, 0, 1][..], &seq.to_be_bytes(), b"hi"].concat(),
}
}
#[tokio::test]
async fn pings_are_listed_with_the_route_they_were_sent_to() -> Result<()> {
let server = TestServer::start(65000).await?;
let (base, _http) = server.serve_http().await?;
let client = reqwest::Client::new();
server
.local_rib
.announce(LocalRoute::new(IpPrefix::try_from("2001:db8:1a::/48").unwrap(), "alice", Default::default()))
.await?;
server.pings.record(ping("2001:67c:10ec::5", "2001:db8:1a::1", 57, 0)).await;
server.pings.record(ping("2001:67c:10ec::5", "2001:db8:1a::1", 57, 1)).await;
server.pings.record(ping("2a02:1210::9", "2001:db8:ff::1", 243, 0)).await;
server.pings.record(ping("2a02:1210::9", "2a0d:d740::1", 243, 0)).await; // not a watched prefix
let view: Value = client.get(format!("{base}/api/pings")).send().await?.error_for_status()?.json().await?;
assert_eq!(view["status"], "listening");
assert_eq!(view["prefixes"][0], "2001:db8::/32");
assert_eq!(view["total_flows"], 2);
let flows = view["flows"].as_array().unwrap();
let alice = flows.iter().find(|f| f["dst"] == "2001:db8:1a::1").unwrap();
assert_eq!(alice["src"], "2001:67c:10ec::5");
assert_eq!(alice["prefix"], "2001:db8:1a::/48");
assert_eq!(alice["owner"], "alice");
assert_eq!(alice["interface"], "swissix0");
assert_eq!((alice["hop_limit"].as_u64(), alice["hops"].as_u64()), (Some(57), Some(7)));
assert_eq!((alice["count"].as_u64(), alice["seq"].as_u64()), (Some(2), Some(1)));
assert_eq!(alice["hex"], "8000abcd000100016869");
assert!(alice["last_seen_unix"].as_u64().unwrap() > 1_700_000_000);
let stray = flows.iter().find(|f| f["dst"] == "2001:db8:ff::1").unwrap();
assert!(stray["prefix"].is_null() && stray["owner"].is_null());
assert_eq!(stray["hops"], 12);
let limited: Value = client.get(format!("{base}/api/pings?limit=1")).send().await?.json().await?;
assert_eq!(limited["total_flows"], 2);
assert_eq!(limited["flows"].as_array().unwrap().len(), 1);
let metrics = client.get(format!("{base}/metricz")).send().await?.text().await?;
assert!(metrics.contains("ping_monitor_echo_requests_total"));
Ok(())
}
#[tokio::test]
async fn pings_appear_on_the_event_stream() -> Result<()> {
let server = TestServer::start(65000).await?;
let (base, _http) = server.serve_http().await?;
let mut resp = reqwest::Client::new().get(format!("{base}/api/events")).send().await?.error_for_status()?;
// The subscription exists once the response has started; a ping recorded
// before that would be missed, so keep pinging until one shows up.
let pings = server.pings.clone();
let pinger = tokio::spawn(async move {
for seq in 0.. {
pings.record(ping("2001:67c:10ec::5", "2001:db8:1a::1", 57, seq)).await;
tokio::time::sleep(Duration::from_millis(50)).await;
}
});
let mut buf = String::new();
let event = tokio::time::timeout(TIMEOUT, async {
loop {
let chunk = resp.chunk().await?.ok_or_else(|| eyre::eyre!("stream ended"))?;
buf.push_str(std::str::from_utf8(&chunk)?);
for line in buf.lines().filter_map(|l| l.strip_prefix("data:")) {
let ev: Value = serde_json::from_str(line.trim())?;
if ev["type"] == "ping_seen" {
return Ok::<_, eyre::Error>(ev);
}
}
}
})
.await??;
pinger.abort();
assert_eq!(event["ping"]["src"], "2001:67c:10ec::5");
assert_eq!(event["ping"]["dst"], "2001:db8:1a::1");
assert_eq!(event["ping"]["interface"], "swissix0");
assert!(event["new_burst"].is_boolean());
Ok(())
}
crates/integration-tests/tests/update_errors.rs
+228
View File
@@ -0,0 +1,228 @@
//! End-to-end tests for errors in UPDATE messages (RFC 7606): most of them
//! cost the routes in the UPDATE, not the session.
use std::net::Ipv4Addr;
use std::time::Duration;
use bgp_integration_tests::test_peer::TestPeer;
use bgp_integration_tests::test_server::{ReceivedRoute, TestServer};
use bgp_packet::errors::{BgpError, UpdateMsgSubcode};
use bgp_packet::message::{LocalPrefPathAttribute, Message, PathAttribute};
use bgp_primitives::ip_prefix::IpPrefix;
use bgp_server::metrics::UPDATE_ERRORS;
use eyre::{Result, bail};
const LOCAL_ASN: u32 = 65000;
const PEER_ASN: u32 = 65001;
const TIMEOUT: Duration = Duration::from_secs(5);
const WELL_KNOWN: u8 = 0x40;
const OPTIONAL_TRANSITIVE: u8 = 0xc0;
fn attr(flags: u8, type_code: u8, value: &[u8]) -> Vec<u8> {
let mut wire = vec![flags, type_code, value.len() as u8];
wire.extend_from_slice(value);
wire
}
fn origin() -> Vec<u8> {
attr(WELL_KNOWN, 1, &[0])
}
/// AS_SEQUENCE of the peer's AS, four octets wide.
fn as_path() -> Vec<u8> {
attr(WELL_KNOWN, 2, &[2, 1, 0, 0, 0xfd, 0xe9])
}
fn next_hop() -> Vec<u8> {
attr(WELL_KNOWN, 3, &[192, 0, 2, 1])
}
/// A whole UPDATE message announcing `nlri` with `attributes`, as bytes, so
/// that it can be wrong in ways the encoder would not write.
fn update(attributes: &[Vec<u8>], nlri: &[u8]) -> Vec<u8> {
let attributes = attributes.concat();
let mut body = vec![0, 0];
body.extend_from_slice(&(attributes.len() as u16).to_be_bytes());
body.extend_from_slice(&attributes);
body.extend_from_slice(nlri);
let mut wire = vec![0xff; 16];
wire.extend_from_slice(&((19 + body.len()) as u16).to_be_bytes());
wire.push(2);
wire.extend_from_slice(&body);
wire
}
/// 10.1.0.0/16 and 10.2.0.0/16.
const ROUTE_A: [u8; 3] = [16, 10, 1];
const ROUTE_B: [u8; 3] = [16, 10, 2];
fn prefix(s: &str) -> IpPrefix {
IpPrefix::try_from(s).unwrap()
}
async fn established(server: &TestServer, asn: u32) -> Result<TestPeer> {
let mut peer = TestPeer::connect(server.addr).await?;
peer.handshake(asn, Ipv4Addr::new(10, 0, 0, 1), 90).await?;
Ok(peer)
}
/// Waits until the server's Adj-RIB-In holds exactly `prefixes`.
async fn wait_for_routes(server: &TestServer, prefixes: &[&str]) -> Result<Vec<ReceivedRoute>> {
let mut expected: Vec<IpPrefix> = prefixes.iter().map(|p| prefix(p)).collect();
expected.sort_by_key(|p| p.to_string());
Ok(tokio::time::timeout(TIMEOUT, async {
loop {
let mut routes = server.received_routes();
routes.sort_by_key(|r| r.prefix.to_string());
if routes.iter().map(|r| &r.prefix).eq(expected.iter()) {
return routes;
}
tokio::time::sleep(Duration::from_millis(10)).await;
}
})
.await?)
}
/// The situation RFC 7606 exists for: a route is in the table, and then an
/// UPDATE for it arrives with a broken attribute. The route goes; the
/// session, and the other routes learned over it, stay.
#[tokio::test]
async fn test_a_malformed_attribute_withdraws_the_route_and_keeps_the_session() -> Result<()> {
let server = TestServer::start(LOCAL_ASN).await?;
let mut peer = established(&server, PEER_ASN).await?;
let handled_before = UPDATE_ERRORS
.with_label_values(&["treat-as-withdraw"])
.get();
let good = [origin(), as_path(), next_hop()];
peer.send_raw(&update(&good, &[ROUTE_A, ROUTE_B].concat()))
.await?;
wait_for_routes(&server, &["10.1.0.0/16", "10.2.0.0/16"]).await?;
// ORIGIN with a value that is not defined.
let bad = [attr(WELL_KNOWN, 1, &[7]), as_path(), next_hop()];
peer.send_raw(&update(&bad, &ROUTE_A)).await?;
wait_for_routes(&server, &["10.2.0.0/16"]).await?;
// The session goes on: the route can be announced again.
peer.send_raw(&update(&good, &ROUTE_A)).await?;
wait_for_routes(&server, &["10.1.0.0/16", "10.2.0.0/16"]).await?;
assert_eq!(server.registry.list().len(), 1);
assert!(
UPDATE_ERRORS
.with_label_values(&["treat-as-withdraw"])
.get()
> handled_before
);
Ok(())
}
/// An announcement without a mandatory attribute is not a route we can use.
#[tokio::test]
async fn test_a_route_without_as_path_is_not_accepted() -> Result<()> {
let server = TestServer::start(LOCAL_ASN).await?;
let mut peer = established(&server, PEER_ASN).await?;
peer.send_raw(&update(&[origin(), next_hop()], &ROUTE_A))
.await?;
// A well-formed one after it shows that the first was dealt with.
peer.send_raw(&update(&[origin(), as_path(), next_hop()], &ROUTE_B))
.await?;
wait_for_routes(&server, &["10.2.0.0/16"]).await?;
assert_eq!(server.registry.list().len(), 1);
Ok(())
}
/// A broken attribute that plays no part in choosing a route is dropped,
/// and the route is kept.
#[tokio::test]
async fn test_a_malformed_aggregator_is_dropped_and_the_route_kept() -> Result<()> {
let server = TestServer::start(LOCAL_ASN).await?;
let mut peer = established(&server, PEER_ASN).await?;
let attributes = [
origin(),
as_path(),
next_hop(),
// AGGREGATOR is eight octets on this session.
attr(OPTIONAL_TRANSITIVE, 7, &[0, 0, 0xfd, 0xe9, 192]),
];
peer.send_raw(&update(&attributes, &ROUTE_A)).await?;
let routes = wait_for_routes(&server, &["10.1.0.0/16"]).await?;
assert_eq!(routes[0].attributes.len(), 3);
assert!(
!routes[0]
.attributes
.iter()
.any(|a| matches!(a, PathAttribute::Aggregator(_)))
);
Ok(())
}
/// LOCAL_PREF is a matter between routers of one AS. From an external peer
/// it is dropped; from an internal one it is kept.
#[tokio::test]
async fn test_local_pref_is_kept_only_from_an_internal_peer() -> Result<()> {
let local_pref = attr(WELL_KNOWN, 5, &[0, 0, 0, 200]);
let has_local_pref = |routes: &[ReceivedRoute]| {
routes[0]
.attributes
.contains(&PathAttribute::LocalPref(LocalPrefPathAttribute(200)))
};
let server = TestServer::start(LOCAL_ASN).await?;
let mut external = established(&server, PEER_ASN).await?;
let attributes = [origin(), as_path(), next_hop(), local_pref.clone()];
external.send_raw(&update(&attributes, &ROUTE_A)).await?;
let routes = wait_for_routes(&server, &["10.1.0.0/16"]).await?;
assert!(!has_local_pref(&routes));
let server = TestServer::start(LOCAL_ASN).await?;
let mut internal = established(&server, LOCAL_ASN).await?;
// An internal peer has not added its AS to the path.
let attributes = [origin(), attr(WELL_KNOWN, 2, &[]), next_hop(), local_pref];
internal.send_raw(&update(&attributes, &ROUTE_A)).await?;
let routes = wait_for_routes(&server, &["10.1.0.0/16"]).await?;
assert!(has_local_pref(&routes));
Ok(())
}
/// When the prefixes themselves cannot be read there is nothing to
/// withdraw, and the session is reset as before.
#[tokio::test]
async fn test_an_unreadable_prefix_still_resets_the_session() -> Result<()> {
let server = TestServer::start(LOCAL_ASN).await?;
let mut peer = established(&server, PEER_ASN).await?;
// An IPv4 prefix of length 33.
peer.send_raw(&update(
&[origin(), as_path(), next_hop()],
&[33, 10, 1, 2, 3, 4],
))
.await?;
loop {
match tokio::time::timeout(TIMEOUT, peer.recv()).await?? {
Message::KeepAlive => {}
Message::Notification(n) => {
assert_eq!(
n.error,
BgpError::UpdateMsg(UpdateMsgSubcode::InvalidNetworkField)
);
break;
}
other => bail!("expected NOTIFICATION, got {other:?}"),
}
}
tokio::time::timeout(TIMEOUT, async {
while !server.registry.list().is_empty() {
tokio::time::sleep(Duration::from_millis(10)).await;
}
})
.await?;
Ok(())
}
crates/packet/Cargo.toml
+9 -3
View File
@@ -1,7 +1,7 @@
[package]
name = "bgp-packet"
description.workspace = true
description = "Parsing and serialisation of BGP messages"
authors.workspace = true
edition.workspace = true
homepage.workspace = true
license.workspace = true
@@ -11,10 +11,16 @@ version.workspace = true
[dependencies]
bitfield.workspace = true
bytes.workspace = true
eyre.workspace = true
nom.workspace = true
serde.workspace = true
serde_repr.workspace = true
thiserror.workspace = true
strum.workspace = true
bgp-primitives.workspace = true
[dev-dependencies]
eyre.workspace = true
proptest.workspace = true
[lints]
workspace = true
crates/web/src/community.rs → crates/packet/src/community.rs
+39 -21
View File
@@ -1,10 +1,22 @@
//! Parsing and formatting of community values as they appear in the API.
//! Parsing and formatting of community values as text, the way they are
//! written in configuration and shown in status output.
//!
//! Standard communities are written `ASN:VALUE` (e.g. `65000:100`) or as one
//! of the well-known names from RFC 1997 / RFC 7999 / RFC 8326. Large
//! communities are written `GLOBAL:LOCAL1:LOCAL2` (RFC 8092).
use bgp_packet::message::LargeCommunity;
use thiserror::Error;
use crate::message::LargeCommunity;
/// Text that is not a community.
#[derive(Debug, Clone, PartialEq, Eq, Error)]
pub enum CommunityError {
#[error("community {0:?} must be ASN:VALUE with 16-bit numbers, or a well-known name")]
Standard(String),
#[error("large community {0:?} must be GLOBAL:LOCAL1:LOCAL2 with 32-bit numbers")]
Large(String),
}
/// Well-known standard communities, by name.
pub const WELL_KNOWN: &[(&str, (u16, u16))] = &[
@@ -16,19 +28,17 @@ pub const WELL_KNOWN: &[(&str, (u16, u16))] = &[
("NOPEER", (0xFFFF, 0xFF04)),
];
pub fn parse_standard(s: &str) -> Result<(u16, u16), String> {
pub fn parse_standard(s: &str) -> Result<(u16, u16), CommunityError> {
let s = s.trim();
if let Some((_, value)) = WELL_KNOWN.iter().find(|(name, _)| name.eq_ignore_ascii_case(s)) {
let invalid = || CommunityError::Standard(s.to_string());
if let Some((_, value)) = WELL_KNOWN
.iter()
.find(|(name, _)| name.eq_ignore_ascii_case(s))
{
return Ok(*value);
}
let (high, low) = s
.split_once(':')
.ok_or_else(|| format!("community {s:?} must be ASN:VALUE or a well-known name"))?;
let parse = |part: &str| {
part.trim()
.parse::<u16>()
.map_err(|_| format!("community {s:?}: {part:?} is not a 16-bit number"))
};
let (high, low) = s.split_once(':').ok_or_else(invalid)?;
let parse = |part: &str| part.trim().parse::<u16>().map_err(|_| invalid());
Ok((parse(high)?, parse(low)?))
}
@@ -36,18 +46,19 @@ pub fn format_standard((high, low): (u16, u16)) -> String {
format!("{high}:{low}")
}
pub fn parse_large(s: &str) -> Result<LargeCommunity, String> {
pub fn parse_large(s: &str) -> Result<LargeCommunity, CommunityError> {
let s = s.trim();
let invalid = || CommunityError::Large(s.to_string());
let parts: Vec<&str> = s.split(':').collect();
if parts.len() != 3 {
return Err(format!("large community {s:?} must be GLOBAL:LOCAL1:LOCAL2"));
return Err(invalid());
}
let parse = |part: &str| {
part.trim()
.parse::<u32>()
.map_err(|_| format!("large community {s:?}: {part:?} is not a 32-bit number"))
};
Ok(LargeCommunity { global_admin: parse(parts[0])?, data_1: parse(parts[1])?, data_2: parse(parts[2])? })
let parse = |part: &str| part.trim().parse::<u32>().map_err(|_| invalid());
Ok(LargeCommunity {
global_admin: parse(parts[0])?,
data_1: parse(parts[1])?,
data_2: parse(parts[2])?,
})
}
pub fn format_large(c: &LargeCommunity) -> String {
@@ -72,7 +83,14 @@ mod tests {
#[test]
fn large_roundtrip() {
let c = parse_large("65000:1:2").unwrap();
assert_eq!(c, LargeCommunity { global_admin: 65000, data_1: 1, data_2: 2 });
assert_eq!(
c,
LargeCommunity {
global_admin: 65000,
data_1: 1,
data_2: 2
}
);
assert_eq!(format_large(&c), "65000:1:2");
assert!(parse_large("65000:1").is_err());
assert!(parse_large("a:b:c").is_err());
crates/packet/src/errors.rs
+68 -95
View File
@@ -1,68 +1,79 @@
use serde::{Deserialize, Serialize};
use serde::Serialize;
use strum::FromRepr;
use thiserror::Error;
#[derive(Debug, Error)]
#[error("Unknown BGP error code {code} subcode {subcode}")]
pub struct UnknownErrorCode {
pub code: u8,
pub subcode: u8,
/// A field holds a number that is not one of the values defined for it.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Error)]
#[error("invalid {what} value {value}")]
pub struct InvalidValue {
pub what: &'static str,
pub value: u32,
}
#[derive(Debug, Clone, PartialEq, Eq, Error, Serialize, Deserialize)]
#[repr(u8)]
/// A message that could not be decoded, together with the NOTIFICATION that
/// reports the problem to the peer (RFC 4271 §6).
#[derive(Debug, Clone, PartialEq, Eq, Error)]
#[error("{detail}")]
pub struct DecodeError {
/// The error code and subcode to send.
pub notification: BgpError,
/// The Data field of that NOTIFICATION, where the RFC defines one.
pub data: Vec<u8>,
/// What was wrong, for the log.
pub detail: String,
}
/// The error a NOTIFICATION reports: its code and subcode (RFC 4271 §4.5).
#[derive(Debug, Clone, PartialEq, Eq, Error, Serialize)]
pub enum BgpError {
#[error("Message header error")]
MsgHeader(MsgHeaderSubcode) = 1,
#[error("Message header error: {0}")]
MsgHeader(MsgHeaderSubcode),
#[error("Open message error")]
OpenMsg(OpenMsgSubcode) = 2,
#[error("Open message error: {0}")]
OpenMsg(OpenMsgSubcode),
#[error("Update message error")]
UpdateMsg(UpdateMsgSubcode) = 3,
#[error("Update message error: {0}")]
UpdateMsg(UpdateMsgSubcode),
#[error("Hold timer expired")]
HoldTimer = 4,
HoldTimer,
#[error("Finite state machine error")]
FsmError(FsmSubcode) = 5,
#[error("Finite state machine error: {0}")]
FsmError(FsmSubcode),
#[error("Cease")]
Cease(CeaseSubcode) = 6,
#[error("Cease: {0}")]
Cease(CeaseSubcode),
#[error("Route refresh message error")]
RouteRefresh(RouteRefreshSubcode) = 7,
#[error("Route refresh message error: {0}")]
RouteRefresh(RouteRefreshSubcode),
#[error("Send hold timer expired")]
SendHoldTimer = 8,
SendHoldTimer,
/// A code or subcode this implementation has no name for. A peer may
/// send any: new ones are assigned, and zero ("unspecific") is always
/// allowed. The session ends the same way, and the numbers are kept
/// for the log.
#[error("error code {code} subcode {subcode}")]
Other { code: u8, subcode: u8 },
}
impl BgpError {
pub fn from_code_subcode(code: u8, subcode: u8) -> Result<Self, UnknownErrorCode> {
let err = || UnknownErrorCode { code, subcode };
match code {
1 => Ok(Self::MsgHeader(
MsgHeaderSubcode::try_from(subcode).map_err(|_| err())?,
)),
2 => Ok(Self::OpenMsg(
OpenMsgSubcode::try_from(subcode).map_err(|_| err())?,
)),
3 => Ok(Self::UpdateMsg(
UpdateMsgSubcode::try_from(subcode).map_err(|_| err())?,
)),
4 => Ok(Self::HoldTimer),
5 => Ok(Self::FsmError(
FsmSubcode::try_from(subcode).map_err(|_| err())?,
)),
6 => Ok(Self::Cease(
CeaseSubcode::try_from(subcode).map_err(|_| err())?,
)),
7 => Ok(Self::RouteRefresh(
RouteRefreshSubcode::try_from(subcode).map_err(|_| err())?,
)),
8 => Ok(Self::SendHoldTimer),
_ => Err(err()),
}
/// The error for a code and subcode off the wire. Every pair has one,
/// so that a peer's NOTIFICATION can always be read.
pub fn from_code_subcode(code: u8, subcode: u8) -> Self {
let known = match code {
1 => MsgHeaderSubcode::from_repr(subcode).map(Self::MsgHeader),
2 => OpenMsgSubcode::from_repr(subcode).map(Self::OpenMsg),
3 => UpdateMsgSubcode::from_repr(subcode).map(Self::UpdateMsg),
4 => (subcode == 0).then_some(Self::HoldTimer),
5 => FsmSubcode::from_repr(subcode).map(Self::FsmError),
6 => CeaseSubcode::from_repr(subcode).map(Self::Cease),
7 => RouteRefreshSubcode::from_repr(subcode).map(Self::RouteRefresh),
8 => (subcode == 0).then_some(Self::SendHoldTimer),
_ => None,
};
known.unwrap_or(Self::Other { code, subcode })
}
pub fn code_subcode(&self) -> (u8, u8) {
@@ -75,11 +86,12 @@ impl BgpError {
Self::Cease(s) => (6, *s as u8),
Self::RouteRefresh(s) => (7, *s as u8),
Self::SendHoldTimer => (8, 0),
Self::Other { code, subcode } => (*code, *subcode),
}
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Error, Serialize, Deserialize, FromRepr)]
#[derive(Debug, Clone, Copy, PartialEq, Eq, Error, Serialize, FromRepr)]
#[repr(u8)]
pub enum MsgHeaderSubcode {
#[error("Connection not synchronized")]
@@ -92,16 +104,12 @@ pub enum MsgHeaderSubcode {
BadMessageType = 3,
}
impl TryFrom<u8> for MsgHeaderSubcode {
type Error = ();
fn try_from(v: u8) -> Result<Self, ()> {
Self::from_repr(v).ok_or(())
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Error, Serialize, Deserialize, FromRepr)]
#[derive(Debug, Clone, Copy, PartialEq, Eq, Error, Serialize, FromRepr)]
#[repr(u8)]
pub enum OpenMsgSubcode {
#[error("Unspecific")]
Unspecific = 0,
#[error("Unsupported BGP version number")]
UnsupportedVersion = 1,
@@ -124,14 +132,7 @@ pub enum OpenMsgSubcode {
RoleMismatch = 8,
}
impl TryFrom<u8> for OpenMsgSubcode {
type Error = ();
fn try_from(v: u8) -> Result<Self, ()> {
Self::from_repr(v).ok_or(())
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Error, Serialize, Deserialize, FromRepr)]
#[derive(Debug, Clone, Copy, PartialEq, Eq, Error, Serialize, FromRepr)]
#[repr(u8)]
pub enum UpdateMsgSubcode {
#[error("Malformed attribute list")]
@@ -165,14 +166,7 @@ pub enum UpdateMsgSubcode {
MalformedAsPath = 11,
}
impl TryFrom<u8> for UpdateMsgSubcode {
type Error = ();
fn try_from(v: u8) -> Result<Self, ()> {
Self::from_repr(v).ok_or(())
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Error, Serialize, Deserialize, FromRepr)]
#[derive(Debug, Clone, Copy, PartialEq, Eq, Error, Serialize, FromRepr)]
#[repr(u8)]
pub enum FsmSubcode {
#[error("Received an unexpected message in OpenSent state")]
@@ -185,14 +179,7 @@ pub enum FsmSubcode {
UnexpectedEstablished = 3,
}
impl TryFrom<u8> for FsmSubcode {
type Error = ();
fn try_from(v: u8) -> Result<Self, ()> {
Self::from_repr(v).ok_or(())
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Error, Serialize, Deserialize, FromRepr)]
#[derive(Debug, Clone, Copy, PartialEq, Eq, Error, Serialize, FromRepr)]
#[repr(u8)]
pub enum CeaseSubcode {
#[error("Maximum number of prefixes reached")]
@@ -226,23 +213,9 @@ pub enum CeaseSubcode {
BfdDown = 10,
}
impl TryFrom<u8> for CeaseSubcode {
type Error = ();
fn try_from(v: u8) -> Result<Self, ()> {
Self::from_repr(v).ok_or(())
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Error, Serialize, Deserialize, FromRepr)]
#[derive(Debug, Clone, Copy, PartialEq, Eq, Error, Serialize, FromRepr)]
#[repr(u8)]
pub enum RouteRefreshSubcode {
#[error("Invalid message length")]
InvalidLength = 1,
}
impl TryFrom<u8> for RouteRefreshSubcode {
type Error = ();
fn try_from(v: u8) -> Result<Self, ()> {
Self::from_repr(v).ok_or(())
}
}
crates/packet/src/lib.rs
+2
View File
@@ -1,3 +1,5 @@
pub mod community;
pub mod errors;
pub mod message;
pub mod open;
pub mod update_decode;
crates/packet/src/message.rs
+1319 -1207
View File
File diff suppressed because it is too large. Load diff
crates/packet/src/open.rs
+844 -595
View File
File diff suppressed because it is too large. Load diff
crates/packet/src/update_decode.rs
+1008
View File
File diff suppressed because it is too large. Load diff
crates/packet/tests/properties.rs
+674
View File
@@ -0,0 +1,674 @@
//! Property tests for the wire codec.
//!
//! Two things are checked for every kind of message:
//!
//! - **Round trip.** A message that can be written can be read back, comes
//! back equal.
//! - **Hostile input.** Whatever bytes arrive, decoding returns a message or
//! an error: it never panics. What it does accept can be shown, written
//! again, and read back as the same message.
//!
//! The second is what a peer can do to us, so it is run on three kinds of
//! input: plain noise, noise behind a valid header (so it reaches the body
//! parsers), and valid messages with a few bytes damaged (so it reaches deep
//! into them).
use std::net::{Ipv4Addr, Ipv6Addr};
use bgp_packet::errors::BgpError;
use bgp_packet::message::{
AggregatorPathAttribute, AsPathAttribute, AsPathSegment, AtomicAggregatePathAttribute,
BGP_HEADER_LEN, BGP_MAX_MSG_LEN, CommunitiesPathAttribute, ExtendedCommunitiesPathAttribute,
ExtendedCommunity, LargeCommunitiesPathAttribute, LargeCommunity, LocalPrefPathAttribute,
Message, MpReachNlriPathAttribute, MpUnreachNlriPathAttribute, MultiExitDiscPathAttribute,
NextHopPathAttribute, NlriNextHop, NotificationMessage, OriginPathAttribute, PathAttribute,
PathAttributeFlags, RouteRefreshMessage, UpdateMessage,
};
use bgp_packet::open::{
AddPathEntry, AddPathSendRecv, BgpRole, BgpSecHeader, Capability, ExtendedNextHop,
GracefulRestartEntry, LongLivedGracefulRestart, MultiLabelEntry, OpenMessage, OpenOption,
OrfFamily, OrfSendRecv, OrfType, OutboundRouteFilterEntry,
};
use bgp_primitives::constants::{Afi, AfiSafi, IpVersion, Safi};
use bgp_primitives::ip_prefix::{IpPrefix, Nlri};
use bgp_primitives::parser::{EncodeError, ParserContext};
use bytes::BytesMut;
use proptest::collection::vec;
use proptest::prelude::*;
/// What a session settled that the codec depends on.
#[derive(Debug, Clone, Copy)]
struct Session {
four_octet_asn: bool,
ibgp: bool,
/// Whether prefixes carry path identifiers (RFC 7911), for IPv4 and
/// for IPv6 unicast.
add_path: [bool; 2],
}
impl Session {
fn ctx(self) -> ParserContext {
ParserContext {
four_octet_asn: self.four_octet_asn,
ibgp: self.ibgp,
add_path: [AfiSafi::IPV4_UNICAST, AfiSafi::IPV6_UNICAST]
.into_iter()
.zip(self.add_path)
.filter_map(|(family, in_use)| in_use.then_some(family))
.collect(),
}
}
fn has_add_path(self, version: IpVersion, safi: Safi) -> bool {
safi == Safi::UNICAST
&& match version {
IpVersion::Ipv4 => self.add_path[0],
IpVersion::Ipv6 => self.add_path[1],
}
}
}
fn session() -> impl Strategy<Value = Session> {
any::<(bool, bool, [bool; 2])>().prop_map(|(four_octet_asn, ibgp, add_path)| Session {
four_octet_asn,
ibgp,
add_path,
})
}
// ---------------------------------------------------------------------------
// Generators
// ---------------------------------------------------------------------------
fn prefix(version: IpVersion) -> impl Strategy<Value = IpPrefix> {
let max_len = match version {
IpVersion::Ipv4 => 32u8,
IpVersion::Ipv6 => 128,
};
(0..=max_len, any::<[u8; 16]>()).prop_map(move |(length, bytes)| {
IpPrefix::new(version, bytes, length).expect("16 bytes are enough for any prefix")
})
}
/// Prefixes as an UPDATE names them. Without ADD-PATH for the family there
/// is no identifier on the wire, and it reads back as zero.
fn nlri(session: Session, version: IpVersion, safi: Safi) -> impl Strategy<Value = Vec<Nlri>> {
let with_path_id = session.has_add_path(version, safi);
vec(
(prefix(version), any::<u32>()).prop_map(move |(prefix, path_id)| Nlri {
prefix,
path_id: if with_path_id { path_id } else { 0 },
}),
0..5,
)
}
fn afi() -> impl Strategy<Value = Afi> {
any::<u16>().prop_map(Afi)
}
fn safi() -> impl Strategy<Value = Safi> {
any::<u8>().prop_map(Safi)
}
/// AS numbers that fit the session: any on a four-octet one, two-octet ones
/// otherwise (wider ones are replaced by AS_TRANS on the wire, by design).
fn asn(four_octet_asn: bool) -> BoxedStrategy<u32> {
if four_octet_asn {
any::<u32>().boxed()
} else {
any::<u16>().prop_map(u32::from).boxed()
}
}
fn as_path(asn: BoxedStrategy<u32>) -> impl Strategy<Value = AsPathAttribute> {
// A segment has at least one AS number.
vec((any::<bool>(), vec(asn, 1..6)), 0..4).prop_map(|segments| AsPathAttribute {
segments: segments
.into_iter()
.map(|(ordered, path)| AsPathSegment { ordered, path })
.collect(),
})
}
fn extended_community() -> impl Strategy<Value = ExtendedCommunity> {
any::<[u8; 8]>().prop_map(ExtendedCommunity)
}
fn mp_reach(session: Session) -> impl Strategy<Value = MpReachNlriPathAttribute> {
let ip_safi = || prop_oneof![Just(Safi::UNICAST), Just(Safi::MULTICAST)];
let v4 = ip_safi()
.prop_flat_map(move |safi| {
(
Just(safi),
any::<[u8; 4]>(),
nlri(session, IpVersion::Ipv4, safi),
)
})
.prop_map(|(safi, next_hop, prefixes)| MpReachNlriPathAttribute {
family: AfiSafi::new(Afi::IPV4, safi),
next_hop: NlriNextHop::Ipv4(Ipv4Addr::from(next_hop)),
prefixes,
});
let v6_next_hop = || {
prop_oneof![
any::<[u8; 16]>().prop_map(|a| NlriNextHop::Ipv6(Ipv6Addr::from(a))),
(any::<[u8; 16]>(), any::<[u8; 16]>()).prop_map(|(global, link_local)| {
NlriNextHop::Ipv6WithLl {
global: Ipv6Addr::from(global),
link_local: Ipv6Addr::from(link_local),
}
}),
]
};
let v6 = ip_safi()
.prop_flat_map(move |safi| {
(
Just(safi),
v6_next_hop(),
nlri(session, IpVersion::Ipv6, safi),
)
})
.prop_map(|(safi, next_hop, prefixes)| MpReachNlriPathAttribute {
family: AfiSafi::new(Afi::IPV6, safi),
next_hop,
prefixes,
});
prop_oneof![v4, v6]
}
fn mp_unreach(session: Session) -> impl Strategy<Value = MpUnreachNlriPathAttribute> {
let of = move |version: IpVersion| {
nlri(session, version, Safi::UNICAST).prop_map(move |prefixes| MpUnreachNlriPathAttribute {
family: AfiSafi::new(version.into(), Safi::UNICAST),
prefixes,
})
};
prop_oneof![of(IpVersion::Ipv4), of(IpVersion::Ipv6)]
}
/// Type codes whose definition the decoder knows; an unknown attribute has
/// none of these.
const DEFINED_ATTRIBUTES: [u8; 16] = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 14, 15, 16, 17, 18, 32];
/// Unknown attributes, each with a type code of its own: a repeated
/// attribute is dropped by the decoder.
fn unknown_attributes() -> impl Strategy<Value = Vec<PathAttribute>> {
proptest::collection::btree_map(
any::<u8>().prop_filter("a type code without a definition", |code| {
!DEFINED_ATTRIBUTES.contains(code)
}),
(any::<u8>(), vec(any::<u8>(), 0..300)),
0..3,
)
.prop_map(|attributes| {
attributes
.into_iter()
.map(|(type_code, (flags, payload))| {
// Optional (bit 7): an unknown well-known attribute is an
// error. Bit 4 selects the two-octet length, which a payload
// this long needs.
let mut flags = flags | 0x80;
if payload.len() > 255 {
flags |= 0x10;
}
PathAttribute::UnknownPathAttribute {
flags: PathAttributeFlags(flags),
type_code,
payload,
}
})
.collect()
})
}
/// The attributes of a well-formed UPDATE on `session`: the mandatory ones,
/// and any of the others at most once.
fn path_attributes(session: Session) -> impl Strategy<Value = Vec<PathAttribute>> {
let mandatory = (
prop_oneof![
Just(OriginPathAttribute::IGP),
Just(OriginPathAttribute::EGP),
Just(OriginPathAttribute::INCOMPLETE)
],
as_path(asn(session.four_octet_asn)),
any::<[u8; 4]>(),
any::<u32>(),
)
.prop_map(move |(origin, as_path, next_hop, local_pref)| {
let mut attributes = vec![
PathAttribute::Origin(origin),
PathAttribute::AsPath(as_path),
PathAttribute::NextHop(NextHopPathAttribute(Ipv4Addr::from(next_hop))),
];
// Mandatory between internal peers, dropped from external ones.
if session.ibgp {
attributes.push(PathAttribute::LocalPref(LocalPrefPathAttribute(local_pref)));
}
attributes
});
let optional = (
proptest::option::of(
any::<u32>().prop_map(|v| PathAttribute::MultiExitDisc(MultiExitDiscPathAttribute(v))),
),
proptest::option::of(Just(PathAttribute::AtomicAggregate(
AtomicAggregatePathAttribute {},
))),
proptest::option::of((asn(session.four_octet_asn), any::<[u8; 4]>()).prop_map(
|(asn, ip)| {
PathAttribute::Aggregator(AggregatorPathAttribute {
asn,
ip: Ipv4Addr::from(ip),
})
},
)),
proptest::option::of(
vec(any::<(u16, u16)>(), 1..8)
.prop_map(|c| PathAttribute::Communities(CommunitiesPathAttribute(c))),
),
proptest::option::of(mp_reach(session).prop_map(PathAttribute::MpReachNlri)),
proptest::option::of(mp_unreach(session).prop_map(PathAttribute::MpUnreachNlri)),
proptest::option::of(
vec(extended_community(), 1..6).prop_map(|extended_communities| {
PathAttribute::ExtendedCommunities(ExtendedCommunitiesPathAttribute {
extended_communities,
})
}),
),
// AS4_PATH is four octets wide, and only a two-octet speaker sends
// it.
proptest::option::of(as_path(any::<u32>().boxed()).prop_map(PathAttribute::As4Path))
.prop_map(move |as4_path| as4_path.filter(|_| !session.four_octet_asn)),
proptest::option::of(vec(any::<(u32, u32, u32)>(), 1..6).prop_map(|communities| {
PathAttribute::LargeCommunities(LargeCommunitiesPathAttribute {
communities: communities
.into_iter()
.map(|(global_admin, data_1, data_2)| LargeCommunity {
global_admin,
data_1,
data_2,
})
.collect(),
})
})),
unknown_attributes(),
);
(mandatory, optional).prop_map(|(mut attributes, optional)| {
let (med, atomic, aggregator, communities, reach, unreach, extended, as4, large, unknown) =
optional;
attributes.extend(
[
med,
atomic,
aggregator,
communities,
reach,
unreach,
extended,
as4,
large,
]
.into_iter()
.flatten(),
);
attributes.extend(unknown);
attributes
})
}
fn update(session: Session) -> impl Strategy<Value = UpdateMessage> {
(
nlri(session, IpVersion::Ipv4, Safi::UNICAST),
path_attributes(session),
nlri(session, IpVersion::Ipv4, Safi::UNICAST),
)
.prop_map(
|(withdrawn_routes, path_attributes, announced_routes)| UpdateMessage {
withdrawn_routes,
path_attributes,
announced_routes,
},
)
}
/// Capability codes that have their own variant.
const KNOWN_CAPABILITIES: [u8; 13] = [1, 2, 3, 5, 6, 7, 8, 9, 64, 65, 69, 70, 71];
fn capability() -> impl Strategy<Value = Capability> {
let orf = (
any::<u8>().prop_map(OrfType),
prop_oneof![
Just(OrfSendRecv::Receive),
Just(OrfSendRecv::Send),
Just(OrfSendRecv::Both)
],
)
.prop_map(|(r#type, send_recv)| OutboundRouteFilterEntry { r#type, send_recv });
let add_path = prop_oneof![
Just(AddPathSendRecv::Receive),
Just(AddPathSendRecv::Send),
Just(AddPathSendRecv::Both)
];
prop_oneof![
(afi(), safi()).prop_map(|(afi, safi)| Capability::MultiProtocol { afi, safi }),
Just(Capability::RouteRefresh),
vec((afi(), safi(), vec(orf, 0..4)), 0..3).prop_map(|families| {
Capability::OutboundRouteFilter(
families
.into_iter()
.map(|(afi, safi, orfs)| OrfFamily { afi, safi, orfs })
.collect(),
)
}),
vec((afi(), safi(), afi()), 1..4).prop_map(|entries| Capability::ExtendedNextHop {
entries: entries
.into_iter()
.map(|(afi, safi, nh_afi)| ExtendedNextHop { afi, safi, nh_afi })
.collect(),
}),
Just(Capability::ExtendedMessage),
(any::<u8>(), afi()).prop_map(|(header, afi)| Capability::BgpSec {
header: BgpSecHeader(header),
afi,
}),
vec((afi(), safi(), any::<u8>()), 1..4).prop_map(|entries| {
Capability::MultiLabelCompat {
entries: entries
.into_iter()
.map(|(afi, safi, count)| MultiLabelEntry { afi, safi, count })
.collect(),
}
}),
any::<u8>().prop_map(|role| Capability::BgpRole {
role: BgpRole::from(role),
}),
(
any::<bool>(),
0u16..4096,
vec((afi(), safi(), any::<bool>()), 0..4)
)
.prop_map(|(restart_state, restart_time, entries)| {
Capability::GracefulRestart {
restart_state,
restart_time,
entries: entries
.into_iter()
.map(|(afi, safi, preserve_forwarding)| GracefulRestartEntry {
afi,
safi,
preserve_forwarding,
})
.collect(),
}
}),
any::<u32>().prop_map(|asn| Capability::FourByteAsn { asn }),
vec((afi(), safi(), add_path), 1..4).prop_map(|entries| Capability::AddPath {
entries: entries
.into_iter()
.map(|(afi, safi, send_recv)| AddPathEntry {
afi,
safi,
send_recv,
})
.collect(),
}),
Just(Capability::EnhancedRouteRefresh),
vec((afi(), safi(), any::<u8>(), 0u32..(1 << 24)), 0..4).prop_map(|entries| {
Capability::LongLivedGracefulRestart(
entries
.into_iter()
.map(|(afi, safi, flags, stale_time)| LongLivedGracefulRestart {
afi,
safi,
flags,
stale_time,
})
.collect(),
)
}),
(
any::<u8>().prop_filter("a code without a variant", |code| {
!KNOWN_CAPABILITIES.contains(code)
}),
vec(any::<u8>(), 0..20)
)
.prop_map(|(type_code, value)| Capability::UnknownCapability { type_code, value }),
]
}
fn open() -> impl Strategy<Value = OpenMessage> {
(
any::<u16>(),
any::<u16>(),
any::<[u8; 4]>(),
// Enough to go past 255 octets of options now and then, which
// switches to the extended length of RFC 9072.
vec(vec(capability(), 0..10), 0..8),
)
.prop_map(|(asn, hold_time, identifier, options)| OpenMessage {
version: 4,
asn,
hold_time,
identifier: Ipv4Addr::from(identifier),
options: options.into_iter().map(OpenOption::Capabilities).collect(),
})
}
fn notification() -> impl Strategy<Value = NotificationMessage> {
// Any code and subcode: a peer may send ones we have no name for.
(any::<u8>(), any::<u8>(), vec(any::<u8>(), 0..40)).prop_map(|(code, subcode, data)| {
NotificationMessage {
error: BgpError::from_code_subcode(code, subcode),
data,
}
})
}
fn message(session: Session) -> impl Strategy<Value = Message> {
prop_oneof![
open().prop_map(Message::Open),
update(session).prop_map(Message::Update),
notification().prop_map(Message::Notification),
Just(Message::KeepAlive),
(afi(), any::<u8>(), safi()).prop_map(|(afi, subtype, safi)| {
Message::RouteRefresh(RouteRefreshMessage { afi, subtype, safi })
}),
]
}
/// A session, and a well-formed message that fits it.
fn session_and_message() -> impl Strategy<Value = (Session, Message)> {
session().prop_flat_map(|session| (Just(session), message(session)))
}
// ---------------------------------------------------------------------------
// Properties
// ---------------------------------------------------------------------------
/// Decodes `wire` and checks what must hold for anything the decoder
/// accepts: it can be shown and written, and unless the decoder had to
/// repair it, it reads back as itself.
fn check_decoding(ctx: &ParserContext, wire: &[u8]) -> Result<(), TestCaseError> {
let Ok(decoded) = Message::decode_with_report(ctx, wire) else {
return Ok(());
};
let (msg, repairs) = (decoded.message, decoded.update_errors);
// Display of what a peer sent must not be able to panic: it is logged.
let _ = format!("{msg:?} {repairs:?}");
if let Message::Update(update) = &msg {
for prefix in update
.withdrawn_routes
.iter()
.chain(&update.announced_routes)
{
let _ = prefix.to_string();
}
for attribute in &update.path_attributes {
match attribute {
PathAttribute::MpReachNlri(mp) => {
mp.prefixes.iter().for_each(|p| drop(p.to_string()))
}
PathAttribute::MpUnreachNlri(mp) => {
mp.prefixes.iter().for_each(|p| drop(p.to_string()))
}
_ => {}
}
}
}
let mut rewritten = BytesMut::new();
match msg.to_wire(ctx, &mut rewritten) {
Ok(()) => {}
// Withdrawing what a full-size UPDATE announced can need more room
// than one message has. The result is acted on, never sent.
Err(EncodeError::TooLong { .. }) if !repairs.is_empty() => return Ok(()),
Err(e) => {
return Err(TestCaseError::fail(format!(
"accepted but cannot be written: {e}"
)));
}
}
if repairs.is_empty() {
let again = Message::decode_with_report(ctx, &rewritten)
.map_err(|e| TestCaseError::fail(format!("written but cannot be read: {e}")))?;
prop_assert!(again.update_errors.is_empty(), "{:?}", again.update_errors);
prop_assert_eq!(again.message, msg);
}
Ok(())
}
/// Writes a generated message. `None` when it is too big for one BGP
/// message, or some part of it for its length field, which the generators
/// can produce and the encoder must refuse.
fn written(ctx: &ParserContext, msg: &Message) -> Option<BytesMut> {
let mut wire = BytesMut::new();
match msg.to_wire(ctx, &mut wire) {
Ok(()) => Some(wire),
Err(EncodeError::TooLong { .. }) => None,
}
}
fn framed(msg_type: u8, body: &[u8]) -> Vec<u8> {
let mut wire = vec![0xff; 16];
let length = u16::try_from(usize::from(BGP_HEADER_LEN) + body.len()).unwrap();
wire.extend_from_slice(&length.to_be_bytes());
wire.push(msg_type);
wire.extend_from_slice(body);
wire
}
/// Ways to damage a message.
#[derive(Debug, Clone)]
enum Damage {
Set { at: prop::sample::Index, to: u8 },
Remove { at: prop::sample::Index },
Insert { at: prop::sample::Index, byte: u8 },
Truncate { to: prop::sample::Index },
}
fn damage() -> impl Strategy<Value = Damage> {
prop_oneof![
(any::<prop::sample::Index>(), any::<u8>()).prop_map(|(at, to)| Damage::Set { at, to }),
any::<prop::sample::Index>().prop_map(|at| Damage::Remove { at }),
(any::<prop::sample::Index>(), any::<u8>())
.prop_map(|(at, byte)| Damage::Insert { at, byte }),
any::<prop::sample::Index>().prop_map(|to| Damage::Truncate { to }),
]
}
/// Applies `damage` to the body of `wire` and makes the header's length
/// right again, so that the damage is the body parsers' problem.
fn damaged(wire: &[u8], damage: &[Damage]) -> Vec<u8> {
let header = usize::from(BGP_HEADER_LEN);
let msg_type = wire[header - 1];
let mut body = wire[header..].to_vec();
for d in damage {
match d {
Damage::Set { at, to } if !body.is_empty() => {
let i = at.index(body.len());
body[i] = *to;
}
Damage::Remove { at } if !body.is_empty() => {
body.remove(at.index(body.len()));
}
Damage::Insert { at, byte } => body.insert(at.index(body.len() + 1), *byte),
Damage::Truncate { to } if !body.is_empty() => body.truncate(to.index(body.len())),
_ => {}
}
}
body.truncate(usize::from(BGP_MAX_MSG_LEN - BGP_HEADER_LEN));
framed(msg_type, &body)
}
/// 2000 cases a property, which takes about a second. For a longer hunt
/// set `PROPTEST_CASES`, e.g. `PROPTEST_CASES=1000000 cargo test --release
/// -p bgp-packet --test properties`.
fn config() -> ProptestConfig {
let cases = std::env::var("PROPTEST_CASES")
.ok()
.and_then(|cases| cases.parse().ok())
.unwrap_or(2000);
ProptestConfig {
cases,
// Failures are not saved to a file: one that is found becomes a
// named test next to the code it broke.
failure_persistence: None,
..ProptestConfig::default()
}
}
proptest! {
#![proptest_config(config())]
#[test]
fn a_message_that_can_be_written_reads_back_equal((session, msg) in session_and_message()) {
let ctx = session.ctx();
let Some(wire) = written(&ctx, &msg) else {
return Ok(());
};
let decoded = Message::decode_with_report(&ctx, &wire)
.map_err(|e| TestCaseError::fail(format!("{e}")))?;
// What we write is well formed: nothing for the decoder to repair.
prop_assert!(decoded.update_errors.is_empty(), "{:?}", decoded.update_errors);
prop_assert_eq!(&decoded.message, &msg);
}
#[test]
fn noise_is_never_more_than_an_error(session in session(), bytes in vec(any::<u8>(), 0..200)) {
check_decoding(&session.ctx(), &bytes)?;
}
#[test]
fn noise_behind_a_valid_header_is_never_more_than_an_error(
session in session(),
msg_type in 0u8..7,
body in vec(any::<u8>(), 0..300),
) {
check_decoding(&session.ctx(), &framed(msg_type, &body))?;
}
#[test]
fn a_damaged_message_is_never_more_than_an_error(
(session, msg) in session_and_message(),
damage in vec(damage(), 1..4),
) {
let ctx = session.ctx();
let Some(wire) = written(&ctx, &msg) else {
return Ok(());
};
check_decoding(&ctx, &damaged(&wire, &damage))?;
}
/// The parsers below the message level are public too.
#[test]
fn noise_is_never_more_than_an_error_for_the_parts(
session in session(),
bytes in vec(any::<u8>(), 0..120),
) {
let ctx = session.ctx();
let _ = OpenMessage::from_wire(&bytes);
let _ = Capability::from_wire(&bytes);
let _ = UpdateMessage::decode(&ctx, &bytes);
let _ = PathAttribute::from_wire(&ctx, &bytes);
}
}
crates/primitives/Cargo.toml
+7 -2
View File
@@ -1,7 +1,7 @@
[package]
name = "bgp-primitives"
description = "Primitive types for working with routing information"
authors.workspace = true
edition.workspace = true
homepage.workspace = true
license.workspace = true
@@ -11,9 +11,14 @@ version.workspace = true
[dependencies]
bitfield.workspace = true
bytes.workspace = true
eyre.workspace = true
nom.workspace = true
serde.workspace = true
serde_repr.workspace = true
thiserror.workspace = true
strum.workspace = true
[dev-dependencies]
proptest.workspace = true
[lints]
workspace = true
crates/primitives/src/constants.rs
+150 -56
View File
@@ -1,80 +1,174 @@
use std::fmt::Display;
use eyre::bail;
use serde_repr::{Deserialize_repr, Serialize_repr};
use serde::Serialize;
use serde_repr::Serialize_repr;
use thiserror::Error;
#[derive(Debug, Copy, Clone, Hash, PartialEq, Eq, Serialize_repr, Deserialize_repr)]
/// An address family number this implementation has no name for.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Error)]
#[error("unknown address family identifier {0}")]
pub struct UnknownAfi(pub u16);
/// The IP version of a prefix. For the address family field of a capability
/// or an MP_REACH_NLRI, which can name families that are not IP, see [`Afi`].
#[derive(Debug, Copy, Clone, Hash, PartialEq, Eq, Serialize_repr)]
#[repr(u16)]
pub enum AddressFamilyId {
pub enum IpVersion {
Ipv4 = 1,
Ipv6 = 2,
}
impl TryFrom<u16> for AddressFamilyId {
type Error = eyre::Error;
fn try_from(value: u16) -> Result<Self, Self::Error> {
Ok(match value {
x if x == Self::Ipv4 as u16 => Self::Ipv4,
x if x == Self::Ipv6 as u16 => Self::Ipv6,
other => bail!("Unknown AddressFamily: {}", other),
})
}
}
impl Display for AddressFamilyId {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
impl IpVersion {
/// How many bits an address of this version has.
pub fn bits(self) -> u8 {
match self {
AddressFamilyId::Ipv4 => write!(f, "Ipv4"),
AddressFamilyId::Ipv6 => write!(f, "Ipv6"),
Self::Ipv4 => 32,
Self::Ipv6 => 128,
}
}
}
/// Represents a Subsequent Address Family Identifier.
#[derive(Debug, Copy, Clone, Hash, PartialEq, Eq, Serialize_repr, Deserialize_repr)]
#[repr(u8)]
pub enum SubsequentAfi {
Unicast = 1,
Multicast = 2,
NlriWithMpls = 4,
MplsLabelledVpn = 128,
MulticastMplsVpn = 129,
impl TryFrom<u16> for IpVersion {
type Error = UnknownAfi;
fn try_from(value: u16) -> Result<Self, Self::Error> {
match value {
x if x == Self::Ipv4 as u16 => Ok(Self::Ipv4),
x if x == Self::Ipv6 as u16 => Ok(Self::Ipv6),
other => Err(UnknownAfi(other)),
}
}
}
impl TryFrom<u8> for SubsequentAfi {
type Error = eyre::Error;
impl Display for IpVersion {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
IpVersion::Ipv4 => write!(f, "Ipv4"),
IpVersion::Ipv6 => write!(f, "Ipv6"),
}
}
}
fn try_from(value: u8) -> Result<Self, Self::Error> {
Ok(match value {
x if x == Self::Unicast as u8 => Self::Unicast,
x if x == Self::Multicast as u8 => Self::Multicast,
x if x == Self::NlriWithMpls as u8 => Self::NlriWithMpls,
x if x == Self::MplsLabelledVpn as u8 => Self::MplsLabelledVpn,
x if x == Self::MulticastMplsVpn as u8 => Self::MulticastMplsVpn,
other => bail!("Unknown SubsequentAfi: {}", other),
/// An Address Family Identifier as it appears on the wire (RFC 4760).
///
/// Any number is a valid value: a peer may name a family this implementation
/// has never heard of, and that must be readable so it can be ignored. Which
/// families are *supported* is a separate question, answered where routes
/// are stored.
#[derive(Debug, Copy, Clone, Hash, PartialEq, Eq, PartialOrd, Ord, Serialize)]
#[serde(transparent)]
pub struct Afi(pub u16);
impl Afi {
pub const IPV4: Self = Self(1);
pub const IPV6: Self = Self(2);
pub const L2VPN: Self = Self(25);
pub const BGP_LS: Self = Self(16388);
}
impl From<IpVersion> for Afi {
fn from(version: IpVersion) -> Self {
Self(version as u16)
}
}
impl TryFrom<Afi> for IpVersion {
type Error = UnknownAfi;
fn try_from(afi: Afi) -> Result<Self, Self::Error> {
Self::try_from(afi.0)
}
}
/// A Subsequent Address Family Identifier as it appears on the wire. Like
/// [`Afi`], any number is a valid value.
#[derive(Debug, Copy, Clone, Hash, PartialEq, Eq, PartialOrd, Ord, Serialize)]
#[serde(transparent)]
pub struct Safi(pub u8);
impl Safi {
pub const UNICAST: Self = Self(1);
pub const MULTICAST: Self = Self(2);
pub const MPLS_LABEL: Self = Self(4);
pub const EVPN: Self = Self(70);
pub const BGP_LS: Self = Self(71);
pub const MPLS_VPN: Self = Self(128);
pub const MULTICAST_MPLS_VPN: Self = Self(129);
pub const FLOWSPEC: Self = Self(133);
}
/// An address family: the pair that says what kind of routes a capability
/// or an MP_REACH_NLRI is about.
#[derive(Debug, Copy, Clone, Hash, PartialEq, Eq, PartialOrd, Ord, Serialize)]
pub struct AfiSafi {
pub afi: Afi,
pub safi: Safi,
}
impl AfiSafi {
pub const IPV4_UNICAST: Self = Self::new(Afi::IPV4, Safi::UNICAST);
pub const IPV6_UNICAST: Self = Self::new(Afi::IPV6, Safi::UNICAST);
pub const fn new(afi: Afi, safi: Safi) -> Self {
Self { afi, safi }
}
/// The unicast family of an IP version.
pub fn unicast(version: IpVersion) -> Self {
Self::new(version.into(), Safi::UNICAST)
}
/// The usual name of the family, if it has one here.
fn name(self) -> Option<&'static str> {
Some(match (self.afi, self.safi) {
(Afi::IPV4, Safi::UNICAST) => "IPv4 unicast",
(Afi::IPV6, Safi::UNICAST) => "IPv6 unicast",
(Afi::IPV4, Safi::MULTICAST) => "IPv4 multicast",
(Afi::IPV6, Safi::MULTICAST) => "IPv6 multicast",
(Afi::IPV4, Safi::MPLS_LABEL) => "IPv4 labeled unicast",
(Afi::IPV6, Safi::MPLS_LABEL) => "IPv6 labeled unicast",
(Afi::IPV4, Safi::MPLS_VPN) => "VPNv4 unicast",
(Afi::IPV6, Safi::MPLS_VPN) => "VPNv6 unicast",
(Afi::IPV4, Safi::FLOWSPEC) => "IPv4 flowspec",
(Afi::IPV6, Safi::FLOWSPEC) => "IPv6 flowspec",
(Afi::L2VPN, Safi::EVPN) => "L2VPN EVPN",
(Afi::BGP_LS, Safi::BGP_LS) => "BGP-LS",
_ => return None,
})
}
}
/// The implementation try_from for u16 just reuses the u8 implementation since
/// we don't have any cases where the SAFI is longer than u8::MAX yet.
impl TryFrom<u16> for SubsequentAfi {
type Error = eyre::Error;
fn try_from(value: u16) -> Result<Self, Self::Error> {
TryFrom::<u8>::try_from(value as u8)
}
}
impl Display for SubsequentAfi {
impl Display for AfiSafi {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
SubsequentAfi::Unicast => write!(f, "Unicast"),
SubsequentAfi::Multicast => write!(f, "Multicast"),
SubsequentAfi::NlriWithMpls => write!(f, "NlriWithMpls"),
SubsequentAfi::MplsLabelledVpn => write!(f, "MplsLabelledVpn"),
SubsequentAfi::MulticastMplsVpn => write!(f, "MulticastMplsVpn"),
match self.name() {
Some(name) => f.write_str(name),
None => write!(f, "AFI {} SAFI {}", self.afi.0, self.safi.0),
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn any_family_is_a_value_and_the_common_ones_have_names() {
assert_eq!(AfiSafi::IPV6_UNICAST.to_string(), "IPv6 unicast");
assert_eq!(
AfiSafi::new(Afi::L2VPN, Safi::EVPN).to_string(),
"L2VPN EVPN"
);
assert_eq!(
AfiSafi::new(Afi(999), Safi(7)).to_string(),
"AFI 999 SAFI 7"
);
assert_eq!(AfiSafi::unicast(IpVersion::Ipv4), AfiSafi::IPV4_UNICAST);
}
#[test]
fn only_ipv4_and_ipv6_are_ip_versions() {
assert_eq!(IpVersion::try_from(Afi::IPV6), Ok(IpVersion::Ipv6));
assert_eq!(IpVersion::try_from(Afi::L2VPN), Err(UnknownAfi(25)));
assert_eq!(IpVersion::try_from(3u16), Err(UnknownAfi(3)));
}
}
crates/primitives/src/ip_prefix.rs
+244 -116
View File
@@ -2,38 +2,92 @@ use std::fmt::Display;
use std::net::{Ipv4Addr, Ipv6Addr};
use std::str::FromStr;
use bytes::{Buf, BufMut, BytesMut};
use eyre::{Result, bail};
use serde::{Deserialize, Serialize};
use serde::Serialize;
use thiserror::Error;
use crate::constants::AddressFamilyId;
use crate::parser::{ParserContext, ToWireError};
use crate::constants::IpVersion;
/// IpPrefix represents some IP address prefix, for a specific AddressFamilyId.
#[derive(Debug, PartialEq, Eq, Clone, Hash, Serialize, Deserialize)]
/// Why something is not a valid prefix.
#[derive(Debug, Clone, PartialEq, Eq, Error)]
pub enum PrefixError {
#[error("prefix {0:?} is not in address/length form")]
Format(String),
#[error("prefix {0:?} has an address that is neither IPv4 nor IPv6")]
Address(String),
#[error("prefix length {length} is longer than the {max} bits of an address")]
LengthTooLong { length: u8, max: u8 },
#[error("prefix length {length} needs {needed} address bytes, got {got}")]
TooFewBytes {
length: u8,
needed: usize,
got: usize,
},
}
/// An IPv4 or IPv6 prefix.
///
/// It is a small value with no allocation behind it, cheap to copy, compare
/// and hash: a routing table holds millions of them and looks them up by
/// value. A prefix has one representation: the bits past its length are
/// always zero, so two prefixes are equal exactly when they are the same
/// prefix.
#[derive(Debug, PartialEq, Eq, Clone, Copy, Hash, Serialize)]
pub struct IpPrefix {
pub address_family: AddressFamilyId,
pub prefix: Vec<u8>,
pub length: u8,
address_family: IpVersion,
length: u8,
/// The address, left-aligned, zero past `length` bits.
address: [u8; 16],
}
impl IpPrefix {
pub fn new(address_family: AddressFamilyId, prefix: Vec<u8>, length: u8) -> Result<Self> {
// Ensure that the prefix we are given contains the right number of bytes corresponding to the prefix length.
if prefix.len() < ((length + 7) / 8).into() {
bail!(
"Mismatched prefix {:?} for given prefix length: {}",
prefix,
length
);
/// Makes a prefix from the leading bytes of its address. `prefix` must
/// hold at least the bytes that `length` bits cover; anything past
/// those bits is ignored (RFC 4271 §4.3: such bits are irrelevant).
pub fn new(
address_family: IpVersion,
prefix: impl AsRef<[u8]>,
length: u8,
) -> Result<Self, PrefixError> {
let prefix = prefix.as_ref();
let max = address_family.bits();
if length > max {
return Err(PrefixError::LengthTooLong { length, max });
}
let needed = usize::from(length.div_ceil(8));
if prefix.len() < needed {
return Err(PrefixError::TooFewBytes {
length,
needed,
got: prefix.len(),
});
}
let mut address = [0u8; 16];
address[..needed].copy_from_slice(&prefix[..needed]);
if let (Some(last), rem @ 1..) = (address[..needed].last_mut(), length % 8) {
*last &= u8::MAX << (8 - rem);
}
Ok(Self {
address_family,
prefix,
length,
address,
})
}
pub fn address_family(&self) -> IpVersion {
self.address_family
}
/// The prefix length in bits.
pub fn length(&self) -> u8 {
self.length
}
/// The bytes the prefix length covers, as they go on the wire: none for
/// a default route, three for a /24.
pub fn bytes(&self) -> &[u8] {
&self.address[..usize::from(self.length.div_ceil(8))]
}
/// Returns true if `other` is equal to or more specific than `self`, i.e.
/// `self` covers `other`. Prefixes of different address families never
/// contain each other.
@@ -41,8 +95,8 @@ impl IpPrefix {
if self.address_family != other.address_family || other.length < self.length {
return false;
}
let full_bytes = (self.length / 8) as usize;
if self.prefix[..full_bytes] != other.prefix[..full_bytes] {
let full_bytes = usize::from(self.length / 8);
if self.address[..full_bytes] != other.address[..full_bytes] {
return false;
}
let rem_bits = self.length % 8;
@@ -50,101 +104,85 @@ impl IpPrefix {
return true;
}
let mask = u8::MAX << (8 - rem_bits);
(self.prefix[full_bytes] & mask) == (other.prefix[full_bytes] & mask)
(self.address[full_bytes] & mask) == (other.address[full_bytes] & mask)
}
}
/// A prefix as an UPDATE names it: the prefix and, on a session where
/// ADD-PATH is in use for the address family (RFC 7911), the identifier
/// that tells apart the several routes a peer may announce for it.
///
/// Whether the identifier is on the wire is a matter of what the session
/// negotiated, not of its value. Where it is not, it is zero here.
#[derive(Debug, PartialEq, Eq, Clone, Copy, Hash, Serialize)]
pub struct Nlri {
pub prefix: IpPrefix,
pub path_id: u32,
}
impl Nlri {
/// `prefixes` as they are named on a session without ADD-PATH.
pub fn plain(prefixes: impl IntoIterator<Item = IpPrefix>) -> Vec<Self> {
prefixes.into_iter().map(Self::from).collect()
}
pub fn to_wire(&self, _: &ParserContext, out: &mut BytesMut) -> Result<(), ToWireError> {
// Verify that there is enough space to write the IpPrefix.
if out.remaining() < (self.prefix.len() + 1) {
Err(ToWireError::OutBufferOverflow)?;
/// The bytes this takes on the wire, with or without its identifier.
pub fn wire_len(&self, with_path_id: bool) -> usize {
1 + self.prefix.bytes().len() + if with_path_id { 4 } else { 0 }
}
}
impl From<IpPrefix> for Nlri {
fn from(prefix: IpPrefix) -> Self {
Self { prefix, path_id: 0 }
}
}
/// So that a list of these compares with a list of plain prefixes.
impl PartialEq<IpPrefix> for Nlri {
fn eq(&self, prefix: &IpPrefix) -> bool {
self.path_id == 0 && self.prefix == *prefix
}
}
impl Display for Nlri {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self.path_id {
0 => write!(f, "{}", self.prefix),
id => write!(f, "{} (path {id})", self.prefix),
}
// Write length and prefix.
out.put_u8(self.length);
out.put(self.prefix.as_slice());
Ok(())
}
}
impl TryFrom<&str> for IpPrefix {
type Error = eyre::Error;
type Error = PrefixError;
fn try_from(value: &str) -> Result<Self, Self::Error> {
let parts: Vec<&str> = value.split("/").collect();
let format = || PrefixError::Format(value.to_string());
let address = || PrefixError::Address(value.to_string());
let (addr, length) = value.split_once('/').ok_or_else(format)?;
let length: u8 = length.parse().map_err(|_| format())?;
if parts.len() != 2 {
bail!(
"Expected IpPrefix in format prefix/length but got: {}",
value
);
}
let length: u8 = u8::from_str_radix(parts[1], 10).map_err(eyre::Error::from)?;
let mut octets;
let afi: AddressFamilyId;
if parts[0].contains(":") {
afi = AddressFamilyId::Ipv6;
let addr: Ipv6Addr = Ipv6Addr::from_str(parts[0]).map_err(eyre::Error::from)?;
octets = addr.octets().to_vec();
} else if parts[0].contains(".") {
afi = AddressFamilyId::Ipv4;
let addr: Ipv4Addr = Ipv4Addr::from_str(parts[0]).map_err(eyre::Error::from)?;
octets = addr.octets().to_vec();
let (afi, octets) = if addr.contains(':') {
let addr = Ipv6Addr::from_str(addr).map_err(|_| address())?;
(IpVersion::Ipv6, addr.octets().to_vec())
} else {
bail!("Could not figure out address type")
}
// Truncate the octets we have to the right number of bytes to match the prefix length..
if length % 8 == 0 {
// We can cleanly truncate the number of bytes since we are at a byte boundary.
octets.truncate((length / 8).into());
} else {
// We need to keep length % 8 bits of the last byte.
let num_bytes = (length / 8) + 1;
let mask = u8::MAX << (8 - (length % 8));
octets.truncate(num_bytes.into());
// Fix up the last byte.
let last_pos = octets.len() - 1;
octets[last_pos] &= mask;
}
Ok(IpPrefix {
address_family: afi,
prefix: octets,
length,
})
let addr = Ipv4Addr::from_str(addr).map_err(|_| address())?;
(IpVersion::Ipv4, addr.octets().to_vec())
};
IpPrefix::new(afi, octets, length)
}
}
impl Display for IpPrefix {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self.address_family {
AddressFamilyId::Ipv4 => {
// Pad the prefix with 0 bytes if it's less than 4 bytes long.
let bytes = &mut self.prefix.clone();
if bytes.len() < 4 {
bytes.extend(std::iter::repeat(0).take(4 - bytes.len()));
}
let four_bytes: [u8; 4] = bytes
.as_slice()
.try_into()
.map_err(|_| std::fmt::Error {})?;
let ipv4_addr = Ipv4Addr::from(four_bytes);
write!(f, "{}/{}", ipv4_addr, self.length)
IpVersion::Ipv4 => {
let [a, b, c, d, ..] = self.address;
write!(f, "{}/{}", Ipv4Addr::new(a, b, c, d), self.length)
}
AddressFamilyId::Ipv6 => {
let bytes = &mut self.prefix.clone();
if bytes.len() < 16 {
bytes.extend(std::iter::repeat(0).take(16 - bytes.len()));
}
let sixteen_bytes: [u8; 16] = bytes
.as_slice()
.try_into()
.map_err(|_| std::fmt::Error {})?;
let ipv6_addr = Ipv6Addr::from(sixteen_bytes);
write!(f, "{}/{}", ipv6_addr, self.length)
IpVersion::Ipv6 => {
write!(f, "{}/{}", Ipv6Addr::from(self.address), self.length)
}
}
}
@@ -152,9 +190,11 @@ impl Display for IpPrefix {
#[cfg(test)]
mod tests {
use crate::constants::AddressFamilyId;
use crate::constants::IpVersion;
use super::IpPrefix;
use proptest::prelude::*;
use super::{IpPrefix, PrefixError};
macro_rules! verify_roundtrip {
($name:ident, $prefix_str:expr, $afi:expr, $bytes:expr, $length:expr) => {
@@ -162,9 +202,9 @@ mod tests {
fn $name() {
let ip_prefix = IpPrefix::try_from($prefix_str).unwrap();
assert_eq!(ip_prefix.address_family, $afi);
assert_eq!(ip_prefix.prefix, $bytes);
assert_eq!(ip_prefix.length, $length);
assert_eq!(ip_prefix.address_family(), $afi);
assert_eq!(ip_prefix.bytes(), $bytes);
assert_eq!(ip_prefix.length(), $length);
let to_str: &str = &ip_prefix.to_string();
assert_eq!(IpPrefix::try_from(to_str).unwrap(), ip_prefix);
@@ -175,7 +215,7 @@ mod tests {
verify_roundtrip!(
verify_roundtrip_ipv4_24,
"10.1.2.0/24",
AddressFamilyId::Ipv4,
IpVersion::Ipv4,
vec![10, 1, 2],
24
);
@@ -184,7 +224,7 @@ mod tests {
verify_roundtrip!(
verify_roundtrip_ipv4_19,
"10.245.123.0/19",
AddressFamilyId::Ipv4,
IpVersion::Ipv4,
vec![10, 245, 96],
19
);
@@ -193,7 +233,7 @@ mod tests {
verify_roundtrip!(
verify_roundtrip_ipv4_3,
"192.168.1.0/3",
AddressFamilyId::Ipv4,
IpVersion::Ipv4,
vec![192],
3
);
@@ -202,7 +242,7 @@ mod tests {
verify_roundtrip!(
verify_roundtrip_ipv4_0,
"0.0.0.0/0",
AddressFamilyId::Ipv4,
IpVersion::Ipv4,
vec![],
0
);
@@ -210,7 +250,7 @@ mod tests {
verify_roundtrip!(
verify_roundtrip_ipv6_48,
"2001:db8:cafe::/48",
AddressFamilyId::Ipv6,
IpVersion::Ipv6,
vec![32, 1, 13, 184, 202, 254],
48
);
@@ -219,11 +259,105 @@ mod tests {
verify_roundtrip!(
verify_roundtrip_ipv6_32,
"2001:db8:cafe::/32",
AddressFamilyId::Ipv6,
IpVersion::Ipv6,
vec![32, 1, 13, 184],
32
);
#[test]
fn text_that_is_not_a_prefix_is_rejected_with_the_reason() {
for (text, expected) in [
("10.0.0.0", PrefixError::Format("10.0.0.0".into())),
("10.0.0.0/x", PrefixError::Format("10.0.0.0/x".into())),
("10.0.0/8", PrefixError::Address("10.0.0/8".into())),
("nonsense/8", PrefixError::Address("nonsense/8".into())),
(
"10.0.0.0/33",
PrefixError::LengthTooLong {
length: 33,
max: 32,
},
),
(
"2001:db8::/129",
PrefixError::LengthTooLong {
length: 129,
max: 128,
},
),
] {
assert_eq!(IpPrefix::try_from(text), Err(expected), "{text}");
}
assert!(IpPrefix::try_from("0.0.0.0/0").is_ok());
assert!(IpPrefix::try_from("2001:db8::1/128").is_ok());
assert_eq!(
IpPrefix::new(IpVersion::Ipv4, vec![10], 16),
Err(PrefixError::TooFewBytes {
length: 16,
needed: 2,
got: 1
})
);
}
#[test]
fn a_prefix_has_one_representation() {
// Extra bytes and stray bits past the length are dropped.
let noisy = IpPrefix::new(IpVersion::Ipv4, vec![10, 0xff, 3, 4], 12).unwrap();
assert_eq!(noisy, IpPrefix::try_from("10.240.0.0/12").unwrap());
assert_eq!(noisy.bytes(), [10, 0xf0]);
assert_eq!(
IpPrefix::new(IpVersion::Ipv4, vec![0xff; 13], 99),
Err(PrefixError::LengthTooLong {
length: 99,
max: 32
})
);
}
#[test]
fn a_prefix_is_a_small_value() {
assert!(size_of::<IpPrefix>() <= 20);
let prefix = IpPrefix::try_from("10.1.2.0/24").unwrap();
let copy = prefix;
assert_eq!(copy, prefix);
assert_eq!(prefix.bytes(), [10, 1, 2]);
assert_eq!(IpPrefix::try_from("0.0.0.0/0").unwrap().bytes(), [0u8; 0]);
}
proptest! {
#[test]
fn text_round_trips(version: bool, bytes: [u8; 16], length in 0u8..=128) {
let version = if version { IpVersion::Ipv6 } else { IpVersion::Ipv4 };
let length = length.min(version.bits());
let prefix = IpPrefix::new(version, bytes, length).unwrap();
let text = prefix.to_string();
prop_assert_eq!(IpPrefix::try_from(text.as_str()), Ok(prefix));
prop_assert!(prefix.contains(&prefix));
}
#[test]
fn text_that_is_noise_is_an_error_or_a_prefix_that_round_trips(text in "\\PC{0,40}") {
if let Ok(prefix) = IpPrefix::try_from(text.as_str()) {
let shown = prefix.to_string();
prop_assert_eq!(IpPrefix::try_from(shown.as_str()), Ok(prefix));
}
}
/// `contains` agrees with comparing the leading bits by hand.
#[test]
fn contains_is_a_comparison_of_leading_bits(
a: [u8; 4], a_len in 0u8..=32, b: [u8; 4], b_len in 0u8..=32,
) {
let outer = IpPrefix::new(IpVersion::Ipv4, a, a_len).unwrap();
let inner = IpPrefix::new(IpVersion::Ipv4, b, b_len).unwrap();
let mask = if a_len == 0 { 0 } else { u32::MAX << (32 - a_len) };
let expected = b_len >= a_len
&& (u32::from_be_bytes(a) & mask) == (u32::from_be_bytes(b) & mask);
prop_assert_eq!(outer.contains(&inner), expected);
}
}
#[test]
fn contains_covers_more_specifics() {
let p = |s: &str| IpPrefix::try_from(s).unwrap();
@@ -240,11 +374,5 @@ mod tests {
assert!(p("2001:db8::/32").contains(&p("2001:db8:cafe::/48")));
}
verify_roundtrip!(
verify_roundtrip_ipv6_0,
"::/0",
AddressFamilyId::Ipv6,
vec![],
0
);
verify_roundtrip!(verify_roundtrip_ipv6_0, "::/0", IpVersion::Ipv6, vec![], 0);
}
crates/primitives/src/parser.rs
+33 -25
View File
@@ -1,35 +1,43 @@
use std::fmt::Display;
use nom::error::{ErrorKind, ParseError};
use crate::constants::AddressFamilyId;
use crate::constants::AfiSafi;
use thiserror::Error;
#[derive(Debug, Default)]
/// What reading or writing a message has to know about the session it
/// belongs to. The default is a session before its OPENs are exchanged;
/// only an UPDATE depends on anything here, and there are none until then.
#[derive(Debug, Clone, Default)]
pub struct ParserContext {
/// Whether thi parser is being run with a peer that is RFC6793 compliant.
pub four_octet_asn: Option<bool>,
/// Which address family should be parsed by default with this parser.
pub address_family: Option<AddressFamilyId>,
/// Whether AS numbers are four octets wide on this session (RFC 6793).
pub four_octet_asn: bool,
/// Whether the peer is in our own AS. Some attributes mean something
/// only between internal peers and are dropped when an external one
/// sends them (RFC 7606 §7).
pub ibgp: bool,
/// The address families whose prefixes carry a path identifier in the
/// direction this context is for (RFC 7911). What a session sends and
/// what it receives are negotiated separately, so a session has one
/// context for reading and one for writing.
pub add_path: Vec<AfiSafi>,
}
#[derive(Debug)]
pub enum ToWireError {
/// There was not enough space in the output buffer to serialize the data into.
OutBufferOverflow,
/// Another error.
Other(eyre::Error),
}
impl Display for ToWireError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
ToWireError::OutBufferOverflow => write!(f, "OutBufferOverflow"),
ToWireError::Other(report) => report.fmt(f),
}
impl ParserContext {
pub fn has_add_path(&self, family: AfiSafi) -> bool {
self.add_path.contains(&family)
}
}
impl std::error::Error for ToWireError {}
/// Why a value could not be written to the wire.
#[derive(Debug, Clone, PartialEq, Eq, Error)]
pub enum EncodeError {
/// A length does not fit the field that has to carry it.
#[error("{what} is too long to encode: {len} exceeds {max}")]
TooLong {
what: &'static str,
len: usize,
max: usize,
},
}
// Custom error type for the parser.
#[derive(Debug)]
@@ -37,8 +45,8 @@ pub enum BgpParserError<I> {
/// ProtocolError encapsulates protocol level semantics problems that require
/// action from the caller of the parser.
ProtocolError(ProtocolErrorValues),
CustomText(&'static str),
Eyre(eyre::ErrReport),
/// The input does not follow the format; the text says how.
Malformed(String),
Nom(I, ErrorKind),
}
crates/primitives/src/poptrie/mod.rs
+14 -5
View File
@@ -1,3 +1,10 @@
//! A Poptrie (Asai and Ohara, SIGCOMM 2015): a compressed multiway trie for
//! longest-prefix matching.
//!
//! Work in progress. Nothing in the daemon uses it yet, and the construction
//! side has no public entry point, hence the lint expectation below.
#![expect(dead_code, reason = "not yet wired into a RIB")]
use std::collections::BTreeMap;
#[derive(Debug)]
@@ -75,7 +82,8 @@ impl<T: Clone> Poptrie<T> {
result
}
fn new(&mut self, rt: BTreeMap<(u32, u8), T>) {
/// Fills the trie from a routing table keyed by (address, prefix length).
fn build(&mut self, rt: BTreeMap<(u32, u8), T>) {
const DEPTH: u8 = 6;
let mut forest = vec![(0, rt)];
@@ -168,9 +176,10 @@ mod tests {
leaves: vec![],
};
poptrie.new(map);
println!("Success");
dbg!(poptrie);
poptrie.build(map);
assert!(
!poptrie.inners.is_empty(),
"building from a non-empty table yields nodes"
);
}
}
crates/rib/Cargo.toml
+14
View File
@@ -1,11 +1,25 @@
[package]
name = "bgp-rib"
authors.workspace = true
edition.workspace = true
homepage.workspace = true
license.workspace = true
repository.workspace = true
version.workspace = true
[dependencies]
bgp-packet.workspace = true
bgp-primitives.workspace = true
bytes.workspace = true
foldhash.workspace = true
hashbrown.workspace = true
serde.workspace = true
thiserror.workspace = true
tokio = { workspace = true, features = ["full"] }
tracing.workspace = true
[dev-dependencies]
proptest.workspace = true
[lints]
workspace = true
crates/rib/proptest-regressions/table/journal_tests.txt
+7
View File
@@ -0,0 +1,7 @@
# Seeds for failure cases proptest has generated in the past. It is
# automatically read and these particular cases re-run before any
# novel cases are generated.
#
# It is recommended to check this file in to source control so that
# everyone who runs the test benefits from these saved cases.
cc 7107cf5580fe7bb95e138f1fead386d89273532f83aa1d11808aa9586aa9da63 # shrinks to ops = [Withdraw { peer: 0, prefix: 1 }, Withdraw { peer: 0, prefix: 1 }, Poll { reader: 2, budget: 2 }, Withdraw { peer: 0, prefix: 10 }, Poll { reader: 2, budget: 4 }, Announce { peer: 2, prefix: 7, route: Route { local_pref: None, path: [1, 2], origin: 0, med: Some(0) } }, Announce { peer: 3, prefix: 6, route: Route { local_pref: None, path: [1], origin: 0, med: None } }, Announce { peer: 3, prefix: 6, route: Route { local_pref: None, path: [1, 1], origin: 0, med: None } }, Withdraw { peer: 1, prefix: 5 }, Withdraw { peer: 0, prefix: 11 }, Withdraw { peer: 3, prefix: 6 }, Announce { peer: 0, prefix: 3, route: Route { local_pref: None, path: [1, 2], origin: 1, med: None } }, Poll { reader: 2, budget: 3 }, Withdraw { peer: 0, prefix: 8 }, Flap { peer: 3 }, Withdraw { peer: 1, prefix: 1 }, Announce { peer: 2, prefix: 5, route: Route { local_pref: None, path: [1], origin: 1, med: Some(0) } }, Announce { peer: 0, prefix: 1, route: Route { local_pref: None, path: [1, 1], origin: 1, med: None } }, Withdraw { peer: 0, prefix: 1 }, Withdraw { peer: 2, prefix: 8 }, Poll { reader: 1, budget: 2 }, Flap { peer: 1 }, Poll { reader: 2, budget: 2 }, Announce { peer: 0, prefix: 9, route: Route { local_pref: None, path: [1], origin: 0, med: Some(1) } }, Announce { peer: 2, prefix: 2, route: Route { local_pref: Some(100), path: [2], origin: 0, med: Some(1) } }, Poll { reader: 1, budget: 3 }, Withdraw { peer: 3, prefix: 1 }, Announce { peer: 1, prefix: 4, route: Route { local_pref: None, path: [2], origin: 1, med: None } }, Announce { peer: 3, prefix: 7, route: Route { local_pref: None, path: [2], origin: 0, med: None } }, Announce { peer: 1, prefix: 11, route: Route { local_pref: None, path: [1], origin: 1, med: Some(1) } }, Withdraw { peer: 0, prefix: 6 }, Withdraw { peer: 0, prefix: 4 }, Announce { peer: 3, prefix: 10, route: Route { local_pref: Some(100), path: [2, 2], origin: 1, med: None } }, Announce { peer: 1, prefix: 2, route: Route { local_pref: None, path: [2], origin: 0, med: Some(2) } }, Announce { peer: 1, prefix: 3, route: Route { local_pref: None, path: [1], origin: 1, med: None } }, Withdraw { peer: 0, prefix: 2 }, Announce { peer: 1, prefix: 4, route: Route { local_pref: None, path: [1, 1], origin: 0, med: None } }, Poll { reader: 1, budget: 2 }, Poll { reader: 1, budget: 4 }, Poll { reader: 2, budget: 1 }, Poll { reader: 1, budget: 3 }, Poll { reader: 1, budget: 5 }, Poll { reader: 1, budget: 1 }, Withdraw { peer: 3, prefix: 5 }, Withdraw { peer: 3, prefix: 9 }, Withdraw { peer: 2, prefix: 10 }], shards = 3
crates/rib/proptest-regressions/table/tests.txt
+7
View File
@@ -0,0 +1,7 @@
# Seeds for failure cases proptest has generated in the past. It is
# automatically read and these particular cases re-run before any
# novel cases are generated.
#
# It is recommended to check this file in to source control so that
# everyone who runs the test benefits from these saved cases.
cc 6bcfc317c976928c2fbb29e943f385127b91770a76f18e7061b80b31f4276955 # shrinks to ops = [Announce { peer: 1, prefix: 1, route: Route { local_pref: None, path: [2], origin: 0, med: None } }, Announce { peer: 2, prefix: 1, route: Route { local_pref: None, path: [1], origin: 0, med: Some(2) } }, Announce { peer: 3, prefix: 1, route: Route { local_pref: None, path: [1], origin: 0, med: None } }, Announce { peer: 3, prefix: 1, route: Route { local_pref: None, path: [1], origin: 1, med: None } }]
crates/rib/src/adj_rib_in.rs
-112
View File
@@ -1,112 +0,0 @@
use std::collections::HashMap;
use bgp_primitives::ip_prefix::IpPrefix;
use tokio::sync::{broadcast, mpsc, oneshot};
use tracing::warn;
use crate::entry::{PeerId, RibEntry, SnapshotEntry};
use crate::event::RibEvent;
const EVENT_CHANNEL_CAPACITY: usize = 1024;
const CMD_CHANNEL_CAPACITY: usize = 256;
enum RibCommand {
Announce { peer: PeerId, prefix: IpPrefix, entry: RibEntry },
Withdraw { peer: PeerId, prefix: IpPrefix },
/// Drop everything learned from `peer`, e.g. because its session ended.
RemovePeer { peer: PeerId, reply: oneshot::Sender<usize> },
Snapshot { reply: oneshot::Sender<Vec<SnapshotEntry>> },
}
/// Handle to the Adj-RIB-In actor. Cheap to clone — all clones share the same
/// backing task. The actor is stopped when all handles are dropped.
#[derive(Clone)]
pub struct AdjRibIn {
tx: mpsc::Sender<RibCommand>,
events: broadcast::Sender<RibEvent>,
}
impl AdjRibIn {
/// Spawns the backing actor and returns a handle.
pub fn spawn() -> Self {
let (tx, rx) = mpsc::channel(CMD_CHANNEL_CAPACITY);
let (events, _) = broadcast::channel(EVENT_CHANNEL_CAPACITY);
let events_clone = events.clone();
tokio::spawn(run_actor(rx, events_clone));
Self { tx, events }
}
pub async fn announce(&self, peer: PeerId, prefix: IpPrefix, entry: RibEntry) {
let _ = self.tx.send(RibCommand::Announce { peer, prefix, entry }).await;
}
pub async fn withdraw(&self, peer: PeerId, prefix: IpPrefix) {
let _ = self.tx.send(RibCommand::Withdraw { peer, prefix }).await;
}
/// Removes every route learned from `peer`, emitting a
/// [`RibEvent::Withdrawn`] for each (RFC 4271 §8.2.2: routes from a
/// terminated session are deleted). Returns how many were removed.
pub async fn remove_peer(&self, peer: PeerId) -> usize {
let (reply_tx, reply_rx) = oneshot::channel();
let _ = self.tx.send(RibCommand::RemovePeer { peer, reply: reply_tx }).await;
reply_rx.await.unwrap_or(0)
}
/// Returns a point-in-time snapshot of all routes in the RIB.
pub async fn snapshot(&self) -> Vec<SnapshotEntry> {
let (reply_tx, reply_rx) = oneshot::channel();
let _ = self.tx.send(RibCommand::Snapshot { reply: reply_tx }).await;
reply_rx.await.unwrap_or_default()
}
/// Subscribe to a stream of RIB mutation events.
///
/// The receiver will see every [`RibEvent`] from the point of subscription.
/// If the consumer falls behind by more than [`EVENT_CHANNEL_CAPACITY`]
/// events it will receive a [`broadcast::error::RecvError::Lagged`] error.
pub fn subscribe(&self) -> broadcast::Receiver<RibEvent> {
self.events.subscribe()
}
}
async fn run_actor(mut rx: mpsc::Receiver<RibCommand>, events: broadcast::Sender<RibEvent>) {
let mut table: HashMap<(PeerId, IpPrefix), RibEntry> = HashMap::new();
while let Some(cmd) = rx.recv().await {
match cmd {
// Send errors below mean there are no subscribers, which is fine.
RibCommand::Announce { peer, prefix, entry } => {
table.insert((peer, prefix.clone()), entry.clone());
let _ = events.send(RibEvent::Announced { peer, prefix, entry });
}
RibCommand::Withdraw { peer, prefix } => {
if table.remove(&(peer, prefix.clone())).is_some() {
let _ = events.send(RibEvent::Withdrawn { peer, prefix });
}
}
RibCommand::RemovePeer { peer, reply } => {
let prefixes: Vec<IpPrefix> =
table.keys().filter(|(p, _)| *p == peer).map(|(_, prefix)| prefix.clone()).collect();
for prefix in &prefixes {
table.remove(&(peer, prefix.clone()));
let _ = events.send(RibEvent::Withdrawn { peer, prefix: prefix.clone() });
}
let _ = reply.send(prefixes.len());
}
RibCommand::Snapshot { reply } => {
let snap = table
.iter()
.map(|((peer, prefix), entry)| SnapshotEntry {
peer: *peer,
prefix: prefix.clone(),
entry: entry.clone(),
})
.collect();
if reply.send(snap).is_err() {
warn!("Snapshot reply channel closed before send");
}
}
}
}
}
crates/rib/src/entry.rs
-39
View File
@@ -1,39 +0,0 @@
use std::net::SocketAddr;
use std::time::SystemTime;
use bgp_packet::message::PathAttribute;
use bgp_primitives::ip_prefix::IpPrefix;
/// Identifies a BGP peer by its TCP address and negotiated ASN.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub struct PeerId {
pub addr: SocketAddr,
pub asn: u32,
}
impl std::fmt::Display for PeerId {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "{}(AS{})", self.addr, self.asn)
}
}
/// A single route stored in the Adj-RIB-In.
#[derive(Debug, Clone)]
pub struct RibEntry {
pub path_attributes: Vec<PathAttribute>,
pub received_at: SystemTime,
}
impl RibEntry {
pub fn new(path_attributes: Vec<PathAttribute>) -> Self {
Self { path_attributes, received_at: SystemTime::now() }
}
}
/// A flattened row returned by [`crate::AdjRibIn::snapshot`].
#[derive(Debug, Clone)]
pub struct SnapshotEntry {
pub peer: PeerId,
pub prefix: IpPrefix,
pub entry: RibEntry,
}
crates/rib/src/error.rs
+14
View File
@@ -0,0 +1,14 @@
/// A RIB's actor is no longer running, so the table can be neither read nor
/// changed.
///
/// The actor runs for as long as a handle to it exists; it stops early only
/// by panicking. That is a bug, and one the server cannot work around: it
/// could no longer say what it originates. Callers pass this up
/// until the process exits and is restarted, rather than carry on with a
/// table that looks empty.
#[derive(Debug, Clone, Copy, PartialEq, Eq, thiserror::Error)]
#[error("the {rib} has stopped")]
pub struct RibStopped {
/// Which table.
pub rib: &'static str,
}
crates/rib/src/event.rs
-9
View File
@@ -1,9 +0,0 @@
use bgp_primitives::ip_prefix::IpPrefix;
use crate::entry::{PeerId, RibEntry};
#[derive(Debug, Clone)]
pub enum RibEvent {
Announced { peer: PeerId, prefix: IpPrefix, entry: RibEntry },
Withdrawn { peer: PeerId, prefix: IpPrefix },
}
crates/rib/src/lib.rs
+8 -8
View File
@@ -1,9 +1,9 @@
mod adj_rib_in;
mod entry;
mod event;
mod local_rib;
//! The router's tables: the routes it has learned and chosen
//! ([`table::RibTable`]) and the routes it originates ([`LocalRib`]).
pub use adj_rib_in::AdjRibIn;
pub use entry::{PeerId, RibEntry, SnapshotEntry};
pub use event::RibEvent;
pub use local_rib::{LocalRib, LocalRibError, LocalRibEvent, LocalRibPolicy, LocalRoute, LocalRouteAttrs};
mod error;
mod local_rib;
pub mod table;
pub use error::RibStopped;
pub use local_rib::{LocalRib, LocalRibEvent, LocalRoute, LocalRouteAttrs};
crates/rib/src/local_rib.rs
+136 -179
View File
@@ -1,24 +1,26 @@
//! Locally originated routes (the "Loc-RIB" of routes this router announces
//! on its own behalf, as opposed to routes learned from peers).
//!
//! The actor stores one route per prefix together with the attributes chosen
//! by whoever announced it. It does not know about peers or sessions: the
//! server subscribes to [`LocalRibEvent`]s and turns them into UPDATEs.
//! The actor stores one route per prefix together with its attributes. It
//! does not know about peers or sessions: the server subscribes to
//! [`LocalRibEvent`]s and turns them into UPDATEs.
use std::collections::HashMap;
use std::time::SystemTime;
use bgp_packet::message::{LargeCommunity, OriginPathAttribute};
use bgp_primitives::ip_prefix::IpPrefix;
use serde::{Deserialize, Serialize};
use serde::Serialize;
use tokio::sync::{broadcast, mpsc, oneshot};
use tracing::warn;
use tokio::task::AbortHandle;
use crate::error::RibStopped;
const EVENT_CHANNEL_CAPACITY: usize = 1024;
const CMD_CHANNEL_CAPACITY: usize = 256;
/// User-selectable attributes for a locally originated route.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
/// Attributes of a locally originated route.
#[derive(Debug, Clone, PartialEq, Eq, Serialize)]
pub struct LocalRouteAttrs {
pub origin: OriginPathAttribute,
/// Number of *additional* copies of the local ASN to prepend to the
@@ -44,197 +46,155 @@ impl Default for LocalRouteAttrs {
}
/// A route originated by this router.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[derive(Debug, Clone, PartialEq, Eq, Serialize)]
pub struct LocalRoute {
pub prefix: IpPrefix,
/// Free-form identity of whoever announced the route (e.g. a student
/// name). Used to stop one owner replacing or withdrawing another's route.
pub owner: String,
pub attrs: LocalRouteAttrs,
pub announced_at: SystemTime,
}
impl LocalRoute {
pub fn new(prefix: IpPrefix, owner: impl Into<String>, attrs: LocalRouteAttrs) -> Self {
Self { prefix, owner: owner.into(), attrs, announced_at: SystemTime::now() }
pub fn new(prefix: IpPrefix, attrs: LocalRouteAttrs) -> Self {
Self {
prefix,
attrs,
announced_at: SystemTime::now(),
}
}
}
/// Limits on what may be announced. Enforced inside the actor so every caller
/// (HTTP API, tests, future CLI) goes through the same checks.
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct LocalRibPolicy {
/// A route is accepted only if it is covered by at least one of these
/// prefixes. Empty means no restriction.
pub allowed_prefixes: Vec<IpPrefix>,
/// Maximum number of routes a single owner may hold. `0` means unlimited.
pub max_routes_per_owner: usize,
}
impl LocalRibPolicy {
fn allows(&self, prefix: &IpPrefix) -> bool {
self.allowed_prefixes.is_empty() || self.allowed_prefixes.iter().any(|p| p.contains(prefix))
}
}
#[derive(Debug, Clone, PartialEq, Eq, thiserror::Error)]
pub enum LocalRibError {
#[error("prefix {0} is not covered by the allowed prefixes")]
NotAllowed(IpPrefix),
#[error("owner {owner:?} already has {limit} routes")]
OwnerLimit { owner: String, limit: usize },
#[error("prefix {prefix} is owned by {owner:?}")]
OwnedByOther { prefix: IpPrefix, owner: String },
#[error("local RIB actor has stopped")]
Closed,
}
const STOPPED: RibStopped = RibStopped { rib: "local RIB" };
#[derive(Debug, Clone)]
pub enum LocalRibEvent {
/// A route was added or its attributes replaced.
Announced(LocalRoute),
Withdrawn { prefix: IpPrefix },
Withdrawn {
prefix: IpPrefix,
},
}
enum Command {
Announce { route: LocalRoute, reply: oneshot::Sender<Result<(), LocalRibError>> },
Announce {
route: LocalRoute,
reply: oneshot::Sender<()>,
},
Withdraw {
prefix: IpPrefix,
requester: Option<String>,
reply: oneshot::Sender<Result<Option<LocalRoute>, LocalRibError>>,
reply: oneshot::Sender<Option<LocalRoute>>,
},
Snapshot {
reply: oneshot::Sender<Vec<LocalRoute>>,
},
Len {
reply: oneshot::Sender<usize>,
},
Snapshot { reply: oneshot::Sender<Vec<LocalRoute>> },
}
/// Handle to the local RIB actor. Cheap to clone; all clones share one table.
///
/// Every operation fails with [`RibStopped`] if the actor is gone; see
/// there for what that means.
#[derive(Clone)]
pub struct LocalRib {
tx: mpsc::Sender<Command>,
events: broadcast::Sender<LocalRibEvent>,
actor: AbortHandle,
}
impl LocalRib {
pub fn spawn(policy: LocalRibPolicy) -> Self {
pub fn spawn() -> Self {
let (tx, rx) = mpsc::channel(CMD_CHANNEL_CAPACITY);
let (events, _) = broadcast::channel(EVENT_CHANNEL_CAPACITY);
tokio::spawn(run_actor(rx, events.clone(), policy));
Self { tx, events }
let actor = tokio::spawn(run_actor(rx, events.clone())).abort_handle();
Self { tx, events, actor }
}
/// Adds a route, or replaces the attributes of an existing route with the
/// same prefix and owner.
pub async fn announce(&self, route: LocalRoute) -> Result<(), LocalRibError> {
let (reply, rx) = oneshot::channel();
self.tx
.send(Command::Announce { route, reply })
.await
.map_err(|_| LocalRibError::Closed)?;
rx.await.map_err(|_| LocalRibError::Closed)?
/// Adds a route, or replaces the existing route for the same prefix.
pub async fn announce(&self, route: LocalRoute) -> Result<(), RibStopped> {
self.ask(|reply| Command::Announce { route, reply }).await
}
/// Removes a route. `requester` must match the route's owner; pass `None`
/// to withdraw regardless of owner (administrative override). Returns the
/// removed route, or `None` if the prefix was not present.
pub async fn withdraw(
&self,
prefix: IpPrefix,
requester: Option<&str>,
) -> Result<Option<LocalRoute>, LocalRibError> {
let (reply, rx) = oneshot::channel();
self.tx
.send(Command::Withdraw { prefix, requester: requester.map(str::to_owned), reply })
.await
.map_err(|_| LocalRibError::Closed)?;
rx.await.map_err(|_| LocalRibError::Closed)?
/// Removes a route. Returns the removed route, or `None` if the prefix
/// was not present.
pub async fn withdraw(&self, prefix: IpPrefix) -> Result<Option<LocalRoute>, RibStopped> {
self.ask(|reply| Command::Withdraw { prefix, reply }).await
}
/// Point-in-time copy of every route, in no particular order.
pub async fn snapshot(&self) -> Vec<LocalRoute> {
let (reply, rx) = oneshot::channel();
let _ = self.tx.send(Command::Snapshot { reply }).await;
rx.await.unwrap_or_default()
pub async fn snapshot(&self) -> Result<Vec<LocalRoute>, RibStopped> {
self.ask(|reply| Command::Snapshot { reply }).await
}
/// Subscribe to mutations. See [`crate::AdjRibIn::subscribe`] for lag
/// semantics.
/// How many routes the RIB holds, without copying them.
pub async fn len(&self) -> Result<usize, RibStopped> {
self.ask(|reply| Command::Len { reply }).await
}
/// Whether the RIB holds no routes.
pub async fn is_empty(&self) -> Result<bool, RibStopped> {
Ok(self.len().await? == 0)
}
/// Sends a command and waits for its answer.
async fn ask<T>(
&self,
command: impl FnOnce(oneshot::Sender<T>) -> Command,
) -> Result<T, RibStopped> {
let (reply, answer) = oneshot::channel();
self.tx.send(command(reply)).await.map_err(|_| STOPPED)?;
answer.await.map_err(|_| STOPPED)
}
/// Resolves when the actor has stopped. It never does in a healthy
/// process; whoever owns the RIB waits on this to notice a failure even
/// when nothing happens to be using the table.
pub async fn stopped(&self) {
self.tx.closed().await
}
/// Stops the actor, as a panic in it would. For tests of how the rest
/// of the server copes.
#[doc(hidden)]
pub fn stop_for_testing(&self) {
self.actor.abort();
}
/// Subscribe to mutations. A subscriber that falls too far behind is
/// told so with [`broadcast::error::RecvError::Lagged`].
pub fn subscribe(&self) -> broadcast::Receiver<LocalRibEvent> {
self.events.subscribe()
}
}
async fn run_actor(
mut rx: mpsc::Receiver<Command>,
events: broadcast::Sender<LocalRibEvent>,
policy: LocalRibPolicy,
) {
async fn run_actor(mut rx: mpsc::Receiver<Command>, events: broadcast::Sender<LocalRibEvent>) {
let mut table: HashMap<IpPrefix, LocalRoute> = HashMap::new();
while let Some(cmd) = rx.recv().await {
match cmd {
Command::Announce { route, reply } => {
let result = check_announce(&table, &policy, &route).map(|()| {
table.insert(route.prefix.clone(), route.clone());
let _ = events.send(LocalRibEvent::Announced(route));
});
let _ = reply.send(result);
table.insert(route.prefix, route.clone());
let _ = events.send(LocalRibEvent::Announced(route));
let _ = reply.send(());
}
Command::Withdraw { prefix, requester, reply } => {
let result = match (table.get(&prefix), requester) {
(Some(existing), Some(req)) if existing.owner != req => {
Err(LocalRibError::OwnedByOther {
prefix: prefix.clone(),
owner: existing.owner.clone(),
})
}
_ => {
let removed = table.remove(&prefix);
if removed.is_some() {
let _ = events.send(LocalRibEvent::Withdrawn { prefix });
}
Ok(removed)
}
};
let _ = reply.send(result);
Command::Withdraw { prefix, reply } => {
let removed = table.remove(&prefix);
if removed.is_some() {
let _ = events.send(LocalRibEvent::Withdrawn { prefix });
}
let _ = reply.send(removed);
}
Command::Snapshot { reply } => {
if reply.send(table.values().cloned().collect()).is_err() {
warn!("Snapshot reply channel closed before send");
}
// A failed reply means the asker stopped waiting.
let _ = reply.send(table.values().cloned().collect());
}
Command::Len { reply } => {
let _ = reply.send(table.len());
}
}
}
}
fn check_announce(
table: &HashMap<IpPrefix, LocalRoute>,
policy: &LocalRibPolicy,
route: &LocalRoute,
) -> Result<(), LocalRibError> {
if !policy.allows(&route.prefix) {
return Err(LocalRibError::NotAllowed(route.prefix.clone()));
}
if let Some(existing) = table.get(&route.prefix) {
if existing.owner != route.owner {
return Err(LocalRibError::OwnedByOther {
prefix: route.prefix.clone(),
owner: existing.owner.clone(),
});
}
// Replacing our own route never changes the per-owner count.
return Ok(());
}
if policy.max_routes_per_owner > 0 {
let held = table.values().filter(|r| r.owner == route.owner).count();
if held >= policy.max_routes_per_owner {
return Err(LocalRibError::OwnerLimit {
owner: route.owner.clone(),
limit: policy.max_routes_per_owner,
});
}
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
@@ -243,63 +203,60 @@ mod tests {
IpPrefix::try_from(s).unwrap()
}
fn route(p: &str, owner: &str) -> LocalRoute {
LocalRoute::new(prefix(p), owner, LocalRouteAttrs::default())
fn route(p: &str) -> LocalRoute {
LocalRoute::new(prefix(p), LocalRouteAttrs::default())
}
#[tokio::test]
async fn announce_and_withdraw_emit_events() {
let rib = LocalRib::spawn(LocalRibPolicy::default());
let rib = LocalRib::spawn();
let mut events = rib.subscribe();
rib.announce(route("10.0.0.0/24", "alice")).await.unwrap();
assert!(matches!(events.recv().await.unwrap(), LocalRibEvent::Announced(r) if r.prefix == prefix("10.0.0.0/24")));
assert_eq!(rib.snapshot().await.len(), 1);
rib.announce(route("10.0.0.0/24")).await.unwrap();
assert!(
matches!(events.recv().await.unwrap(), LocalRibEvent::Announced(r) if r.prefix == prefix("10.0.0.0/24"))
);
assert_eq!(rib.len().await, Ok(1));
let removed = rib.withdraw(prefix("10.0.0.0/24"), Some("alice")).await.unwrap();
assert_eq!(removed.unwrap().owner, "alice");
assert!(matches!(events.recv().await.unwrap(), LocalRibEvent::Withdrawn { prefix: p } if p == prefix("10.0.0.0/24")));
assert!(rib.snapshot().await.is_empty());
let removed = rib.withdraw(prefix("10.0.0.0/24")).await.unwrap();
assert_eq!(removed.unwrap().prefix, prefix("10.0.0.0/24"));
assert!(
matches!(events.recv().await.unwrap(), LocalRibEvent::Withdrawn { prefix: p } if p == prefix("10.0.0.0/24"))
);
assert_eq!(rib.is_empty().await, Ok(true));
// Withdrawing an absent prefix is a no-op, not an error.
assert_eq!(rib.withdraw(prefix("10.0.0.0/24"), None).await.unwrap(), None);
assert_eq!(rib.withdraw(prefix("10.0.0.0/24")).await.unwrap(), None);
}
#[tokio::test]
async fn policy_allowlist_is_enforced() {
let rib = LocalRib::spawn(LocalRibPolicy {
allowed_prefixes: vec![prefix("10.100.0.0/16")],
max_routes_per_owner: 0,
});
rib.announce(route("10.100.3.0/24", "alice")).await.unwrap();
let err = rib.announce(route("192.0.2.0/24", "alice")).await.unwrap_err();
assert_eq!(err, LocalRibError::NotAllowed(prefix("192.0.2.0/24")));
async fn announcing_a_prefix_again_replaces_its_attributes() {
let rib = LocalRib::spawn();
rib.announce(route("10.0.0.0/24")).await.unwrap();
let attrs = LocalRouteAttrs {
med: Some(50),
..Default::default()
};
rib.announce(LocalRoute::new(prefix("10.0.0.0/24"), attrs.clone()))
.await
.unwrap();
let table = rib.snapshot().await.unwrap();
assert_eq!(table.len(), 1);
assert_eq!(table[0].attrs, attrs);
}
#[tokio::test]
async fn owner_limit_ignores_replacements() {
let rib = LocalRib::spawn(LocalRibPolicy { allowed_prefixes: vec![], max_routes_per_owner: 1 });
rib.announce(route("10.0.0.0/24", "alice")).await.unwrap();
// Replacing the same prefix is fine.
rib.announce(route("10.0.0.0/24", "alice")).await.unwrap();
let err = rib.announce(route("10.0.1.0/24", "alice")).await.unwrap_err();
assert!(matches!(err, LocalRibError::OwnerLimit { limit: 1, .. }));
// Other owners are unaffected.
rib.announce(route("10.0.1.0/24", "bob")).await.unwrap();
}
async fn a_stopped_actor_is_an_error_and_not_an_empty_table() {
let rib = LocalRib::spawn();
rib.announce(route("10.0.0.0/24")).await.unwrap();
#[tokio::test]
async fn other_owner_cannot_replace_or_withdraw() {
let rib = LocalRib::spawn(LocalRibPolicy::default());
rib.announce(route("10.0.0.0/24", "alice")).await.unwrap();
rib.stop_for_testing();
rib.stopped().await;
let err = rib.announce(route("10.0.0.0/24", "bob")).await.unwrap_err();
assert!(matches!(err, LocalRibError::OwnedByOther { .. }));
let err = rib.withdraw(prefix("10.0.0.0/24"), Some("bob")).await.unwrap_err();
assert!(matches!(err, LocalRibError::OwnedByOther { .. }));
// Administrative override works.
assert!(rib.withdraw(prefix("10.0.0.0/24"), None).await.unwrap().is_some());
assert_eq!(rib.announce(route("10.0.1.0/24")).await, Err(STOPPED));
assert_eq!(rib.withdraw(prefix("10.0.0.0/24")).await, Err(STOPPED));
assert_eq!(rib.snapshot().await, Err(STOPPED));
assert_eq!(rib.len().await, Err(STOPPED));
}
}
crates/rib/src/table/attrs.rs
+449
View File
@@ -0,0 +1,449 @@
//! Attribute sets, stored once.
//!
//! A full table has millions of routes and far fewer distinct lists of path
//! attributes: a peer announces prefixes that share their attributes
//! together, about six to an UPDATE on the Internet today. A route in the
//! table is therefore a pointer to an [`AttrSet`], and each distinct set
//! exists once, in the [`AttrStore`].
//!
//! A set is kept as its bytes on the wire, which is a fifth the size of the
//! parsed form. What choosing between routes needs from it is worked out
//! once, when the set is stored, and kept beside the bytes.
use std::hash::BuildHasher;
use std::sync::{Arc, Mutex, MutexGuard, PoisonError};
use bgp_packet::message::{NlriNextHop, PathAttribute, attr_type, split_attribute};
use bgp_primitives::parser::{EncodeError, ParserContext};
use bytes::{BufMut, BytesMut};
use foldhash::fast::RandomState;
use hashbrown::HashTable;
const OPTIONAL: u8 = 0x80;
/// What route selection reads from a set of attributes (RFC 4271 §9.1.2.2).
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct SelectionKey {
/// LOCAL_PREF, if the set has one.
pub local_pref: Option<u32>,
/// MULTI_EXIT_DISC, if the set has one.
pub med: Option<u32>,
/// The AS that the route was learned from: the first AS of the path, or
/// zero if the path is empty or begins with a set. MEDs are comparable
/// only between routes with the same one.
pub neighbor_as: u32,
/// The length of the AS path as selection counts it: an AS_SET is one
/// hop however many AS numbers it holds.
pub as_path_len: u16,
/// ORIGIN: 0 for IGP, 1 for EGP, 2 for INCOMPLETE.
pub origin: u8,
}
/// Reads the selection key straight off attributes in the form an
/// [`AttrSet`] keeps, without building anything. `None` if the bytes are
/// not a well-formed list of attributes.
fn scan(mut wire: &[u8]) -> Option<SelectionKey> {
let mut key = SelectionKey {
local_pref: None,
med: None,
neighbor_as: 0,
as_path_len: 0,
// A route without ORIGIN does not get this far; if one did, it
// would be the least preferred.
origin: 2,
};
let number = |value: &[u8]| Some(u32::from_be_bytes(<[u8; 4]>::try_from(value).ok()?));
while !wire.is_empty() {
let (attribute, rest) = split_attribute(wire).ok()?;
wire = rest;
let value = attribute.value;
match attribute.type_code {
attr_type::ORIGIN => key.origin = *value.first()?,
attr_type::AS_PATH => (key.as_path_len, key.neighbor_as) = scan_as_path(value)?,
attr_type::MULTI_EXIT_DISC => key.med = Some(number(value)?),
attr_type::LOCAL_PREF => key.local_pref = Some(number(value)?),
_ => {}
}
}
Some(key)
}
/// The length of an AS_PATH as selection counts it, and the neighbouring
/// AS, from the attribute's value with four-octet AS numbers.
fn scan_as_path(mut segments: &[u8]) -> Option<(u16, u32)> {
const AS_SEQUENCE: u8 = 2;
let mut hops = 0usize;
let mut neighbor_as = 0;
let mut first = true;
while let [kind, count, rest @ ..] = segments {
let (asns, rest) = rest.split_at_checked(usize::from(*count) * 4)?;
let ordered = *kind == AS_SEQUENCE;
hops += if ordered { usize::from(*count) } else { 1 };
if first && ordered {
neighbor_as = asns
.first_chunk::<4>()
.map_or(0, |asn| u32::from_be_bytes(*asn));
}
first = false;
segments = rest;
}
if !segments.is_empty() {
return None;
}
Some((u16::try_from(hops).unwrap_or(u16::MAX), neighbor_as))
}
/// One distinct list of path attributes, with the next hop that goes with
/// it. Made by an [`AttrStore`]; two routes have the same attributes
/// exactly when they point to the same `AttrSet`.
#[derive(Debug)]
pub struct AttrSet {
hash: u64,
key: SelectionKey,
/// The attributes as on the wire, with four-octet AS numbers. A next
/// hop that arrived in MP_REACH_NLRI is kept as an MP_REACH_NLRI that
/// holds nothing else, the form RFC 6396 §4.3.4 uses in table dumps.
wire: Box<[u8]>,
}
impl AttrSet {
pub fn key(&self) -> &SelectionKey {
&self.key
}
/// The attributes as they are on the wire.
pub fn wire(&self) -> &[u8] {
&self.wire
}
/// The attributes, parsed, and the next hop if it came in
/// MP_REACH_NLRI.
///
/// `None` if they do not parse. Sets made from parsed attributes or
/// from an UPDATE the decoder accepted always do; one made from bytes
/// of another source ([`AttrStore::intern_wire`]) was checked only as
/// far as selection reads it.
pub fn attributes(&self) -> Option<(Vec<PathAttribute>, Option<NlriNextHop>)> {
parse(&self.wire)
}
}
/// How the attributes of a set are written: with four-octet AS numbers.
fn four_octet() -> ParserContext {
ParserContext {
four_octet_asn: true,
ibgp: true,
add_path: Vec::new(),
}
}
/// Writes `attributes` and `next_hop` in the form an [`AttrSet`] keeps.
fn encode(
attributes: &[PathAttribute],
next_hop: Option<&NlriNextHop>,
out: &mut BytesMut,
) -> Result<(), EncodeError> {
let ctx = four_octet();
for attribute in attributes {
// These name prefixes, which are not a property of the route.
if matches!(
attribute,
PathAttribute::MpReachNlri(_) | PathAttribute::MpUnreachNlri(_)
) {
continue;
}
attribute.to_wire(&ctx, out)?;
}
if let Some(next_hop) = next_hop {
encode_next_hop(next_hop, out);
}
Ok(())
}
/// Writes a next hop as an MP_REACH_NLRI that holds nothing else.
fn encode_next_hop(next_hop: &NlriNextHop, out: &mut BytesMut) {
out.put_u8(OPTIONAL);
out.put_u8(attr_type::MP_REACH_NLRI);
out.put_u8(1 + next_hop.wire_len());
out.put_u8(next_hop.wire_len());
next_hop.to_wire(out);
}
/// Parses path attributes in the form [`AttrSet`] keeps and table dumps
/// use (RFC 6396 §4.3.4): four-octet AS numbers, and an MP_REACH_NLRI that
/// holds only a next hop. `None` if they are malformed.
fn parse(mut wire: &[u8]) -> Option<(Vec<PathAttribute>, Option<NlriNextHop>)> {
let ctx = four_octet();
let mut attributes = Vec::new();
let mut next_hop = None;
while !wire.is_empty() {
let (raw, rest) = split_attribute(wire).ok()?;
if raw.type_code == attr_type::MP_REACH_NLRI {
let (length, address) = raw.value.split_first()?;
if usize::from(*length) != address.len() {
return None;
}
next_hop = Some(NlriNextHop::from_bytes(address)?);
} else {
let (_, attribute) = PathAttribute::from_wire(&ctx, raw.wire).ok()?;
attributes.push(attribute);
}
wire = rest;
}
Some((attributes, next_hop))
}
/// How many locks the store is split over. Sessions intern concurrently,
/// once per UPDATE; this many locks keep them out of each other's way.
const STRIPES: usize = 64;
/// The one place attribute sets are kept. Safe to share between threads.
pub struct AttrStore {
stripes: Box<[Mutex<HashTable<Arc<AttrSet>>>]>,
hasher: RandomState,
}
impl Default for AttrStore {
fn default() -> Self {
Self::new()
}
}
impl AttrStore {
pub fn new() -> Self {
Self {
stripes: (0..STRIPES).map(|_| Mutex::default()).collect(),
hasher: RandomState::default(),
}
}
/// The stored set equal to `attributes` and `next_hop`, added if it was
/// not there. `next_hop` is the one from MP_REACH_NLRI; routes whose
/// next hop is the NEXT_HOP attribute pass `None`.
///
/// Fails only if the attributes cannot be encoded, which attributes that
/// were decoded from a message always can.
pub fn intern(
&self,
attributes: &[PathAttribute],
next_hop: Option<&NlriNextHop>,
) -> Result<Arc<AttrSet>, EncodeError> {
let mut wire = BytesMut::with_capacity(256);
encode(attributes, next_hop, &mut wire)?;
Ok(self
.intern_wire(&wire)
.expect("attributes written here are well formed"))
}
/// As [`AttrStore::intern`], for attributes that are already in the
/// form a set keeps: on the wire, with four-octet AS numbers, no
/// MP_UNREACH_NLRI, and an MP_REACH_NLRI (if any) that holds only the
/// next hop. That is how a table dump has them (RFC 6396 §4.3.4).
///
/// Nothing is parsed: finding a set that is already stored costs a hash
/// of the bytes and a comparison. `None` if the bytes are not a
/// well-formed list of attributes.
pub fn intern_wire(&self, wire: &[u8]) -> Option<Arc<AttrSet>> {
let hash = self.hasher.hash_one(wire);
let mut stripe = self.stripe(hash);
if let Some(existing) = stripe.find(hash, |set| *set.wire == *wire) {
return Some(Arc::clone(existing));
}
let set = Arc::new(AttrSet {
hash,
key: scan(wire)?,
wire: wire.into(),
});
stripe.insert_unique(hash, Arc::clone(&set), |set| set.hash);
Some(set)
}
/// The set for attributes as a session received them: `wire` is what
/// the decoder kept of the UPDATE's attributes
/// (`DecodedUpdate::attributes_wire`), and `next_hop` the next hop of
/// the routes if they came in MP_REACH_NLRI.
pub fn intern_received(
&self,
wire: &[u8],
next_hop: Option<&NlriNextHop>,
) -> Option<Arc<AttrSet>> {
let Some(next_hop) = next_hop else {
return self.intern_wire(wire);
};
let mut with_next_hop = BytesMut::with_capacity(wire.len() + 36);
with_next_hop.put_slice(wire);
encode_next_hop(next_hop, &mut with_next_hop);
self.intern_wire(&with_next_hop)
}
/// How many distinct sets are stored.
pub fn len(&self) -> usize {
(0..STRIPES).map(|i| self.stripe_at(i).len()).sum()
}
pub fn is_empty(&self) -> bool {
self.len() == 0
}
/// Drops the sets that no route refers to any more, and returns how
/// many that was. Sets are not dropped as their last route goes, since
/// that would mean a lock here for every route removed; this is called
/// now and then instead.
pub fn sweep(&self) -> usize {
let mut dropped = 0;
for i in 0..STRIPES {
let mut stripe = self.stripe_at(i);
let before = stripe.len();
// The store's own reference is the only one left. Nobody can
// be taking another: that is done under this lock.
stripe.retain(|set| Arc::strong_count(set) > 1);
dropped += before - stripe.len();
}
dropped
}
fn stripe(&self, hash: u64) -> MutexGuard<'_, HashTable<Arc<AttrSet>>> {
self.stripe_at(super::spread(hash, STRIPES))
}
fn stripe_at(&self, index: usize) -> MutexGuard<'_, HashTable<Arc<AttrSet>>> {
// A panic while holding the lock cannot leave the table half
// changed in a way that matters: see `IgnorePoison` in the server.
self.stripes[index]
.lock()
.unwrap_or_else(PoisonError::into_inner)
}
}
#[cfg(test)]
mod tests {
use std::net::Ipv4Addr;
use bgp_packet::message::{
AsPathAttribute, AsPathSegment, CommunitiesPathAttribute, LocalPrefPathAttribute,
MultiExitDiscPathAttribute, NextHopPathAttribute, OriginPathAttribute,
};
use super::*;
fn attributes(path: Vec<u32>) -> Vec<PathAttribute> {
vec![
PathAttribute::Origin(OriginPathAttribute::EGP),
AsPathAttribute::from_asns(path),
PathAttribute::NextHop(NextHopPathAttribute(Ipv4Addr::new(192, 0, 2, 1))),
PathAttribute::MultiExitDisc(MultiExitDiscPathAttribute(50)),
PathAttribute::Communities(CommunitiesPathAttribute(vec![(65000, 1)])),
]
}
#[test]
fn equal_attributes_are_one_set() {
let store = AttrStore::new();
let a = store.intern(&attributes(vec![65001, 65002]), None).unwrap();
let b = store.intern(&attributes(vec![65001, 65002]), None).unwrap();
let c = store.intern(&attributes(vec![65001, 65003]), None).unwrap();
assert!(Arc::ptr_eq(&a, &b));
assert!(!Arc::ptr_eq(&a, &c));
assert_eq!(store.len(), 2);
// The next hop is part of what a set is.
let next_hop = NlriNextHop::Ipv6("2001:db8::1".parse().unwrap());
let d = store
.intern(&attributes(vec![65001, 65002]), Some(&next_hop))
.unwrap();
assert!(!Arc::ptr_eq(&a, &d));
assert_eq!(store.len(), 3);
}
#[test]
fn a_set_gives_back_what_went_in() {
let store = AttrStore::new();
let next_hop = NlriNextHop::Ipv6WithLl {
global: "2001:db8::1".parse().unwrap(),
link_local: "fe80::1".parse().unwrap(),
};
let original = attributes(vec![65001, 4_200_000_001]);
let set = store.intern(&original, Some(&next_hop)).unwrap();
assert_eq!(set.attributes(), Some((original, Some(next_hop))));
}
#[test]
fn the_selection_key_is_read_off_the_attributes() {
let store = AttrStore::new();
let mut attrs = vec![
PathAttribute::Origin(OriginPathAttribute::INCOMPLETE),
PathAttribute::AsPath(AsPathAttribute {
segments: vec![
AsPathSegment {
ordered: true,
path: vec![65001, 65002],
},
// A set counts as one hop.
AsPathSegment {
ordered: false,
path: vec![65003, 65004, 65005],
},
],
}),
PathAttribute::LocalPref(LocalPrefPathAttribute(200)),
];
let key = *store.intern(&attrs, None).unwrap().key();
assert_eq!(
key,
SelectionKey {
local_pref: Some(200),
med: None,
neighbor_as: 65001,
as_path_len: 3,
origin: 2,
}
);
attrs.push(PathAttribute::MultiExitDisc(MultiExitDiscPathAttribute(7)));
assert_eq!(store.intern(&attrs, None).unwrap().key().med, Some(7));
}
#[test]
fn attributes_on_the_wire_are_interned_without_parsing_them() {
let store = AttrStore::new();
let next_hop = NlriNextHop::Ipv6("2001:db8::1".parse().unwrap());
let mut attrs = attributes(vec![65001, 65002, 65003]);
attrs.push(PathAttribute::LocalPref(LocalPrefPathAttribute(300)));
let parsed = store.intern(&attrs, Some(&next_hop)).unwrap();
// The same set by its bytes is the same set, key and all.
let by_wire = store.intern_wire(parsed.wire()).unwrap();
assert!(Arc::ptr_eq(&parsed, &by_wire));
// And one that arrives as bytes first gets the same key as it would
// have had from the parsed form.
let other = AttrStore::new();
let fresh = other.intern_wire(parsed.wire()).unwrap();
assert_eq!(fresh.key(), parsed.key());
assert_eq!(fresh.key().as_path_len, 3);
assert_eq!(fresh.key().neighbor_as, 65001);
assert_eq!(fresh.attributes(), Some((attrs, Some(next_hop))));
// Bytes that are not attributes are refused.
assert!(other.intern_wire(&[0x40, 0x01, 0x05, 0x00]).is_none());
assert!(
other
.intern_wire(&[0x40, 0x02, 0x03, 0x02, 0x01, 0x00])
.is_none()
);
}
#[test]
fn sweeping_drops_only_sets_nobody_holds() {
let store = AttrStore::new();
let kept = store.intern(&attributes(vec![65001]), None).unwrap();
drop(store.intern(&attributes(vec![65002]), None).unwrap());
assert_eq!(store.len(), 2);
assert_eq!(store.sweep(), 1);
assert_eq!(store.len(), 1);
// The one that is left is still the one that is found.
let again = store.intern(&attributes(vec![65001]), None).unwrap();
assert!(Arc::ptr_eq(&kept, &again));
}
}
crates/rib/src/table/journal.rs
+755
View File
@@ -0,0 +1,755 @@
//! The journal: how the rest of the router follows the best routes.
//!
//! Whatever acts on the table's choices (the peers that are sent the best
//! routes, the forwarding table, the status page) has to learn when a best
//! route changes, at its own pace, without the table waiting for it. Each
//! shard keeps a journal for that, and each follower reads it through a
//! [`Reader`].
//!
//! # What the journal holds
//!
//! It records *that* a prefix's best route changed, not what it changed to.
//! A reader that comes to an entry looks the prefix up and is given the best
//! route as it is at that moment. A follower does not want the history; it
//! wants to end up with what the table has now.
//!
//! This is what keeps the journal small. If a prefix changes again before a
//! reader has got to its entry, the older entry no longer says anything: the
//! newer one will deliver the current route. Each row remembers the
//! sequence number of its latest change, an entry with any other number is
//! dead, and dead entries are thrown out when the journal is tidied. So a
//! prefix that flaps a thousand times costs a reader one event, and the
//! journal never has more live entries than the table has prefixes.
//!
//! # Readers that fall behind
//!
//! Entries are dropped once every reader has passed them. A reader that
//! stops reading would hold them all back, so there is a limit: when the
//! journal is too long even after tidying, its oldest half is dropped
//! regardless. A reader that was behind that point is told so the next time
//! it reads ([`Event::Reset`]) and is given the whole table again. Reading
//! too slowly costs the slow reader a resynchronisation; it costs the
//! table, and the other readers, nothing.
//!
//! A new reader starts the same way: `Reset`, every best route, `Synced`,
//! and changes from then on.
//!
//! # All routes, or only the best
//!
//! There are two journals in each shard. One records changes of best route,
//! for followers that want one route for each prefix: the forwarding table,
//! and peers that are sent the best route. The other records every change
//! to any route, for followers that want them all: peers that are sent
//! several routes for a prefix (ADD-PATH, RFC 7911), or a choice made for
//! them alone. A [`Reader`] follows one or the other. The second journal
//! costs nothing while nobody follows it.
//!
//! # Order
//!
//! For any one prefix, a reader is given its states in the order they came
//! about, and always, in the end, the latest. Between prefixes there is no
//! order worth relying on.
use std::collections::VecDeque;
use std::hash::BuildHasher;
use std::sync::Arc;
use std::sync::atomic::{AtomicU64, Ordering};
use bgp_primitives::ip_prefix::IpPrefix;
use foldhash::fast::RandomState;
use hashbrown::{HashMap, HashTable};
use super::{AttrSet, PeerIndex, RibTable, RouteRef, Row, Shard, read, write};
/// A cursor that holds nothing back: a reader slot that is free, or a
/// reader that has not read this shard yet.
const NO_CURSOR: u64 = u64::MAX;
/// When a journal is tidied, and when it is too long.
#[derive(Debug, Clone, Copy)]
pub(super) struct Limits {
/// Below this many entries a journal is not worth tidying.
pub(super) tidy_at: usize,
/// Room allowed beyond the number of prefixes before the journal is
/// thought too long.
pub(super) slack: usize,
}
impl Default for Limits {
fn default() -> Self {
Self {
tidy_at: 1024,
slack: 64,
}
}
}
struct Entry {
seq: u64,
prefix: IpPrefix,
}
/// One journal of one shard.
struct Journal {
kind: Kind,
/// The sequence number of the latest change. Counts every change this
/// journal is for, whether or not anyone was reading.
seq: u64,
/// Entries up to and including this number were dropped without regard
/// to the readers. A reader whose cursor is below it missed something.
floor: u64,
/// In order of sequence number. Not every number is here: dead entries
/// are removed, and nothing is recorded while there are no readers.
entries: VecDeque<Entry>,
/// Prefixes that have no route any more and whose disappearance is
/// still to be read, with the sequence number of the entry that says
/// so. A row keeps this number itself; a prefix without a row cannot.
gone: HashMap<IpPrefix, u64, RandomState>,
/// Where each reader has read to, by reader slot. Written by the
/// readers themselves while they hold the shard for reading.
cursors: Vec<AtomicU64>,
/// How many reader slots are in use.
readers: usize,
limits: Limits,
}
impl Journal {
fn new(kind: Kind, limits: Limits) -> Self {
Self {
kind,
limits,
seq: 0,
floor: 0,
entries: VecDeque::new(),
gone: HashMap::default(),
cursors: Vec::new(),
readers: 0,
}
}
/// Notes a change to `prefix`, which has a row. Returns the sequence
/// number for the row to remember.
fn changed(&mut self, prefix: IpPrefix) -> u64 {
self.seq += 1;
if self.readers > 0 {
self.entries.push_back(Entry {
seq: self.seq,
prefix,
});
}
self.seq
}
/// As [`Journal::changed`], for a prefix whose row was just made:
/// if it had gone and is back, the note that it had gone is void.
fn appeared(&mut self, prefix: IpPrefix) -> u64 {
if !self.gone.is_empty() {
self.gone.remove(&prefix);
}
self.changed(prefix)
}
/// Notes that `prefix` lost its last route and has no row any more.
fn disappeared(&mut self, prefix: IpPrefix) {
self.seq += 1;
if self.readers > 0 {
self.entries.push_back(Entry {
seq: self.seq,
prefix,
});
self.gone.insert(prefix, self.seq);
}
}
/// Forgets that `entry`'s prefix had gone, if `entry` is what said so.
fn drop_entry(gone: &mut HashMap<IpPrefix, u64, RandomState>, entry: &Entry) {
if gone.get(&entry.prefix) == Some(&entry.seq) {
gone.remove(&entry.prefix);
}
}
/// What `entry` says, if it still says something: it is the latest
/// change of its prefix. The row, or `None` for a prefix that has gone.
fn live<'a>(
&self,
entry: &Entry,
rows: &'a HashTable<Row>,
hasher: &RandomState,
) -> Option<Option<&'a Row>> {
let hash = hasher.hash_one(entry.prefix);
match rows.find(hash, |row| row.prefix == entry.prefix) {
Some(row) => (row.seqs.of(self.kind) == entry.seq).then_some(Some(row)),
None => (self.gone.get(&entry.prefix) == Some(&entry.seq)).then_some(None),
}
}
/// Keeps the journal from growing. Called after every change to the
/// shard; does nothing until there is something worth doing.
fn tidy(&mut self, rows: &HashTable<Row>, hasher: &RandomState) {
if self.entries.len() <= self.limits.tidy_at {
return;
}
// What every reader has passed can go.
let passed = self
.cursors
.iter()
.map(|cursor| cursor.load(Ordering::Acquire))
.min()
.unwrap_or(NO_CURSOR);
while let Some(front) = self.entries.front() {
if front.seq > passed {
break;
}
Self::drop_entry(&mut self.gone, front);
self.entries.pop_front();
}
// Entries for prefixes that have a row are at most as many as the
// rows once the dead ones are out. Entries for prefixes that have
// gone are not counted in: they are what a reader that has stopped
// reading makes pile up.
let limit = 2 * (rows.len() + self.limits.slack);
if self.entries.len() <= limit {
return;
}
// Dead entries: their prefix has changed again since. This looks
// every entry up, but only once the journal has doubled, so the
// cost per change stays small.
let mut entries = std::mem::take(&mut self.entries);
entries.retain(|entry| self.live(entry, rows, hasher).is_some());
self.entries = entries;
if self.entries.len() <= limit {
return;
}
// Still too long: a reader is not reading while prefixes come and
// go, or more prefixes went at once than are left. The older half
// goes, and a reader that had not read it loses its place. Half,
// so that readers that are only a little behind keep theirs, and
// so that getting a long journal down to size takes few passes.
for entry in self.entries.drain(..self.entries.len() / 2) {
Self::drop_entry(&mut self.gone, &entry);
self.floor = entry.seq;
}
}
}
/// Which of a shard's two journals.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub(super) enum Kind {
/// Changes of best route.
Best,
/// Changes of any route.
All,
}
/// The sequence numbers a row remembers, one for each journal: of the
/// latest change of its best route, and of any of its routes.
#[derive(Debug, Clone, Copy)]
pub(super) struct Seqs {
best: u64,
all: u64,
}
impl Seqs {
fn of(self, kind: Kind) -> u64 {
match kind {
Kind::Best => self.best,
Kind::All => self.all,
}
}
}
/// The two journals of a shard, written together: every change to a row is
/// a change of one of its routes, and some are a change of its best route.
pub(super) struct Journals {
best: Journal,
all: Journal,
}
impl Journals {
pub(super) fn new(limits: Limits) -> Self {
Self {
best: Journal::new(Kind::Best, limits),
all: Journal::new(Kind::All, limits),
}
}
fn of(&self, kind: Kind) -> &Journal {
match kind {
Kind::Best => &self.best,
Kind::All => &self.all,
}
}
fn of_mut(&mut self, kind: Kind) -> &mut Journal {
match kind {
Kind::Best => &mut self.best,
Kind::All => &mut self.all,
}
}
/// Notes that `prefix` has its first route, and so a row. Returns the
/// sequence numbers for the new row.
pub(super) fn appeared(&mut self, prefix: IpPrefix) -> Seqs {
Seqs {
best: self.best.appeared(prefix),
all: self.all.appeared(prefix),
}
}
/// Notes that a route of `row` changed, and whether its best route did.
pub(super) fn changed(&mut self, row: &mut Row, best_changed: bool) {
if best_changed {
row.seqs.best = self.best.changed(row.prefix);
}
row.seqs.all = self.all.changed(row.prefix);
}
/// Notes that `prefix` lost its last route and has no row any more.
pub(super) fn disappeared(&mut self, prefix: IpPrefix) {
self.best.disappeared(prefix);
self.all.disappeared(prefix);
}
/// Keeps the journals from growing; see [`Journal::tidy`].
pub(super) fn tidy(&mut self, rows: &HashTable<Row>, hasher: &RandomState) {
self.best.tidy(rows, hasher);
self.all.tidy(rows, hasher);
}
/// Where the journals are now, to ask [`Journals::has_news_since`]
/// about later.
pub(super) fn mark(&self) -> Mark {
Mark(self.best.seq, self.all.seq)
}
/// Whether anything was written since `mark` that some reader is there
/// to read.
pub(super) fn has_news_since(&self, mark: Mark) -> bool {
(self.best.readers > 0 && self.best.seq != mark.0)
|| (self.all.readers > 0 && self.all.seq != mark.1)
}
fn len(&self) -> usize {
self.best.entries.len() + self.all.entries.len()
}
}
/// A point in a shard's journals; see [`Journals::mark`].
#[derive(Clone, Copy)]
pub(super) struct Mark(u64, u64);
/// The routes of one prefix, as a follower of all routes is given them.
#[derive(Clone, Copy)]
pub struct Routes<'a>(&'a Row);
impl<'a> Routes<'a> {
pub fn iter(&self) -> impl Iterator<Item = RouteRef<'a>> + use<'a> {
self.0.routes()
}
/// How many there are: at least one.
pub fn len(&self) -> usize {
self.0.paths.len()
}
pub fn is_empty(&self) -> bool {
self.len() == 0
}
}
impl std::fmt::Debug for Routes<'_> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_list().entries(self.iter()).finish()
}
}
/// What a [`Reader`] is given.
#[derive(Debug, Clone, Copy)]
pub enum Event<'a> {
/// Forget what you know. The whole table follows, as [`Event::Best`]
/// or [`Event::Routes`] for every prefix, and then [`Event::Synced`].
/// The first thing a new
/// reader is given, and what a reader that fell too far behind is
/// given in place of what it missed.
Reset,
/// This is now the best route for `prefix`. Given to a follower of the
/// best routes.
Best {
prefix: IpPrefix,
peer: PeerIndex,
/// The peer's identifier for the route: zero unless the peer
/// sends several routes for a prefix (ADD-PATH, RFC 7911).
path_id: u32,
attrs: &'a Arc<AttrSet>,
},
/// These are now all the routes for `prefix`: whatever was known of its
/// routes before is replaced. Given to a follower of all routes.
Routes {
prefix: IpPrefix,
routes: Routes<'a>,
},
/// `prefix` has no route any more.
Withdrawn { prefix: IpPrefix },
/// The table that began with [`Event::Reset`] is complete: a prefix
/// not heard of since the reset has no route.
Synced,
}
/// What a call to [`Reader::poll`] got done.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Progress {
/// How many events were given out.
pub events: usize,
/// Whether the reader has now been given everything there was when the
/// call began. If not, call again.
pub caught_up: bool,
}
enum Phase {
/// Giving out the whole table; `next` is the shard to do next, and
/// `announced` whether [`Event::Reset`] has been given.
Resync { next: usize, announced: bool },
/// Following the journals; `next` is the shard to read next.
Follow { next: usize },
}
impl Phase {
/// The beginning of a resynchronisation.
const RESYNC: Self = Self::Resync {
next: 0,
announced: false,
};
}
/// Follows the routes of a table: the best of each prefix, or all of them.
/// Made by [`RibTable::reader`] and its siblings.
///
/// A reader holds its place in every shard's journal. Dropping it gives
/// the place up, so that the journals need not keep entries for it.
pub struct Reader {
table: Arc<RibTable>,
/// Which journal it follows.
kind: Kind,
slot: usize,
/// Where this reader has read to in each shard; the copy in the shard
/// is for the shard's benefit.
cursors: Vec<u64>,
phase: Phase,
/// The table's version when the reader was last found caught up.
seen: u64,
caught_up: bool,
}
impl RibTable {
/// A new follower of this table's best routes. It is given the whole
/// table first and every change after.
pub fn reader(self: &Arc<Self>) -> Reader {
self.register(Kind::Best, Phase::RESYNC)
}
/// A new follower that is given only the changes from now on, for
/// those who want the news and not the table. If it falls too far
/// behind it is reset like any other, and is then given the table.
pub fn reader_from_now(self: &Arc<Self>) -> Reader {
let mut reader = self.register(Kind::Best, Phase::Follow { next: 0 });
for (index, shard) in self.shards.iter().enumerate() {
let shard = read(shard);
reader.set_cursor(index, &shard, shard.journals.best.seq);
}
reader
}
/// A new follower of every route in the table, not only the best: it
/// is told whenever any route of a prefix changes, with all the
/// prefix's routes ([`Event::Routes`]). The whole table first, like
/// [`RibTable::reader`].
pub fn reader_of_all_routes(self: &Arc<Self>) -> Reader {
self.register(Kind::All, Phase::RESYNC)
}
fn register(self: &Arc<Self>, kind: Kind, phase: Phase) -> Reader {
let mut slots = self
.reader_slots
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
let slot = match slots.iter().position(|used| !used) {
Some(free) => free,
None => {
slots.push(false);
slots.len() - 1
}
};
slots[slot] = true;
for shard in &self.shards {
let mut shard = write(shard);
let journal = shard.journals.of_mut(kind);
while journal.cursors.len() <= slot {
journal.cursors.push(AtomicU64::new(NO_CURSOR));
}
journal.cursors[slot].store(NO_CURSOR, Ordering::Release);
journal.readers += 1;
}
self.readers.fetch_add(1, Ordering::AcqRel);
Reader {
table: Arc::clone(self),
kind,
slot,
cursors: vec![NO_CURSOR; self.shards.len()],
phase,
seen: 0,
caught_up: false,
}
}
/// How many entries the journals hold, over all shards. For tests and
/// for watching: it stays bounded whatever the readers do.
pub fn journal_len(&self) -> usize {
self.shards
.iter()
.map(|shard| read(shard).journals.len())
.sum()
}
/// Tells readers that are waiting that there is something to read.
pub(super) fn wake_readers(&self) {
if self.readers.load(Ordering::Acquire) > 0 {
self.version.fetch_add(1, Ordering::AcqRel);
self.wake.notify_waiters();
}
}
}
impl Drop for Reader {
fn drop(&mut self) {
for shard in &self.table.shards {
let mut shard = write(shard);
let journal = shard.journals.of_mut(self.kind);
journal.cursors[self.slot].store(NO_CURSOR, Ordering::Release);
journal.readers -= 1;
if journal.readers == 0 {
// Nobody left to read it.
journal.entries.clear();
journal.gone.clear();
}
}
self.table.readers.fetch_sub(1, Ordering::AcqRel);
let mut slots = self
.table
.reader_slots
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
slots[self.slot] = false;
}
}
enum Drained {
/// Entries were dropped that this reader had not read.
Lagged,
/// `events` were given out; `more` if the budget ran out first.
Read { events: usize, more: bool },
}
impl Reader {
/// Gives `visit` what has changed since the last call, in up to about
/// `budget` events (a shard's part of a resynchronisation is not cut
/// short, so it can be more).
///
/// `visit` is called with a shard of the table held for reading. It
/// should be quick, and it must not call into the table, even to read:
/// taking the shard's lock again can wait for ever on a writer that is
/// waiting for this hold to end.
pub fn poll(&mut self, budget: usize, mut visit: impl FnMut(Event<'_>)) -> Progress {
// A call that may give nothing out would never get anywhere.
let budget = budget.max(1);
let shards = self.table.shards.len();
let mut events = 0;
loop {
match self.phase {
Phase::Resync { next, announced } => {
if !announced {
self.forget_place();
visit(Event::Reset);
events += 1;
}
if next == shards {
visit(Event::Synced);
events += 1;
self.phase = Phase::Follow { next: 0 };
continue;
}
events += self.scan(next, &mut visit);
self.phase = Phase::Resync {
next: next + 1,
announced: true,
};
if events >= budget {
return self.progress(events, false);
}
}
Phase::Follow { next } => {
// Read before the pass: a change made during it shows as
// a newer version, and `changed` then returns at once.
let version = if next == 0 {
self.table.version.load(Ordering::Acquire)
} else {
self.seen
};
self.seen = version;
let mut shard = next;
let mut lagged = false;
while shard < shards {
if events >= budget {
self.phase = Phase::Follow { next: shard };
return self.progress(events, false);
}
match self.drain(shard, budget - events, &mut visit) {
Drained::Lagged => {
lagged = true;
break;
}
Drained::Read { events: read, more } => {
events += read;
if more {
self.phase = Phase::Follow { next: shard };
return self.progress(events, false);
}
}
}
shard += 1;
}
if lagged {
self.phase = Phase::RESYNC;
continue;
}
self.phase = Phase::Follow { next: 0 };
if next != 0 {
// This pass took up where an earlier call stopped.
// The shards before that point have not been looked
// at in this call, so go round again from the first.
continue;
}
return self.progress(events, true);
}
}
}
}
fn progress(&mut self, events: usize, caught_up: bool) -> Progress {
self.caught_up = caught_up;
Progress { events, caught_up }
}
/// Waits until there may be something to read. Returns at once if the
/// last [`Reader::poll`] did not catch up.
pub async fn changed(&mut self) {
loop {
// Registered before the check, so that a change between the
// check and the wait is not missed.
let woken = self.table.wake.notified();
tokio::pin!(woken);
woken.as_mut().enable();
if !self.caught_up || self.table.version.load(Ordering::Acquire) != self.seen {
return;
}
woken.await;
}
}
/// What this reader is told about a row.
fn event_for<'a>(&self, row: &'a Row) -> Event<'a> {
match self.kind {
Kind::Best => {
let best = row.route(row.best());
Event::Best {
prefix: row.prefix,
peer: best.peer,
path_id: best.path_id,
attrs: best.attrs,
}
}
Kind::All => Event::Routes {
prefix: row.prefix,
routes: Routes(row),
},
}
}
/// Gives up this reader's place in every journal, before a
/// resynchronisation takes a new one.
fn forget_place(&mut self) {
for (index, shard) in self.table.shards.iter().enumerate() {
read(shard).journals.of(self.kind).cursors[self.slot]
.store(NO_CURSOR, Ordering::Release);
self.cursors[index] = NO_CURSOR;
}
}
fn set_cursor(&mut self, index: usize, shard: &Shard, cursor: u64) {
self.cursors[index] = cursor;
shard.journals.of(self.kind).cursors[self.slot].store(cursor, Ordering::Release);
}
/// Gives out every prefix in a shard and takes a place at the end of
/// its journal, in one hold of the shard: what changes afterwards is in
/// the journal after that place.
fn scan(&mut self, index: usize, visit: &mut impl FnMut(Event<'_>)) -> usize {
let table = Arc::clone(&self.table);
let shard = read(&table.shards[index]);
for row in &shard.rows {
visit(self.event_for(row));
}
self.set_cursor(index, &shard, shard.journals.of(self.kind).seq);
shard.rows.len()
}
/// Reads a shard's journal from this reader's place, for at most `max`
/// events.
fn drain(&mut self, index: usize, max: usize, visit: &mut impl FnMut(Event<'_>)) -> Drained {
let table = Arc::clone(&self.table);
let shard = read(&table.shards[index]);
let journal = shard.journals.of(self.kind);
let mut cursor = self.cursors[index];
if cursor < journal.floor {
return Drained::Lagged;
}
if cursor == journal.seq {
return Drained::Read {
events: 0,
more: false,
};
}
let start = journal.entries.partition_point(|entry| entry.seq <= cursor);
let mut events = 0;
let mut more = false;
for entry in journal.entries.range(start..) {
if events == max {
more = true;
break;
}
cursor = entry.seq;
let event = match journal.live(entry, &shard.rows, &shard.hasher) {
Some(Some(row)) => self.event_for(row),
Some(None) => Event::Withdrawn {
prefix: entry.prefix,
},
// The prefix has changed again; a later entry has it.
None => continue,
};
visit(event);
events += 1;
}
if !more {
// Past the last entry there is nothing up to the latest change.
cursor = journal.seq;
}
self.set_cursor(index, &shard, cursor);
Drained::Read { events, more }
}
}
#[cfg(test)]
#[path = "journal_tests.rs"]
mod tests;
crates/rib/src/table/journal_tests.rs
+713
View File
@@ -0,0 +1,713 @@
use std::collections::HashMap;
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, Ordering};
use bgp_primitives::constants::IpVersion;
use bgp_primitives::ip_prefix::IpPrefix;
use proptest::prelude::*;
use super::super::tests::{Fixture, Route, external, internal, prefix, route};
use super::super::{AttrSet, PeerIndex, PeerInfo, RibTable};
use super::{Event, Limits, Reader};
/// What a follower would keep: its own copy of the best routes, brought
/// up to date from the events it is given.
#[derive(Default)]
struct Mirror {
best: HashMap<IpPrefix, (PeerIndex, Arc<AttrSet>)>,
/// What was held before a reset and has not been heard of since.
stale: HashMap<IpPrefix, (PeerIndex, Arc<AttrSet>)>,
/// For a follower of all routes: every route of every prefix, as
/// (peer, path identifier, attributes, whether best).
routes: HashMap<IpPrefix, Vec<Held>>,
resets: usize,
synced: usize,
best_events: usize,
routes_events: usize,
withdrawn_events: usize,
}
/// A route as a mirror holds it. The attributes are held, so that the
/// address used to compare them stays theirs.
type Held = (PeerIndex, u32, Arc<AttrSet>, bool);
fn same_routes(a: &[Held], b: &[Held]) -> bool {
a.len() == b.len()
&& a.iter()
.zip(b)
.all(|(a, b)| (a.0, a.1, a.3) == (b.0, b.1, b.3) && Arc::ptr_eq(&a.2, &b.2))
}
fn sorted(mut routes: Vec<Held>) -> Vec<Held> {
routes.sort_by_key(|(peer, path_id, _, _)| (*peer, *path_id));
routes
}
impl Mirror {
fn apply(&mut self, event: Event<'_>) {
match event {
Event::Reset => {
self.resets += 1;
self.stale = std::mem::take(&mut self.best);
self.routes.clear();
}
Event::Routes { prefix, routes } => {
self.routes_events += 1;
assert!(!routes.is_empty(), "a prefix with no routes is withdrawn");
self.routes.insert(
prefix,
sorted(
routes
.iter()
.map(|r| (r.peer, r.path_id, Arc::clone(r.attrs), r.best))
.collect(),
),
);
}
Event::Best {
prefix,
peer,
attrs,
..
} => {
self.best_events += 1;
self.stale.remove(&prefix);
self.best.insert(prefix, (peer, Arc::clone(attrs)));
}
Event::Withdrawn { prefix } => {
self.withdrawn_events += 1;
self.best.remove(&prefix);
self.routes.remove(&prefix);
}
Event::Synced => {
self.synced += 1;
self.stale.clear();
}
}
}
/// Reads until there is nothing more, in steps of `budget`.
fn catch_up(&mut self, reader: &mut Reader, budget: usize) {
while !reader.poll(budget, |event| self.apply(event)).caught_up {}
}
#[track_caller]
fn assert_matches(&self, table: &RibTable) {
let mut expected = HashMap::new();
table.for_each_best(|prefix, peer, attrs| {
expected.insert(*prefix, (peer, Arc::clone(attrs)));
});
assert_eq!(
self.best.len(),
expected.len(),
"the mirror has {} prefixes, the table {}",
self.best.len(),
expected.len()
);
for (prefix, (peer, attrs)) in &expected {
let Some((mirrored_peer, mirrored_attrs)) = self.best.get(prefix) else {
panic!("{prefix} is in the table and not in the mirror");
};
assert_eq!(mirrored_peer, peer, "{prefix}: from the wrong peer");
assert!(
Arc::ptr_eq(mirrored_attrs, attrs),
"{prefix}: wrong attributes"
);
}
}
}
impl Mirror {
/// For the mirror of a follower of all routes: it holds every route in
/// the table, and knows which is best.
#[track_caller]
fn assert_has_all_routes_of(&self, table: &RibTable) {
let mut expected: HashMap<IpPrefix, Vec<Held>> = HashMap::new();
table.for_each_route(|r| {
expected.entry(r.prefix).or_default().push((
r.peer,
r.path_id,
Arc::clone(r.attrs),
r.best,
));
});
assert_eq!(self.routes.len(), expected.len(), "prefixes");
for (prefix, routes) in expected {
let mirrored = self
.routes
.get(&prefix)
.unwrap_or_else(|| panic!("{prefix} is in the table and not in the mirror"));
assert!(
same_routes(mirrored, &sorted(routes)),
"{prefix}: the mirror's routes are not the table's"
);
}
}
}
fn nth_prefix(n: u32) -> IpPrefix {
IpPrefix::new(IpVersion::Ipv4, (n << 8).to_be_bytes(), 24).unwrap()
}
#[test]
fn a_new_reader_is_given_the_table_and_then_the_changes() {
let fixture = Fixture::new(&[external(1), external(2)]);
let (a, b) = (prefix("10.0.0.0/8"), prefix("2001:db8::/32"));
fixture.announce(0, &a, &Route::via(&[1, 2]));
fixture.announce(0, &b, &Route::via(&[1, 2]));
let mut reader = fixture.table.reader();
let mut mirror = Mirror::default();
let mut events = Vec::new();
let progress = reader.poll(100, |event| {
events.push(
format!("{event:?}")
.split([' ', '{'])
.next()
.unwrap()
.to_string(),
);
mirror.apply(event);
});
assert!(progress.caught_up);
assert_eq!(events, ["Reset", "Best", "Best", "Synced"]);
mirror.assert_matches(&fixture.table);
// Nothing has happened: nothing to read.
assert_eq!(reader.poll(100, |_| panic!("no event expected")).events, 0);
// A change that does not change the best route is not an event.
fixture.announce(1, &a, &Route::via(&[3, 4, 5]));
assert_eq!(reader.poll(100, |_| panic!("no event expected")).events, 0);
fixture.announce(1, &a, &Route::via(&[3]));
fixture.withdraw(0, &b);
mirror.catch_up(&mut reader, 100);
assert_eq!((mirror.best_events, mirror.withdrawn_events), (3, 1));
assert_eq!((mirror.resets, mirror.synced), (1, 1));
mirror.assert_matches(&fixture.table);
}
#[test]
fn a_reader_from_now_is_given_the_changes_and_not_the_table() {
let fixture = Fixture::new(&[external(1)]);
let (old, new) = (prefix("10.0.0.0/8"), prefix("10.1.0.0/16"));
fixture.announce(0, &old, &Route::via(&[1]));
let mut reader = fixture.table.reader_from_now();
assert_eq!(
reader
.poll(100, |event| panic!("unexpected {event:?}"))
.events,
0
);
fixture.announce(0, &new, &Route::via(&[1]));
fixture.withdraw(0, &old);
let (mut best, mut withdrawn) = (Vec::new(), Vec::new());
reader.poll(100, |event| match event {
Event::Best { prefix, .. } => best.push(prefix),
Event::Withdrawn { prefix } => withdrawn.push(prefix),
other => panic!("unexpected {other:?}"),
});
assert_eq!((best, withdrawn), (vec![new], vec![old]));
}
/// A change to a route that is not the best is nothing to a follower of
/// the best routes, and is what a follower of all routes is there for.
#[test]
fn a_follower_of_all_routes_is_told_of_every_change() {
let fixture = Fixture::new(&[external(1), external(2)]);
let p = prefix("10.0.0.0/8");
fixture.announce(0, &p, &Route::via(&[1]));
let mut best = fixture.table.reader();
let mut all = fixture.table.reader_of_all_routes();
let (mut best_mirror, mut all_mirror) = (Mirror::default(), Mirror::default());
best_mirror.catch_up(&mut best, 100);
all_mirror.catch_up(&mut all, 100);
assert_eq!((best_mirror.best_events, all_mirror.routes_events), (1, 1));
// A worse route from another peer, then two more from that peer under
// other path identifiers, one of them changed, one withdrawn.
fixture.announce(1, &p, &Route::via(&[2, 3, 4]));
fixture.announce_path(1, &p, 7, &Route::via(&[2, 3, 5]));
fixture.announce_path(1, &p, 8, &Route::via(&[2, 3, 6]));
fixture.announce_path(1, &p, 7, &Route::via(&[2, 9, 5]));
fixture.withdraw_path(1, &p, 8);
assert_eq!(
best.poll(100, |event| panic!("unexpected {event:?}"))
.events,
0
);
all_mirror.catch_up(&mut all, 100);
// All of it between two reads is one event, with the routes as they
// are now.
assert_eq!(all_mirror.routes_events, 2);
assert_eq!(all_mirror.routes[&p].len(), 3);
all_mirror.assert_has_all_routes_of(&fixture.table);
// The last route going is a withdrawal for both.
fixture.table.remove_peer(fixture.peers[1]);
fixture.withdraw(0, &p);
best_mirror.catch_up(&mut best, 100);
all_mirror.catch_up(&mut all, 100);
assert_eq!(
(best_mirror.withdrawn_events, all_mirror.withdrawn_events),
(1, 1)
);
all_mirror.assert_has_all_routes_of(&fixture.table);
// Nobody following all routes: that journal keeps nothing. A change
// that is not of the best route is then recorded nowhere.
drop(all);
fixture.announce(0, &p, &Route::via(&[1]));
fixture.announce(1, &p, &Route::via(&[2, 3]));
let before = fixture.table.journal_len();
fixture.announce(1, &p, &Route::via(&[2, 4]));
fixture.announce_path(1, &p, 7, &Route::via(&[2, 5]));
assert_eq!(fixture.table.journal_len(), before);
}
/// A follower wants what the table has now, not how it got there.
#[test]
fn many_changes_to_a_prefix_between_reads_are_one_event() {
let fixture = Fixture::new(&[external(1), external(2)]);
let p = prefix("10.0.0.0/8");
let mut reader = fixture.table.reader();
let mut mirror = Mirror::default();
mirror.catch_up(&mut reader, 100);
for round in 0..50 {
fixture.announce(0, &p, &Route::via(&[1, 2, 3]));
fixture.announce(1, &p, &Route::via(&[4, 5]));
fixture.withdraw(1, &p);
if round % 2 == 0 {
fixture.withdraw(0, &p);
}
}
// It ended with peer 0's route in place.
mirror.catch_up(&mut reader, 100);
assert_eq!((mirror.best_events, mirror.withdrawn_events), (1, 0));
mirror.assert_matches(&fixture.table);
// And a prefix that came and went again is, at most, a withdrawal.
let q = prefix("10.1.0.0/16");
for _ in 0..50 {
fixture.announce(0, &q, &Route::via(&[1]));
fixture.withdraw(0, &q);
}
mirror.catch_up(&mut reader, 100);
assert_eq!((mirror.best_events, mirror.withdrawn_events), (1, 1));
mirror.assert_matches(&fixture.table);
}
#[test]
fn reading_in_small_steps_gives_the_same_as_reading_at_once() {
let fixture = Fixture::new(&[external(1)]);
for n in 0..500 {
fixture.announce(0, &nth_prefix(n), &Route::via(&[1, n % 7 + 1]));
}
let mut reader = fixture.table.reader();
let mut mirror = Mirror::default();
mirror.catch_up(&mut reader, 7);
mirror.assert_matches(&fixture.table);
for n in 0..500 {
if n % 3 == 0 {
fixture.withdraw(0, &nth_prefix(n));
} else {
fixture.announce(0, &nth_prefix(n), &Route::via(&[2, n % 5 + 1]));
}
}
let mut steps = 0;
while !reader.poll(7, |event| mirror.apply(event)).caught_up {
steps += 1;
}
assert!(steps > 50, "the budget was kept to: {steps} steps");
mirror.assert_matches(&fixture.table);
assert_eq!(mirror.resets, 1);
}
#[test]
fn with_no_reader_nothing_is_kept() {
let fixture = Fixture::new(&[external(1)]);
for n in 0..5000 {
fixture.announce(0, &nth_prefix(n), &Route::via(&[1]));
fixture.withdraw(0, &nth_prefix(n));
}
assert_eq!(fixture.table.journal_len(), 0);
// And what a reader was holding back goes when the reader does.
let reader = fixture.table.reader();
for n in 0..100 {
fixture.announce(0, &nth_prefix(n), &Route::via(&[1]));
}
assert_eq!(fixture.table.journal_len(), 100);
drop(reader);
assert_eq!(fixture.table.journal_len(), 0);
}
/// A reader that stops reading must not make the table keep a growing
/// journal for it. It is the one that pays: it is given the table again.
#[test]
fn a_reader_that_stops_reading_is_reset_and_holds_nothing_up() {
// One shard, so that the journal's length is that shard's.
let fixture = Fixture::on(RibTable::new(1), &[external(1)]);
fixture.announce(0, &prefix("192.0.2.0/24"), &Route::via(&[1]));
let mut stuck = fixture.table.reader();
let mut stuck_mirror = Mirror::default();
stuck_mirror.catch_up(&mut stuck, 100);
let mut diligent = fixture.table.reader();
let mut diligent_mirror = Mirror::default();
// Prefixes come and go, each a different one, so no entry is made
// dead by a later one for the same prefix.
let limits = Limits::default();
let mut longest = 0;
for n in 0..20_000 {
fixture.announce(0, &nth_prefix(n), &Route::via(&[1]));
fixture.withdraw(0, &nth_prefix(n));
longest = longest.max(fixture.table.journal_len());
if n % 100 == 0 {
diligent_mirror.catch_up(&mut diligent, 1000);
}
}
assert!(
longest <= limits.tidy_at.max(2 * (1 + limits.slack)) + 2,
"the journal reached {longest} entries"
);
// The one that kept up never noticed.
diligent_mirror.catch_up(&mut diligent, 1000);
assert_eq!(
diligent_mirror.resets, 1,
"only the reset every reader starts with"
);
diligent_mirror.assert_matches(&fixture.table);
// The one that did not is told, and ends up right all the same.
stuck_mirror.catch_up(&mut stuck, 1000);
assert_eq!(stuck_mirror.resets, 2);
assert_eq!(stuck_mirror.synced, 2);
stuck_mirror.assert_matches(&fixture.table);
}
/// When a peer that was the only source of most prefixes goes, more
/// prefixes disappear at once than are left. The journal still comes down
/// to its bound, and a reader ends up with what the table has.
#[test]
fn many_prefixes_going_at_once_leaves_the_journal_bounded() {
let fixture = Fixture::on(RibTable::new(1), &[external(1), external(2)]);
for n in 0..5000 {
fixture.announce(0, &nth_prefix(n), &Route::via(&[1]));
}
fixture.announce(1, &nth_prefix(0), &Route::via(&[2, 3]));
let mut reader = fixture.table.reader();
let mut mirror = Mirror::default();
mirror.catch_up(&mut reader, 1000);
fixture.table.remove_peer(fixture.peers[0]);
// Some later change, as there always is, lets the journal be tidied.
for _ in 0..8 {
fixture.announce(1, &nth_prefix(1), &Route::via(&[2]));
fixture.withdraw(1, &nth_prefix(1));
}
let limits = Limits::default();
assert!(
fixture.table.journal_len() <= limits.tidy_at.max(2 * (1 + limits.slack)) + 2,
"the journal holds {} entries",
fixture.table.journal_len()
);
mirror.catch_up(&mut reader, 1000);
mirror.assert_matches(&fixture.table);
assert_eq!(mirror.best.len(), 1);
}
/// Readers are woken to read. An announcement that changes nothing, or
/// nothing a reader follows, writes nothing for them and must not wake
/// them: each would look through every shard to find that out.
#[test]
fn readers_are_not_woken_when_nothing_was_written_for_them() {
let fixture = Fixture::new(&[external(1), external(2)]);
let p = prefix("10.0.0.0/8");
let version = || fixture.table.version.load(Ordering::Acquire);
// Nobody reading: nobody to wake.
fixture.announce(0, &p, &Route::via(&[1]));
assert_eq!(version(), 0);
let _best = fixture.table.reader();
let before = version();
// The same route again.
fixture.announce(0, &p, &Route::via(&[1]));
assert_eq!(version(), before);
// A worse route from another peer: the best route stands.
fixture.announce(1, &p, &Route::via(&[2, 3, 4]));
assert_eq!(version(), before);
// A better one.
fixture.announce(1, &p, &Route::via(&[]));
assert_eq!(version(), before + 1);
// A follower of all routes is told of any change to a route.
let _all = fixture.table.reader_of_all_routes();
let before = version();
fixture.announce(0, &p, &Route::via(&[1, 1, 1]));
assert_eq!(version(), before + 1);
fixture.announce(0, &p, &Route::via(&[1, 1, 1]));
assert_eq!(version(), before + 1);
}
/// A budget of nothing would let a reader ask for ever without getting
/// anywhere; it is taken as the smallest that does.
#[test]
fn a_budget_of_nothing_still_makes_progress() {
let fixture = Fixture::new(&[external(1)]);
for n in 0..20 {
fixture.announce(0, &nth_prefix(n), &Route::via(&[1]));
}
let mut reader = fixture.table.reader();
let mut mirror = Mirror::default();
mirror.catch_up(&mut reader, 0);
for n in 0..20 {
fixture.withdraw(0, &nth_prefix(n));
}
mirror.catch_up(&mut reader, 0);
mirror.assert_matches(&fixture.table);
assert!(mirror.best.is_empty());
}
#[tokio::test]
async fn a_waiting_reader_is_woken_by_a_change_of_best_route() {
let fixture = Fixture::new(&[external(1), external(2)]);
let p = prefix("10.0.0.0/8");
let mut reader = fixture.table.reader();
let mut mirror = Mirror::default();
// Not caught up yet: no waiting.
reader.changed().await;
mirror.catch_up(&mut reader, 100);
let waiting = tokio::time::timeout(std::time::Duration::from_millis(50), reader.changed());
assert!(waiting.await.is_err(), "nothing changed, so it waits");
let table = Arc::clone(&fixture.table);
let attrs = table
.attrs()
.intern(&Route::via(&[1]).attributes(), None)
.unwrap();
let peer = fixture.peers[0];
let writer = tokio::spawn(async move {
tokio::time::sleep(std::time::Duration::from_millis(20)).await;
table.announce(peer, &attrs, &[p]);
});
tokio::time::timeout(std::time::Duration::from_secs(5), reader.changed())
.await
.expect("woken by the announcement");
writer.await.unwrap();
mirror.catch_up(&mut reader, 100);
mirror.assert_matches(&fixture.table);
// A change made before the wait began is not missed either.
fixture.withdraw(0, &p);
tokio::time::timeout(std::time::Duration::from_secs(5), reader.changed())
.await
.expect("the change was already there");
}
/// Writers on several threads, a reader on another, reading while they
/// write. When the writers are done and the reader has caught up, it has
/// what the table has.
#[test]
fn a_reader_keeps_up_with_writers_on_other_threads() {
const WRITERS: usize = 4;
const PREFIXES: u32 = 300;
let infos: Vec<PeerInfo> = (0..WRITERS).map(|i| external(i as u8 + 1)).collect();
// Small journals, so that tidying and falling behind happen too.
let fixture = Fixture::on(
RibTable::with_journal_limits(
8,
Limits {
tidy_at: 16,
slack: 4,
},
),
&infos,
);
let table = &fixture.table;
let done = AtomicBool::new(false);
let mut mirror = Mirror::default();
std::thread::scope(|scope| {
let writers: Vec<_> = (0..WRITERS)
.map(|writer| {
let peer = fixture.peers[writer];
scope.spawn(move || {
// A simple generator; each writer its own sequence.
let mut state = 0x9e37_79b9_7f4a_7c15u64.wrapping_mul(writer as u64 + 1);
let mut next = || {
state ^= state << 13;
state ^= state >> 7;
state ^= state << 17;
state
};
for _ in 0..30_000 {
let p = nth_prefix((next() % u64::from(PREFIXES)) as u32);
if next() % 3 == 0 {
table.withdraw(peer, &[p]);
} else {
let route = Route {
med: Some((next() % 3) as u32),
..Route::via(
&[(next() % 3) as u32 + 1, 9][..(next() % 2 + 1) as usize],
)
};
let attrs = table.attrs().intern(&route.attributes(), None).unwrap();
table.announce(peer, &attrs, &[p]);
}
}
})
})
.collect();
let mirror = &mut mirror;
let done = &done;
let reading = scope.spawn(move || {
let mut reader = table.reader();
while !done.load(Ordering::Acquire) {
reader.poll(50, |event| mirror.apply(event));
}
mirror.catch_up(&mut reader, 50);
});
for writer in writers {
writer.join().unwrap();
}
done.store(true, Ordering::Release);
reading.join().unwrap();
});
mirror.assert_matches(table);
assert!(mirror.best_events > 0);
}
// ---------------------------------------------------------------------------
// Readers against the table, under any interleaving
// ---------------------------------------------------------------------------
#[derive(Debug, Clone)]
enum Op {
Announce {
peer: usize,
prefix: usize,
path_id: u32,
route: Route,
},
Withdraw {
peer: usize,
prefix: usize,
path_id: u32,
},
Flap {
peer: usize,
},
Poll {
reader: usize,
budget: usize,
},
/// The reader is dropped and a new one takes its place.
Replace {
reader: usize,
},
}
const PEERS: usize = 4;
const PREFIXES: usize = 12;
// The first three follow the best routes, the others all routes.
const READERS: usize = 5;
const OF_BEST: usize = 3;
fn op() -> impl Strategy<Value = Op> {
prop_oneof![
8 => (0..PEERS, 0..PREFIXES, 0u32..3, route()).prop_map(
|(peer, prefix, path_id, route)| Op::Announce { peer, prefix, path_id, route }
),
5 => (0..PEERS, 0..PREFIXES, 0u32..3)
.prop_map(|(peer, prefix, path_id)| Op::Withdraw { peer, prefix, path_id }),
1 => (0..PEERS).prop_map(|peer| Op::Flap { peer }),
// Reader 0 is polled rarely: it is the one that falls behind.
4 => (1..READERS, 1usize..6).prop_map(|(reader, budget)| Op::Poll { reader, budget }),
1 => (0..READERS, 1usize..6).prop_map(|(reader, budget)| Op::Poll { reader, budget }),
1 => (1..READERS).prop_map(|reader| Op::Replace { reader }),
]
}
proptest! {
#![proptest_config(ProptestConfig::with_cases(1500))]
/// However changes and reads are interleaved, and however small the
/// journals: a reader that has caught up holds exactly the table's best
/// routes, and every reader gets there in the end.
#[test]
fn a_reader_that_has_caught_up_has_what_the_table_has(
ops in proptest::collection::vec(op(), 1..150),
shards in 1usize..4,
) {
let infos: Vec<PeerInfo> = (0..PEERS)
.map(|i| if i == 0 { internal(9) } else { external(i as u8) })
.collect();
// Journals that are tidied, and found too long, almost at once.
let limits = Limits { tidy_at: 3, slack: 1 };
let mut fixture = Fixture::on(RibTable::with_journal_limits(shards, limits), &infos);
let prefixes: Vec<IpPrefix> = (0..PREFIXES as u32).map(nth_prefix).collect();
let new_reader = |table: &Arc<RibTable>, index: usize| {
if index < OF_BEST { table.reader() } else { table.reader_of_all_routes() }
};
let check = |index: usize, mirror: &Mirror, table: &RibTable| {
if index < OF_BEST {
mirror.assert_matches(table);
} else {
mirror.assert_has_all_routes_of(table);
}
};
let mut readers: Vec<(Reader, Mirror)> = (0..READERS)
.map(|index| (new_reader(&fixture.table, index), Mirror::default()))
.collect();
for op in ops {
match op {
Op::Announce { peer, prefix, path_id, route } => {
fixture.announce_path(peer, &prefixes[prefix], path_id, &route);
}
Op::Withdraw { peer, prefix, path_id } => {
fixture.withdraw_path(peer, &prefixes[prefix], path_id);
}
Op::Flap { peer } => {
fixture.table.remove_peer(fixture.peers[peer]);
fixture.peers[peer] = fixture.table.add_peer(infos[peer]).unwrap();
}
Op::Poll { reader: index, budget } => {
let (reader, mirror) = &mut readers[index];
let progress = reader.poll(budget, |event| mirror.apply(event));
if progress.caught_up {
check(index, mirror, &fixture.table);
}
}
Op::Replace { reader } => {
readers[reader] = (new_reader(&fixture.table, reader), Mirror::default());
}
}
// Never much longer than the table, whatever the readers do.
// Two journals to a shard.
let bound = 2 * shards * (2 * (PREFIXES + limits.slack) + limits.tidy_at + PREFIXES);
prop_assert!(fixture.table.journal_len() <= bound);
}
for (index, (reader, mirror)) in readers.iter_mut().enumerate() {
mirror.catch_up(reader, 5);
check(index, mirror, &fixture.table);
prop_assert_eq!(mirror.resets, mirror.synced);
}
}
}
crates/rib/src/table/mod.rs
+728
View File
@@ -0,0 +1,728 @@
//! The routing table: every route every peer has announced, and for each
//! prefix the one chosen as best.
//!
//! This is the Adj-RIBs-In and the Loc-RIB of RFC 4271 §3.2 in one
//! structure, organised by prefix: a prefix's row holds the routes for it
//! from all peers, side by side, and which of them is best. Choosing again
//! after a change looks at one row and nothing else.
//!
//! It is built for a table of hundreds of full feeds, hundreds of millions
//! of routes, taken in on many cores at once:
//!
//! - **A route is small**: sixteen bytes. A pointer to its attribute set,
//! which is shared with every other route that has the same attributes
//! ([`attrs`]), and one word that holds the peer it is from and the first
//! steps of route selection as a single number to compare.
//! - **The table is in shards**, by hash of the prefix, each behind its own
//! lock. A session applies an UPDATE itself, on its own thread, taking
//! the locks of the few shards its prefixes fall in. There is no queue in
//! front of the table and no thread that owns it; sessions that touch
//! different shards do not wait for each other, and one that has to wait
//! is slowed down, which is the back-pressure that is wanted.
//! - **Readers do not stop the table.** Reading takes a shard's lock for
//! reading, one shard at a time, and looks at the rows in place.
//! - **Followers go at their own pace.** What acts on the best routes reads
//! the changes from a [`journal`], and a slow reader holds nothing up.
pub mod attrs;
mod journal;
mod select;
#[cfg(test)]
mod tests;
use std::hash::BuildHasher;
use std::net::{IpAddr, Ipv4Addr};
use std::sync::atomic::{AtomicU64, AtomicUsize};
use std::sync::{Arc, Mutex, PoisonError, RwLock, RwLockReadGuard, RwLockWriteGuard};
use bgp_primitives::ip_prefix::{IpPrefix, Nlri};
use foldhash::fast::RandomState;
use hashbrown::HashTable;
use hashbrown::hash_table::Entry;
use tokio::sync::Notify;
pub use attrs::{AttrSet, AttrStore, SelectionKey};
pub use journal::{Event, Progress, Reader, Routes};
/// What route selection needs to know about a peer.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct PeerInfo {
pub asn: u32,
/// The peer's BGP Identifier.
pub router_id: Ipv4Addr,
pub address: IpAddr,
/// Whether the peer is in our own AS.
pub ibgp: bool,
}
/// A peer's number in a table, given out by [`RibTable::add_peer`]. Small,
/// because every route carries one.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Ord)]
pub struct PeerIndex(u16);
impl PeerIndex {
pub fn as_usize(self) -> usize {
self.0.into()
}
}
/// The LOCAL_PREF of a route that does not carry one: every route from an
/// external peer, until import policy says otherwise.
pub const DEFAULT_LOCAL_PREF: u32 = 100;
/// What a change to the table did.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub struct Outcome {
/// Routes that were not in the table before.
pub added: usize,
/// Routes that took the place of one from the same peer.
pub replaced: usize,
/// Routes that were taken out.
pub removed: usize,
/// Prefixes whose best route is now another one, or none.
pub best_changed: usize,
}
impl std::ops::AddAssign for Outcome {
fn add_assign(&mut self, other: Self) {
self.added += other.added;
self.replaced += other.replaced;
self.removed += other.removed;
self.best_changed += other.best_changed;
}
}
/// One route of a prefix: sixteen bytes.
struct Path {
/// Two things in one word. The high 48 bits are the first steps of
/// route selection as one number, the larger the better; see
/// [`select::preference`]. The low 16 are the peer the route is from.
word: u64,
attrs: Arc<AttrSet>,
}
impl Path {
fn new(preference: u64, peer: PeerIndex, attrs: &Arc<AttrSet>) -> Self {
Self {
word: (preference << 16) | u64::from(peer.0),
attrs: Arc::clone(attrs),
}
}
fn preference(&self) -> u64 {
self.word >> 16
}
fn peer(&self) -> PeerIndex {
PeerIndex(self.word as u16)
}
}
/// Everything about one prefix.
struct Row {
prefix: IpPrefix,
/// Index of the best route in `paths`. A row always has at least one
/// route, and so always a best one; and the best one always has the
/// highest [`Path::preference`] in the row, which is what lets most
/// changes be settled without looking at the other routes
/// ([`select`]).
best: u32,
/// The sequence numbers of the latest change of this row's best route
/// and of any of its routes, by which the journals tell their entries
/// for it that still say something from those that do not.
seqs: journal::Seqs,
/// The routes, in one run of memory: finding a peer's route and
/// choosing the best are scans of it. One from each peer that has
/// announced the prefix, or several from a peer that sends more than one
/// (ADD-PATH, RFC 7911).
paths: Vec<Path>,
/// The path identifier of each route in `paths`, for telling apart the
/// routes of a peer that sends several. Empty while every route has
/// the identifier zero, which is all routes of peers that send one
/// each: they pay nothing for this.
path_ids: Vec<u32>,
}
impl Row {
fn new(prefix: IpPrefix, path: Path, path_id: u32, seqs: journal::Seqs) -> Self {
Self {
prefix,
best: 0,
seqs,
paths: vec![path],
path_ids: if path_id == 0 { vec![] } else { vec![path_id] },
}
}
fn path_id(&self, index: usize) -> u32 {
self.path_ids.get(index).copied().unwrap_or(0)
}
/// Where the route from `peer` with the identifier `path_id` is.
fn position(&self, peer: PeerIndex, path_id: u32) -> Option<usize> {
self.paths
.iter()
.enumerate()
.position(|(index, path)| path.peer() == peer && self.path_id(index) == path_id)
}
/// Where a route from `peer` is, whatever its identifier.
fn position_of_peer(&self, peer: PeerIndex) -> Option<usize> {
self.paths.iter().position(|path| path.peer() == peer)
}
fn push(&mut self, path: Path, path_id: u32) {
if path_id != 0 && self.path_ids.is_empty() {
self.path_ids = vec![0; self.paths.len()];
}
push_tight(&mut self.paths, path);
if !self.path_ids.is_empty() {
push_tight(&mut self.path_ids, path_id);
}
}
/// Takes out the route at `at`; the last route moves into its place.
fn swap_remove(&mut self, at: usize) -> Path {
if !self.path_ids.is_empty() {
self.path_ids.swap_remove(at);
}
self.paths.swap_remove(at)
}
fn best(&self) -> usize {
self.best as usize
}
fn set_best(&mut self, index: usize) {
// An index into `paths`, which cannot have that many: each is
// sixteen bytes.
self.best = index as u32;
}
fn route(&self, index: usize) -> RouteRef<'_> {
let path = &self.paths[index];
RouteRef {
prefix: self.prefix,
peer: path.peer(),
path_id: self.path_id(index),
attrs: &path.attrs,
best: index == self.best(),
}
}
fn routes(&self) -> impl Iterator<Item = RouteRef<'_>> {
(0..self.paths.len()).map(|index| self.route(index))
}
}
/// A route in the table, looked at in place.
#[derive(Debug, Clone, Copy)]
pub struct RouteRef<'a> {
pub prefix: IpPrefix,
/// The peer the route is from.
pub peer: PeerIndex,
/// The peer's identifier for the route: zero unless the peer sends
/// several routes for a prefix (ADD-PATH, RFC 7911).
pub path_id: u32,
pub attrs: &'a Arc<AttrSet>,
/// Whether it is the route chosen for its prefix.
pub best: bool,
}
impl RouteRef<'_> {
/// The route, to keep.
pub fn to_owned(&self) -> Route {
Route {
prefix: self.prefix,
peer: self.peer,
path_id: self.path_id,
attrs: Arc::clone(self.attrs),
best: self.best,
}
}
}
/// A route taken out of the table to keep; see [`RouteRef`].
#[derive(Debug, Clone)]
pub struct Route {
pub prefix: IpPrefix,
pub peer: PeerIndex,
pub path_id: u32,
pub attrs: Arc<AttrSet>,
pub best: bool,
}
struct Shard {
rows: HashTable<Row>,
routes: usize,
/// A copy of the peer list, so that route selection reads it under the
/// lock it already holds.
peers: Vec<Option<PeerInfo>>,
hasher: RandomState,
/// Reused between selections.
scratch: select::Scratch,
/// Which prefixes have changed, for those who follow.
journals: journal::Journals,
}
/// The routing table. Shared between threads by reference; every method
/// takes `&self`.
pub struct RibTable {
shards: Box<[RwLock<Shard>]>,
hasher: RandomState,
attrs: AttrStore,
/// Which peer numbers are in use. Changing the peer list also takes
/// every shard's lock in turn; this serialises that.
peers: Mutex<Vec<Option<PeerInfo>>>,
/// Which reader slots are in use; see [`journal`].
reader_slots: Mutex<Vec<bool>>,
/// How many readers there are, to skip waking them when there are none.
readers: AtomicUsize,
/// Goes up whenever something is written to a journal that has
/// readers.
version: AtomicU64,
wake: Notify,
}
/// No more peers can be added: every [`PeerIndex`] is in use.
#[derive(Debug, Clone, Copy, PartialEq, Eq, thiserror::Error)]
#[error("the table has no room for another peer")]
pub struct TooManyPeers;
impl RibTable {
/// A table in `shards` shards. More shards let more sessions write at
/// once; a few times the number of cores is plenty.
pub fn new(shards: usize) -> Self {
Self::with_journal_limits(shards, journal::Limits::default())
}
/// A table whose journals are tidied, and found too long, at the given
/// sizes. Tests use small ones to get there quickly.
fn with_journal_limits(shards: usize, limits: journal::Limits) -> Self {
let hasher = RandomState::default();
Self {
shards: (0..shards.max(1))
.map(|_| {
RwLock::new(Shard {
rows: HashTable::new(),
routes: 0,
peers: Vec::new(),
hasher: hasher.clone(),
scratch: select::Scratch::default(),
journals: journal::Journals::new(limits),
})
})
.collect(),
hasher,
attrs: AttrStore::new(),
peers: Mutex::default(),
reader_slots: Mutex::default(),
readers: AtomicUsize::new(0),
version: AtomicU64::new(0),
wake: Notify::new(),
}
}
/// Where attribute sets for this table's routes are made.
pub fn attrs(&self) -> &AttrStore {
&self.attrs
}
/// Makes room for a peer's routes and returns the number to announce
/// them under.
///
/// The number is the peer's until [`RibTable::remove_peer`] is called
/// with it, and may then be given to another. Whoever was given it must
/// not use it after that: the table cannot tell a late announcement
/// from one of the number's next owner.
pub fn add_peer(&self, info: PeerInfo) -> Result<PeerIndex, TooManyPeers> {
let mut peers = self.peers.lock().unwrap_or_else(PoisonError::into_inner);
let index = match peers.iter().position(Option::is_none) {
Some(free) => free,
None => {
// A peer number shares a word with other things; see `Path`.
if peers.len() > usize::from(u16::MAX) {
return Err(TooManyPeers);
}
peers.push(None);
peers.len() - 1
}
};
peers[index] = Some(info);
for shard in &self.shards {
let mut shard = write(shard);
if shard.peers.len() <= index {
shard.peers.resize(index + 1, None);
}
shard.peers[index] = Some(info);
}
Ok(PeerIndex(index as u16))
}
/// Takes out every route from `peer` and gives its number back for
/// reuse (RFC 4271 §8.2.2: the routes of a session that ended are
/// deleted).
///
/// This looks at every prefix in the table. The list of peers is not
/// held while it does, so other peers can come and go meanwhile.
pub fn remove_peer(&self, peer: PeerIndex) -> Outcome {
let mut outcome = Outcome::default();
let mut news = false;
for shard in &self.shards {
let mut shard = write(shard);
let before = shard.journals.mark();
outcome += shard.remove_peer(peer);
// From here on the shard takes no more routes from the peer.
if let Some(slot) = shard.peers.get_mut(peer.as_usize()) {
*slot = None;
}
shard.tidy_journals();
news |= shard.journals.has_news_since(before);
}
// Only now can the number be given out again: no shard knows it.
let mut peers = self.peers.lock().unwrap_or_else(PoisonError::into_inner);
if let Some(slot) = peers.get_mut(peer.as_usize()) {
*slot = None;
}
drop(peers);
if news {
self.wake_readers();
}
outcome
}
/// Announces `routes` from `peer`, all with the attributes `attrs`:
/// what one UPDATE does. A route is known by its prefix and path
/// identifier together ([`Nlri`]; a plain [`IpPrefix`] is the route
/// with identifier zero, which is every route of a peer that sends one
/// for each prefix). Announcing a route the peer had announced before
/// replaces it.
pub fn announce<N>(&self, peer: PeerIndex, attrs: &Arc<AttrSet>, routes: &[N]) -> Outcome
where
N: Copy + Into<Nlri>,
{
let mut outcome = Outcome::default();
self.by_shard(routes, |shard, placed| {
let Some(info) = shard.peers.get(peer.as_usize()).copied().flatten() else {
// The peer was removed while this was on its way.
return;
};
let preference = select::preference(attrs.key(), &info);
for (hash, nlri) in placed {
outcome += shard.announce(*hash, *nlri, peer, preference, attrs);
}
});
outcome
}
/// Withdraws `routes` of `peer`. Withdrawing a route that is not there
/// does nothing.
pub fn withdraw<N>(&self, peer: PeerIndex, routes: &[N]) -> Outcome
where
N: Copy + Into<Nlri>,
{
let mut outcome = Outcome::default();
self.by_shard(routes, |shard, placed| {
for (hash, nlri) in placed {
outcome += shard.withdraw(*hash, nlri, peer);
}
});
outcome
}
fn shard_of(&self, hash: u64) -> usize {
spread(hash, self.shards.len())
}
/// Calls `apply` once for each shard that some of `routes` fall in,
/// with that shard locked and those routes. Wakes the readers if a
/// journal one of them follows was written to.
fn by_shard<N>(&self, routes: &[N], mut apply: impl FnMut(&mut Shard, &[(u64, Nlri)]))
where
N: Copy + Into<Nlri>,
{
let Some(first) = routes.first() else {
return;
};
let hashed = |route: &N| {
let nlri: Nlri = (*route).into();
(self.hasher.hash_one(nlri.prefix), nlri)
};
// An UPDATE has a handful of prefixes; they are sorted on the stack.
const INLINE: usize = 32;
let mut inline = [hashed(first); INLINE];
let mut spilled = Vec::new();
let placed: &mut [(u64, Nlri)] = if routes.len() <= INLINE {
for (slot, route) in inline.iter_mut().zip(routes) {
*slot = hashed(route);
}
&mut inline[..routes.len()]
} else {
spilled.extend(routes.iter().map(hashed));
&mut spilled
};
placed.sort_unstable_by_key(|(hash, _)| self.shard_of(*hash));
let mut news = false;
for run in placed.chunk_by(|a, b| self.shard_of(a.0) == self.shard_of(b.0)) {
let mut shard = write(&self.shards[self.shard_of(run[0].0)]);
let before = shard.journals.mark();
apply(&mut shard, run);
shard.tidy_journals();
news |= shard.journals.has_news_since(before);
}
if news {
self.wake_readers();
}
}
/// How many routes the table holds, from all peers.
pub fn len(&self) -> usize {
self.shards.iter().map(|shard| read(shard).routes).sum()
}
pub fn is_empty(&self) -> bool {
self.len() == 0
}
/// How many prefixes have at least one route.
pub fn prefix_count(&self) -> usize {
self.shards.iter().map(|shard| read(shard).rows.len()).sum()
}
/// The best route for `prefix`: who it is from, and its attributes.
pub fn best(&self, prefix: &IpPrefix) -> Option<(PeerIndex, Arc<AttrSet>)> {
self.with_row(prefix, |row| {
let best = &row.paths[row.best()];
(best.peer(), Arc::clone(&best.attrs))
})
}
/// Every route for `prefix`.
pub fn routes(&self, prefix: &IpPrefix) -> Vec<Route> {
self.with_row(prefix, |row| row.routes().map(|r| r.to_owned()).collect())
.unwrap_or_default()
}
fn with_row<T>(&self, prefix: &IpPrefix, read_row: impl FnOnce(&Row) -> T) -> Option<T> {
let hash = self.hasher.hash_one(prefix);
let shard = read(&self.shards[self.shard_of(hash)]);
shard
.rows
.find(hash, |row| row.prefix == *prefix)
.map(read_row)
}
/// Calls `visit` with the best route of every prefix, in no particular
/// order. The table is not stopped: one shard at a time is held for
/// reading, so the routes seen are not all of one moment.
///
/// `visit` is called with a shard held, and must not call into the
/// table itself: taking the shard's lock again can wait for ever on a
/// writer that is waiting for this hold to end.
pub fn for_each_best(&self, mut visit: impl FnMut(&IpPrefix, PeerIndex, &Arc<AttrSet>)) {
for shard in &self.shards {
for row in &read(shard).rows {
let best = &row.paths[row.best()];
visit(&row.prefix, best.peer(), &best.attrs);
}
}
}
/// Up to `limit` routes: every route of as many prefixes as fit, in no
/// particular order. For showing some of a table that is too large to
/// show all of.
pub fn some_routes(&self, limit: usize) -> Vec<Route> {
let mut routes = Vec::new();
for shard in &self.shards {
for row in &read(shard).rows {
for route in row.routes() {
if routes.len() == limit {
return routes;
}
routes.push(route.to_owned());
}
}
}
routes
}
/// Calls `visit` with every route in the table. As
/// [`RibTable::for_each_best`]: the table is not stopped, and `visit`
/// must not call into it.
pub fn for_each_route(&self, mut visit: impl FnMut(RouteRef<'_>)) {
for shard in &self.shards {
for row in &read(shard).rows {
row.routes().for_each(&mut visit);
}
}
}
}
fn read(shard: &RwLock<Shard>) -> RwLockReadGuard<'_, Shard> {
// A panic under the lock cannot leave a row half changed in a way that
// is unsafe to read; the poison is ignored, as for the server's tables.
shard.read().unwrap_or_else(PoisonError::into_inner)
}
fn write(shard: &RwLock<Shard>) -> RwLockWriteGuard<'_, Shard> {
shard.write().unwrap_or_else(PoisonError::into_inner)
}
/// Picks one of `count` buckets by a hash, for spreading keys over shards
/// or lock stripes that each hold a hash table keyed by the same hash.
///
/// That table takes a key's slot from the low bits of the hash and, from
/// the top seven, the tag it compares before it looks at the key itself
/// (hashbrown). The bucket is taken from bits in between: taken from
/// either end, the keys of one bucket would all share those bits, and the
/// table inside would lose them.
fn spread(hash: u64, count: usize) -> usize {
let middle = (hash << 7) >> 32;
((middle * count as u64) >> 32) as usize
}
/// Adds to a vector without the doubling that `push` does: a row of a few
/// hundred routes should not hold room for as many again. Grows by a
/// quarter, which keeps adding cheap on average.
fn push_tight<T>(vec: &mut Vec<T>, value: T) {
if vec.len() == vec.capacity() {
vec.reserve_exact((vec.len() / 4).max(1));
}
vec.push(value);
}
impl Shard {
fn tidy_journals(&mut self) {
self.journals.tidy(&self.rows, &self.hasher);
}
fn announce(
&mut self,
hash: u64,
nlri: Nlri,
peer: PeerIndex,
preference: u64,
attrs: &Arc<AttrSet>,
) -> Outcome {
let Nlri { prefix, path_id } = nlri;
let hasher = &self.hasher;
let entry = self.rows.entry(
hash,
|row| row.prefix == prefix,
|row| hasher.hash_one(row.prefix),
);
let path = Path::new(preference, peer, attrs);
let row = match entry {
Entry::Vacant(vacant) => {
let seqs = self.journals.appeared(prefix);
vacant.insert(Row::new(prefix, path, path_id, seqs));
self.routes += 1;
return Outcome {
added: 1,
best_changed: 1,
..Outcome::default()
};
}
Entry::Occupied(occupied) => occupied.into_mut(),
};
let (outcome, best_changed) = match row.position(peer, path_id) {
None => {
row.push(path, path_id);
self.routes += 1;
let outcome = Outcome {
added: 1,
..Outcome::default()
};
(
outcome,
select::after_add(row, &self.peers, &mut self.scratch),
)
}
Some(at) => {
let before = std::mem::replace(&mut row.paths[at], path);
let outcome = Outcome {
replaced: 1,
..Outcome::default()
};
if Arc::ptr_eq(&before.attrs, attrs) && before.preference() == preference {
// The same route again: nothing changed at all.
return outcome;
}
let best_changed = select::after_replace(
row,
at,
before.preference(),
&self.peers,
&mut self.scratch,
);
(outcome, best_changed)
}
};
self.journals.changed(row, best_changed);
Outcome {
best_changed: best_changed.into(),
..outcome
}
}
fn withdraw(&mut self, hash: u64, nlri: &Nlri, peer: PeerIndex) -> Outcome {
let prefix = nlri.prefix;
let Ok(mut entry) = self.rows.find_entry(hash, |row| row.prefix == prefix) else {
return Outcome::default();
};
let row = entry.get_mut();
let Some(at) = row.position(peer, nlri.path_id) else {
return Outcome::default();
};
self.routes -= 1;
if row.paths.len() == 1 {
entry.remove();
self.journals.disappeared(prefix);
return Outcome {
removed: 1,
best_changed: 1,
..Outcome::default()
};
}
let best_changed = select::after_remove(row, at, &self.peers, &mut self.scratch);
self.journals.changed(row, best_changed);
Outcome {
removed: 1,
best_changed: best_changed.into(),
..Outcome::default()
}
}
fn remove_peer(&mut self, peer: PeerIndex) -> Outcome {
let mut outcome = Outcome::default();
let peers = &self.peers;
let scratch = &mut self.scratch;
let journals = &mut self.journals;
self.rows.retain(|row| {
let mut removed = 0;
let mut best_changed = false;
// A peer that sends several routes for a prefix has several
// here.
while let Some(at) = row.position_of_peer(peer) {
removed += 1;
if row.paths.len() == 1 {
outcome.removed += removed;
outcome.best_changed += 1;
journals.disappeared(row.prefix);
return false;
}
best_changed |= select::after_remove(row, at, peers, scratch);
}
if removed > 0 {
outcome.removed += removed;
outcome.best_changed += usize::from(best_changed);
journals.changed(row, best_changed);
}
true
});
self.routes -= outcome.removed;
outcome
}
}
crates/rib/src/table/select.rs
+249
View File
@@ -0,0 +1,249 @@
//! Choosing the best route for a prefix (RFC 4271 §9.1.2.2).
//!
//! The steps, in order. A route wins at the first step where it differs:
//!
//! 1. highest LOCAL_PREF;
//! 2. shortest AS path;
//! 3. lowest ORIGIN;
//! 4. lowest MULTI_EXIT_DISC, among routes from the same neighbouring AS;
//! 5. a route from an external peer over one from an internal peer;
//! 6. the peer with the lowest BGP Identifier;
//! 7. the peer with the lowest address.
//!
//! (The RFC's step between 5 and 6, the interior cost to the next hop, has
//! nothing to compare until there is an IGP to ask.)
//!
//! Steps 1 to 3 are put into one number per route when it is stored,
//! [`preference`]. Most of the time that number alone settles it: a new
//! route with a larger one than the best is the new best, with a smaller
//! one changes nothing, and neither needs a look at any other route. Only a
//! tie goes on to [`choose`], which looks at the whole row, and that begins
//! with a pass over the numbers to find the routes still in the running.
//!
//! [`after_add`], [`after_replace`] and [`after_remove`] are what the table
//! calls when a row has changed. Each says whether the best route is now
//! another, and takes the short way where there is one. They rely on, and
//! keep, one fact about a row: its best route has the highest preference
//! of all its routes.
use super::attrs::SelectionKey;
use super::{DEFAULT_LOCAL_PREF, PeerInfo, Row};
/// Steps 1 to 3 as one number of 48 bits: the larger, the better.
///
/// LOCAL_PREF fills the high 32 bits. Below it come the AS path length in
/// 12 bits and the ORIGIN in 2, each turned upside down so that shorter and
/// lower compare as larger. Twelve bits are enough for any path: a BGP
/// message of 4096 octets has room for about a thousand AS numbers.
pub(super) fn preference(key: &SelectionKey, peer: &PeerInfo) -> u64 {
// LOCAL_PREF is a matter between routers of one AS. A route from an
// external peer has the default until import policy gives it another.
let local_pref = if peer.ibgp {
key.local_pref.unwrap_or(DEFAULT_LOCAL_PREF)
} else {
DEFAULT_LOCAL_PREF
};
const LONGEST: u16 = (1 << 12) - 1;
(u64::from(local_pref) << 16)
| (u64::from(LONGEST - key.as_path_len.min(LONGEST)) << 4)
| (u64::from(3 - key.origin.min(3)) << 2)
}
/// Buffers that [`choose`] reuses.
#[derive(Default)]
pub(super) struct Scratch {
/// Routes still in the running: (neighbouring AS, MED, index in row).
candidates: Vec<(u32, u32, u32)>,
}
/// The best route changed after a route was added at the end of `row`?
pub(super) fn after_add(row: &mut Row, peers: &[Option<PeerInfo>], scratch: &mut Scratch) -> bool {
let added = row.paths.len() - 1;
let preference = row.paths[added].preference();
let best = row.paths[row.best()].preference();
// Better than the best in the first steps of selection is better than
// everything; worse than it changes nothing. A tie has to be settled by
// the later steps.
if preference > best {
row.set_best(added);
true
} else if preference < best {
false
} else if let Some(wins) = settles_tie(row, added, peers) {
if wins {
row.set_best(added);
}
wins
} else {
choose(row, peers, scratch)
}
}
/// The best route changed after the route at `at` was replaced by another
/// from the same peer? `before` is the preference of the route it replaced.
/// The two are known to differ.
pub(super) fn after_replace(
row: &mut Row,
at: usize,
before: u64,
peers: &[Option<PeerInfo>],
scratch: &mut Scratch,
) -> bool {
let preference = row.paths[at].preference();
if at == row.best() {
// The best route itself changed. If it got better in the first
// steps it is still the best, but it is a different route now.
if preference <= before {
choose(row, peers, scratch);
}
return true;
}
let best = row.paths[row.best()].preference();
if preference > best {
row.set_best(at);
true
} else if preference < best && before < best {
// It was not in the running and still is not.
false
} else {
choose(row, peers, scratch)
}
}
/// Takes the route at `at` out of `row`, which has others, and returns
/// whether the best route changed.
pub(super) fn after_remove(
row: &mut Row,
at: usize,
peers: &[Option<PeerInfo>],
scratch: &mut Scratch,
) -> bool {
let best = row.best();
let best_preference = row.paths[best].preference();
let last = row.paths.len() - 1;
let removed = row.swap_remove(at);
if at == best {
choose(row, peers, scratch);
return true;
}
// The last route moved into the gap.
if best == last {
row.set_best(at);
}
// A route that tied with the best in the first steps may have been
// what kept another from being best, by its MED.
removed.preference() == best_preference && choose(row, peers, scratch)
}
/// Chooses the best route of `row` from all its routes, records it in the
/// row, and returns whether it is another route than before.
///
/// The row must have at least one route.
fn choose(row: &mut Row, peers: &[Option<PeerInfo>], scratch: &mut Scratch) -> bool {
let before = row.best;
let top = row
.paths
.iter()
.map(|path| path.preference())
.max()
.unwrap_or(0);
let candidates = &mut scratch.candidates;
candidates.clear();
for (index, path) in row.paths.iter().enumerate() {
if path.preference() == top {
let key = path.attrs.key();
// A route without a MED is taken to have the lowest (RFC 4271
// §9.1.2.2 c).
candidates.push((key.neighbor_as, key.med.unwrap_or(0), index as u32));
}
}
// Step 4. MEDs are compared only between routes from the same
// neighbouring AS, so "lower MED" is not an order over all the routes,
// and comparing them pair by pair can go round in circles. Instead:
// within each neighbouring AS only the routes with its lowest MED stay
// in, and the steps after are an order over what is left.
candidates.sort_unstable();
let mut best: Option<(TieBreak, u32)> = None;
let mut group = None;
for &(neighbor_as, med, index) in candidates.iter() {
// Sorted by AS and then MED: the first of each AS has its lowest.
let lowest = match group {
Some((group_as, lowest)) if group_as == neighbor_as => lowest,
_ => {
group = Some((neighbor_as, med));
med
}
};
if med != lowest {
continue;
}
let tie_break = TieBreak::of(row, index as usize, peers);
if best.as_ref().is_none_or(|(least, _)| tie_break < *least) {
best = Some((tie_break, index));
}
}
row.best = best.map_or(0, |(_, index)| index);
row.best != before
}
/// For a route just added at `added` that ties with the best in steps 1 to
/// 3: whether it is the new best, if that can be said from the two routes
/// alone. `None` if the whole row has to be looked at.
///
/// It can be said when MED cannot knock either of them out. The new route
/// has the lowest MED there is, so nothing knocks it out. It knocks out
/// only routes of its own neighbouring AS with a higher MED, and if the
/// best is not one of those, the best is still the best of all the others.
/// So the better of the two is the best of all. This is the usual case:
/// most routes have no MED.
fn settles_tie(row: &Row, added: usize, peers: &[Option<PeerInfo>]) -> Option<bool> {
let new = row.paths[added].attrs.key();
let best = row.paths[row.best()].attrs.key();
if new.med.unwrap_or(0) != 0 {
return None;
}
if best.neighbor_as == new.neighbor_as && best.med.unwrap_or(0) != 0 {
return None;
}
Some(TieBreak::of(row, added, peers) < TieBreak::of(row, row.best(), peers))
}
/// Steps 5 to 7, then the peer's number, and then the path identifier for
/// a peer that sent several routes: so that the choice never depends on the
/// order routes arrived in. The least wins.
#[derive(PartialEq, Eq, PartialOrd, Ord)]
struct TieBreak {
internal: bool,
router_id: u32,
address: std::net::IpAddr,
peer: u16,
path_id: u32,
}
impl TieBreak {
fn of(row: &Row, index: usize, peers: &[Option<PeerInfo>]) -> Self {
let peer = row.paths[index].peer();
let path_id = row.path_id(index);
match peers.get(peer.as_usize()).copied().flatten() {
Some(info) => Self {
internal: info.ibgp,
router_id: info.router_id.into(),
address: info.address,
peer: peer.0,
path_id,
},
// Not expected: a peer is forgotten only once its routes are
// gone. If it happened, anything known would be preferred.
None => Self {
internal: true,
router_id: u32::MAX,
address: std::net::Ipv6Addr::from(u128::MAX).into(),
peer: peer.0,
path_id,
},
}
}
}
crates/rib/src/table/tests.rs
+759
View File
@@ -0,0 +1,759 @@
use std::collections::{BTreeMap, HashMap};
use std::net::{IpAddr, Ipv4Addr};
use std::sync::Arc;
use bgp_packet::message::{
AsPathAttribute, LocalPrefPathAttribute, MultiExitDiscPathAttribute, NextHopPathAttribute,
OriginPathAttribute, PathAttribute,
};
use bgp_primitives::ip_prefix::{IpPrefix, Nlri};
use proptest::prelude::*;
use super::*;
pub(super) fn prefix(s: &str) -> IpPrefix {
IpPrefix::try_from(s).unwrap()
}
/// What a test says about a route; everything selection looks at.
#[derive(Debug, Clone, PartialEq, Eq)]
pub(super) struct Route {
pub(super) local_pref: Option<u32>,
pub(super) path: Vec<u32>,
pub(super) origin: u8,
pub(super) med: Option<u32>,
}
impl Route {
pub(super) fn via(path: &[u32]) -> Self {
Self {
local_pref: None,
path: path.to_vec(),
origin: 0,
med: None,
}
}
pub(super) fn attributes(&self) -> Vec<PathAttribute> {
let mut attributes = vec![
PathAttribute::Origin(match self.origin {
0 => OriginPathAttribute::IGP,
1 => OriginPathAttribute::EGP,
_ => OriginPathAttribute::INCOMPLETE,
}),
AsPathAttribute::from_asns(self.path.clone()),
PathAttribute::NextHop(NextHopPathAttribute(Ipv4Addr::new(192, 0, 2, 1))),
];
attributes.extend(
self.med
.map(|med| PathAttribute::MultiExitDisc(MultiExitDiscPathAttribute(med))),
);
attributes.extend(
self.local_pref
.map(|lp| PathAttribute::LocalPref(LocalPrefPathAttribute(lp))),
);
attributes
}
}
pub(super) fn external(number: u8) -> PeerInfo {
PeerInfo {
asn: 65000 + u32::from(number),
router_id: Ipv4Addr::new(10, 0, 0, number),
address: IpAddr::from(Ipv4Addr::new(192, 0, 2, number)),
ibgp: false,
}
}
pub(super) fn internal(number: u8) -> PeerInfo {
PeerInfo {
asn: 64999,
ibgp: true,
..external(number)
}
}
/// A table and the peers added to it, for tests that speak of peers by the
/// order they were added in.
pub(super) struct Fixture {
pub(super) table: Arc<RibTable>,
pub(super) peers: Vec<PeerIndex>,
}
impl Fixture {
pub(super) fn new(peers: &[PeerInfo]) -> Self {
// More than one shard, so that a test's prefixes are spread.
Self::on(RibTable::new(4), peers)
}
pub(super) fn on(table: RibTable, peers: &[PeerInfo]) -> Self {
let table = Arc::new(table);
let peers = peers.iter().map(|p| table.add_peer(*p).unwrap()).collect();
Self { table, peers }
}
pub(super) fn announce(&self, peer: usize, prefix: &IpPrefix, route: &Route) -> Outcome {
let attrs = self
.table
.attrs()
.intern(&route.attributes(), None)
.unwrap();
self.table.announce(self.peers[peer], &attrs, &[*prefix])
}
pub(super) fn withdraw(&self, peer: usize, prefix: &IpPrefix) -> Outcome {
self.table.withdraw(self.peers[peer], &[*prefix])
}
/// As `announce`, from a peer that sends several routes for a prefix
/// and calls this one `path_id`.
pub(super) fn announce_path(
&self,
peer: usize,
prefix: &IpPrefix,
path_id: u32,
route: &Route,
) -> Outcome {
let attrs = self
.table
.attrs()
.intern(&route.attributes(), None)
.unwrap();
let nlri = Nlri {
prefix: *prefix,
path_id,
};
self.table.announce(self.peers[peer], &attrs, &[nlri])
}
pub(super) fn withdraw_path(&self, peer: usize, prefix: &IpPrefix, path_id: u32) -> Outcome {
let nlri = Nlri {
prefix: *prefix,
path_id,
};
self.table.withdraw(self.peers[peer], &[nlri])
}
/// The best route for `prefix`: which peer (by order added) it is from
/// and that peer's identifier for it.
pub(super) fn best_path(&self, prefix: &IpPrefix) -> Option<(usize, u32)> {
let best = self.table.routes(prefix).into_iter().find(|r| r.best)?;
let peer = self.peers.iter().position(|p| *p == best.peer)?;
Some((peer, best.path_id))
}
/// The routes held for `prefix`, as (peer, path identifier), in order.
pub(super) fn held(&self, prefix: &IpPrefix) -> Vec<(usize, u32)> {
let mut held: Vec<(usize, u32)> = self
.table
.routes(prefix)
.iter()
.map(|r| {
let peer = self.peers.iter().position(|p| *p == r.peer).unwrap();
(peer, r.path_id)
})
.collect();
held.sort_unstable();
held
}
/// Which peer (by order added) the best route for `prefix` is from.
fn best(&self, prefix: &IpPrefix) -> Option<usize> {
let (peer, _) = self.table.best(prefix)?;
self.peers.iter().position(|p| *p == peer)
}
}
#[test]
fn each_step_of_selection_decides_when_the_ones_before_tie() {
let p = prefix("10.0.0.0/8");
// Peer 0 has the lower identifier, so it wins whenever nothing else
// decides; each case makes peer 1's route better in one step.
let cases: [(&str, [PeerInfo; 2], Route, Route); 6] = [
(
"higher LOCAL_PREF, despite a longer path",
[internal(1), internal(2)],
Route {
local_pref: Some(100),
..Route::via(&[1])
},
Route {
local_pref: Some(200),
..Route::via(&[1, 2, 3])
},
),
(
"shorter AS path",
[external(1), external(2)],
Route::via(&[1, 2, 3]),
Route::via(&[4, 5]),
),
(
"lower ORIGIN",
[external(1), external(2)],
Route {
origin: 2,
..Route::via(&[1, 2])
},
Route {
origin: 0,
..Route::via(&[3, 4])
},
),
(
"lower MED from the same neighbouring AS",
[external(1), external(2)],
Route {
med: Some(20),
..Route::via(&[7, 2])
},
Route {
med: Some(10),
..Route::via(&[7, 3])
},
),
(
"no MED counts as the lowest",
[external(1), external(2)],
Route {
med: Some(1),
..Route::via(&[7, 2])
},
Route::via(&[7, 3]),
),
(
"external over internal",
[internal(1), external(2)],
Route::via(&[7, 2]),
Route::via(&[8, 3]),
),
];
for (case, peers, first, second) in cases {
// In both orders of arrival: the choice must not depend on it.
for order in [[0, 1], [1, 0]] {
let fixture = Fixture::new(&peers);
for peer in order {
fixture.announce(peer, &p, [&first, &second][peer]);
}
assert_eq!(fixture.best(&p), Some(1), "{case}, arriving {order:?}");
}
}
// And when nothing decides: the lowest identifier.
let fixture = Fixture::new(&[external(2), external(1)]);
fixture.announce(0, &p, &Route::via(&[1, 2]));
fixture.announce(1, &p, &Route::via(&[3, 4]));
assert_eq!(fixture.best(&p), Some(1));
}
/// MEDs are compared only within a neighbouring AS. Taken pair by pair
/// that is not an order: A beats B, B beats C, C beats A. Whatever the
/// order of arrival, the answer must be the same.
#[test]
fn med_is_compared_only_within_a_neighbouring_as_and_never_goes_in_circles() {
let p = prefix("10.0.0.0/8");
let routes = [
// From AS 1, MED 10, and the highest identifier.
Route {
med: Some(10),
..Route::via(&[1, 9])
},
// From AS 2: its MED is not comparable with the others'.
Route {
med: Some(50),
..Route::via(&[2, 9])
},
// From AS 1, MED 20: loses to the first on MED.
Route {
med: Some(20),
..Route::via(&[1, 8])
},
];
// Identifiers: the third route's peer lowest, then the second's, then
// the first's. Without MED the third would win; it is knocked out by
// the first, and of the two left the second has the lower identifier.
let peers = [external(3), external(2), external(1)];
for order in [
[0, 1, 2],
[0, 2, 1],
[1, 0, 2],
[1, 2, 0],
[2, 0, 1],
[2, 1, 0],
] {
let fixture = Fixture::new(&peers);
for peer in order {
fixture.announce(peer, &p, &routes[peer]);
}
assert_eq!(fixture.best(&p), Some(1), "arriving {order:?}");
}
// Take away the route that knocked the third out, and the third wins.
let fixture = Fixture::new(&peers);
for (peer, route) in routes.iter().enumerate() {
fixture.announce(peer, &p, route);
}
let outcome = fixture.withdraw(0, &p);
assert_eq!(outcome.best_changed, 1);
assert_eq!(fixture.best(&p), Some(2));
}
/// A route that is not the best can still decide which is: by its MED it
/// keeps another route of its neighbouring AS out. When it gets worse or
/// goes, that other route is back in the running. The shortcuts must not
/// skip this.
#[test]
fn a_route_that_was_keeping_another_out_matters_when_it_changes() {
let p = prefix("10.0.0.0/8");
// Peer 0: from AS 1 with the lowest identifier, but MED 1.
// Peer 1: from AS 2. Best while peer 0's route is kept out.
// Peer 2: from AS 1 with MED 0, which keeps peer 0's route out.
let peers = [external(1), external(2), external(3)];
let held_out = Route {
med: Some(1),
..Route::via(&[1, 9])
};
let best = Route::via(&[2, 9]);
let holding_out = Route {
med: Some(0),
..Route::via(&[1, 8])
};
let set_up = || {
let fixture = Fixture::new(&peers);
fixture.announce(0, &p, &held_out);
fixture.announce(1, &p, &best);
fixture.announce(2, &p, &holding_out);
assert_eq!(fixture.best(&p), Some(1));
fixture
};
// It is replaced by a route with a longer path.
let fixture = set_up();
let outcome = fixture.announce(2, &p, &Route::via(&[1, 8, 7]));
assert_eq!(fixture.best(&p), Some(0));
assert_eq!(outcome.best_changed, 1);
// It is withdrawn.
let fixture = set_up();
let outcome = fixture.withdraw(2, &p);
assert_eq!(fixture.best(&p), Some(0));
assert_eq!(outcome.best_changed, 1);
// Its peer goes away.
let fixture = set_up();
let outcome = fixture.table.remove_peer(fixture.peers[2]);
assert_eq!(fixture.best(&p), Some(0));
assert_eq!(outcome.best_changed, 1);
// Its MED gets worse than the one it was keeping out.
let fixture = set_up();
fixture.announce(
2,
&p,
&Route {
med: Some(5),
..Route::via(&[1, 8])
},
);
assert_eq!(fixture.best(&p), Some(0));
}
#[test]
fn what_a_change_did_is_reported() {
let fixture = Fixture::new(&[external(1), external(2)]);
let p = prefix("10.0.0.0/8");
let long = Route::via(&[1, 2, 3]);
let short = Route::via(&[4]);
let first = fixture.announce(0, &p, &long);
assert_eq!((first.added, first.best_changed), (1, 1));
let worse = fixture.announce(1, &p, &Route::via(&[1, 2, 3, 4]));
assert_eq!((worse.added, worse.best_changed), (1, 0));
// The same route again changes nothing.
let again = fixture.announce(0, &p, &long);
assert_eq!((again.replaced, again.best_changed), (1, 0));
// The best route replaced by a better one is still a change of best.
let better = fixture.announce(0, &p, &short);
assert_eq!((better.replaced, better.best_changed), (1, 1));
assert_eq!((fixture.table.len(), fixture.table.prefix_count()), (2, 1));
assert_eq!(fixture.withdraw(1, &p).best_changed, 0);
let last = fixture.withdraw(0, &p);
assert_eq!((last.removed, last.best_changed), (1, 1));
assert_eq!(fixture.withdraw(0, &p), Outcome::default(), "nothing there");
assert!(fixture.table.is_empty());
assert_eq!(fixture.best(&p), None);
}
#[test]
fn removing_a_peer_takes_its_routes_and_frees_its_number() {
let fixture = Fixture::new(&[external(1), external(2)]);
let prefixes: Vec<IpPrefix> = (0..200u32)
.map(|i| {
IpPrefix::new(
bgp_primitives::constants::IpVersion::Ipv4,
(i << 8).to_be_bytes(),
24,
)
.unwrap()
})
.collect();
let attrs = |route: &Route| {
fixture
.table
.attrs()
.intern(&route.attributes(), None)
.unwrap()
};
// Peer 0 is best everywhere; peer 1 has only the first hundred.
fixture
.table
.announce(fixture.peers[0], &attrs(&Route::via(&[1])), &prefixes);
fixture.table.announce(
fixture.peers[1],
&attrs(&Route::via(&[2, 3])),
&prefixes[..100],
);
assert_eq!(fixture.table.len(), 300);
let outcome = fixture.table.remove_peer(fixture.peers[0]);
assert_eq!(outcome.removed, 200);
assert_eq!(outcome.best_changed, 200);
assert_eq!(
(fixture.table.len(), fixture.table.prefix_count()),
(100, 100)
);
assert_eq!(fixture.best(&prefixes[0]), Some(1));
assert_eq!(fixture.best(&prefixes[150]), None);
// Its routes are gone, so its attribute set can be dropped.
assert_eq!(fixture.table.attrs().sweep(), 1);
// And its number is given to the next peer.
assert_eq!(fixture.table.add_peer(external(3)), Ok(fixture.peers[0]));
}
#[test]
fn readers_see_every_route_and_which_is_best() {
let fixture = Fixture::new(&[external(1), external(2)]);
let (a, b) = (prefix("10.0.0.0/8"), prefix("2001:db8::/32"));
fixture.announce(0, &a, &Route::via(&[1, 2]));
fixture.announce(1, &a, &Route::via(&[3]));
fixture.announce(0, &b, &Route::via(&[1, 2]));
let mut best = Vec::new();
fixture
.table
.for_each_best(|prefix, peer, _| best.push((*prefix, peer)));
best.sort_by_key(|(prefix, _)| prefix.to_string());
assert_eq!(best, [(a, fixture.peers[1]), (b, fixture.peers[0])]);
let mut all = 0;
let mut marked_best = 0;
fixture.table.for_each_route(|route| {
all += 1;
marked_best += usize::from(route.best);
});
assert_eq!((all, marked_best), (3, 2));
let routes = fixture.table.routes(&a);
assert_eq!(routes.len(), 2);
assert!(
routes
.iter()
.any(|route| route.peer == fixture.peers[1] && route.best)
);
// Routes with the same attributes share one set.
let shared = &fixture.table.routes(&b)[0].attrs;
assert!(routes.iter().any(|route| Arc::ptr_eq(&route.attrs, shared)));
}
/// With ADD-PATH a peer sends several routes for one prefix and tells them
/// apart by a path identifier. Each is a route of its own: kept, replaced
/// and withdrawn by prefix and identifier together.
#[test]
fn a_peer_can_have_several_routes_for_a_prefix() {
let fixture = Fixture::new(&[external(1), external(2)]);
let p = prefix("10.0.0.0/8");
fixture.announce_path(0, &p, 1, &Route::via(&[1, 2, 3]));
let second = fixture.announce_path(0, &p, 2, &Route::via(&[1, 4]));
assert_eq!(
(second.added, second.replaced, second.best_changed),
(1, 0, 1)
);
fixture.announce(1, &p, &Route::via(&[2, 5, 6, 7]));
assert_eq!(fixture.held(&p), [(0, 1), (0, 2), (1, 0)]);
assert_eq!(fixture.best_path(&p), Some((0, 2)), "the shorter path");
assert_eq!((fixture.table.len(), fixture.table.prefix_count()), (3, 1));
// Announcing a path again replaces that path and no other.
let replaced = fixture.announce_path(0, &p, 1, &Route::via(&[1]));
assert_eq!((replaced.added, replaced.replaced), (0, 1));
assert_eq!(fixture.held(&p), [(0, 1), (0, 2), (1, 0)]);
assert_eq!(fixture.best_path(&p), Some((0, 1)));
// Withdrawing one leaves the peer's other.
assert_eq!(fixture.withdraw_path(0, &p, 1).removed, 1);
assert_eq!(fixture.held(&p), [(0, 2), (1, 0)]);
assert_eq!(fixture.best_path(&p), Some((0, 2)));
// A path that is not there is not withdrawn: not the same prefix
// under another identifier, and not the peer's plain route.
assert_eq!(fixture.withdraw_path(0, &p, 9), Outcome::default());
assert_eq!(fixture.withdraw(0, &p), Outcome::default());
// When the session ends, all of the peer's routes go.
fixture.announce_path(0, &p, 3, &Route::via(&[1, 8, 9]));
let outcome = fixture.table.remove_peer(fixture.peers[0]);
assert_eq!((outcome.removed, outcome.best_changed), (2, 1));
assert_eq!(fixture.held(&p), [(1, 0)]);
}
/// Two routes from one peer that nothing else tells apart: the lower path
/// identifier is chosen, whichever arrived first.
#[test]
fn equal_routes_from_one_peer_are_chosen_by_path_identifier() {
let p = prefix("10.0.0.0/8");
for order in [[5, 3], [3, 5]] {
let fixture = Fixture::new(&[external(1)]);
for path_id in order {
fixture.announce_path(0, &p, path_id, &Route::via(&[1, path_id]));
}
assert_eq!(fixture.best_path(&p), Some((0, 3)), "arriving {order:?}");
}
}
// ---------------------------------------------------------------------------
// The table against a model
// ---------------------------------------------------------------------------
/// Route selection written the plain way: all routes of a prefix, step by
/// step, with no shortcuts. The table takes shortcuts and must agree.
/// Routes are known by peer and path identifier.
fn chosen_plainly(
routes: &BTreeMap<(usize, u32), Route>,
peers: &[PeerInfo],
) -> Option<(usize, u32)> {
let local_pref = |peer: usize, route: &Route| {
if peers[peer].ibgp {
route.local_pref.unwrap_or(DEFAULT_LOCAL_PREF)
} else {
DEFAULT_LOCAL_PREF
}
};
let mut left: Vec<((usize, u32), &Route)> =
routes.iter().map(|(k, route)| (*k, route)).collect();
let most = left.iter().map(|(k, r)| local_pref(k.0, r)).max()?;
left.retain(|(k, r)| local_pref(k.0, r) == most);
let shortest = left.iter().map(|(_, r)| r.path.len()).min()?;
left.retain(|(_, r)| r.path.len() == shortest);
let lowest = left.iter().map(|(_, r)| r.origin).min()?;
left.retain(|(_, r)| r.origin == lowest);
let neighbor = |route: &Route| route.path.first().copied().unwrap_or(0);
let med = |route: &Route| route.med.unwrap_or(0);
let mut lowest_med: HashMap<u32, u32> = HashMap::new();
for (_, route) in &left {
let entry = lowest_med.entry(neighbor(route)).or_insert(u32::MAX);
*entry = (*entry).min(med(route));
}
left.retain(|(_, r)| med(r) == lowest_med[&neighbor(r)]);
left.into_iter()
.map(|(key, _)| key)
.min_by_key(|&(peer, path_id)| {
(
peers[peer].ibgp,
peers[peer].router_id,
peers[peer].address,
peer,
path_id,
)
})
}
#[derive(Debug, Clone)]
enum Op {
Announce {
peer: usize,
prefix: usize,
path_id: u32,
route: Route,
},
Withdraw {
peer: usize,
prefix: usize,
path_id: u32,
},
/// The session goes down and comes straight back.
Flap { peer: usize },
}
const PEERS: usize = 6;
// Few prefixes, so that each gathers routes from most peers.
const PREFIXES: usize = 2;
// Most routes have the identifier zero, as without ADD-PATH; some peers
// send a second and third route.
const PATH_IDS: u32 = 3;
pub(super) fn route() -> impl Strategy<Value = Route> {
// Few values of everything, so that ties at every step are common:
// ties are where the shortcuts could be wrong.
(
proptest::option::of(100u32..102),
proptest::collection::vec(1u32..3, 1..3),
0u8..2,
proptest::option::of(0u32..3),
)
.prop_map(|(local_pref, path, origin, med)| Route {
local_pref,
path,
origin,
med,
})
}
fn path_id() -> impl Strategy<Value = u32> {
prop_oneof![3 => Just(0), 2 => 1..PATH_IDS]
}
fn op() -> impl Strategy<Value = Op> {
prop_oneof![
6 => (0..PEERS, 0..PREFIXES, path_id(), route()).prop_map(
|(peer, prefix, path_id, route)| Op::Announce { peer, prefix, path_id, route }
),
3 => (0..PEERS, 0..PREFIXES, path_id())
.prop_map(|(peer, prefix, path_id)| Op::Withdraw { peer, prefix, path_id }),
1 => (0..PEERS).prop_map(|peer| Op::Flap { peer }),
]
}
proptest! {
#![proptest_config(ProptestConfig::with_cases(2000))]
/// After any sequence of changes the table holds exactly the routes it
/// should, has chosen for every prefix what plain selection chooses,
/// and has reported a change of best route exactly when there was one.
#[test]
fn the_table_agrees_with_plain_selection(ops in proptest::collection::vec(op(), 1..80)) {
// Internal and external peers, and identifiers that are not in the
// order the peers are added in.
let infos: Vec<PeerInfo> = (0..PEERS)
.map(|i| {
let number = (PEERS - i) as u8;
if i % 3 == 0 { internal(number) } else { external(number) }
})
.collect();
let mut fixture = Fixture::new(&infos);
let prefixes: Vec<IpPrefix> = (0..PREFIXES)
.map(|i| prefix(&format!("10.{i}.0.0/16")))
.collect();
let mut model: Vec<BTreeMap<(usize, u32), Route>> = vec![BTreeMap::new(); PREFIXES];
for op in ops {
let before: Vec<Option<((usize, u32), Route)>> = model
.iter()
.map(|routes| chosen_plainly(routes, &infos).map(|key| (key, routes[&key].clone())))
.collect();
let outcome = match &op {
Op::Announce { peer, prefix, path_id, route } => {
model[*prefix].insert((*peer, *path_id), route.clone());
fixture.announce_path(*peer, &prefixes[*prefix], *path_id, route)
}
Op::Withdraw { peer, prefix, path_id } => {
model[*prefix].remove(&(*peer, *path_id));
fixture.withdraw_path(*peer, &prefixes[*prefix], *path_id)
}
Op::Flap { peer } => {
for routes in &mut model {
routes.retain(|(from, _), _| from != peer);
}
let outcome = fixture.table.remove_peer(fixture.peers[*peer]);
fixture.peers[*peer] = fixture.table.add_peer(infos[*peer]).unwrap();
outcome
}
};
let mut best_changes = 0;
for (i, routes) in model.iter().enumerate() {
let expected = chosen_plainly(routes, &infos);
prop_assert_eq!(fixture.best_path(&prefixes[i]), expected, "after {:?}", &op);
let now = expected.map(|key| (key, routes[&key].clone()));
best_changes += usize::from(now != before[i]);
prop_assert_eq!(
fixture.held(&prefixes[i]),
routes.keys().copied().collect::<Vec<_>>()
);
}
prop_assert_eq!(outcome.best_changed, best_changes, "after {:?}", &op);
prop_assert_eq!(
fixture.table.len(),
model.iter().map(BTreeMap::len).sum::<usize>()
);
}
}
}
/// Shards and lock stripes are picked by bits of the hash that the hash
/// table inside does not use, so that keys which share a shard still
/// differ where that table looks.
#[test]
fn keys_of_one_shard_differ_in_the_bits_their_table_uses() {
let hasher = RandomState::default();
let shards = 64;
let mut counts = vec![0usize; shards];
let mut tags = vec![std::collections::HashSet::new(); shards];
for n in 0..64_000u32 {
let hash = hasher.hash_one(n);
let shard = spread(hash, shards);
counts[shard] += 1;
// The seven bits hashbrown keeps as a tag.
tags[shard].insert(hash >> 57);
}
for shard in 0..shards {
assert!(
(700..1300).contains(&counts[shard]),
"shard {shard} has {} of 64000 keys",
counts[shard]
);
assert_eq!(tags[shard].len(), 128, "shard {shard}");
}
assert_eq!(spread(u64::MAX, 3), 2);
assert_eq!(spread(0, 3), 0);
}
/// A peer that sends several routes for a prefix can send more than fit a
/// sixteen-bit index. The best route was kept as one, and wrapped.
#[test]
fn a_row_can_hold_more_routes_than_a_sixteen_bit_index_counts() {
let store = AttrStore::new();
let worse = store
.intern(&Route::via(&[1, 2]).attributes(), None)
.unwrap();
let better = store.intern(&Route::via(&[1]).attributes(), None).unwrap();
let info = external(1);
let peers = [Some(info)];
let peer = PeerIndex(0);
let path =
|attrs: &Arc<AttrSet>| Path::new(select::preference(attrs.key(), &info), peer, attrs);
let seqs = journal::Journals::new(journal::Limits::default()).appeared(prefix("10.0.0.0/8"));
let mut row = Row::new(prefix("10.0.0.0/8"), path(&worse), 1, seqs);
let mut scratch = select::Scratch::default();
let count = usize::from(u16::MAX) + 10;
for path_id in 2..=count as u32 {
row.push(path(&worse), path_id);
}
// Equal routes from one peer: the lowest identifier is the best.
assert!(!select::after_add(&mut row, &peers, &mut scratch));
assert_eq!(row.best(), 0);
row.push(path(&better), count as u32 + 1);
assert!(select::after_add(&mut row, &peers, &mut scratch));
assert_eq!(row.best(), count);
assert_eq!(row.route(row.best()).path_id, count as u32 + 1);
// With the better one gone, the choice among all the rest is the
// lowest identifier again.
assert!(select::after_remove(&mut row, count, &peers, &mut scratch));
assert_eq!(row.route(row.best()).path_id, 1);
}
crates/server/Cargo.toml
+10 -3
View File
@@ -1,6 +1,11 @@
[package]
name = "bgp-server"
authors.workspace = true
edition.workspace = true
homepage.workspace = true
license.workspace = true
repository.workspace = true
version.workspace = true
[dependencies]
bgp-packet.workspace = true
@@ -9,9 +14,8 @@ bgp-rib.workspace = true
bytes.workspace = true
clap.workspace = true
futures.workspace = true
libc.workspace = true
eyre.workspace = true
prometheus.workspace = true
socket2.workspace = true
tokio.workspace = true
tracing-subscriber.workspace = true
tracing.workspace = true
@@ -22,4 +26,7 @@ thiserror.workspace = true
protobuf-codegen.workspace = true
[dev-dependencies]
tokio = { workspace = true, features = ["test-util"] }
proptest.workspace = true
[lints]
workspace = true
crates/server/build.rs
+1 -1
View File
@@ -6,7 +6,7 @@ fn main() {
// on the machine (routers and minimal containers rarely have one).
.pure()
.input("proto/config.proto")
.includes(&["proto"])
.includes(["proto"])
.cargo_out_dir("protos")
.run_from_script();
}
crates/server/proto/config.proto
+50 -25
View File
@@ -18,8 +18,8 @@ message BgpConfig {
// Peers configured on this server.
repeated Peer peers = 4;
// Limits on routes originated locally (e.g. via the demo UI).
LocalRoutePolicy local_route_policy = 5;
// Routes this router originates and announces to every peer.
repeated OriginatedRoute originate = 5;
// Where to serve the HTTP API, status page and metrics. The port is
// required. Unset disables the HTTP server.
@@ -37,33 +37,29 @@ message BgpConfig {
// restarted take over from a session we have not yet noticed is dead,
// instead of being refused until the hold timer expires.
bool collision_detect_established_state = 8;
// Watch for pings (ICMPv6 echo requests) sent to the lab prefixes and show
// them on the status page. Unset disables it.
PingMonitor ping_monitor = 9;
}
// Shows who is pinging addresses that this host answers for.
//
// The host must already own the addresses, e.g. through an AnyIP route
// ("ip -6 route add local 2001:db8::/40 dev lo"); the kernel keeps answering
// the pings and we only receive a copy. Opening the raw ICMPv6 socket this
// needs takes CAP_NET_RAW on Linux. IPv6 only.
message PingMonitor {
// Only pings to destinations inside these prefixes are shown. Defaults to
// the IPv6 prefixes in local_route_policy.allowed_prefixes.
repeated string prefixes = 1;
}
// A route originated by this router. It is announced with an AS_PATH of the
// local AS and, on each session, that session's local address as next hop.
message OriginatedRoute {
// The prefix in CIDR notation, e.g. "192.0.2.0/24" or "2001:db8::/32".
string prefix = 1;
// Limits on locally originated routes.
message LocalRoutePolicy {
// A locally originated route is accepted only if it is covered by one of
// these prefixes, written in CIDR notation (e.g. "10.100.0.0/16").
// Empty means any prefix may be announced.
repeated string allowed_prefixes = 1;
// ORIGIN attribute: "IGP" (the default), "EGP" or "INCOMPLETE".
string origin = 2;
// Maximum number of routes a single owner may announce. 0 means unlimited.
uint32 max_routes_per_owner = 2;
// Extra copies of the local AS to prepend to the AS_PATH. At most 16.
uint32 as_path_prepend = 3;
// MULTI_EXIT_DISC. Left out of the announcement when unset.
optional uint32 med = 4;
// Standard communities (RFC 1997): "ASN:VALUE", or a well-known name such
// as "NO_EXPORT".
repeated string communities = 5;
// Large communities (RFC 8092): "GLOBAL:LOCAL1:LOCAL2".
repeated string large_communities = 6;
}
// An IP address and port pair.
@@ -95,4 +91,33 @@ message Peer {
// same machine. Must be the same address family as the endpoint. Unset
// lets the operating system choose.
string local_address = 5;
// The address families to exchange with this peer. Each is offered to the
// peer in our OPEN, and is in use on the session only if the peer offers
// it too (RFC 4760). Defaults to all of them when empty. A session with
// no family in common is refused.
repeated AddressFamily address_families = 6;
// ADD-PATH (RFC 7911): whether to take several routes for one prefix
// from this peer, to send it several, or both. It is in use, for each
// direction and address family, only if the peer agrees. Defaults to
// off: one route for each prefix, as in plain BGP.
AddPath add_path = 7;
}
// The directions ADD-PATH can be offered for.
enum AddPath {
ADD_PATH_OFF = 0;
// Accept several routes for a prefix from the peer.
ADD_PATH_RECEIVE = 1;
// Send the peer several routes for a prefix.
ADD_PATH_SEND = 2;
ADD_PATH_BOTH = 3;
}
// An address family routes can be exchanged for.
enum AddressFamily {
ADDRESS_FAMILY_UNSPECIFIED = 0;
IPV4_UNICAST = 1;
IPV6_UNICAST = 2;
}
crates/server/src/advertise.rs
+383 -121
View File
@@ -10,172 +10,283 @@
use std::net::IpAddr;
use bgp_packet::message::{
AsPathAttribute, CommunitiesPathAttribute, LargeCommunitiesPathAttribute, Message,
MpReachNlriPathAttribute, MpUnreachNlriPathAttribute, MultiExitDiscPathAttribute,
NextHopPathAttribute, NlriNextHop, PathAttribute, UpdateMessage,
AsPathAttribute, AsPathSegment, CommunitiesPathAttribute, LargeCommunitiesPathAttribute,
LocalPrefPathAttribute, Message, MpReachNlriPathAttribute, MpUnreachNlriPathAttribute,
MultiExitDiscPathAttribute, NextHopPathAttribute, NlriNextHop, PathAttribute, UpdateMessage,
};
use bgp_primitives::constants::{AddressFamilyId, SubsequentAfi};
use bgp_primitives::ip_prefix::IpPrefix;
use bgp_rib::{LocalRib, LocalRibEvent, LocalRoute};
use bgp_primitives::constants::{AfiSafi, IpVersion};
use bgp_primitives::ip_prefix::{IpPrefix, Nlri};
use bgp_rib::table::DEFAULT_LOCAL_PREF;
use bgp_rib::{LocalRib, LocalRibEvent, LocalRoute, RibStopped};
use tokio::sync::broadcast::Receiver;
use tokio::sync::broadcast::error::RecvError;
use tokio::task::JoinHandle;
use tracing::{debug, info, warn};
use crate::session::{SessionEvent, SessionHandle, SessionRegistry};
/// Starts the advertiser. It hears of every session that comes up after
/// this returns, so call it before anything that can bring one up.
pub fn spawn_advertiser(
local_rib: LocalRib,
registry: SessionRegistry,
local_asn: u32,
) -> JoinHandle<()> {
tokio::spawn(run(local_rib, registry, local_asn))
let advertiser = Advertiser {
rib_events: local_rib.subscribe(),
session_events: registry.subscribe(),
local_rib,
registry,
local_asn,
};
tokio::spawn(advertiser.run())
}
async fn run(local_rib: LocalRib, registry: SessionRegistry, local_asn: u32) {
let mut rib_events = local_rib.subscribe();
let mut session_events = registry.subscribe();
loop {
tokio::select! {
ev = rib_events.recv() => match ev {
Ok(LocalRibEvent::Announced(route)) => {
for session in registry.list() {
send_route(&session, &route, local_asn).await;
}
}
Ok(LocalRibEvent::Withdrawn { prefix }) => {
for session in registry.list() {
send_withdraw(&session, &prefix).await;
}
}
Err(RecvError::Lagged(n)) => {
warn!(missed = n, "Advertiser lagged on local RIB events, resyncing all sessions");
resync_all(&local_rib, &registry, local_asn).await;
}
Err(RecvError::Closed) => return,
},
ev = session_events.recv() => match ev {
Ok(SessionEvent::Up(session)) => {
let routes = local_rib.snapshot().await;
info!(peer = %session.peer, routes = routes.len(), "Session up, sending local routes");
for route in &routes {
send_route(&session, route, local_asn).await;
}
}
Ok(SessionEvent::RefreshRequested(session)) => {
let routes = local_rib.snapshot().await;
info!(peer = %session.peer, routes = routes.len(), "Route refresh requested, re-sending local routes");
for route in &routes {
send_route(&session, route, local_asn).await;
}
}
Ok(SessionEvent::Down(_)) => {}
Err(RecvError::Lagged(n)) => {
warn!(missed = n, "Advertiser lagged on session events, resyncing all sessions");
resync_all(&local_rib, &registry, local_asn).await;
}
Err(RecvError::Closed) => return,
},
}
}
struct Advertiser {
local_rib: LocalRib,
registry: SessionRegistry,
local_asn: u32,
rib_events: Receiver<LocalRibEvent>,
session_events: Receiver<SessionEvent>,
}
async fn resync_all(local_rib: &LocalRib, registry: &SessionRegistry, local_asn: u32) {
let routes = local_rib.snapshot().await;
for session in registry.list() {
for route in &routes {
send_route(&session, route, local_asn).await;
}
}
}
async fn send_route(session: &SessionHandle, route: &LocalRoute, local_asn: u32) {
match build_announce(route, local_asn, session.local_addr.ip()) {
Some(update) => {
debug!(peer = %session.peer, prefix = %route.prefix, "Announcing local route");
if let Err(e) = session.send(Message::Update(update)).await {
warn!(peer = %session.peer, err = %e, "Failed to queue announce");
impl Advertiser {
/// Runs until the local RIB stops. The server watches for this task
/// ending: without it nothing is announced.
async fn run(mut self) {
loop {
let handled = tokio::select! {
event = self.rib_events.recv() => self.on_route_event(event).await,
event = self.session_events.recv() => self.on_session_event(event).await,
};
if handled.is_err() {
return;
}
}
None => debug!(
peer = %session.peer,
prefix = %route.prefix,
"Skipping route: address family does not match session transport"
),
}
async fn on_route_event(
&self,
event: Result<LocalRibEvent, RecvError>,
) -> Result<(), RibStopped> {
match event {
Ok(LocalRibEvent::Announced(route)) => {
for session in self.registry.list() {
self.send_route(&session, &route).await;
}
Ok(())
}
Ok(LocalRibEvent::Withdrawn { prefix }) => {
for session in self.registry.list() {
send_withdraw(&session, &prefix).await;
}
Ok(())
}
Err(RecvError::Lagged(missed)) => {
warn!(
missed,
"Advertiser lagged on local RIB events, resyncing all sessions"
);
self.send_all_to(&self.registry.list()).await
}
Err(RecvError::Closed) => Err(RibStopped { rib: "local RIB" }),
}
}
async fn on_session_event(
&self,
event: Result<SessionEvent, RecvError>,
) -> Result<(), RibStopped> {
match event {
Ok(SessionEvent::Up(session)) => {
info!(peer = %session.peer, "Session up, sending local routes");
self.send_all_to(&[session]).await
}
Ok(SessionEvent::RefreshRequested(session)) => {
info!(peer = %session.peer, "Route refresh requested, re-sending local routes");
self.send_all_to(&[session]).await
}
Ok(SessionEvent::Down(_)) => Ok(()),
Err(RecvError::Lagged(missed)) => {
warn!(
missed,
"Advertiser lagged on session events, resyncing all sessions"
);
self.send_all_to(&self.registry.list()).await
}
// The registry outlives the server's tasks; nothing more will
// come up, and nothing more needs sending.
Err(RecvError::Closed) => Err(RibStopped { rib: "local RIB" }),
}
}
/// Sends every originated route to each of `sessions`.
async fn send_all_to(&self, sessions: &[SessionHandle]) -> Result<(), RibStopped> {
let routes = self.local_rib.snapshot().await?;
for session in sessions {
for route in &routes {
self.send_route(session, route).await;
}
}
Ok(())
}
async fn send_route(&self, session: &SessionHandle, route: &LocalRoute) {
if !session.families.carries(&route.prefix) {
debug!(
peer = %session.peer,
prefix = %route.prefix,
"Skipping route: its address family was not negotiated"
);
return;
}
let towards = Towards {
local_asn: self.local_asn,
next_hop: session.local_addr.ip(),
four_octet_asn: session.four_octet_asn,
ibgp: session.ibgp,
};
match build_announce(route, &towards) {
Some(update) => {
debug!(peer = %session.peer, prefix = %route.prefix, "Announcing local route");
if let Err(e) = session.send(Message::Update(update)).await {
warn!(peer = %session.peer, err = %e, "Failed to queue announce");
}
}
None => debug!(
peer = %session.peer,
prefix = %route.prefix,
"Skipping route: address family does not match session transport"
),
}
}
}
async fn send_withdraw(session: &SessionHandle, prefix: &IpPrefix) {
if !session.families.carries(prefix) {
// It was never announced on this session either.
return;
}
debug!(peer = %session.peer, %prefix, "Withdrawing local route");
if let Err(e) = session.send(Message::Update(build_withdraw(prefix))).await {
warn!(peer = %session.peer, err = %e, "Failed to queue withdraw");
}
}
/// Builds the UPDATE announcing `route` with `next_hop` as the next hop.
/// What about a session decides how a route is announced on it.
#[derive(Debug, Clone, Copy)]
pub struct Towards {
pub local_asn: u32,
/// Our address on the session, which is the routes' next hop.
pub next_hop: IpAddr,
/// Whether the session negotiated four-octet AS numbers. If it did
/// not, and an AS number in the path needs four octets, the path is
/// also sent as AS4_PATH so that it survives the two-octet speaker
/// (RFC 6793 §4.2.2).
pub four_octet_asn: bool,
/// Whether the peer is in our own AS.
pub ibgp: bool,
}
/// Builds the UPDATE announcing `route` on a session.
///
/// Returns `None` when the prefix family does not match the next-hop family:
/// carrying IPv4 NLRI over an IPv6 next hop (RFC 8950) or vice versa is not
/// supported yet.
pub fn build_announce(route: &LocalRoute, local_asn: u32, next_hop: IpAddr) -> Option<UpdateMessage> {
pub fn build_announce(route: &LocalRoute, towards: &Towards) -> Option<UpdateMessage> {
let attrs = &route.attrs;
// Our AS number goes on the path as a route leaves the AS, and not
// before (RFC 4271 §5.1.2): a route we originate has an empty path
// inside it. A peer in our AS that found our AS number on the path
// would take the route for one that had looped.
let as_path = AsPathAttribute {
segments: if towards.ibgp {
vec![]
} else {
vec![AsPathSegment {
ordered: true,
path: vec![towards.local_asn; 1 + usize::from(attrs.prepend)],
}]
},
};
let as4_path = if towards.four_octet_asn {
None
} else {
as_path.as4_path()
};
// Path attributes in ascending type-code order, as most implementations
// expect them.
let mut path_attributes = vec![
PathAttribute::Origin(attrs.origin),
AsPathAttribute::from_asns(vec![local_asn; 1 + attrs.prepend as usize]),
PathAttribute::AsPath(as_path),
];
// IPv4 routes go in the body of the UPDATE with their next hop in
// NEXT_HOP; IPv6 routes go in MP_REACH_NLRI, which carries its own.
let mut announced_routes = vec![];
match (route.prefix.address_family, next_hop) {
(AddressFamilyId::Ipv4, IpAddr::V4(nh)) => {
path_attributes.push(PathAttribute::NextHop(NextHopPathAttribute(nh)));
announced_routes.push(route.prefix.clone());
let mp_reach = match (route.prefix.address_family(), towards.next_hop) {
(IpVersion::Ipv4, IpAddr::V4(next_hop)) => {
path_attributes.push(PathAttribute::NextHop(NextHopPathAttribute(next_hop)));
announced_routes.push(route.prefix.into());
None
}
(IpVersion::Ipv6, IpAddr::V6(next_hop)) => {
Some(PathAttribute::MpReachNlri(MpReachNlriPathAttribute {
family: AfiSafi::IPV6_UNICAST,
next_hop: NlriNextHop::Ipv6(next_hop),
prefixes: vec![route.prefix.into()],
}))
}
(AddressFamilyId::Ipv6, IpAddr::V6(_)) => {}
_ => return None,
}
};
if let Some(med) = attrs.med {
path_attributes.push(PathAttribute::MultiExitDisc(MultiExitDiscPathAttribute(med)));
path_attributes.push(PathAttribute::MultiExitDisc(MultiExitDiscPathAttribute(
med,
)));
}
// LOCAL_PREF is sent to peers in our AS and to no others (RFC 4271
// §5.1.5).
if towards.ibgp {
path_attributes.push(PathAttribute::LocalPref(LocalPrefPathAttribute(
DEFAULT_LOCAL_PREF,
)));
}
if !attrs.communities.is_empty() {
path_attributes.push(PathAttribute::Communities(CommunitiesPathAttribute(
attrs.communities.clone(),
)));
}
if let (AddressFamilyId::Ipv6, IpAddr::V6(nh)) = (route.prefix.address_family, next_hop) {
path_attributes.push(PathAttribute::MpReachNlri(MpReachNlriPathAttribute {
afi: AddressFamilyId::Ipv6,
safi: SubsequentAfi::Unicast,
next_hop: NlriNextHop::Ipv6(nh),
prefixes: vec![route.prefix.clone()],
}));
}
path_attributes.extend(mp_reach);
path_attributes.extend(as4_path);
if !attrs.large_communities.is_empty() {
path_attributes.push(PathAttribute::LargeCommunities(LargeCommunitiesPathAttribute {
communities: attrs.large_communities.clone(),
}));
path_attributes.push(PathAttribute::LargeCommunities(
LargeCommunitiesPathAttribute {
communities: attrs.large_communities.clone(),
},
));
}
Some(UpdateMessage { withdrawn_routes: vec![], path_attributes, announced_routes })
Some(UpdateMessage {
withdrawn_routes: vec![],
path_attributes,
announced_routes,
})
}
/// Builds the UPDATE withdrawing `prefix`.
pub fn build_withdraw(prefix: &IpPrefix) -> UpdateMessage {
match prefix.address_family {
AddressFamilyId::Ipv4 => UpdateMessage {
withdrawn_routes: vec![prefix.clone()],
match prefix.address_family() {
IpVersion::Ipv4 => UpdateMessage {
withdrawn_routes: vec![Nlri::from(*prefix)],
path_attributes: vec![],
announced_routes: vec![],
},
AddressFamilyId::Ipv6 => UpdateMessage {
IpVersion::Ipv6 => UpdateMessage {
withdrawn_routes: vec![],
path_attributes: vec![PathAttribute::MpUnreachNlri(MpUnreachNlriPathAttribute {
afi: AddressFamilyId::Ipv6,
safi: SubsequentAfi::Unicast,
prefixes: vec![prefix.clone()],
family: AfiSafi::IPV6_UNICAST,
prefixes: vec![Nlri::from(*prefix)],
})],
announced_routes: vec![],
},
@@ -197,20 +308,37 @@ mod tests {
IpPrefix::try_from(s).unwrap()
}
/// An external peer.
fn towards(local_asn: u32, next_hop: IpAddr, four_octet_asn: bool) -> Towards {
Towards {
local_asn,
next_hop,
four_octet_asn,
ibgp: false,
}
}
#[test]
fn ipv4_announce_has_expected_attributes_and_roundtrips() {
let route = LocalRoute::new(
prefix("10.100.1.0/24"),
"alice",
LocalRouteAttrs {
origin: OriginPathAttribute::IGP,
prepend: 2,
med: Some(50),
communities: vec![(65000, 100), (65535, 65281)],
large_communities: vec![LargeCommunity { global_admin: 65000, data_1: 1, data_2: 2 }],
large_communities: vec![LargeCommunity {
global_admin: 65000,
data_1: 1,
data_2: 2,
}],
},
);
let update = build_announce(&route, 65000, IpAddr::V4(Ipv4Addr::new(192, 0, 2, 1))).unwrap();
let update = build_announce(
&route,
&towards(65000, IpAddr::V4(Ipv4Addr::new(192, 0, 2, 1)), true),
)
.unwrap();
assert_eq!(update.announced_routes, vec![prefix("10.100.1.0/24")]);
assert!(update.withdrawn_routes.is_empty());
@@ -221,31 +349,48 @@ mod tests {
AsPathAttribute::from_asns(vec![65000, 65000, 65000]),
PathAttribute::NextHop(NextHopPathAttribute(Ipv4Addr::new(192, 0, 2, 1))),
PathAttribute::MultiExitDisc(MultiExitDiscPathAttribute(50)),
PathAttribute::Communities(CommunitiesPathAttribute(vec![(65000, 100), (65535, 65281)])),
PathAttribute::Communities(CommunitiesPathAttribute(vec![
(65000, 100),
(65535, 65281)
])),
PathAttribute::LargeCommunities(LargeCommunitiesPathAttribute {
communities: vec![LargeCommunity { global_admin: 65000, data_1: 1, data_2: 2 }],
communities: vec![LargeCommunity {
global_admin: 65000,
data_1: 1,
data_2: 2
}],
}),
]
);
// Serialise and parse back to make sure the wire encoding is sound.
let ctx = ParserContext { four_octet_asn: Some(true), address_family: None };
let ctx = ParserContext {
four_octet_asn: true,
ibgp: false,
add_path: Vec::new(),
};
let msg = Message::Update(update.clone());
let mut buf = BytesMut::new();
msg.to_wire(&ctx, &mut buf).unwrap();
let (_, parsed) = Message::from_wire(&ctx, &buf).unwrap();
let Message::Update(parsed) = parsed else { panic!("expected UPDATE") };
let Message::Update(parsed) = Message::decode(&ctx, &buf).unwrap() else {
panic!("expected UPDATE")
};
assert_eq!(parsed, update);
}
#[test]
fn ipv6_announce_uses_mp_reach() {
let route = LocalRoute::new(prefix("2001:db8:1::/48"), "alice", LocalRouteAttrs::default());
let route = LocalRoute::new(prefix("2001:db8:1::/48"), LocalRouteAttrs::default());
let nh = Ipv6Addr::new(0x2001, 0xdb8, 0, 0, 0, 0, 0, 1);
let update = build_announce(&route, 65000, IpAddr::V6(nh)).unwrap();
let update = build_announce(&route, &towards(65000, IpAddr::V6(nh), true)).unwrap();
assert!(update.announced_routes.is_empty());
assert!(!update.path_attributes.iter().any(|a| matches!(a, PathAttribute::NextHop(_))));
assert!(
!update
.path_attributes
.iter()
.any(|a| matches!(a, PathAttribute::NextHop(_)))
);
assert!(update.path_attributes.iter().any(|a| matches!(
a,
PathAttribute::MpReachNlri(MpReachNlriPathAttribute { next_hop: NlriNextHop::Ipv6(h), prefixes, .. })
@@ -253,12 +398,62 @@ mod tests {
)));
}
/// A two-octet session cannot carry a four-octet AS number in AS_PATH:
/// it goes out as AS_TRANS there, and in full in AS4_PATH.
#[test]
fn a_wide_as_number_reaches_a_two_octet_peer_in_as4_path() {
let route = LocalRoute::new(
prefix("10.100.1.0/24"),
LocalRouteAttrs {
prepend: 1,
..LocalRouteAttrs::default()
},
);
let next_hop = IpAddr::V4(Ipv4Addr::new(192, 0, 2, 1));
let path = |update: &UpdateMessage, as4: bool| {
update.path_attributes.iter().find_map(|a| match a {
PathAttribute::As4Path(p) if as4 => Some(p.segments[0].path.clone()),
PathAttribute::AsPath(p) if !as4 => Some(p.segments[0].path.clone()),
_ => None,
})
};
let update = build_announce(&route, &towards(4_200_000_001, next_hop, false)).unwrap();
assert_eq!(path(&update, true), Some(vec![4_200_000_001; 2]));
// What the peer reads from the wire.
let ctx = ParserContext {
four_octet_asn: false,
ibgp: false,
add_path: Vec::new(),
};
let mut buf = BytesMut::new();
Message::Update(update).to_wire(&ctx, &mut buf).unwrap();
let Message::Update(seen) = Message::decode(&ctx, &buf).unwrap() else {
panic!("expected UPDATE")
};
assert_eq!(path(&seen, false), Some(vec![23456; 2]), "AS_TRANS");
assert_eq!(path(&seen, true), Some(vec![4_200_000_001; 2]));
// No AS4_PATH when it would say nothing new: the peer handles
// four-octet numbers, or ours fits in two.
let update = build_announce(&route, &towards(4_200_000_001, next_hop, true)).unwrap();
assert_eq!(path(&update, true), None);
let update = build_announce(&route, &towards(65000, next_hop, false)).unwrap();
assert_eq!(path(&update, true), None);
assert_eq!(path(&update, false), Some(vec![65000; 2]));
}
#[test]
fn family_mismatch_is_skipped() {
let v6 = LocalRoute::new(prefix("2001:db8::/32"), "a", LocalRouteAttrs::default());
assert!(build_announce(&v6, 65000, IpAddr::V4(Ipv4Addr::LOCALHOST)).is_none());
let v4 = LocalRoute::new(prefix("10.0.0.0/8"), "a", LocalRouteAttrs::default());
assert!(build_announce(&v4, 65000, IpAddr::V6(Ipv6Addr::LOCALHOST)).is_none());
let v6 = LocalRoute::new(prefix("2001:db8::/32"), LocalRouteAttrs::default());
assert!(
build_announce(&v6, &towards(65000, IpAddr::V4(Ipv4Addr::LOCALHOST), true)).is_none()
);
let v4 = LocalRoute::new(prefix("10.0.0.0/8"), LocalRouteAttrs::default());
assert!(
build_announce(&v4, &towards(65000, IpAddr::V6(Ipv6Addr::LOCALHOST), true)).is_none()
);
}
#[test]
@@ -269,6 +464,73 @@ mod tests {
let v6 = build_withdraw(&prefix("2001:db8::/32"));
assert!(v6.withdrawn_routes.is_empty());
assert!(matches!(&v6.path_attributes[..], [PathAttribute::MpUnreachNlri(m)] if m.prefixes == vec![prefix("2001:db8::/32")]));
assert!(
matches!(&v6.path_attributes[..], [PathAttribute::MpUnreachNlri(m)] if m.prefixes == vec![prefix("2001:db8::/32")])
);
}
/// Inside our own AS a route we originate has an empty AS path and a
/// LOCAL_PREF. With our AS number on the path, a peer in our AS takes
/// the route for a loop; without LOCAL_PREF it treats the route as
/// withdrawn, as our own decoder does.
#[test]
fn a_route_announced_inside_our_as_has_an_empty_path_and_a_local_pref() {
let route = LocalRoute::new(
prefix("10.100.1.0/24"),
LocalRouteAttrs {
prepend: 3,
..LocalRouteAttrs::default()
},
);
let internal = Towards {
ibgp: true,
..towards(65000, IpAddr::V4(Ipv4Addr::new(192, 0, 2, 1)), true)
};
let update = build_announce(&route, &internal).unwrap();
assert_eq!(
update.path_attributes,
vec![
PathAttribute::Origin(OriginPathAttribute::IGP),
PathAttribute::AsPath(AsPathAttribute { segments: vec![] }),
PathAttribute::NextHop(NextHopPathAttribute(Ipv4Addr::new(192, 0, 2, 1))),
PathAttribute::LocalPref(LocalPrefPathAttribute(100)),
]
);
// An internal peer accepts it as it stands.
let ctx = ParserContext {
four_octet_asn: true,
ibgp: true,
add_path: Vec::new(),
};
let mut buf = BytesMut::new();
Message::Update(update.clone())
.to_wire(&ctx, &mut buf)
.unwrap();
let decoded = Message::decode_with_report(&ctx, &buf).unwrap();
assert!(
decoded.update_errors.is_empty(),
"{:?}",
decoded.update_errors
);
assert_eq!(decoded.message, Message::Update(update));
// To an external peer the same route has our AS on the path, as
// many times as configured, and no LOCAL_PREF.
let external = Towards {
ibgp: false,
..internal
};
let update = build_announce(&route, &external).unwrap();
assert_eq!(
update.path_attributes[1],
AsPathAttribute::from_asns(vec![65000; 4])
);
assert!(
!update
.path_attributes
.iter()
.any(|a| matches!(a, PathAttribute::LocalPref(_)))
);
}
}
crates/server/src/collision.rs
+119 -30
View File
@@ -8,9 +8,14 @@
//! This cannot be a check local to one connection: the rival connection is
//! usually still mid-handshake (so it is not in the session registry yet),
//! and sometimes it is the rival, running in another task, that must close.
//! Hence one small shared map, keyed by the peer's BGP Identifier, holding
//! the single connection currently preferred for that peer and a way to
//! tell it to stop.
//! Hence one small shared map, holding the single connection currently
//! preferred for each peer and a way to tell it to stop.
//!
//! A peer here is an address together with the BGP Identifier it announced.
//! The identifier alone would not do: a router commonly has one session
//! over IPv4 and another over IPv6 with the same neighbour, which are two
//! sessions and not a collision; and it would let any host that can reach
//! us displace a peer's connection by claiming the peer's identifier.
//!
//! A collision with an *Established* connection closes the new one, unless
//! `detect_established` is set (the RFC's optional
@@ -19,13 +24,14 @@
//! by the same side the newer wins: the initiator has abandoned the older.
use std::collections::HashMap;
use std::net::Ipv4Addr;
use std::net::{IpAddr, Ipv4Addr};
use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::{Arc, Mutex};
use tokio::sync::oneshot;
use crate::metrics::COLLISIONS;
use crate::sync::IgnorePoison;
/// Which side opened the TCP connection.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
@@ -67,31 +73,56 @@ fn new_loses(
return false;
}
// Identifiers compare as unsigned 32-bit integers.
let keep = if u32::from(local_id) > u32::from(remote_id) { Direction::Outbound } else { Direction::Inbound };
let keep = if u32::from(local_id) > u32::from(remote_id) {
Direction::Outbound
} else {
Direction::Inbound
};
new != keep
}
/// Who a connection is with: the address at the other end, and the BGP
/// Identifier announced from it.
type Peer = (IpAddr, Ipv4Addr);
/// Shared by every connection of one server.
#[derive(Clone)]
pub struct CollisionTable {
entries: Arc<Mutex<HashMap<Ipv4Addr, Entry>>>,
entries: Arc<Mutex<HashMap<Peer, Entry>>>,
next_id: Arc<AtomicU64>,
detect_established: bool,
}
impl CollisionTable {
pub fn new(detect_established: bool) -> Self {
Self { entries: Default::default(), next_id: Default::default(), detect_established }
Self {
entries: Default::default(),
next_id: Default::default(),
detect_established,
}
}
/// Call when a connection receives the peer's OPEN. `None` means this
/// connection lost and must be closed with Cease / Connection Collision
/// Resolution. Otherwise any rival has been told to close and this
/// connection holds the slot until the guard drops.
pub fn admit(&self, local_id: Ipv4Addr, remote_id: Ipv4Addr, direction: Direction) -> Option<ConnectionGuard> {
let mut entries = self.entries.lock().expect("collision table poisoned");
if let Some(existing) = entries.get_mut(&remote_id) {
if new_loses(local_id, remote_id, existing, direction, self.detect_established) {
pub fn admit(
&self,
local_id: Ipv4Addr,
remote_addr: IpAddr,
remote_id: Ipv4Addr,
direction: Direction,
) -> Option<ConnectionGuard> {
let peer = (remote_addr, remote_id);
let mut entries = self.entries.locked();
if let Some(existing) = entries.get_mut(&peer) {
if new_loses(
local_id,
remote_id,
existing,
direction,
self.detect_established,
) {
COLLISIONS.with_label_values(&["new"]).inc();
return None;
}
@@ -102,37 +133,58 @@ impl CollisionTable {
}
let id = self.next_id.fetch_add(1, Ordering::Relaxed);
let (kill, killed) = oneshot::channel();
entries.insert(remote_id, Entry { id, direction, established: false, kill: Some(kill) });
Some(ConnectionGuard { table: self.clone(), remote_id, id, killed })
entries.insert(
peer,
Entry {
id,
direction,
established: false,
kill: Some(kill),
},
);
Some(ConnectionGuard {
table: self.clone(),
peer,
id,
killed,
})
}
}
/// Holds a connection's slot in the table; releases it on drop.
pub struct ConnectionGuard {
table: CollisionTable,
remote_id: Ipv4Addr,
peer: Peer,
id: u64,
/// Completes when a later connection won against this one; the owner
/// must then send Cease / Connection Collision Resolution and close.
pub killed: oneshot::Receiver<()>,
killed: oneshot::Receiver<()>,
}
impl ConnectionGuard {
pub fn set_established(&self) {
let mut entries = self.table.entries.lock().expect("collision table poisoned");
if let Some(e) = entries.get_mut(&self.remote_id).filter(|e| e.id == self.id) {
let mut entries = self.table.entries.locked();
if let Some(e) = entries.get_mut(&self.peer).filter(|e| e.id == self.id) {
e.established = true;
}
}
/// Completes when a later connection has won against this one; the
/// owner must then send Cease / Connection Collision Resolution and
/// close. Cancel-safe.
pub async fn killed(&mut self) {
// The sending half is dropped only when it has been used: the entry
// that holds it is removed by this guard alone, or replaced by a
// connection that signals first. Either way, the wait is over.
let _ = (&mut self.killed).await;
}
}
impl Drop for ConnectionGuard {
fn drop(&mut self) {
// A connection that lost has already been replaced in the map; only
// remove the slot if it is still ours.
let mut entries = self.table.entries.lock().expect("collision table poisoned");
if entries.get(&self.remote_id).is_some_and(|e| e.id == self.id) {
entries.remove(&self.remote_id);
let mut entries = self.table.entries.locked();
if entries.get(&self.peer).is_some_and(|e| e.id == self.id) {
entries.remove(&self.peer);
}
}
}
@@ -145,8 +197,20 @@ mod tests {
const LOW: Ipv4Addr = Ipv4Addr::new(10, 0, 0, 1);
const HIGH: Ipv4Addr = Ipv4Addr::new(192, 0, 2, 1);
fn loses(local: Ipv4Addr, remote: Ipv4Addr, existing: Direction, established: bool, new: Direction, detect: bool) -> bool {
let e = Entry { id: 0, direction: existing, established, kill: None };
fn loses(
local: Ipv4Addr,
remote: Ipv4Addr,
existing: Direction,
established: bool,
new: Direction,
detect: bool,
) -> bool {
let e = Entry {
id: 0,
direction: existing,
established,
kill: None,
};
new_loses(local, remote, &e, new, detect)
}
@@ -169,7 +233,10 @@ mod tests {
for (a, b) in [(LOW, HIGH), (HIGH, LOW)] {
let a_closes_dial_from_b = loses(a, b, Outbound, false, Inbound, false);
let b_closes_dial_from_a = loses(b, a, Outbound, false, Inbound, false);
assert_ne!(a_closes_dial_from_b, b_closes_dial_from_a, "ends disagree for a={a} b={b}");
assert_ne!(
a_closes_dial_from_b, b_closes_dial_from_a,
"ends disagree for a={a} b={b}"
);
}
}
@@ -196,20 +263,42 @@ mod tests {
#[tokio::test]
async fn table_signals_loser_and_releases_slots() {
let table = CollisionTable::new(false);
let addr: IpAddr = Ipv4Addr::new(192, 0, 2, 1).into();
// We are LOW: outbound first, then the peer's inbound wins.
let mut out = table.admit(LOW, HIGH, Outbound).unwrap();
let inb = table.admit(LOW, HIGH, Inbound).unwrap();
let mut out = table.admit(LOW, addr, HIGH, Outbound).unwrap();
let inb = table.admit(LOW, addr, HIGH, Inbound).unwrap();
assert!(out.killed.try_recv().is_ok());
// The loser's guard dropping must not free the winner's slot.
drop(out);
inb.set_established();
assert!(table.admit(LOW, HIGH, Outbound).is_none());
assert!(table.admit(LOW, addr, HIGH, Outbound).is_none());
// A different identifier never collides.
assert!(table.admit(LOW, Ipv4Addr::new(192, 0, 2, 2), Inbound).is_some());
assert!(
table
.admit(LOW, addr, Ipv4Addr::new(192, 0, 2, 2), Inbound)
.is_some()
);
// Once the survivor goes away the identifier is free again.
drop(inb);
assert!(table.admit(LOW, HIGH, Outbound).is_some());
assert!(table.admit(LOW, addr, HIGH, Outbound).is_some());
}
/// A neighbour reached over IPv4 and over IPv6 announces the same
/// identifier on both sessions. They are two sessions, and neither
/// closes the other. Nor can a connection from elsewhere that claims a
/// peer's identifier take the place of the peer's own.
#[tokio::test]
async fn the_same_identifier_from_another_address_is_another_peer() {
let table = CollisionTable::new(true);
let v4: IpAddr = Ipv4Addr::new(192, 0, 2, 1).into();
let v6: IpAddr = "2001:db8::1".parse().unwrap();
let mut over_v4 = table.admit(LOW, v4, HIGH, Outbound).unwrap();
over_v4.set_established();
let over_v6 = table.admit(LOW, v6, HIGH, Inbound);
assert!(over_v6.is_some());
assert!(over_v4.killed.try_recv().is_err(), "the first is untouched");
}
}
crates/server/src/config.rs
+412 -104
View File
@@ -2,16 +2,21 @@ use std::net::{IpAddr, Ipv4Addr, SocketAddr};
use std::str::FromStr;
use std::time::Duration;
use bgp_primitives::constants::AddressFamilyId;
use bgp_packet::community;
use bgp_packet::message::OriginPathAttribute;
use bgp_packet::open::AddPathSendRecv;
use bgp_primitives::constants::AfiSafi;
use bgp_primitives::ip_prefix::IpPrefix;
use bgp_rib::LocalRibPolicy;
use eyre::{Result, WrapErr, bail, eyre};
use bgp_rib::{LocalRoute, LocalRouteAttrs};
use crate::proto::config::BgpConfig;
use crate::error::{ConfigError, OriginateProblem, PeerProblem};
use crate::family::FamilySet;
use crate::proto::config::{self as proto, BgpConfig, OriginatedRoute, Peer};
const DEFAULT_BGP_PORT: u16 = 179;
const DEFAULT_HOLD_TIME_SECS: u16 = 90;
const DEFAULT_CONNECT_RETRY: Duration = Duration::from_secs(5);
const MAX_AS_PATH_PREPEND: u32 = 16;
/// Validated, Rust-native server configuration derived from the proto config.
pub struct Config {
@@ -28,13 +33,10 @@ pub struct Config {
/// instead of always closing the new connection (RFC 4271 §8.1.1
/// `CollisionDetectEstablishedState`).
pub collision_detect_established_state: bool,
/// Limits on locally originated routes.
pub local_route_policy: LocalRibPolicy,
/// Routes this router originates.
pub originate: Vec<LocalRoute>,
/// Address for the HTTP API; `None` disables it.
pub http_endpoint: Option<SocketAddr>,
/// IPv6 prefixes whose incoming pings are shown; `None` disables the
/// ping monitor.
pub ping_monitor: Option<Vec<IpPrefix>>,
}
#[derive(Debug, Clone)]
@@ -46,109 +48,87 @@ pub struct PeerConfig {
/// Source address for connections we open to this peer; `None` lets the
/// operating system choose.
pub local_address: Option<IpAddr>,
/// The address families offered to this peer.
pub families: FamilySet,
/// The directions ADD-PATH (RFC 7911) is offered to this peer for, on
/// every one of `families`. `None` offers it for neither.
pub add_path: Option<AddPathSendRecv>,
}
impl Config {
pub fn from_proto(proto: BgpConfig) -> Result<Self> {
/// The configured peer at `address`, if there is one.
pub fn peer_at(&self, address: IpAddr) -> Option<&PeerConfig> {
self.peers.iter().find(|peer| peer.endpoint.ip() == address)
}
pub fn from_proto(proto: BgpConfig) -> Result<Self, ConfigError> {
// Peers refuse an identifier of 0.0.0.0, as we do theirs.
let router_id = Ipv4Addr::from_str(&proto.router_id)
.map_err(|_| eyre!("Invalid router_id {:?}: must be a dotted-quad IPv4 address", proto.router_id))?;
.ok()
.filter(|id| !id.is_unspecified())
.ok_or_else(|| ConfigError::RouterId(proto.router_id.clone()))?;
if proto.local_asn == 0 {
bail!("local_asn must be non-zero");
return Err(ConfigError::LocalAsn);
}
let listen_endpoints = if proto.listen_endpoints.is_empty() {
vec![
SocketAddr::new(IpAddr::V4(Ipv4Addr::UNSPECIFIED), DEFAULT_BGP_PORT),
SocketAddr::new(IpAddr::V6(std::net::Ipv6Addr::UNSPECIFIED), DEFAULT_BGP_PORT),
SocketAddr::new(
IpAddr::V6(std::net::Ipv6Addr::UNSPECIFIED),
DEFAULT_BGP_PORT,
),
]
} else {
proto.listen_endpoints
.into_iter()
proto
.listen_endpoints
.iter()
.map(|ep| parse_endpoint(&ep.address, ep.port))
.collect::<Result<_>>()?
.collect::<Result<_, _>>()?
};
let peers = proto.peers
.into_iter()
.map(|p| {
let ep = p.endpoint.as_ref()
.ok_or_else(|| eyre!("Peer {:?} is missing endpoint", p.name))?;
let endpoint = parse_endpoint(&ep.address, ep.port)?;
let mut peers: Vec<PeerConfig> = Vec::with_capacity(proto.peers.len());
for peer in proto.peers {
let name = peer.name.clone();
let rejected = |problem| ConfigError::Peer {
peer: name.clone(),
problem,
};
let peer = parse_peer(peer).map_err(rejected)?;
// A peer is told from the others by its name in what the
// dialer reports, and by its address when it connects to us.
if peers.iter().any(|earlier| earlier.name == peer.name) {
return Err(rejected(PeerProblem::DuplicateName));
}
if peers
.iter()
.any(|earlier| earlier.endpoint.ip() == peer.endpoint.ip())
{
return Err(rejected(PeerProblem::DuplicateAddress));
}
peers.push(peer);
}
if p.remote_asn == 0 {
bail!("Peer {:?} remote_asn must be non-zero", p.name);
}
let hold_time_secs = if p.hold_time_seconds == 0 {
DEFAULT_HOLD_TIME_SECS
} else if p.hold_time_seconds < 3 {
bail!("Peer {:?} hold_time_seconds must be 0 or >= 3", p.name);
} else {
p.hold_time_seconds as u16
};
let local_address = if p.local_address.is_empty() {
None
} else {
let ip = IpAddr::from_str(&p.local_address)
.map_err(|_| eyre!("Peer {:?} has invalid local_address {:?}", p.name, p.local_address))?;
if ip.is_ipv4() != endpoint.is_ipv4() {
bail!("Peer {:?}: local_address {ip} and endpoint {endpoint} are different address families", p.name);
}
Some(ip)
};
Ok(PeerConfig {
name: p.name,
endpoint,
remote_asn: p.remote_asn,
hold_time_secs,
local_address,
})
})
.collect::<Result<_>>()?;
let local_route_policy = match proto.local_route_policy.into_option() {
Some(policy) => LocalRibPolicy {
allowed_prefixes: policy
.allowed_prefixes
.iter()
.map(|p| IpPrefix::try_from(p.as_str()).wrap_err_with(|| format!("Invalid allowed prefix {p:?}")))
.collect::<Result<_>>()?,
max_routes_per_owner: policy.max_routes_per_owner as usize,
},
None => LocalRibPolicy::default(),
};
let mut originate: Vec<LocalRoute> = Vec::with_capacity(proto.originate.len());
for entry in &proto.originate {
let rejected = |problem| ConfigError::Originate {
prefix: entry.prefix.clone(),
problem,
};
let route = parse_originated_route(entry).map_err(rejected)?;
if originate
.iter()
.any(|earlier| earlier.prefix == route.prefix)
{
return Err(rejected(OriginateProblem::Duplicate));
}
originate.push(route);
}
let http_endpoint = match proto.http_endpoint.into_option() {
Some(ep) => {
if ep.port == 0 {
bail!("http_endpoint.port is required");
}
Some(parse_endpoint(&ep.address, ep.port).wrap_err("Invalid http_endpoint")?)
}
None => None,
};
let ping_monitor = match proto.ping_monitor.into_option() {
Some(monitor) => {
let prefixes: Vec<IpPrefix> = if monitor.prefixes.is_empty() {
local_route_policy.allowed_prefixes.clone()
} else {
monitor
.prefixes
.iter()
.map(|p| IpPrefix::try_from(p.as_str()).wrap_err_with(|| format!("Invalid ping_monitor prefix {p:?}")))
.collect::<Result<_>>()?
};
let prefixes: Vec<IpPrefix> =
prefixes.into_iter().filter(|p| p.address_family == AddressFamilyId::Ipv6).collect();
if prefixes.is_empty() {
bail!("ping_monitor needs IPv6 prefixes, either its own or local_route_policy.allowed_prefixes");
}
Some(prefixes)
}
Some(ep) if ep.port == 0 => return Err(ConfigError::HttpPort),
Some(ep) => Some(parse_endpoint(&ep.address, ep.port)?),
None => None,
};
@@ -166,24 +146,352 @@ impl Config {
peers,
connect_retry,
collision_detect_established_state: proto.collision_detect_established_state,
local_route_policy,
originate,
http_endpoint,
ping_monitor,
})
}
}
fn parse_endpoint(address: &str, port: u32) -> Result<SocketAddr> {
let port = if port == 0 {
DEFAULT_BGP_PORT
} else if port > u16::MAX as u32 {
bail!("Port {port} is out of range");
} else {
port as u16
fn parse_peer(peer: Peer) -> Result<PeerConfig, PeerProblem> {
let ep = peer.endpoint.as_ref().ok_or(PeerProblem::MissingEndpoint)?;
let endpoint =
parse_endpoint(&ep.address, ep.port).map_err(|e| PeerProblem::Endpoint(Box::new(e)))?;
if peer.remote_asn == 0 {
return Err(PeerProblem::RemoteAsn);
}
let hold_time_secs = match peer.hold_time_seconds {
0 => DEFAULT_HOLD_TIME_SECS,
// RFC 4271 §4.2: zero, or at least three seconds.
1 | 2 => return Err(PeerProblem::HoldTime),
secs => u16::try_from(secs).map_err(|_| PeerProblem::HoldTime)?,
};
let ip = IpAddr::from_str(address)
.map_err(|_| eyre!("Invalid IP address {address:?}"))?;
let local_address = if peer.local_address.is_empty() {
None
} else {
let ip = IpAddr::from_str(&peer.local_address)
.map_err(|_| PeerProblem::LocalAddress(peer.local_address.clone()))?;
if ip.is_ipv4() != endpoint.is_ipv4() {
return Err(PeerProblem::AddressFamilyMismatch);
}
Some(ip)
};
let families = if peer.address_families.is_empty() {
FamilySet::supported()
} else {
peer.address_families
.iter()
.map(|family| match family.enum_value() {
Ok(proto::AddressFamily::IPV4_UNICAST) => Ok(AfiSafi::IPV4_UNICAST),
Ok(proto::AddressFamily::IPV6_UNICAST) => Ok(AfiSafi::IPV6_UNICAST),
Ok(proto::AddressFamily::ADDRESS_FAMILY_UNSPECIFIED) | Err(_) => {
Err(PeerProblem::AddressFamily)
}
})
.collect::<Result<_, _>>()?
};
let add_path = match peer.add_path.enum_value() {
Ok(proto::AddPath::ADD_PATH_OFF) => None,
Ok(proto::AddPath::ADD_PATH_RECEIVE) => Some(AddPathSendRecv::Receive),
Ok(proto::AddPath::ADD_PATH_SEND) => Some(AddPathSendRecv::Send),
Ok(proto::AddPath::ADD_PATH_BOTH) => Some(AddPathSendRecv::Both),
Err(_) => return Err(PeerProblem::AddPath),
};
Ok(PeerConfig {
// A peer without a name goes by its address.
name: if peer.name.is_empty() {
endpoint.ip().to_string()
} else {
peer.name
},
endpoint,
remote_asn: peer.remote_asn,
hold_time_secs,
local_address,
families,
add_path,
})
}
fn parse_endpoint(address: &str, port: u32) -> Result<SocketAddr, ConfigError> {
let port = match port {
0 => DEFAULT_BGP_PORT,
port => u16::try_from(port).map_err(|_| ConfigError::Port(port))?,
};
let ip = IpAddr::from_str(address).map_err(|_| ConfigError::Address(address.to_string()))?;
Ok(SocketAddr::new(ip, port))
}
fn parse_originated_route(route: &OriginatedRoute) -> Result<LocalRoute, OriginateProblem> {
let prefix = IpPrefix::try_from(route.prefix.as_str())?;
let origin = match route.origin.to_ascii_uppercase().as_str() {
"" | "IGP" => OriginPathAttribute::IGP,
"EGP" => OriginPathAttribute::EGP,
"INCOMPLETE" => OriginPathAttribute::INCOMPLETE,
_ => return Err(OriginateProblem::Origin(route.origin.clone())),
};
let prepend = u8::try_from(route.as_path_prepend)
.ok()
.filter(|&n| u32::from(n) <= MAX_AS_PATH_PREPEND)
.ok_or(OriginateProblem::Prepend {
max: MAX_AS_PATH_PREPEND,
})?;
let communities = route
.communities
.iter()
.map(|c| community::parse_standard(c))
.collect::<Result<_, _>>()?;
let large_communities = route
.large_communities
.iter()
.map(|c| community::parse_large(c))
.collect::<Result<_, _>>()?;
Ok(LocalRoute::new(
prefix,
LocalRouteAttrs {
origin,
prepend,
med: route.med,
communities,
large_communities,
},
))
}
#[cfg(test)]
mod tests {
use bgp_primitives::ip_prefix::PrefixError;
use super::*;
fn parse(text: &str) -> Result<Config, ConfigError> {
let proto = protobuf::text_format::parse_from_str::<BgpConfig>(text)
.expect("the test's textproto is well formed");
Config::from_proto(proto)
}
fn rejected(text: &str) -> ConfigError {
parse(text).err().expect(text)
}
const BASE: &str = "router_id: \"192.0.2.1\" local_asn: 65000\n";
#[test]
fn originated_routes_are_parsed_with_their_attributes() {
let config = parse(&format!(
"{BASE}originate {{ prefix: \"192.0.2.0/24\" }}
originate {{
prefix: \"2001:db8::/32\" origin: \"incomplete\" as_path_prepend: 2 med: 0
communities: \"65000:100\" communities: \"NO_EXPORT\"
large_communities: \"65000:1:2\"
}}"
))
.unwrap();
assert_eq!(config.originate.len(), 2);
let plain = &config.originate[0];
assert_eq!(plain.prefix, IpPrefix::try_from("192.0.2.0/24").unwrap());
assert_eq!(plain.attrs, LocalRouteAttrs::default());
let full = &config.originate[1].attrs;
assert_eq!(full.origin, OriginPathAttribute::INCOMPLETE);
assert_eq!(full.prepend, 2);
assert_eq!(
full.med,
Some(0),
"an explicit MED of 0 is not the same as no MED"
);
assert_eq!(full.communities, vec![(65000, 100), (65535, 65281)]);
assert_eq!(full.large_communities.len(), 1);
}
#[test]
fn bad_originated_routes_are_rejected_with_the_entry_and_the_reason() {
let problem = |body: &str| match rejected(&format!("{BASE}originate {{ {body} }}")) {
ConfigError::Originate { problem, .. } => problem,
other => panic!("{body}: {other}"),
};
assert_eq!(
problem("prefix: \"not-a-prefix\""),
OriginateProblem::Prefix(PrefixError::Format("not-a-prefix".into()))
);
assert_eq!(
problem("prefix: \"192.0.2.0/24\" origin: \"static\""),
OriginateProblem::Origin("static".into())
);
assert_eq!(
problem("prefix: \"192.0.2.0/24\" as_path_prepend: 17"),
OriginateProblem::Prepend { max: 16 }
);
assert!(matches!(
problem("prefix: \"192.0.2.0/24\" communities: \"70000:1\""),
OriginateProblem::Community(_)
));
assert!(matches!(
problem("prefix: \"192.0.2.0/24\" large_communities: \"1:2\""),
OriginateProblem::Community(_)
));
let duplicate = rejected(&format!(
"{BASE}originate {{ prefix: \"192.0.2.0/24\" }} originate {{ prefix: \"192.0.2.0/24\" med: 5 }}"
));
assert_eq!(
duplicate,
ConfigError::Originate {
prefix: "192.0.2.0/24".into(),
problem: OriginateProblem::Duplicate
}
);
// The message names the entry, so the operator can find it.
assert_eq!(
duplicate.to_string(),
"originate \"192.0.2.0/24\": listed more than once"
);
}
#[test]
fn a_peer_is_offered_every_address_family_unless_some_are_listed() {
let families = |body: &str| {
let config = parse(&format!(
"{BASE}peers {{ name: \"r1\" endpoint {{ address: \"192.0.2.2\" }} remote_asn: 1 {body} }}"
))
.unwrap();
config.peers[0].families.clone()
};
assert_eq!(families(""), FamilySet::supported());
assert_eq!(
families("address_families: IPV6_UNICAST"),
[AfiSafi::IPV6_UNICAST].into_iter().collect()
);
assert_eq!(
families("address_families: IPV4_UNICAST address_families: IPV6_UNICAST"),
FamilySet::supported()
);
}
#[test]
fn add_path_is_off_unless_a_peer_asks_for_it() {
let add_path = |body: &str| {
let config = parse(&format!(
"{BASE}peers {{ name: \"r1\" endpoint {{ address: \"192.0.2.2\" }} remote_asn: 1 {body} }}"
))
.unwrap();
config.peers[0].add_path
};
assert_eq!(add_path(""), None);
assert_eq!(add_path("add_path: ADD_PATH_OFF"), None);
assert_eq!(
add_path("add_path: ADD_PATH_RECEIVE"),
Some(AddPathSendRecv::Receive)
);
assert_eq!(
add_path("add_path: ADD_PATH_BOTH"),
Some(AddPathSendRecv::Both)
);
}
#[test]
fn bad_peers_and_endpoints_are_rejected_with_the_peer_named() {
assert_eq!(
rejected("router_id: \"nope\" local_asn: 1"),
ConfigError::RouterId("nope".into())
);
assert_eq!(rejected("router_id: \"192.0.2.1\""), ConfigError::LocalAsn);
let problem = |body: &str| match rejected(&format!("{BASE}peers {{ name: \"r1\" {body} }}"))
{
ConfigError::Peer { peer, problem } => {
assert_eq!(peer, "r1");
problem
}
other => panic!("{body}: {other}"),
};
let endpoint = "endpoint { address: \"192.0.2.2\" }";
assert_eq!(problem("remote_asn: 1"), PeerProblem::MissingEndpoint);
assert_eq!(problem(endpoint), PeerProblem::RemoteAsn);
assert_eq!(
problem(&format!("{endpoint} remote_asn: 1 hold_time_seconds: 2")),
PeerProblem::HoldTime
);
assert_eq!(
problem(&format!(
"{endpoint} remote_asn: 1 hold_time_seconds: 70000"
)),
PeerProblem::HoldTime
);
assert_eq!(
problem(&format!("{endpoint} remote_asn: 1 local_address: \"::1\"")),
PeerProblem::AddressFamilyMismatch
);
assert_eq!(
problem(&format!(
"{endpoint} remote_asn: 1 address_families: ADDRESS_FAMILY_UNSPECIFIED"
)),
PeerProblem::AddressFamily
);
assert_eq!(
problem("endpoint { address: \"192.0.2.2\" port: 70000 } remote_asn: 1"),
PeerProblem::Endpoint(Box::new(ConfigError::Port(70000)))
);
assert_eq!(
rejected(&format!("{BASE}http_endpoint {{ address: \"127.0.0.1\" }}")),
ConfigError::HttpPort
);
}
/// Peers are told apart by name where the dialer reports on them and by
/// address when they connect to us; two that share either would be
/// taken for each other.
#[test]
fn peers_must_differ_in_name_and_address() {
let peer = |name: &str, address: &str| {
format!(
"peers {{ name: \"{name}\" endpoint {{ address: \"{address}\" }} remote_asn: 1 }}\n"
)
};
let problem = |peers: String| match rejected(&format!("{BASE}{peers}")) {
ConfigError::Peer { problem, .. } => problem,
other => panic!("{other}"),
};
assert_eq!(
problem(peer("r1", "192.0.2.2") + &peer("r1", "192.0.2.3")),
PeerProblem::DuplicateName
);
assert_eq!(
problem(peer("r1", "192.0.2.2") + &peer("r2", "192.0.2.2")),
PeerProblem::DuplicateAddress
);
// A peer without a name goes by its address, so two of them differ.
let config = parse(&format!(
"{BASE}{}{}",
peer("", "192.0.2.2"),
peer("", "192.0.2.3")
))
.unwrap();
assert_eq!(config.peers[0].name, "192.0.2.2");
assert_eq!(config.peers[1].name, "192.0.2.3");
assert_eq!(
config
.peer_at("192.0.2.3".parse().unwrap())
.map(|p| &*p.name),
Some("192.0.2.3")
);
assert!(config.peer_at("192.0.2.4".parse().unwrap()).is_none());
}
/// A peer would refuse our OPEN.
#[test]
fn a_router_id_of_zero_is_rejected() {
assert_eq!(
rejected("router_id: \"0.0.0.0\" local_asn: 1"),
ConfigError::RouterId("0.0.0.0".into())
);
}
}
crates/server/src/connection.rs
+1039 -8
View File
File diff suppressed because it is too large. Load diff
crates/server/src/dialer.rs
+42 -28
View File
@@ -17,8 +17,8 @@ use tracing::{info, warn};
use crate::collision::Direction;
use crate::config::PeerConfig;
use crate::fsm::{SessionConfig, run_session};
use crate::session::{DialState, SessionRegistry};
use crate::connection::{PeerSettings, ServerContext, run_session};
use crate::session::DialState;
/// Upper bound for exponential backoff between connection attempts.
const MAX_BACKOFF: Duration = Duration::from_secs(120);
@@ -26,21 +26,12 @@ const CONNECT_TIMEOUT: Duration = Duration::from_secs(10);
pub fn spawn_dialers(
peers: &[PeerConfig],
base: &SessionConfig,
server: &Arc<ServerContext>,
connect_retry: Duration,
) -> Vec<JoinHandle<()>> {
peers
.iter()
.map(|peer| {
let config = SessionConfig {
hold_time_secs: peer.hold_time_secs,
direction: Direction::Outbound,
peer_name: Some(peer.name.clone()),
expected_asn: Some(peer.remote_asn),
..base.clone()
};
tokio::spawn(dial_loop(Arc::new(peer.clone()), config, connect_retry))
})
.map(|peer| tokio::spawn(dial_loop(peer.clone(), Arc::clone(server), connect_retry)))
.collect()
}
@@ -49,15 +40,21 @@ async fn connect(peer: &PeerConfig) -> std::io::Result<TcpStream> {
let Some(local) = peer.local_address else {
return TcpStream::connect(peer.endpoint).await;
};
let socket = if peer.endpoint.is_ipv4() { TcpSocket::new_v4()? } else { TcpSocket::new_v6()? };
let socket = if peer.endpoint.is_ipv4() {
TcpSocket::new_v4()?
} else {
TcpSocket::new_v6()?
};
socket
.bind(std::net::SocketAddr::new(local, 0))
.map_err(|e| std::io::Error::new(e.kind(), format!("cannot use local_address {local}: {e}")))?;
.map_err(|e| {
std::io::Error::new(e.kind(), format!("cannot use local_address {local}: {e}"))
})?;
socket.connect(peer.endpoint).await
}
async fn dial_loop(peer: Arc<PeerConfig>, config: SessionConfig, connect_retry: Duration) {
let registry: SessionRegistry = config.registry.clone();
async fn dial_loop(peer: PeerConfig, server: Arc<ServerContext>, connect_retry: Duration) {
let registry = &server.registry;
let mut backoff = connect_retry;
loop {
@@ -74,27 +71,44 @@ async fn dial_loop(peer: Arc<PeerConfig>, config: SessionConfig, connect_retry:
registry.set_dial_state(&peer.name, DialState::Connecting);
info!(peer = %peer.name, endpoint = %peer.endpoint, "Connecting to peer");
let outcome = match tokio::time::timeout(CONNECT_TIMEOUT, connect(&peer)).await {
let ended = match tokio::time::timeout(CONNECT_TIMEOUT, connect(&peer)).await {
Ok(Ok(stream)) => {
info!(peer = %peer.name, endpoint = %peer.endpoint, "TCP connected, starting session");
backoff = connect_retry;
run_session(stream, peer.endpoint, config.clone()).await.map(|()| "session closed by peer".to_string())
let end = run_session(
stream,
peer.endpoint,
Direction::Outbound,
Arc::clone(&server),
PeerSettings::from(&peer),
)
.await;
// A session that came up is a peer worth dialling again
// soon. One that connects and is then refused, for its AS
// number say, will be refused again: it is retried ever
// more slowly, like one that cannot be reached at all.
if end.established {
backoff = connect_retry;
}
end.result.map_err(|e| e.to_string())
}
Ok(Err(e)) => Err(eyre::eyre!("connect failed: {e}")),
Err(_) => Err(eyre::eyre!("connect timed out after {CONNECT_TIMEOUT:?}")),
Ok(Err(e)) => Err(format!("connect failed: {e}")),
Err(_) => Err(format!("connect timed out after {CONNECT_TIMEOUT:?}")),
};
let reason = match outcome {
Ok(msg) => msg,
Err(e) => {
warn!(peer = %peer.name, endpoint = %peer.endpoint, err = %e, "Session attempt ended");
e.to_string()
let reason = match ended {
Ok(()) => "session closed at our request".to_string(),
Err(reason) => {
warn!(peer = %peer.name, endpoint = %peer.endpoint, err = %reason, "Session attempt ended");
reason
}
};
registry.set_dial_state(
&peer.name,
DialState::Backoff { until: SystemTime::now() + backoff, last_error: reason },
DialState::Backoff {
until: SystemTime::now() + backoff,
last_error: reason,
},
);
tokio::time::sleep(backoff).await;
backoff = (backoff * 2).min(MAX_BACKOFF);
crates/server/src/error.rs
+248
View File
@@ -0,0 +1,248 @@
//! Error types of the server crate.
use std::io;
use std::net::SocketAddr;
use std::time::Duration;
use bgp_packet::errors::{BgpError, CeaseSubcode, DecodeError, FsmSubcode, OpenMsgSubcode};
use bgp_packet::message::NotificationMessage;
use bgp_primitives::parser::EncodeError;
use bgp_rib::RibStopped;
use bgp_rib::table::TooManyPeers;
use thiserror::Error;
use crate::family::FamilySet;
use crate::fsm::State;
/// Why a session ended abnormally.
#[derive(Debug, Error)]
pub enum SessionError {
#[error("connection failed: {0}")]
Io(#[from] io::Error),
#[error("received a message that could not be decoded: {0}")]
Decode(#[from] DecodeError),
#[error("could not encode a message: {0}")]
Encode(#[from] EncodeError),
/// The peer sent nothing for as long as the pre-negotiation hold time.
#[error("timed out waiting for a message in the {0} state")]
Timeout(State),
#[error("hold timer expired")]
HoldTimerExpired,
#[error("received {received} in the {state} state")]
UnexpectedMessage {
state: State,
received: &'static str,
},
/// The peer ended the session. Nothing is sent in return.
#[error("peer sent NOTIFICATION: {0}")]
NotifiedByPeer(BgpError),
#[error("refused the peer's OPEN: {0}")]
OpenRefused(BgpError),
#[error("peer announced AS{actual} but AS{expected} was configured")]
WrongPeerAs { expected: u32, actual: u32 },
#[error("closed by connection collision resolution")]
Collision,
/// The routing table has no room for another peer.
#[error(transparent)]
TableFull(#[from] TooManyPeers),
/// A bug: the session's own code panicked. The peer is told the session
/// is being reset, and the session is cleaned up like any other.
#[error("the session panicked: {0}")]
Panicked(String),
/// The peer offers none of the address families configured for it, so
/// the session could carry nothing.
#[error("no address family in common: we offer {ours}, the peer offers {theirs}")]
NoCommonAddressFamily { ours: FamilySet, theirs: FamilySet },
/// The peer took nothing we sent for this long (RFC 9687): it has
/// stopped reading, and there is no telling it so.
#[error("the peer accepted no data for {0:?}")]
SendHoldTimerExpired(Duration),
}
impl SessionError {
/// The NOTIFICATION that tells the peer about this error (RFC 4271 §6),
/// or `None` when there is nothing to say or no way to say it.
pub fn notification(&self) -> Option<NotificationMessage> {
let error = match self {
Self::Io(_) | Self::SendHoldTimerExpired(_) | Self::NotifiedByPeer(_) => return None,
// Something of ours could not be written out, which is a bug
// here: as for a panic, the session is reset.
Self::Encode(_) => BgpError::Cease(CeaseSubcode::AdminReset),
Self::Decode(e) => {
return Some(NotificationMessage {
error: e.notification.clone(),
data: e.data.clone(),
});
}
Self::Timeout(_) | Self::HoldTimerExpired => BgpError::HoldTimer,
Self::UnexpectedMessage { state, .. } => BgpError::FsmError(match state {
State::OpenSent => FsmSubcode::UnexpectedOpenSent,
State::OpenConfirm => FsmSubcode::UnexpectedOpenConfirm,
// A closed machine handles no messages; Established is the
// only other state one can arrive in.
State::Established | State::Closed => FsmSubcode::UnexpectedEstablished,
}),
Self::OpenRefused(error) => error.clone(),
Self::WrongPeerAs { .. } => BgpError::OpenMsg(OpenMsgSubcode::BadPeerAs),
Self::Collision => BgpError::Cease(CeaseSubcode::ConnCollisionResolution),
Self::TableFull(_) => BgpError::Cease(CeaseSubcode::OutOfResources),
// The session will be set up again; nothing is wrong at the
// peer's end.
Self::Panicked(_) => BgpError::Cease(CeaseSubcode::AdminReset),
// RFC 5492 §5: the Data lists the capabilities we need and the
// peer did not offer.
Self::NoCommonAddressFamily { ours, .. } => {
return Some(NotificationMessage {
error: BgpError::OpenMsg(OpenMsgSubcode::UnsupportedCapability),
data: ours.capabilities_wire(),
});
}
};
Some(NotificationMessage {
error,
data: vec![],
})
}
}
/// Why the server could not start, or could not go on.
#[derive(Debug, Error)]
pub enum ServerError {
#[error("failed to bind listener on {addr}: {source}")]
Bind {
addr: SocketAddr,
#[source]
source: io::Error,
},
#[error("listener failed: {0}")]
Io(#[from] io::Error),
/// A RIB's actor died. The server cannot work without its tables.
#[error(transparent)]
RibStopped(#[from] RibStopped),
/// A task the server depends on ended, which none of them do unless
/// they panic.
#[error("the {task} task stopped unexpectedly: {reason}")]
TaskStopped { task: &'static str, reason: String },
}
/// Why a configuration was rejected.
#[derive(Debug, Clone, PartialEq, Eq, Error)]
pub enum ConfigError {
#[error("invalid router_id {0:?}: must be a dotted-quad IPv4 address other than 0.0.0.0")]
RouterId(String),
#[error("local_asn must be non-zero")]
LocalAsn,
#[error("invalid IP address {0:?}")]
Address(String),
#[error("port {0} is out of range")]
Port(u32),
#[error("http_endpoint.port is required")]
HttpPort,
#[error("peer {peer:?}: {problem}")]
Peer { peer: String, problem: PeerProblem },
#[error("originate {prefix:?}: {problem}")]
Originate {
prefix: String,
problem: OriginateProblem,
},
}
/// What is wrong with one `peers` entry.
#[derive(Debug, Clone, PartialEq, Eq, Error)]
pub enum PeerProblem {
#[error("endpoint is missing")]
MissingEndpoint,
#[error("endpoint is invalid: {0}")]
Endpoint(Box<ConfigError>),
#[error("remote_asn must be non-zero")]
RemoteAsn,
#[error("hold_time_seconds must be 0 or at least 3")]
HoldTime,
#[error("there is another peer of this name")]
DuplicateName,
#[error("there is another peer at this address")]
DuplicateAddress,
#[error("local_address {0:?} is not an IP address")]
LocalAddress(String),
#[error("local_address and endpoint are different address families")]
AddressFamilyMismatch,
#[error("address_families has a value that is not an address family")]
AddressFamily,
#[error("add_path has a value that is not one of its settings")]
AddPath,
}
/// What is wrong with one `originate` entry.
#[derive(Debug, Clone, PartialEq, Eq, Error)]
pub enum OriginateProblem {
#[error("{0}")]
Prefix(#[from] bgp_primitives::ip_prefix::PrefixError),
#[error("origin {0:?} must be IGP, EGP or INCOMPLETE")]
Origin(String),
#[error("as_path_prepend must be at most {max}")]
Prepend { max: u32 },
#[error("{0}")]
Community(#[from] bgp_packet::community::CommunityError),
#[error("listed more than once")]
Duplicate,
}
#[cfg(test)]
mod tests {
use bgp_packet::errors::MsgHeaderSubcode;
use super::*;
#[test]
fn each_session_error_names_the_notification_rfc_4271_assigns_it() {
let notification = |e: SessionError| e.notification().map(|n| (n.error, n.data));
assert_eq!(
notification(SessionError::Decode(DecodeError {
notification: BgpError::MsgHeader(MsgHeaderSubcode::BadMessageLength),
data: vec![0x13, 0x88],
detail: "too long".into(),
})),
Some((
BgpError::MsgHeader(MsgHeaderSubcode::BadMessageLength),
vec![0x13, 0x88]
))
);
assert_eq!(
notification(SessionError::Timeout(State::OpenSent)),
Some((BgpError::HoldTimer, vec![]))
);
assert_eq!(
notification(SessionError::UnexpectedMessage {
state: State::OpenConfirm,
received: "update"
}),
Some((
BgpError::FsmError(FsmSubcode::UnexpectedOpenConfirm),
vec![]
))
);
assert_eq!(
notification(SessionError::WrongPeerAs {
expected: 1,
actual: 2
}),
Some((BgpError::OpenMsg(OpenMsgSubcode::BadPeerAs), vec![]))
);
// Something of ours that cannot be written is our fault: the peer
// is told the session is being reset, not left to time it out.
assert_eq!(
notification(SessionError::Encode(EncodeError::TooLong {
what: "path attribute",
len: 70000,
max: 65535
})),
Some((BgpError::Cease(CeaseSubcode::AdminReset), vec![]))
);
// A broken connection cannot carry a NOTIFICATION.
assert_eq!(
notification(SessionError::Io(io::ErrorKind::BrokenPipe.into())),
None
);
}
}
crates/server/src/family.rs
+292
View File
@@ -0,0 +1,292 @@
//! The address families a session can carry, and which of them it does.
//!
//! Each side lists the families it is willing to exchange in its OPEN, as
//! Multiprotocol capabilities (RFC 4760 §8). A family is in use on a session
//! only if both sides listed it.
//!
//! Any family can be named, read from a peer and held in a [`FamilySet`].
//! The ones this speaker can actually exchange routes for are [`SUPPORTED`],
//! and only those are ever offered to a peer.
use std::collections::BTreeSet;
use std::fmt;
use bgp_packet::message::{PathAttribute, UpdateMessage};
use bgp_packet::open::{Capability, OpenMessage};
use bgp_primitives::constants::AfiSafi;
use bgp_primitives::ip_prefix::IpPrefix;
use bytes::BytesMut;
/// The address families this speaker can exchange routes for. Supporting a
/// new one means being able to parse, store and advertise its routes; then
/// it is added here.
pub const SUPPORTED: [AfiSafi; 2] = [AfiSafi::IPV4_UNICAST, AfiSafi::IPV6_UNICAST];
/// A set of address families.
#[derive(Debug, Clone, PartialEq, Eq, Default)]
pub struct FamilySet(BTreeSet<AfiSafi>);
impl FamilySet {
/// Every family in [`SUPPORTED`].
pub fn supported() -> Self {
SUPPORTED.into_iter().collect()
}
pub fn insert(&mut self, family: AfiSafi) {
self.0.insert(family);
}
pub fn contains(&self, family: AfiSafi) -> bool {
self.0.contains(&family)
}
/// Whether a unicast route for `prefix` may be exchanged.
pub fn carries(&self, prefix: &IpPrefix) -> bool {
self.contains(AfiSafi::unicast(prefix.address_family()))
}
pub fn is_empty(&self) -> bool {
self.0.is_empty()
}
pub fn intersection(&self, other: &Self) -> Self {
Self(self.0.intersection(&other.0).copied().collect())
}
pub fn iter(&self) -> impl Iterator<Item = AfiSafi> + '_ {
self.0.iter().copied()
}
/// The families a peer offered in its OPEN, whether or not we support
/// them.
///
/// A peer that sends no Multiprotocol capability at all is a speaker of
/// plain RFC 4271 BGP, which carries IPv4 unicast and nothing else. One
/// that sends any has listed everything it wants, IPv4 unicast included.
pub fn offered_by(open: &OpenMessage) -> Self {
let listed: Self = open
.capabilities()
.filter_map(|capability| match capability {
Capability::MultiProtocol { afi, safi } => Some(AfiSafi::new(*afi, *safi)),
_ => None,
})
.collect();
if listed.is_empty() {
[AfiSafi::IPV4_UNICAST].into_iter().collect()
} else {
listed
}
}
/// These families as Multiprotocol capabilities in wire form: the Data
/// of an Unsupported Capability NOTIFICATION (RFC 5492 §5).
pub fn capabilities_wire(&self) -> Vec<u8> {
let mut out = BytesMut::new();
for family in self.iter() {
// A Multiprotocol capability always encodes; if that ever
// changed, a NOTIFICATION with less Data is still correct.
if Capability::multiprotocol(family).to_wire(&mut out).is_err() {
return vec![];
}
}
out.to_vec()
}
/// Removes from `update` every route of a family that is not in this
/// set, and returns how many prefixes that was.
///
/// The withdrawn routes and NLRI in the body of an UPDATE are IPv4
/// unicast; every other family travels in MP_REACH_NLRI and
/// MP_UNREACH_NLRI, which name theirs.
pub fn restrict(&self, update: &mut UpdateMessage) -> usize {
let mut removed = 0;
if !self.contains(AfiSafi::IPV4_UNICAST) {
removed += update.withdrawn_routes.len() + update.announced_routes.len();
update.withdrawn_routes.clear();
update.announced_routes.clear();
}
update.path_attributes.retain(|attribute| {
let (family, prefixes) = match attribute {
PathAttribute::MpReachNlri(mp) => (mp.family, mp.prefixes.len()),
PathAttribute::MpUnreachNlri(mp) => (mp.family, mp.prefixes.len()),
_ => return true,
};
let keep = self.contains(family);
if !keep {
removed += prefixes;
}
keep
});
removed
}
}
impl FromIterator<AfiSafi> for FamilySet {
fn from_iter<I: IntoIterator<Item = AfiSafi>>(families: I) -> Self {
Self(families.into_iter().collect())
}
}
impl fmt::Display for FamilySet {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
if self.is_empty() {
return f.write_str("none");
}
for (i, family) in self.iter().enumerate() {
if i > 0 {
f.write_str(", ")?;
}
write!(f, "{family}")?;
}
Ok(())
}
}
#[cfg(test)]
mod tests {
use std::net::{Ipv4Addr, Ipv6Addr};
use bgp_packet::message::{
MpReachNlriPathAttribute, MpUnreachNlriPathAttribute, NlriNextHop, OriginPathAttribute,
};
use bgp_packet::open::OpenOption;
use bgp_primitives::constants::{Afi, Safi};
use super::*;
fn open(capabilities: Vec<Capability>) -> OpenMessage {
OpenMessage {
version: 4,
asn: 65001,
hold_time: 90,
identifier: Ipv4Addr::new(10, 0, 0, 2),
options: if capabilities.is_empty() {
vec![]
} else {
vec![OpenOption::Capabilities(capabilities)]
},
}
}
fn prefix(s: &str) -> IpPrefix {
IpPrefix::try_from(s).unwrap()
}
fn only(family: AfiSafi) -> FamilySet {
[family].into_iter().collect()
}
#[test]
fn a_peer_without_multiprotocol_capabilities_speaks_ipv4_unicast() {
assert_eq!(
FamilySet::offered_by(&open(vec![])),
only(AfiSafi::IPV4_UNICAST)
);
// Other capabilities do not change that.
assert_eq!(
FamilySet::offered_by(&open(vec![Capability::RouteRefresh])),
only(AfiSafi::IPV4_UNICAST)
);
}
#[test]
fn a_peer_that_lists_families_gets_exactly_those() {
assert_eq!(
FamilySet::offered_by(&open(vec![Capability::multiprotocol(
AfiSafi::IPV6_UNICAST
)])),
only(AfiSafi::IPV6_UNICAST),
"IPv4 unicast is not implied once a peer lists its families"
);
// Capabilities may be spread over several optional parameters.
let mut spread = open(vec![Capability::multiprotocol(AfiSafi::IPV4_UNICAST)]);
spread
.options
.push(OpenOption::Capabilities(vec![Capability::multiprotocol(
AfiSafi::IPV6_UNICAST,
)]));
assert_eq!(FamilySet::offered_by(&spread), FamilySet::supported());
// A family we do not support is still what the peer offered. It
// drops out when intersected with what we offer.
let evpn = AfiSafi::new(Afi::L2VPN, Safi::EVPN);
let offered = FamilySet::offered_by(&open(vec![
Capability::multiprotocol(evpn),
Capability::multiprotocol(AfiSafi::IPV6_UNICAST),
]));
assert_eq!(offered.to_string(), "IPv6 unicast, L2VPN EVPN");
assert_eq!(
FamilySet::supported().intersection(&offered),
only(AfiSafi::IPV6_UNICAST)
);
assert!(
FamilySet::supported()
.intersection(&FamilySet::offered_by(&open(vec![
Capability::multiprotocol(evpn)
])))
.is_empty()
);
}
#[test]
fn sets_intersect_and_describe_themselves() {
let v6 = only(AfiSafi::IPV6_UNICAST);
assert_eq!(FamilySet::supported().intersection(&v6), v6);
assert!(only(AfiSafi::IPV4_UNICAST).intersection(&v6).is_empty());
assert_eq!(
FamilySet::supported().to_string(),
"IPv4 unicast, IPv6 unicast"
);
assert_eq!(FamilySet::default().to_string(), "none");
assert!(v6.carries(&prefix("2001:db8::/32")));
assert!(!v6.carries(&prefix("192.0.2.0/24")));
// One capability each: code 1, length 4, AFI, reserved, SAFI.
assert_eq!(
FamilySet::supported().capabilities_wire(),
[1, 4, 0, 1, 0, 1, 1, 4, 0, 2, 0, 1]
);
}
fn mixed_update() -> UpdateMessage {
UpdateMessage {
withdrawn_routes: vec![prefix("198.51.100.0/24").into()],
path_attributes: vec![
PathAttribute::Origin(OriginPathAttribute::IGP),
PathAttribute::MpReachNlri(MpReachNlriPathAttribute {
family: AfiSafi::IPV6_UNICAST,
next_hop: NlriNextHop::Ipv6(Ipv6Addr::LOCALHOST),
prefixes: vec![
prefix("2001:db8:1::/48").into(),
prefix("2001:db8:2::/48").into(),
],
}),
PathAttribute::MpUnreachNlri(MpUnreachNlriPathAttribute {
family: AfiSafi::IPV6_UNICAST,
prefixes: vec![prefix("2001:db8:3::/48").into()],
}),
],
announced_routes: vec![prefix("192.0.2.0/24").into()],
}
}
#[test]
fn restrict_removes_the_routes_of_families_not_in_the_set() {
let mut update = mixed_update();
assert_eq!(FamilySet::supported().restrict(&mut update), 0);
assert_eq!(update, mixed_update());
let mut update = mixed_update();
assert_eq!(only(AfiSafi::IPV4_UNICAST).restrict(&mut update), 3);
assert_eq!(update.announced_routes, vec![prefix("192.0.2.0/24")]);
assert_eq!(update.withdrawn_routes, vec![prefix("198.51.100.0/24")]);
assert_eq!(
update.path_attributes,
vec![PathAttribute::Origin(OriginPathAttribute::IGP)]
);
let mut update = mixed_update();
assert_eq!(only(AfiSafi::IPV6_UNICAST).restrict(&mut update), 2);
assert!(update.announced_routes.is_empty());
assert!(update.withdrawn_routes.is_empty());
assert_eq!(update.path_attributes.len(), 3);
}
}
crates/server/src/fsm.rs
+975 -421
View File
File diff suppressed because it is too large. Load diff
crates/server/src/lib.rs
+5 -1
View File
@@ -3,11 +3,15 @@ pub mod collision;
pub mod config;
pub mod connection;
pub mod dialer;
pub mod error;
pub mod family;
pub mod fsm;
pub mod metrics;
pub mod ping;
mod panic;
mod routes;
pub mod server;
pub mod session;
mod sync;
pub mod tracing;
pub mod proto {
crates/server/src/metrics.rs
+62 -25
View File
@@ -9,23 +9,46 @@ use prometheus::{
};
pub static MESSAGES_SENT: LazyLock<IntCounterVec> = LazyLock::new(|| {
register_int_counter_vec!("bgp_messages_sent_total", "BGP messages sent, by type", &["type"])
.expect("metric registration")
register_int_counter_vec!(
"bgp_messages_sent_total",
"BGP messages sent, by type",
&["type"]
)
.expect("metric registration")
});
pub static MESSAGES_RECEIVED: LazyLock<IntCounterVec> = LazyLock::new(|| {
register_int_counter_vec!("bgp_messages_received_total", "BGP messages received, by type", &["type"])
.expect("metric registration")
register_int_counter_vec!(
"bgp_messages_received_total",
"BGP messages received, by type",
&["type"]
)
.expect("metric registration")
});
pub static UPDATE_ERRORS: LazyLock<IntCounterVec> = LazyLock::new(|| {
register_int_counter_vec!(
"bgp_update_errors_total",
"Errors in received UPDATE messages that did not reset the session, by how they were handled (RFC 7606)",
&["handling"]
)
.expect("metric registration")
});
pub static SESSIONS_ESTABLISHED_TOTAL: LazyLock<IntCounter> = LazyLock::new(|| {
register_int_counter!("bgp_sessions_established_total", "Sessions that reached Established")
.expect("metric registration")
register_int_counter!(
"bgp_sessions_established_total",
"Sessions that reached Established"
)
.expect("metric registration")
});
pub static SESSIONS_ESTABLISHED: LazyLock<IntGauge> = LazyLock::new(|| {
register_int_gauge!("bgp_sessions_established", "Sessions currently in Established")
.expect("metric registration")
register_int_gauge!(
"bgp_sessions_established",
"Sessions currently in Established"
)
.expect("metric registration")
});
pub static COLLISIONS: LazyLock<IntCounterVec> = LazyLock::new(|| {
@@ -38,26 +61,38 @@ pub static COLLISIONS: LazyLock<IntCounterVec> = LazyLock::new(|| {
});
pub static LOCAL_ROUTES: LazyLock<IntGauge> = LazyLock::new(|| {
register_int_gauge!("bgp_local_routes", "Locally originated routes").expect("metric registration")
});
pub static ADJ_RIB_IN_ROUTES: LazyLock<IntGauge> = LazyLock::new(|| {
register_int_gauge!("bgp_adj_rib_in_routes", "Routes held in Adj-RIB-In across all peers")
register_int_gauge!("bgp_local_routes", "Locally originated routes")
.expect("metric registration")
});
pub static PING_ECHO_REQUESTS: LazyLock<IntCounter> = LazyLock::new(|| {
register_int_counter!(
"ping_monitor_echo_requests_total",
"ICMPv6 echo requests seen for the watched prefixes"
pub static RIB_ROUTES: LazyLock<IntGauge> = LazyLock::new(|| {
register_int_gauge!(
"bgp_rib_routes",
"Routes learned from peers, counting each peer's separately"
)
.expect("metric registration")
});
pub static PING_EVENTS_SUPPRESSED: LazyLock<IntCounter> = LazyLock::new(|| {
register_int_counter!(
"ping_monitor_events_suppressed_total",
"Ping events not published because of the per-second limit"
pub static RIB_PREFIXES: LazyLock<IntGauge> = LazyLock::new(|| {
register_int_gauge!(
"bgp_rib_prefixes",
"Prefixes with at least one route, and so with a best route"
)
.expect("metric registration")
});
pub static RIB_ATTRIBUTE_SETS: LazyLock<IntGauge> = LazyLock::new(|| {
register_int_gauge!(
"bgp_rib_attribute_sets",
"Distinct sets of path attributes held for the routes"
)
.expect("metric registration")
});
pub static RIB_JOURNAL_ENTRIES: LazyLock<IntGauge> = LazyLock::new(|| {
register_int_gauge!(
"bgp_rib_journal_entries",
"Changes of best route waiting for a follower to read them"
)
.expect("metric registration")
});
@@ -67,13 +102,15 @@ pub static PING_EVENTS_SUPPRESSED: LazyLock<IntCounter> = LazyLock::new(|| {
pub fn init() {
LazyLock::force(&MESSAGES_SENT);
LazyLock::force(&MESSAGES_RECEIVED);
LazyLock::force(&UPDATE_ERRORS);
LazyLock::force(&SESSIONS_ESTABLISHED_TOTAL);
LazyLock::force(&SESSIONS_ESTABLISHED);
LazyLock::force(&COLLISIONS);
LazyLock::force(&LOCAL_ROUTES);
LazyLock::force(&ADJ_RIB_IN_ROUTES);
LazyLock::force(&PING_ECHO_REQUESTS);
LazyLock::force(&PING_EVENTS_SUPPRESSED);
LazyLock::force(&RIB_ROUTES);
LazyLock::force(&RIB_PREFIXES);
LazyLock::force(&RIB_ATTRIBUTE_SETS);
LazyLock::force(&RIB_JOURNAL_ENTRIES);
}
pub fn message_type_name(msg: &Message) -> &'static str {
@@ -82,6 +119,6 @@ pub fn message_type_name(msg: &Message) -> &'static str {
Message::Update(_) => "update",
Message::Notification(_) => "notification",
Message::KeepAlive => "keepalive",
Message::RouteRefresh => "route_refresh",
Message::RouteRefresh(_) => "route_refresh",
}
}
crates/server/src/panic.rs
+60
View File
@@ -0,0 +1,60 @@
//! Containing panics.
//!
//! A panic is a bug. It unwinds the task it happens in, and what that task
//! was responsible for still has to be finished properly: a session tells
//! its peer it is closing and gives up its routes; the server, if it lost
//! something it cannot do without, closes every session and exits.
use std::any::Any;
use std::panic::AssertUnwindSafe;
use futures::FutureExt;
/// Runs `future` to completion. If it panics, the panic is caught and its
/// message returned, so that the caller can clean up after it.
///
/// The caller must not go on using what the future was working on as if
/// nothing had happened: a panic can leave it half changed. Closing it down
/// is what this is for.
pub(crate) async fn contained<F: Future>(future: F) -> Result<F::Output, String> {
AssertUnwindSafe(future)
.catch_unwind()
.await
.map_err(|payload| message(payload.as_ref()))
}
/// The text of a panic, from what `catch_unwind` or a `JoinError` carries.
pub(crate) fn message(payload: &(dyn Any + Send)) -> String {
if let Some(text) = payload.downcast_ref::<&'static str>() {
(*text).to_string()
} else if let Some(text) = payload.downcast_ref::<String>() {
text.clone()
} else {
"a panic that carried no message".to_string()
}
}
#[cfg(test)]
mod tests {
use super::*;
#[tokio::test]
async fn a_panic_is_caught_with_its_message() {
assert_eq!(contained(async { 7 }).await, Ok(7));
let literal: Result<(), _> = contained(async { panic!("the table is gone") }).await;
assert_eq!(literal, Err("the table is gone".to_string()));
let formatted: Result<(), _> =
contained(async { panic!("index {} out of range", 9) }).await;
assert_eq!(formatted, Err("index 9 out of range".to_string()));
// After a point where the future had been suspended, too.
let later: Result<(), _> = contained(async {
tokio::task::yield_now().await;
panic!("later")
})
.await;
assert_eq!(later, Err("later".to_string()));
}
}
crates/server/src/ping.rs
-412
View File
@@ -1,412 +0,0 @@
//! Shows who is pinging the prefixes this router originates.
//!
//! The host answers for the lab address space itself (an AnyIP route on the
//! loopback interface), so the kernel already replies to every ping. A raw
//! ICMPv6 socket gets a copy of each echo request as well; we never reply and
//! never see forwarded traffic. For every packet we learn the source, the
//! destination, the interface it arrived on and the hop limit it arrived
//! with, which is enough to show a student that their route is in use and
//! which way the traffic came in.
//!
//! Anyone on the Internet can send these packets, with any source address, so
//! everything here is bounded: a fixed number of flows is remembered and the
//! number of events published per second is capped.
#[cfg(unix)]
mod sys;
use std::collections::HashMap;
use std::net::Ipv6Addr;
use std::sync::Arc;
use std::time::{Duration, SystemTime};
use bgp_primitives::constants::AddressFamilyId;
use bgp_primitives::ip_prefix::IpPrefix;
use bgp_rib::{LocalRib, LocalRoute};
use tokio::sync::{Mutex, broadcast};
use tokio::time::Instant;
use tracing::{error, info, warn};
use crate::metrics::{PING_ECHO_REQUESTS, PING_EVENTS_SUPPRESSED};
/// Flows (source and destination pairs) remembered. When full, the flow that
/// has been quiet the longest makes room.
const MAX_FLOWS: usize = 2048;
/// A flow that keeps pinging publishes an event this often, carrying the
/// running count.
const EVENT_INTERVAL: Duration = Duration::from_secs(2);
/// A flow that was quiet for this long starts a new burst, which is what the
/// event log reports.
const BURST_GAP: Duration = Duration::from_secs(10);
/// How much of each packet is kept for display.
pub const MAX_PACKET_BYTES: usize = 256;
/// Events published per second across all flows. Packets beyond this are
/// still counted and show up in [`PingMonitor::flows`].
const MAX_EVENTS_PER_SECOND: u32 = 50;
/// One echo request, as seen by the socket.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct PingObservation {
pub src: Ipv6Addr,
pub dst: Ipv6Addr,
/// Name of the interface the packet arrived on.
pub interface: Option<String>,
/// Hop limit left when the packet arrived. Senders start at 64, 128 or
/// 255, so this tells roughly how many routers the packet crossed.
pub hop_limit: Option<u8>,
pub ident: u16,
pub seq: u16,
/// Size of the ICMPv6 message, header included.
pub bytes: usize,
/// The ICMPv6 message as received, from its type byte on, cut off after
/// [`MAX_PACKET_BYTES`]. The IPv6 header is not part of it: a raw ICMPv6
/// socket never sees one, which is why the fields above come separately.
pub packet: Vec<u8>,
}
/// The local route a ping was addressed to.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct PingedRoute {
pub prefix: IpPrefix,
pub owner: String,
}
/// All pings from one source to one destination.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct PingFlow {
/// The most recent packet.
pub last: PingObservation,
pub count: u64,
pub first_seen: SystemTime,
pub last_seen: SystemTime,
/// The most specific local route covering the destination right now;
/// `None` when nobody announces it.
pub route: Option<PingedRoute>,
}
#[derive(Debug, Clone)]
pub struct PingEvent {
pub flow: PingFlow,
/// True for the first event of a burst: a new flow, or one that had gone
/// quiet. False for the periodic updates while it keeps pinging.
pub new_burst: bool,
}
/// Whether the monitor is watching, for display.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum PingMonitorStatus {
/// Not configured.
Disabled,
Listening,
/// Configured, but the socket could not be opened.
Failed(String),
}
struct FlowState {
flow: PingFlow,
last_packet: Instant,
last_event: Option<Instant>,
/// A burst started and its first event has not gone out yet.
burst_pending: bool,
}
#[derive(Default)]
struct Table {
flows: HashMap<(Ipv6Addr, Ipv6Addr), FlowState>,
window_start: Option<Instant>,
window_events: u32,
}
impl Table {
fn take_event_slot(&mut self, now: Instant) -> bool {
if self.window_start.is_none_or(|start| now.duration_since(start) >= Duration::from_secs(1)) {
self.window_start = Some(now);
self.window_events = 0;
}
if self.window_events >= MAX_EVENTS_PER_SECOND {
return false;
}
self.window_events += 1;
true
}
fn make_room(&mut self) {
if self.flows.len() < MAX_FLOWS {
return;
}
if let Some(key) = self.flows.iter().min_by_key(|(_, s)| s.last_packet).map(|(k, _)| *k) {
self.flows.remove(&key);
}
}
}
struct Inner {
prefixes: Vec<IpPrefix>,
local_rib: Option<LocalRib>,
status: PingMonitorStatus,
table: Mutex<Table>,
events: broadcast::Sender<PingEvent>,
}
/// Handle to the ping monitor. Cheap to clone.
#[derive(Clone)]
pub struct PingMonitor {
inner: Arc<Inner>,
}
impl PingMonitor {
fn new(prefixes: Vec<IpPrefix>, local_rib: Option<LocalRib>, status: PingMonitorStatus) -> Self {
let (events, _) = broadcast::channel(256);
Self { inner: Arc::new(Inner { prefixes, local_rib, status, table: Mutex::default(), events }) }
}
/// A monitor that watches nothing.
pub fn disabled() -> Self {
Self::new(vec![], None, PingMonitorStatus::Disabled)
}
/// A monitor without a socket, fed through [`PingMonitor::record`].
pub fn without_socket(prefixes: Vec<IpPrefix>, local_rib: Option<LocalRib>) -> Self {
Self::new(prefixes, local_rib, PingMonitorStatus::Listening)
}
/// Opens the socket and starts watching for pings to `prefixes`. Failing
/// to open it (usually: no CAP_NET_RAW) is not fatal; the daemon runs on
/// without the monitor and the status page says why.
pub fn start(prefixes: Vec<IpPrefix>, local_rib: LocalRib) -> Self {
#[cfg(unix)]
let socket = sys::EchoSocket::open();
#[cfg(not(unix))]
let socket: std::io::Result<()> = Err(std::io::Error::other("not supported on this platform"));
match socket {
Ok(socket) => {
let shown: Vec<String> = prefixes.iter().map(ToString::to_string).collect();
info!(prefixes = ?shown, "Ping monitor listening for ICMPv6 echo requests");
let monitor = Self::new(prefixes, Some(local_rib), PingMonitorStatus::Listening);
#[cfg(unix)]
tokio::spawn(monitor.clone().run(socket));
monitor
}
Err(e) => {
warn!(err = %e, "Ping monitor disabled: cannot open a raw ICMPv6 socket (needs CAP_NET_RAW)");
Self::new(prefixes, None, PingMonitorStatus::Failed(e.to_string()))
}
}
}
#[cfg(unix)]
async fn run(self, mut socket: sys::EchoSocket) {
loop {
match socket.recv().await {
Ok(observation) => self.record(observation).await,
Err(e) => {
error!(err = %e, "Ping monitor socket failed");
tokio::time::sleep(Duration::from_secs(1)).await;
}
}
}
}
pub fn status(&self) -> &PingMonitorStatus {
&self.inner.status
}
/// The prefixes being watched.
pub fn prefixes(&self) -> &[IpPrefix] {
&self.inner.prefixes
}
pub fn subscribe(&self) -> broadcast::Receiver<PingEvent> {
self.inner.events.subscribe()
}
/// Accounts for one echo request. Packets to destinations outside the
/// watched prefixes are ignored.
pub async fn record(&self, observation: PingObservation) {
let destination = host_prefix(observation.dst);
if !self.inner.prefixes.iter().any(|p| p.contains(&destination)) {
return;
}
PING_ECHO_REQUESTS.inc();
let now = Instant::now();
let wall = SystemTime::now();
let key = (observation.src, observation.dst);
let mut guard = self.inner.table.lock().await;
let table = &mut *guard;
let is_new = !table.flows.contains_key(&key);
if is_new {
table.make_room();
}
let state = table.flows.entry(key).or_insert_with(|| FlowState {
flow: PingFlow { last: observation.clone(), count: 0, first_seen: wall, last_seen: wall, route: None },
last_packet: now,
last_event: None,
burst_pending: false,
});
if is_new || now.duration_since(state.last_packet) >= BURST_GAP {
state.burst_pending = true;
}
state.flow.last = observation;
state.flow.count += 1;
state.flow.last_seen = wall;
state.last_packet = now;
let due = state.burst_pending || state.last_event.is_none_or(|t| now.duration_since(t) >= EVENT_INTERVAL);
if !due {
return;
}
let new_burst = state.burst_pending;
let mut flow = state.flow.clone();
if !table.take_event_slot(now) {
PING_EVENTS_SUPPRESSED.inc();
return;
}
if let Some(state) = table.flows.get_mut(&key) {
state.last_event = Some(now);
state.burst_pending = false;
}
drop(guard);
flow.route = resolve(&self.local_routes().await, &destination);
// No subscribers is fine.
let _ = self.inner.events.send(PingEvent { flow, new_burst });
}
/// Every remembered flow, most recently seen first.
pub async fn flows(&self) -> Vec<PingFlow> {
let mut flows: Vec<PingFlow> = self.inner.table.lock().await.flows.values().map(|s| s.flow.clone()).collect();
let routes = self.local_routes().await;
for flow in &mut flows {
flow.route = resolve(&routes, &host_prefix(flow.last.dst));
}
flows.sort_by(|a, b| b.last_seen.cmp(&a.last_seen).then_with(|| a.last.dst.cmp(&b.last.dst)));
flows
}
async fn local_routes(&self) -> Vec<LocalRoute> {
match &self.inner.local_rib {
Some(rib) => rib.snapshot().await,
None => vec![],
}
}
}
fn host_prefix(addr: Ipv6Addr) -> IpPrefix {
IpPrefix { address_family: AddressFamilyId::Ipv6, prefix: addr.octets().to_vec(), length: 128 }
}
fn resolve(routes: &[LocalRoute], destination: &IpPrefix) -> Option<PingedRoute> {
routes
.iter()
.filter(|r| r.prefix.contains(destination))
.max_by_key(|r| r.prefix.length)
.map(|r| PingedRoute { prefix: r.prefix.clone(), owner: r.owner.clone() })
}
#[cfg(test)]
mod tests {
use super::*;
use bgp_rib::{LocalRibPolicy, LocalRouteAttrs};
fn lab() -> Vec<IpPrefix> {
vec![IpPrefix::try_from("2a0d:d741::/40").unwrap()]
}
fn ping(src: &str, dst: &str, seq: u16) -> PingObservation {
PingObservation {
src: src.parse().unwrap(),
dst: dst.parse().unwrap(),
interface: Some("swissix0".into()),
hop_limit: Some(57),
ident: 7,
seq,
bytes: 64,
packet: vec![128, 0, 0, 0, 0, 7, (seq >> 8) as u8, seq as u8],
}
}
#[tokio::test(start_paused = true)]
async fn counts_flows_and_names_the_route_owner() {
let rib = LocalRib::spawn(LocalRibPolicy::default());
let prefix = IpPrefix::try_from("2a0d:d741:1a::/48").unwrap();
rib.announce(LocalRoute::new(prefix.clone(), "alice", LocalRouteAttrs::default())).await.unwrap();
let monitor = PingMonitor::without_socket(lab(), Some(rib.clone()));
let mut events = monitor.subscribe();
monitor.record(ping("2001:67c:10ec::5", "2a0d:d741:1a::1", 0)).await;
monitor.record(ping("2001:67c:10ec::5", "2a0d:d741:1a::1", 1)).await;
monitor.record(ping("2001:67c:10ec::5", "2a0d:d741:5::1", 0)).await;
let first = events.try_recv().unwrap();
assert!(first.new_burst);
assert_eq!(first.flow.route, Some(PingedRoute { prefix, owner: "alice".into() }));
let other = events.try_recv().unwrap();
assert_eq!(other.flow.route, None, "nobody announces 2a0d:d741:5::/48");
assert!(events.try_recv().is_err(), "the second packet of a flow is not an event of its own");
let flows = monitor.flows().await;
assert_eq!(flows.len(), 2);
let alice = flows.iter().find(|f| f.route.is_some()).unwrap();
assert_eq!((alice.count, alice.last.seq), (2, 1));
// The owner shown follows the routing table, not the first packet.
rib.withdraw(alice.route.as_ref().unwrap().prefix.clone(), None).await.unwrap();
assert!(monitor.flows().await.iter().all(|f| f.route.is_none()));
}
#[tokio::test(start_paused = true)]
async fn ignores_destinations_outside_the_watched_prefixes() {
let monitor = PingMonitor::without_socket(lab(), None);
monitor.record(ping("2001:db8::1", "2a0d:d740::1", 0)).await;
monitor.record(ping("2001:db8::1", "2001:7f8:24::c9", 0)).await;
assert!(monitor.flows().await.is_empty());
}
#[tokio::test(start_paused = true)]
async fn a_steady_ping_is_an_occasional_update_and_a_pause_starts_a_new_burst() {
let monitor = PingMonitor::without_socket(lab(), None);
let mut events = monitor.subscribe();
let mut seen = vec![];
for seq in 0..6 {
monitor.record(ping("2001:db8::1", "2a0d:d741:1::1", seq)).await;
while let Ok(ev) = events.try_recv() {
seen.push((ev.flow.count, ev.new_burst));
}
tokio::time::advance(Duration::from_secs(1)).await;
}
assert_eq!(seen, vec![(1, true), (3, false), (5, false)]);
tokio::time::advance(BURST_GAP).await;
monitor.record(ping("2001:db8::1", "2a0d:d741:1::1", 6)).await;
let ev = events.try_recv().unwrap();
assert_eq!((ev.flow.count, ev.new_burst), (7, true));
}
#[tokio::test(start_paused = true)]
async fn a_flood_of_spoofed_sources_is_bounded() {
let monitor = PingMonitor::without_socket(lab(), None);
let mut events = monitor.subscribe();
for n in 0..(MAX_FLOWS as u32 + 500) {
let src = Ipv6Addr::from(0x2001_0db8_0000_0000_0000_0000_0000_0000u128 + n as u128);
monitor.record(ping(&src.to_string(), "2a0d:d741:1::1", 0)).await;
}
assert_eq!(monitor.flows().await.len(), MAX_FLOWS);
let mut published = 0;
while events.try_recv().is_ok() {
published += 1;
}
assert_eq!(published, MAX_EVENTS_PER_SECOND, "time stood still, so this was all one second");
// A real student still gets through once the flood's second is over,
// even though the first attempt at announcing the burst was dropped.
monitor.record(ping("2001:67c:10ec::5", "2a0d:d741:1::1", 0)).await;
assert!(events.try_recv().is_err());
tokio::time::advance(Duration::from_secs(1)).await;
monitor.record(ping("2001:67c:10ec::5", "2a0d:d741:1::1", 1)).await;
let ev = events.try_recv().unwrap();
assert_eq!((ev.flow.count, ev.new_burst), (2, true));
}
}
crates/server/src/ping/sys.rs
-265
View File
@@ -1,265 +0,0 @@
//! The raw ICMPv6 socket behind the ping monitor.
//!
//! All the `unsafe` in the daemon lives here: the socket options and the
//! ancillary data we need (`ICMP6_FILTER`, `IPV6_PKTINFO`, `IPV6_HOPLIMIT`)
//! have no safe wrapper in the standard library or tokio.
use std::collections::HashMap;
use std::ffi::CStr;
use std::io;
use std::mem::{size_of, size_of_val, zeroed};
use std::net::Ipv6Addr;
use std::os::fd::{AsRawFd, FromRawFd, OwnedFd, RawFd};
use libc::{c_int, c_void};
use tokio::io::Interest;
use tokio::io::unix::AsyncFd;
use super::{MAX_PACKET_BYTES, PingObservation};
const ICMP6_ECHO_REQUEST: u8 = 128;
#[cfg(any(target_os = "linux", target_os = "android"))]
const ICMP6_FILTER: c_int = 1;
#[cfg(not(any(target_os = "linux", target_os = "android")))]
const ICMP6_FILTER: c_int = 18;
/// `struct icmp6_filter`: one bit per ICMPv6 type.
type Icmp6Filter = [u32; 8];
/// A filter that lets only echo requests through. Linux and the BSDs agree on
/// the layout and disagree on the meaning: on Linux a set bit blocks the
/// type, on the BSDs (and macOS) a set bit passes it.
fn echo_request_only() -> Icmp6Filter {
let (word, bit) = ((ICMP6_ECHO_REQUEST >> 5) as usize, 1u32 << (ICMP6_ECHO_REQUEST & 31));
if cfg!(any(target_os = "linux", target_os = "android")) {
let mut filter = [u32::MAX; 8];
filter[word] &= !bit;
filter
} else {
let mut filter = [0u32; 8];
filter[word] |= bit;
filter
}
}
fn check(ret: c_int) -> io::Result<c_int> {
if ret < 0 { Err(io::Error::last_os_error()) } else { Ok(ret) }
}
fn setsockopt<T>(fd: RawFd, level: c_int, name: c_int, value: &T) -> io::Result<()> {
// SAFETY: `value` is a live reference and the length given is its size.
check(unsafe {
libc::setsockopt(fd, level, name, (value as *const T).cast::<c_void>(), size_of::<T>() as libc::socklen_t)
})?;
Ok(())
}
/// Receives a copy of every ICMPv6 echo request the kernel delivers to this
/// host. The kernel still answers them; we never send anything.
pub struct EchoSocket {
fd: AsyncFd<OwnedFd>,
/// Interface names by index. Looked up once: interfaces on a router do
/// not change name, and under a ping flood the lookup would be three
/// system calls per packet.
interfaces: HashMap<u32, Option<String>>,
}
impl EchoSocket {
/// Opens the socket. Needs CAP_NET_RAW on Linux. macOS also hands echo
/// requests to unprivileged ICMPv6 datagram sockets, which is enough to
/// try the monitor out on a laptop. (On Linux those only ever see the
/// replies to their own pings, so there is no such fallback there.)
pub fn open() -> io::Result<Self> {
Self::open_as(libc::SOCK_RAW).or_else(|e| {
if cfg!(target_os = "macos") { Self::open_as(libc::SOCK_DGRAM) } else { Err(e) }
})
}
fn open_as(socket_type: c_int) -> io::Result<Self> {
// SAFETY: plain system call; the result is checked before use.
let raw = check(unsafe { libc::socket(libc::AF_INET6, socket_type, libc::IPPROTO_ICMPV6) })?;
// SAFETY: `raw` is a descriptor we just opened and nobody else owns.
let fd = unsafe { OwnedFd::from_raw_fd(raw) };
// SAFETY: fcntl on a descriptor we own, with integer arguments.
unsafe {
let flags = check(libc::fcntl(raw, libc::F_GETFL))?;
check(libc::fcntl(raw, libc::F_SETFL, flags | libc::O_NONBLOCK))?;
check(libc::fcntl(raw, libc::F_SETFD, libc::FD_CLOEXEC))?;
}
setsockopt(raw, libc::IPPROTO_ICMPV6, ICMP6_FILTER, &echo_request_only())?;
let on: c_int = 1;
setsockopt(raw, libc::IPPROTO_IPV6, libc::IPV6_RECVPKTINFO, &on)?;
setsockopt(raw, libc::IPPROTO_IPV6, libc::IPV6_RECVHOPLIMIT, &on)?;
Ok(Self { fd: AsyncFd::with_interest(fd, Interest::READABLE)?, interfaces: HashMap::new() })
}
/// Waits for the next echo request.
pub async fn recv(&mut self) -> io::Result<PingObservation> {
loop {
let mut guard = self.fd.readable().await?;
let packet = match guard.try_io(|fd| recv_packet(fd.as_raw_fd())) {
Ok(result) => result?,
Err(_would_block) => continue,
};
let Some(packet) = packet else { continue };
let interface = self.interfaces.entry(packet.ifindex).or_insert_with(|| interface_name(packet.ifindex)).clone();
return Ok(PingObservation {
src: packet.src,
dst: packet.dst,
interface,
hop_limit: packet.hop_limit,
ident: packet.ident,
seq: packet.seq,
bytes: packet.bytes,
packet: packet.data,
});
}
}
}
struct Packet {
src: Ipv6Addr,
dst: Ipv6Addr,
ifindex: u32,
hop_limit: Option<u8>,
ident: u16,
seq: u16,
bytes: usize,
data: Vec<u8>,
}
/// Reads one packet. `Ok(None)` is a packet we cannot use (not an echo
/// request, or the kernel did not say where it was going).
fn recv_packet(fd: RawFd) -> io::Result<Option<Packet>> {
// A raw ICMPv6 socket delivers the ICMPv6 message without the IPv6
// header, so everything about the header comes from ancillary data.
let mut data = [0u8; 2048];
let mut control = [0u64; 16]; // u64 for the alignment cmsghdr needs
// SAFETY: all-zero is a valid value for these plain C structs.
let mut from: libc::sockaddr_in6 = unsafe { zeroed() };
let mut iov = libc::iovec { iov_base: data.as_mut_ptr().cast(), iov_len: data.len() };
// SAFETY: as above. msghdr has private padding on some targets, so it
// cannot be built with a struct literal.
let mut msg: libc::msghdr = unsafe { zeroed() };
msg.msg_name = (&mut from as *mut libc::sockaddr_in6).cast();
msg.msg_namelen = size_of::<libc::sockaddr_in6>() as _;
msg.msg_iov = &mut iov;
msg.msg_iovlen = 1 as _;
msg.msg_control = control.as_mut_ptr().cast();
msg.msg_controllen = size_of_val(&control) as _;
// SAFETY: every pointer in `msg` refers to a local that outlives the call
// and the lengths given match the buffers.
let n = unsafe { libc::recvmsg(fd, &mut msg, 0) };
if n < 0 {
return Err(io::Error::last_os_error());
}
let n = n as usize;
if n < 8 || data[0] != ICMP6_ECHO_REQUEST {
return Ok(None);
}
let mut dst = None;
let mut hop_limit = None;
// SAFETY: the CMSG_* macros walk the control buffer the kernel just
// filled in, bounded by msg_controllen. Payloads are read unaligned.
unsafe {
let mut cmsg = libc::CMSG_FIRSTHDR(&msg);
while !cmsg.is_null() {
let header = &*cmsg;
if header.cmsg_level == libc::IPPROTO_IPV6 {
if header.cmsg_type == libc::IPV6_PKTINFO {
let info: libc::in6_pktinfo = std::ptr::read_unaligned(libc::CMSG_DATA(cmsg).cast());
dst = Some((Ipv6Addr::from(info.ipi6_addr.s6_addr), info.ipi6_ifindex as u32));
} else if header.cmsg_type == libc::IPV6_HOPLIMIT {
let value: c_int = std::ptr::read_unaligned(libc::CMSG_DATA(cmsg).cast());
hop_limit = u8::try_from(value).ok();
}
}
cmsg = libc::CMSG_NXTHDR(&msg, cmsg);
}
}
let Some((dst, ifindex)) = dst else { return Ok(None) };
Ok(Some(Packet {
src: Ipv6Addr::from(from.sin6_addr.s6_addr),
dst,
ifindex,
hop_limit,
ident: u16::from_be_bytes([data[4], data[5]]),
seq: u16::from_be_bytes([data[6], data[7]]),
bytes: n,
data: data[..n.min(MAX_PACKET_BYTES)].to_vec(),
}))
}
fn interface_name(index: u32) -> Option<String> {
let mut name = [0 as libc::c_char; libc::IF_NAMESIZE];
// SAFETY: the buffer is IF_NAMESIZE bytes, as if_indextoname requires,
// and on success holds a NUL-terminated string.
unsafe {
if libc::if_indextoname(index, name.as_mut_ptr()).is_null() {
return None;
}
Some(CStr::from_ptr(name.as_ptr()).to_string_lossy().into_owned())
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn filter_passes_only_echo_requests() {
let filter = echo_request_only();
let blocked = |t: u8| {
let set = filter[(t >> 5) as usize] & (1 << (t & 31)) != 0;
if cfg!(any(target_os = "linux", target_os = "android")) { set } else { !set }
};
assert!(!blocked(128), "echo request passes");
for t in [1u8, 3, 129, 133, 134, 135, 136] {
assert!(blocked(t), "type {t} is dropped in the kernel");
}
}
/// Pings ::1 and expects to see the request. Skipped where the socket
/// cannot be opened, i.e. on Linux without CAP_NET_RAW.
#[tokio::test]
async fn sees_a_ping_to_loopback() {
let mut socket = match EchoSocket::open() {
Ok(socket) => socket,
Err(e) => return eprintln!("skipped: cannot open an ICMPv6 socket: {e}"),
};
let ping = if cfg!(target_os = "macos") { "ping6" } else { "ping" };
let Ok(mut child) = std::process::Command::new(ping)
.args(["-c", "1", "::1"])
.stdout(std::process::Stdio::null())
.spawn()
else {
return eprintln!("skipped: no {ping} command");
};
let seen = tokio::time::timeout(std::time::Duration::from_secs(5), async {
loop {
let ping = socket.recv().await.unwrap();
if ping.dst == Ipv6Addr::LOCALHOST {
return ping;
}
}
})
.await;
let _ = child.wait();
let seen = seen.expect("echo request to ::1 was not seen");
assert_eq!(seen.src, Ipv6Addr::LOCALHOST);
assert_eq!(seen.hop_limit, Some(64));
assert!(seen.seq <= 1, "first sequence number is 0 (BSD ping6) or 1 (iputils)");
assert!(seen.interface.as_deref().is_some_and(|name| name.starts_with("lo")), "{:?}", seen.interface);
assert!(seen.bytes >= 8);
assert_eq!(seen.packet[..2], [128, 0], "echo request, code 0");
assert_eq!(seen.packet[4..8], [seen.ident.to_be_bytes(), seen.seq.to_be_bytes()].concat()[..]);
}
}
crates/server/src/routes.rs
+64
View File
@@ -0,0 +1,64 @@
//! Putting the routes of a received UPDATE into the routing table.
use std::sync::Arc;
use bgp_packet::message::{NlriNextHop, PathAttribute, UpdateMessage};
use bgp_primitives::ip_prefix::Nlri;
use bgp_primitives::parser::EncodeError;
use bgp_rib::table::{AttrSet, Outcome, PeerIndex, RibTable};
/// Applies `update`, received from `peer`, to `table`: its withdrawals
/// first and then its announcements, so that a prefix named in both ends up
/// announced (RFC 4271 §4.3).
///
/// An UPDATE names prefixes in four places: withdrawn routes and NLRI in the
/// message body, which are IPv4, and the MP_UNREACH_NLRI and MP_REACH_NLRI
/// attributes for every other family (RFC 4760). Where the peer sends
/// several routes for a prefix (ADD-PATH, RFC 7911), each is named by its
/// prefix and path identifier together, and is a route of its own.
///
/// `attributes_wire` is the attributes as the decoder kept them, if it did
/// (see `DecodedUpdate::attributes_wire`). With it the table finds the
/// attribute set from those bytes, without writing the attributes out again.
pub(crate) fn apply(
table: &RibTable,
peer: PeerIndex,
update: &UpdateMessage,
attributes_wire: Option<&[u8]>,
) -> Result<Outcome, EncodeError> {
let mut outcome = Outcome::default();
if !update.withdrawn_routes.is_empty() {
outcome += table.withdraw(peer, &update.withdrawn_routes);
}
for attribute in &update.path_attributes {
if let PathAttribute::MpUnreachNlri(mp) = attribute {
outcome += table.withdraw(peer, &mp.prefixes);
}
}
let attributes = |next_hop: Option<&NlriNextHop>| -> Result<Arc<AttrSet>, EncodeError> {
if let Some(set) =
attributes_wire.and_then(|wire| table.attrs().intern_received(wire, next_hop))
{
return Ok(set);
}
table.attrs().intern(&update.path_attributes, next_hop)
};
let mut announce = |nlri: &[Nlri], next_hop| -> Result<(), EncodeError> {
if !nlri.is_empty() {
outcome += table.announce(peer, &attributes(next_hop)?, nlri);
}
Ok(())
};
// Routes in the body have their next hop in the NEXT_HOP attribute,
// which is one of the set's attributes; routes in MP_REACH_NLRI bring
// theirs along.
announce(&update.announced_routes, None)?;
for attribute in &update.path_attributes {
if let PathAttribute::MpReachNlri(mp) = attribute {
announce(&mp.prefixes, Some(&mp.next_hop))?;
}
}
Ok(outcome)
}
crates/server/src/server.rs
+227 -70
View File
@@ -1,80 +1,96 @@
//! The server: listeners, the tasks that run for as long as it does, and
//! what happens when one of them stops.
use std::io;
use std::net::SocketAddr;
use std::pin::Pin;
use std::sync::Arc;
use std::time::Duration;
use eyre::{Result, WrapErr};
use futures::stream::{FuturesUnordered, StreamExt};
use tokio::net::{TcpListener, TcpStream};
use tracing::{error, info};
use tokio::net::{TcpListener, TcpSocket, TcpStream};
use tokio::task::{JoinError, JoinHandle};
use tracing::{error, info, warn};
use bgp_rib::{AdjRibIn, LocalRib};
use bgp_rib::table::RibTable;
use bgp_rib::{LocalRib, RibStopped};
use crate::advertise::spawn_advertiser;
use crate::collision::{CollisionTable, Direction};
use crate::config::Config;
use crate::connection::handle_connection;
use crate::connection::{PeerSettings, ServerContext, run_session};
use crate::dialer::spawn_dialers;
use crate::fsm::SessionConfig;
use crate::error::ServerError;
use crate::panic;
use crate::session::{ConfiguredPeer, SessionRegistry};
pub struct BgpServerBuilder {
config: Config,
}
/// How long sessions get to send their Cease when the server stops.
const SHUTDOWN_GRACE: Duration = Duration::from_secs(2);
/// A `BgpServerBuilder` that has bound all its listeners and is ready to accept.
/// Call [`BgpServer::run`] to start the accept loop.
/// How long to wait before accepting again after an accept failed. What
/// makes it fail (no file descriptors left, usually) does not pass at once,
/// and trying again in a loop would only fill the log.
const ACCEPT_RETRY: Duration = Duration::from_millis(200);
/// How often attribute sets that no route uses any more are dropped. They
/// are not dropped as their last route goes (see `AttrStore::sweep`), and a
/// table whose routes keep changing attributes collects them.
const SWEEP_INTERVAL: Duration = Duration::from_secs(60);
/// How many connections may wait to be accepted on a listener.
const LISTEN_BACKLOG: u32 = 1024;
/// A BGP speaker that has bound its listeners and is ready to accept.
/// Call [`BgpServer::run`] to start it.
pub struct BgpServer {
config: Config,
listeners: Vec<(SocketAddr, TcpListener)>,
/// Shared RIB handle — created at bind time so callers can clone it before
/// the accept loop starts (e.g. tests that want to introspect the RIB).
pub adj_rib_in: AdjRibIn,
/// The routing table: every route learned from peers, and the best one
/// for each prefix. Created at bind time so that callers can share it
/// before the accept loop starts.
pub rib: Arc<RibTable>,
/// Routes this router originates; announced to every established peer.
pub local_rib: LocalRib,
/// Live view of established sessions.
pub registry: SessionRegistry,
}
impl BgpServerBuilder {
pub fn new(config: Config) -> Self {
Self { config }
}
/// Binds all configured listen endpoints and returns a [`BgpServer`].
/// Separating bind from run lets callers (e.g. tests) inspect the actual
/// bound addresses before the accept loop starts.
pub async fn bind(self) -> Result<BgpServer> {
let mut listeners = Vec::with_capacity(self.config.listen_endpoints.len());
for &addr in &self.config.listen_endpoints {
let listener = TcpListener::bind(addr)
.await
.wrap_err_with(|| format!("Failed to bind listener on {addr}"))?;
impl BgpServer {
/// Binds all configured listen endpoints. Separate from
/// [`BgpServer::run`] so that callers (tests, say) can learn the
/// addresses actually bound before any connection is accepted.
pub async fn bind(config: Config) -> Result<Self, ServerError> {
let mut listeners = Vec::with_capacity(config.listen_endpoints.len());
for &addr in &config.listen_endpoints {
let listener = listen(addr).map_err(|source| ServerError::Bind { addr, source })?;
let bound_addr = listener.local_addr()?;
info!(%bound_addr, "Listening for BGP connections");
listeners.push((bound_addr, listener));
}
// Routes from the configuration go in before any session exists;
// each session is sent the whole table when it comes up.
let local_rib = LocalRib::spawn();
for route in &config.originate {
local_rib.announce(route.clone()).await?;
}
let configured = config
.peers
.iter()
.map(|p| ConfiguredPeer {
name: p.name.clone(),
endpoint: p.endpoint,
remote_asn: p.remote_asn,
})
.collect();
Ok(BgpServer {
listeners,
adj_rib_in: AdjRibIn::spawn(),
local_rib: LocalRib::spawn(self.config.local_route_policy.clone()),
registry: SessionRegistry::with_configured(
self.config
.peers
.iter()
.map(|p| ConfiguredPeer { name: p.name.clone(), endpoint: p.endpoint, remote_asn: p.remote_asn })
.collect(),
),
config: self.config,
rib: Arc::new(RibTable::new(table_shards())),
local_rib,
registry: SessionRegistry::with_configured(configured),
config,
})
}
/// Convenience method: bind then run immediately.
pub async fn run(self) -> Result<()> {
self.bind().await?.run().await
}
}
impl BgpServer {
pub fn config(&self) -> &Config {
&self.config
}
@@ -84,7 +100,9 @@ impl BgpServer {
self.listeners.iter().map(|(addr, _)| *addr).collect()
}
pub async fn run(self) -> Result<()> {
/// Runs the server. Returns only if something it cannot do without has
/// failed, after closing its sessions.
pub async fn run(self) -> Result<(), ServerError> {
crate::metrics::init();
info!(
router_id = %self.config.router_id,
@@ -93,24 +111,30 @@ impl BgpServer {
"BGP server started"
);
let session_config = SessionConfig {
let context = Arc::new(ServerContext {
router_id: self.config.router_id,
local_asn: self.config.local_asn,
hold_time_secs: self.config.hold_time_secs,
adj_rib_in: self.adj_rib_in.clone(),
rib: Arc::clone(&self.rib),
registry: self.registry.clone(),
collisions: CollisionTable::new(self.config.collision_detect_established_state),
direction: Direction::Inbound,
peer_name: None,
expected_asn: None,
};
});
let _advertiser = spawn_advertiser(
self.local_rib.clone(),
self.registry.clone(),
self.config.local_asn,
);
let _dialers = spawn_dialers(&self.config.peers, &session_config, self.config.connect_retry);
// The tasks the server cannot do without. None of them ends while
// the server runs. The advertiser comes first: it has to be
// listening before a dialer brings up the first session.
let mut tasks: FuturesUnordered<NamedTask> = FuturesUnordered::new();
tasks.push(named(
"advertiser",
spawn_advertiser(
self.local_rib.clone(),
self.registry.clone(),
self.config.local_asn,
),
));
tasks.push(named("sweeper", spawn_sweeper(Arc::clone(&self.rib))));
for dialer in spawn_dialers(&self.config.peers, &context, self.config.connect_retry) {
tasks.push(named("dialer", dialer));
}
let mut accepting: FuturesUnordered<_> = self
.listeners
@@ -118,27 +142,123 @@ impl BgpServer {
.map(|(addr, listener)| accept_one(addr, listener))
.collect();
while let Some((listen_addr, result, listener)) = accepting.next().await {
loop {
let (listen_addr, result, listener) = tokio::select! {
Some(accepted) = accepting.next() => accepted,
failure = failed(&self.local_rib, &mut tasks) => {
error!(err = %failure, "Shutting down: the server cannot continue");
close_sessions(&self.registry).await;
return Err(failure);
}
};
match result {
Ok((stream, peer_addr)) => {
// Inbound connections from a configured peer inherit its
// name, expected ASN and hold time.
let mut config = session_config.clone();
if let Some(peer) = self.config.peers.iter().find(|p| p.endpoint.ip() == peer_addr.ip()) {
config.peer_name = Some(peer.name.clone());
config.expected_asn = Some(peer.remote_asn);
config.hold_time_secs = peer.hold_time_secs;
}
tokio::spawn(handle_connection(stream, peer_addr, Arc::new(config)));
// A connection from a configured peer is run with what
// is configured for that peer.
let peer = self.config.peer_at(peer_addr.ip()).map_or_else(
|| PeerSettings::unconfigured(self.config.hold_time_secs),
PeerSettings::from,
);
tokio::spawn(serve(stream, peer_addr, Arc::clone(&context), peer));
}
Err(e) => {
error!(%listen_addr, err = %e, "Accept error");
tokio::time::sleep(ACCEPT_RETRY).await;
}
}
accepting.push(accept_one(listen_addr, listener));
}
}
}
Ok(())
/// Closes every session politely and waits a moment for them to go: each
/// peer is told we are going (Cease / Administrative Shutdown), so that it
/// stops using our routes now and not when its hold timer expires.
pub async fn close_sessions(registry: &SessionRegistry) {
let left = registry.shutdown(SHUTDOWN_GRACE).await;
if left > 0 {
warn!(sessions = left, "Some sessions did not close in time");
}
}
/// Opens a listening socket on `addr`.
fn listen(addr: SocketAddr) -> io::Result<TcpListener> {
let socket = match addr {
SocketAddr::V4(_) => TcpSocket::new_v4()?,
SocketAddr::V6(_) => {
let socket = TcpSocket::new_v6()?;
// IPv6 only. Where a system's default is to take IPv4
// connections on such a socket too, listening on both 0.0.0.0
// and [::] would fail on the second, and IPv4 peers would come
// in under IPv4-mapped addresses that match no configured peer.
socket2::SockRef::from(&socket).set_only_v6(true)?;
socket
}
};
socket.set_reuseaddr(true)?;
socket.bind(addr)?;
socket.listen(LISTEN_BACKLOG)
}
/// Runs a session on a connection a peer opened, and logs how it ended.
async fn serve(
stream: TcpStream,
peer_addr: SocketAddr,
context: Arc<ServerContext>,
peer: PeerSettings,
) {
let end = run_session(stream, peer_addr, Direction::Inbound, context, peer).await;
if let Err(e) = end.result {
warn!(%peer_addr, err = %e, "BGP session terminated with error");
}
}
/// Drops, every so often, the attribute sets no route uses any more.
fn spawn_sweeper(rib: Arc<RibTable>) -> JoinHandle<()> {
tokio::spawn(async move {
let mut tick = tokio::time::interval(SWEEP_INTERVAL);
loop {
tick.tick().await;
let rib = Arc::clone(&rib);
// Takes each of the store's locks in turn and looks at every
// set: not for a thread that sessions run on.
let _ = tokio::task::spawn_blocking(move || rib.attrs().sweep()).await;
}
})
}
/// How many shards the routing table has. Sessions write to it from every
/// core at once; a good many more shards than cores keeps two of them from
/// wanting the same one.
fn table_shards() -> usize {
16 * std::thread::available_parallelism().map_or(1, |cores| cores.get())
}
type NamedTask = Pin<Box<dyn Future<Output = (&'static str, Result<(), JoinError>)> + Send>>;
fn named(name: &'static str, task: JoinHandle<()>) -> NamedTask {
Box::pin(async move { (name, task.await) })
}
/// Resolves when something the server depends on has stopped: the local
/// RIB's actor or one of `tasks`. In a healthy server it never resolves.
///
/// Nothing else would notice. A stopped RIB is only found out by whoever
/// next uses it, and a stopped task by nobody at all: the server would go
/// on accepting sessions while storing or announcing nothing.
async fn failed(local_rib: &LocalRib, tasks: &mut FuturesUnordered<NamedTask>) -> ServerError {
tokio::select! {
() = local_rib.stopped() => RibStopped { rib: "local RIB" }.into(),
Some((task, result)) = tasks.next() => ServerError::TaskStopped {
task,
reason: match result {
Ok(()) => "it returned".into(),
Err(e) if e.is_panic() => {
format!("it panicked: {}", panic::message(e.into_panic().as_ref()))
}
Err(e) => e.to_string(),
},
},
}
}
@@ -149,3 +269,40 @@ async fn accept_one(
let result = listener.accept().await;
(addr, result, listener)
}
#[cfg(test)]
mod tests {
use super::*;
/// The advertiser and the dialers run for as long as the server does.
/// If one of them panics, nothing else would ever find out.
#[tokio::test]
async fn a_task_that_panics_is_a_failure_of_the_server() {
let local_rib = LocalRib::spawn();
let mut tasks: FuturesUnordered<NamedTask> = FuturesUnordered::new();
tasks.push(named("dialer", tokio::spawn(std::future::pending())));
tasks.push(named(
"advertiser",
tokio::spawn(async { panic!("no route to announce") }),
));
let failure = failed(&local_rib, &mut tasks).await;
assert_eq!(
failure.to_string(),
"the advertiser task stopped unexpectedly: it panicked: no route to announce"
);
}
#[tokio::test]
async fn a_stopped_rib_is_a_failure_of_the_server() {
let local_rib = LocalRib::spawn();
let mut tasks: FuturesUnordered<NamedTask> = FuturesUnordered::new();
tasks.push(named("advertiser", tokio::spawn(std::future::pending())));
local_rib.stop_for_testing();
let failure = failed(&local_rib, &mut tasks).await;
assert!(
matches!(failure, ServerError::RibStopped(RibStopped { rib }) if rib == "local RIB")
);
}
}
crates/server/src/session.rs
+87 -32
View File
@@ -15,14 +15,30 @@ use bgp_packet::errors::{BgpError, CeaseSubcode};
use bgp_packet::message::NotificationMessage;
use bgp_packet::message::Message;
use bgp_rib::PeerId;
use bgp_rib::table::PeerIndex;
use tokio::sync::{broadcast, mpsc};
use crate::collision::Direction;
use crate::family::FamilySet;
use crate::fsm::Negotiated;
use crate::metrics::SESSIONS_ESTABLISHED;
use crate::sync::IgnorePoisonRw;
const EVENT_CHANNEL_CAPACITY: usize = 256;
/// Identifies a session by the peer's TCP address and its AS number.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub struct PeerId {
pub addr: SocketAddr,
pub asn: u32,
}
impl std::fmt::Display for PeerId {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "{}(AS{})", self.addr, self.asn)
}
}
/// Where the dialer is in bringing up a configured peer.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum DialState {
@@ -30,7 +46,11 @@ pub enum DialState {
Idle,
Connecting,
/// The last attempt failed or the session ended; waiting to retry.
Backoff { until: SystemTime, last_error: String },
Backoff {
until: SystemTime,
/// How the last attempt or session ended.
last_error: String,
},
}
/// A peer from the configuration file, whether or not it is currently up.
@@ -53,9 +73,22 @@ pub struct SessionHandle {
/// originate to this peer.
pub local_addr: SocketAddr,
pub established_at: SystemTime,
/// Whether the peer is in our own AS.
pub ibgp: bool,
/// Whether the peer negotiated RFC 6793 four-octet ASNs. Determines how
/// AS_PATH is encoded on the wire for this session.
pub four_octet_asn: bool,
/// The address families negotiated with the peer. Routes of any other
/// family are neither sent to it nor accepted from it.
pub families: FamilySet,
/// The families for which the peer sends us several routes for a
/// prefix (ADD-PATH, RFC 7911), and those for which we send it path
/// identifiers.
pub add_path_receive: FamilySet,
pub add_path_send: FamilySet,
/// The peer's number in the routing table: what its routes there are
/// marked with.
pub rib_peer: PeerIndex,
outbound: mpsc::Sender<Message>,
}
@@ -64,21 +97,38 @@ pub struct SessionHandle {
pub struct SessionClosed(pub PeerId);
impl SessionHandle {
pub fn new(
pub(crate) fn new(
peer: PeerId,
name: Option<String>,
direction: Direction,
local_addr: SocketAddr,
four_octet_asn: bool,
negotiated: &Negotiated,
rib_peer: PeerIndex,
outbound: mpsc::Sender<Message>,
) -> Self {
Self { peer, name, direction, local_addr, established_at: SystemTime::now(), four_octet_asn, outbound }
Self {
peer,
name,
direction,
local_addr,
established_at: SystemTime::now(),
ibgp: negotiated.ibgp,
four_octet_asn: negotiated.four_octet_asn,
families: negotiated.families.clone(),
add_path_receive: negotiated.add_path_receive.clone(),
add_path_send: negotiated.add_path_send.clone(),
rib_peer,
outbound,
}
}
/// Queues a message for transmission on this session. Waits for queue
/// space if the session's writer is behind.
pub async fn send(&self, msg: Message) -> Result<(), SessionClosed> {
self.outbound.send(msg).await.map_err(|_| SessionClosed(self.peer))
self.outbound
.send(msg)
.await
.map_err(|_| SessionClosed(self.peer))
}
}
@@ -114,7 +164,10 @@ impl SessionRegistry {
pub fn with_configured(configured: Vec<ConfiguredPeer>) -> Self {
let (events, _) = broadcast::channel(EVENT_CHANNEL_CAPACITY);
let dial_states = configured.iter().map(|p| (p.name.clone(), DialState::Idle)).collect();
let dial_states = configured
.iter()
.map(|p| (p.name.clone(), DialState::Idle))
.collect();
Self {
inner: Arc::new(RwLock::new(HashMap::new())),
configured: Arc::new(configured),
@@ -127,28 +180,27 @@ impl SessionRegistry {
/// Peers from the configuration file, each with the dialer's current
/// state for it.
pub fn configured_peers(&self) -> Vec<(ConfiguredPeer, DialState)> {
let states = self.dial_states.read().expect("dial states poisoned");
let states = self.dial_states.reading();
self.configured
.iter()
.map(|p| (p.clone(), states.get(&p.name).cloned().unwrap_or(DialState::Idle)))
.map(|p| {
(
p.clone(),
states.get(&p.name).cloned().unwrap_or(DialState::Idle),
)
})
.collect()
}
/// The configured peer whose endpoint IP is `ip`, if any.
pub fn configured_by_ip(&self, ip: IpAddr) -> Option<&ConfiguredPeer> {
self.configured.iter().find(|p| p.endpoint.ip() == ip)
}
pub fn set_dial_state(&self, name: &str, state: DialState) {
self.dial_states.write().expect("dial states poisoned").insert(name.to_string(), state);
pub(crate) fn set_dial_state(&self, name: &str, state: DialState) {
self.dial_states.writing().insert(name.to_string(), state);
}
/// An established session with this remote IP and ASN, regardless of
/// which side opened the TCP connection.
pub fn find(&self, ip: IpAddr, asn: u32) -> Option<SessionHandle> {
self.inner
.read()
.expect("session registry poisoned")
.reading()
.values()
.find(|h| h.peer.addr.ip() == ip && h.peer.asn == asn)
.cloned()
@@ -156,26 +208,26 @@ impl SessionRegistry {
/// Adds a session and emits [`SessionEvent::Up`]. The returned guard
/// removes the session again when dropped.
pub fn register(&self, handle: SessionHandle) -> SessionGuard {
pub(crate) fn register(&self, handle: SessionHandle) -> SessionGuard {
let peer = handle.peer;
self.inner.write().expect("session registry poisoned").insert(peer, handle.clone());
self.inner.writing().insert(peer, handle.clone());
SESSIONS_ESTABLISHED.inc();
let _ = self.events.send(SessionEvent::Up(handle));
SessionGuard { registry: self.clone(), peer }
}
pub fn get(&self, peer: &PeerId) -> Option<SessionHandle> {
self.inner.read().expect("session registry poisoned").get(peer).cloned()
SessionGuard {
registry: self.clone(),
peer,
}
}
/// All currently established sessions, in no particular order.
pub fn list(&self) -> Vec<SessionHandle> {
self.inner.read().expect("session registry poisoned").values().cloned().collect()
self.inner.reading().values().cloned().collect()
}
/// Called by a session when its peer asks for a route refresh.
pub fn request_refresh(&self, peer: &PeerId) {
if let Some(handle) = self.get(peer) {
pub(crate) fn request_refresh(&self, peer: &PeerId) {
let handle = self.inner.reading().get(peer).cloned();
if let Some(handle) = handle {
let _ = self.events.send(SessionEvent::RefreshRequested(handle));
}
}
@@ -193,12 +245,15 @@ impl SessionRegistry {
pub async fn shutdown(&self, grace: Duration) -> usize {
self.shutting_down.store(true, Ordering::Relaxed);
for session in self.list() {
// Not waited for: a session whose queue is full is one whose
// peer is not reading, and it would hold up the goodbye to all
// the others. It ends when the process does.
let _ = session
.send(Message::Notification(NotificationMessage {
.outbound
.try_send(Message::Notification(NotificationMessage {
error: BgpError::Cease(CeaseSubcode::AdminShutdown),
data: vec![],
}))
.await;
}));
}
let deadline = tokio::time::Instant::now() + grace;
while !self.list().is_empty() && tokio::time::Instant::now() < deadline {
@@ -212,7 +267,7 @@ impl SessionRegistry {
}
fn remove(&self, peer: &PeerId) {
let removed = self.inner.write().expect("session registry poisoned").remove(peer);
let removed = self.inner.writing().remove(peer);
if removed.is_some() {
SESSIONS_ESTABLISHED.dec();
let _ = self.events.send(SessionEvent::Down(*peer));
crates/server/src/sync.rs
+66
View File
@@ -0,0 +1,66 @@
//! Locks that survive a panic.
//!
//! A standard lock is poisoned when a thread panics while holding it, and
//! every later `lock().unwrap()` then panics too. For the tables shared by
//! all sessions that would turn a bug in one session into the end of every
//! session, one panic at a time.
//!
//! The tables here are maps that are changed by single inserts and removes,
//! so a panic part-way through an update cannot leave one in a state that
//! is unsafe to read. The poison is therefore ignored: a session that
//! panics is cleaned up by its guards like one that ended any other way.
use std::sync::{Mutex, MutexGuard, PoisonError, RwLock, RwLockReadGuard, RwLockWriteGuard};
pub(crate) trait IgnorePoison<T> {
/// Locks, whether or not an earlier holder panicked.
fn locked(&self) -> MutexGuard<'_, T>;
}
impl<T> IgnorePoison<T> for Mutex<T> {
fn locked(&self) -> MutexGuard<'_, T> {
self.lock().unwrap_or_else(PoisonError::into_inner)
}
}
pub(crate) trait IgnorePoisonRw<T> {
/// Locks for reading, whether or not an earlier holder panicked.
fn reading(&self) -> RwLockReadGuard<'_, T>;
/// Locks for writing, whether or not an earlier holder panicked.
fn writing(&self) -> RwLockWriteGuard<'_, T>;
}
impl<T> IgnorePoisonRw<T> for RwLock<T> {
fn reading(&self) -> RwLockReadGuard<'_, T> {
self.read().unwrap_or_else(PoisonError::into_inner)
}
fn writing(&self) -> RwLockWriteGuard<'_, T> {
self.write().unwrap_or_else(PoisonError::into_inner)
}
}
#[cfg(test)]
mod tests {
use std::sync::Arc;
use super::*;
#[test]
fn a_lock_whose_holder_panicked_can_still_be_taken() {
let mutex = Arc::new(Mutex::new(1));
let rwlock = Arc::new(RwLock::new(1));
let (m, r) = (mutex.clone(), rwlock.clone());
let panicked = std::thread::spawn(move || {
let _m = m.lock().unwrap();
let _r = r.write().unwrap();
panic!("while holding both");
})
.join();
assert!(panicked.is_err());
assert!(mutex.is_poisoned() && rwlock.is_poisoned());
*mutex.locked() += 1;
*rwlock.writing() += 1;
assert_eq!((*mutex.locked(), *rwlock.reading()), (2, 2));
}
}
crates/server/src/tracing.rs
+6 -8
View File
@@ -1,26 +1,24 @@
use clap::Args;
use eyre::Result;
use tracing::Level;
use tracing_subscriber::EnvFilter;
use tracing_subscriber::filter::ParseError;
/// Default targets that we enable logging for.
const DEFAULT_TARGETS: [&str; 3] = ["bgp", "bgp_server", "bgp_web"];
#[derive(Debug, Args)]
pub struct TracingArgs {
/// Set the minimum log level.
#[arg(short, long, global = true, default_value_t = Level::TRACE, help_heading = "Tracing")]
/// Set the minimum log level. At `debug` and below there is a line for
/// every UPDATE and every route announced.
#[arg(short, long, global = true, default_value_t = Level::INFO, help_heading = "Tracing")]
level: Level,
/// Disable logging to stdout.
#[arg(long, global = true, default_value_t = false, help_heading = "Tracing")]
quiet: bool,
}
impl TracingArgs {
pub fn install(&self) -> Result<()> {
pub fn install(&self) -> Result<(), ParseError> {
let filter = DEFAULT_TARGETS.into_iter().try_fold(
EnvFilter::builder().from_env_lossy(),
|filter, target| -> Result<_> {
|filter, target| -> Result<_, ParseError> {
Ok(filter.add_directive(format!("{}={}", target, self.level).parse()?))
},
)?;
crates/web/Cargo.toml
+7 -5
View File
@@ -1,7 +1,12 @@
[package]
name = "bgp-web"
description = "HTTP API, event stream and metrics for the BGP daemon"
authors.workspace = true
edition.workspace = true
homepage.workspace = true
license.workspace = true
repository.workspace = true
version.workspace = true
[dependencies]
axum.workspace = true
@@ -9,15 +14,12 @@ bgp-packet.workspace = true
bgp-primitives.workspace = true
bgp-rib.workspace = true
bgp-server.workspace = true
bytes.workspace = true
eyre.workspace = true
futures.workspace = true
prometheus.workspace = true
serde.workspace = true
serde_json.workspace = true
tokio.workspace = true
tokio-stream.workspace = true
tracing.workspace = true
[dev-dependencies]
reqwest.workspace = true
[lints]
workspace = true
crates/web/src/api.rs
+178 -359
View File
@@ -1,68 +1,31 @@
//! JSON handlers and their view types.
//! JSON handlers and their view types. Everything here is read-only: the
//! daemon is configured through its configuration file.
use std::time::{SystemTime, UNIX_EPOCH};
use axum::Json;
use std::collections::HashMap;
use axum::extract::{Query, State};
use axum::http::StatusCode;
use axum::response::{IntoResponse, Response};
use bgp_packet::message::{Message, OriginPathAttribute, PathAttribute, UpdateMessage};
use bgp_primitives::ip_prefix::IpPrefix;
use bgp_primitives::parser::ParserContext;
use bgp_rib::{LocalRibError, LocalRoute, LocalRouteAttrs, SnapshotEntry};
use bgp_server::advertise::build_announce;
use bgp_server::ping::{PingFlow, PingMonitorStatus};
use bgp_server::session::{ConfiguredPeer, DialState, SessionHandle, SessionRegistry};
use bytes::BytesMut;
use bgp_packet::community;
use bgp_packet::message::{NlriNextHop, OriginPathAttribute, PathAttribute};
use bgp_rib::table::{PeerIndex, Route};
use bgp_rib::{LocalRoute, RibStopped};
use bgp_server::session::{ConfiguredPeer, DialState, PeerId, SessionHandle, SessionRegistry};
use serde::{Deserialize, Serialize};
use serde_json::json;
use crate::AppState;
use crate::community;
const MAX_OWNER_LEN: usize = 64;
const MAX_PREPEND: u8 = 16;
// ---------------------------------------------------------------------------
// Errors
// ---------------------------------------------------------------------------
#[derive(Debug)]
pub struct ApiError {
status: StatusCode,
message: String,
}
impl ApiError {
fn bad_request(message: impl Into<String>) -> Self {
Self { status: StatusCode::BAD_REQUEST, message: message.into() }
}
}
impl IntoResponse for ApiError {
fn into_response(self) -> Response {
(self.status, Json(json!({ "error": self.message }))).into_response()
}
}
impl From<LocalRibError> for ApiError {
fn from(e: LocalRibError) -> Self {
let status = match e {
LocalRibError::NotAllowed(_) => StatusCode::FORBIDDEN,
LocalRibError::OwnedByOther { .. } => StatusCode::CONFLICT,
LocalRibError::OwnerLimit { .. } => StatusCode::UNPROCESSABLE_ENTITY,
LocalRibError::Closed => StatusCode::SERVICE_UNAVAILABLE,
};
Self { status, message: e.to_string() }
}
}
// ---------------------------------------------------------------------------
// Views
// ---------------------------------------------------------------------------
fn unix_secs(t: SystemTime) -> u64 {
t.duration_since(UNIX_EPOCH).map(|d| d.as_secs()).unwrap_or(0)
t.duration_since(UNIX_EPOCH)
.map(|d| d.as_secs())
.unwrap_or(0)
}
#[derive(Debug, Serialize)]
@@ -75,7 +38,10 @@ pub struct StatusView {
pub uptime_secs: u64,
pub sessions: usize,
pub local_routes: usize,
/// Routes learned from peers, counting each peer's separately.
pub received_routes: usize,
/// Prefixes with at least one such route, and so with a best route.
pub received_prefixes: usize,
}
#[derive(Debug, Clone, Serialize)]
@@ -92,6 +58,13 @@ pub struct PeerView {
pub established_at_unix: Option<u64>,
pub uptime_secs: Option<u64>,
pub four_octet_asn: Option<bool>,
/// The address families negotiated with the peer, e.g. `IPv4 unicast`.
pub address_families: Option<Vec<String>>,
/// The families for which the peer sends us several routes for a
/// prefix (ADD-PATH, RFC 7911), and those for which we send it path
/// identifiers.
pub add_path_receive: Option<Vec<String>>,
pub add_path_send: Option<Vec<String>>,
/// Why the last connection attempt or session ended, while in Backoff.
pub last_error: Option<String>,
pub retry_at_unix: Option<u64>,
@@ -109,6 +82,9 @@ impl From<&SessionHandle> for PeerView {
established_at_unix: Some(unix_secs(s.established_at)),
uptime_secs: Some(s.established_at.elapsed().map(|d| d.as_secs()).unwrap_or(0)),
four_octet_asn: Some(s.four_octet_asn),
address_families: Some(s.families.iter().map(|f| f.to_string()).collect()),
add_path_receive: Some(s.add_path_receive.iter().map(|f| f.to_string()).collect()),
add_path_send: Some(s.add_path_send.iter().map(|f| f.to_string()).collect()),
last_error: None,
retry_at_unix: None,
}
@@ -120,7 +96,9 @@ impl PeerView {
let (state_name, last_error, retry_at_unix) = match state {
DialState::Idle => ("Idle", None, None),
DialState::Connecting => ("Connecting", None, None),
DialState::Backoff { until, last_error } => ("Backoff", Some(last_error.clone()), Some(unix_secs(*until))),
DialState::Backoff { until, last_error } => {
("Backoff", Some(last_error.clone()), Some(unix_secs(*until)))
}
};
Self {
name: Some(peer.name.clone()),
@@ -132,6 +110,9 @@ impl PeerView {
established_at_unix: None,
uptime_secs: None,
four_octet_asn: None,
address_families: None,
add_path_receive: None,
add_path_send: None,
last_error,
retry_at_unix,
}
@@ -147,17 +128,26 @@ pub fn peer_views(registry: &SessionRegistry) -> Vec<PeerView> {
.configured_peers()
.iter()
.map(|(peer, state)| {
match sessions.iter().find(|s| s.peer.addr.ip() == peer.endpoint.ip() && s.peer.asn == peer.remote_asn) {
match sessions
.iter()
.find(|s| s.peer.addr.ip() == peer.endpoint.ip() && s.peer.asn == peer.remote_asn)
{
Some(session) => {
claimed.insert(session.peer);
PeerView { name: Some(peer.name.clone()), ..PeerView::from(session) }
PeerView {
name: Some(peer.name.clone()),
..PeerView::from(session)
}
}
None => PeerView::from_configured(peer, state),
}
})
.collect();
let mut rest: Vec<PeerView> =
sessions.iter().filter(|s| !claimed.contains(&s.peer)).map(PeerView::from).collect();
let mut rest: Vec<PeerView> = sessions
.iter()
.filter(|s| !claimed.contains(&s.peer))
.map(PeerView::from)
.collect();
rest.sort_by(|a, b| a.peer_addr.cmp(&b.peer_addr));
views.extend(rest);
views
@@ -166,7 +156,6 @@ pub fn peer_views(registry: &SessionRegistry) -> Vec<PeerView> {
#[derive(Debug, Clone, Serialize)]
pub struct LocalRouteView {
pub prefix: String,
pub owner: String,
pub origin: String,
pub prepend: u8,
pub med: Option<u32>,
@@ -179,12 +168,22 @@ impl From<&LocalRoute> for LocalRouteView {
fn from(r: &LocalRoute) -> Self {
Self {
prefix: r.prefix.to_string(),
owner: r.owner.clone(),
origin: origin_name(r.attrs.origin).to_string(),
prepend: r.attrs.prepend,
med: r.attrs.med,
communities: r.attrs.communities.iter().copied().map(community::format_standard).collect(),
large_communities: r.attrs.large_communities.iter().map(community::format_large).collect(),
communities: r
.attrs
.communities
.iter()
.copied()
.map(community::format_standard)
.collect(),
large_communities: r
.attrs
.large_communities
.iter()
.map(community::format_large)
.collect(),
announced_at_unix: unix_secs(r.announced_at),
}
}
@@ -203,20 +202,32 @@ pub struct AttrSummary {
pub large_communities: Vec<String>,
}
pub fn summarize(attrs: &[PathAttribute]) -> AttrSummary {
let mut s = AttrSummary::default();
for attr in attrs {
pub fn summarize(attributes: &[PathAttribute], next_hop: Option<&NlriNextHop>) -> AttrSummary {
let mut s = AttrSummary {
// A multiprotocol route has its next hop here; an IPv4 route has it
// in the NEXT_HOP attribute below.
next_hop: next_hop.map(ToString::to_string),
..Default::default()
};
for attr in attributes {
match attr {
PathAttribute::Origin(o) => s.origin = Some(origin_name(*o).to_string()),
PathAttribute::AsPath(p) => {
s.as_path = p.segments.iter().flat_map(|seg| seg.path.iter().copied()).collect()
s.as_path = p
.segments
.iter()
.flat_map(|seg| seg.path.iter().copied())
.collect()
}
PathAttribute::NextHop(nh) => s.next_hop = Some(nh.0.to_string()),
PathAttribute::MpReachNlri(mp) => s.next_hop = Some(format!("{:?}", mp.next_hop)),
PathAttribute::MultiExitDisc(m) => s.med = Some(m.0),
PathAttribute::LocalPref(l) => s.local_pref = Some(l.0),
PathAttribute::Communities(c) => {
s.communities = c.0.iter().copied().map(community::format_standard).collect()
s.communities =
c.0.iter()
.copied()
.map(community::format_standard)
.collect()
}
PathAttribute::LargeCommunities(l) => {
s.large_communities = l.communities.iter().map(community::format_large).collect()
@@ -232,38 +243,48 @@ pub struct ReceivedRouteView {
pub peer_addr: String,
pub peer_asn: u32,
pub prefix: String,
pub received_at_unix: u64,
/// The peer's identifier for the route: zero unless the peer sends
/// several routes for a prefix (ADD-PATH, RFC 7911).
pub path_id: u32,
/// Whether this is the route chosen for its prefix.
pub best: bool,
pub summary: AttrSummary,
/// The full decoded attribute list, for the curious.
/// The decoded attribute list. MP_REACH_NLRI is not in it: its next hop
/// is in the summary and its prefixes are routes of their own.
pub attributes: Vec<PathAttribute>,
}
impl From<&SnapshotEntry> for ReceivedRouteView {
fn from(e: &SnapshotEntry) -> Self {
Self {
peer_addr: e.peer.addr.to_string(),
peer_asn: e.peer.asn,
prefix: e.prefix.to_string(),
received_at_unix: unix_secs(e.entry.received_at),
summary: summarize(&e.entry.path_attributes),
attributes: e.entry.path_attributes.clone(),
}
}
}
/// Who the routes in the table are from: the table knows peers by number,
/// the sessions know which number is whose.
pub(crate) struct PeerNames(HashMap<PeerIndex, PeerId>);
/// The exact UPDATE a route produces on one session.
#[derive(Debug, Clone, Serialize)]
pub struct WireView {
pub peer_addr: String,
pub peer_asn: u32,
pub next_hop: String,
/// Whether AS numbers in AS_PATH are 4 bytes wide on this session.
pub four_octet_asn: bool,
/// Complete BGP message including the 19-byte header, as lowercase hex.
pub hex: String,
pub length: usize,
pub summary: AttrSummary,
pub update: UpdateMessage,
impl PeerNames {
pub(crate) fn of(registry: &SessionRegistry) -> Self {
Self(
registry
.list()
.into_iter()
.map(|session| (session.rib_peer, session.peer))
.collect(),
)
}
/// A view of one route. `None` if the route's session has just ended:
/// the route is on its way out of the table.
pub(crate) fn view(&self, route: &Route) -> Option<ReceivedRouteView> {
let peer = self.0.get(&route.peer)?;
// What a session stored was accepted by the decoder, and parses.
let (attributes, next_hop) = route.attrs.attributes().unwrap_or_default();
Some(ReceivedRouteView {
peer_addr: peer.addr.to_string(),
peer_asn: peer.asn,
prefix: route.prefix.to_string(),
path_id: route.path_id,
best: route.best,
summary: summarize(&attributes, next_hop.as_ref()),
attributes,
})
}
}
fn origin_name(o: OriginPathAttribute) -> &'static str {
@@ -274,69 +295,28 @@ fn origin_name(o: OriginPathAttribute) -> &'static str {
}
}
fn parse_origin(s: &str) -> Result<OriginPathAttribute, ApiError> {
match s.trim().to_ascii_uppercase().as_str() {
"IGP" => Ok(OriginPathAttribute::IGP),
"EGP" => Ok(OriginPathAttribute::EGP),
"INCOMPLETE" => Ok(OriginPathAttribute::INCOMPLETE),
other => Err(ApiError::bad_request(format!("origin {other:?} must be IGP, EGP or INCOMPLETE"))),
}
}
fn parse_prefix(s: &str) -> Result<IpPrefix, ApiError> {
IpPrefix::try_from(s.trim()).map_err(|e| ApiError::bad_request(format!("invalid prefix {s:?}: {e}")))
}
fn validate_owner(owner: &str) -> Result<String, ApiError> {
let owner = owner.trim();
if owner.is_empty() {
return Err(ApiError::bad_request("owner must not be empty"));
}
if owner.chars().count() > MAX_OWNER_LEN {
return Err(ApiError::bad_request(format!("owner must be at most {MAX_OWNER_LEN} characters")));
}
Ok(owner.to_string())
}
// ---------------------------------------------------------------------------
// Requests
// ---------------------------------------------------------------------------
#[derive(Debug, Clone, Deserialize)]
pub struct AnnounceRequest {
pub prefix: String,
pub owner: String,
#[serde(default)]
pub origin: Option<String>,
#[serde(default)]
pub prepend: u8,
#[serde(default)]
pub med: Option<u32>,
#[serde(default)]
pub communities: Vec<String>,
#[serde(default)]
pub large_communities: Vec<String>,
}
#[derive(Debug, Clone, Deserialize)]
pub struct WithdrawRequest {
pub prefix: String,
/// Omit to withdraw regardless of owner.
#[serde(default)]
pub owner: Option<String>,
}
#[derive(Debug, Deserialize)]
pub struct WireQuery {
pub prefix: String,
}
// ---------------------------------------------------------------------------
// Handlers
// ---------------------------------------------------------------------------
pub async fn status(State(state): State<AppState>) -> Json<StatusView> {
Json(StatusView {
/// A table could not be read because its actor has stopped. The daemon is
/// on its way down; saying so beats answering with an empty table.
pub struct Unavailable(RibStopped);
impl From<RibStopped> for Unavailable {
fn from(e: RibStopped) -> Self {
Self(e)
}
}
impl IntoResponse for Unavailable {
fn into_response(self) -> Response {
(StatusCode::SERVICE_UNAVAILABLE, self.0.to_string()).into_response()
}
}
pub async fn status(State(state): State<AppState>) -> Result<Json<StatusView>, Unavailable> {
Ok(Json(StatusView {
router_id: state.info.router_id.to_string(),
local_asn: state.info.local_asn,
version: state.info.version.clone(),
@@ -344,9 +324,10 @@ pub async fn status(State(state): State<AppState>) -> Json<StatusView> {
started_at_unix: unix_secs(state.info.started_at),
uptime_secs: state.info.started.elapsed().as_secs(),
sessions: state.registry.list().len(),
local_routes: state.local_rib.snapshot().await.len(),
received_routes: state.adj_rib_in.snapshot().await.len(),
})
local_routes: state.local_rib.len().await?,
received_routes: state.rib.len(),
received_prefixes: state.rib.prefix_count(),
}))
}
/// The configuration file the daemon was started with.
@@ -367,221 +348,59 @@ pub async fn peers(State(state): State<AppState>) -> Json<Vec<PeerView>> {
Json(peer_views(&state.registry))
}
pub async fn local_routes(State(state): State<AppState>) -> Json<Vec<LocalRouteView>> {
let mut routes: Vec<LocalRouteView> =
state.local_rib.snapshot().await.iter().map(LocalRouteView::from).collect();
routes.sort_by(|a, b| a.prefix.cmp(&b.prefix));
Json(routes)
}
pub async fn announce(
pub async fn local_routes(
State(state): State<AppState>,
Json(req): Json<AnnounceRequest>,
) -> Result<Json<LocalRouteView>, ApiError> {
let prefix = parse_prefix(&req.prefix)?;
let owner = validate_owner(&req.owner)?;
if req.prepend > MAX_PREPEND {
return Err(ApiError::bad_request(format!("prepend must be at most {MAX_PREPEND}")));
}
let attrs = LocalRouteAttrs {
origin: req.origin.as_deref().map(parse_origin).transpose()?.unwrap_or(OriginPathAttribute::IGP),
prepend: req.prepend,
med: req.med,
communities: req
.communities
.iter()
.map(|c| community::parse_standard(c))
.collect::<Result<_, _>>()
.map_err(ApiError::bad_request)?,
large_communities: req
.large_communities
.iter()
.map(|c| community::parse_large(c))
.collect::<Result<_, _>>()
.map_err(ApiError::bad_request)?,
};
let route = LocalRoute::new(prefix, owner, attrs);
state.local_rib.announce(route.clone()).await?;
Ok(Json(LocalRouteView::from(&route)))
}
pub async fn withdraw(
State(state): State<AppState>,
Json(req): Json<WithdrawRequest>,
) -> Result<Json<Option<LocalRouteView>>, ApiError> {
let prefix = parse_prefix(&req.prefix)?;
let owner = req.owner.as_deref().map(validate_owner).transpose()?;
let removed = state.local_rib.withdraw(prefix, owner.as_deref()).await?;
Ok(Json(removed.as_ref().map(LocalRouteView::from)))
}
/// Shows the UPDATE bytes a local route produces on each established session.
pub async fn wire(
State(state): State<AppState>,
Query(q): Query<WireQuery>,
) -> Result<Json<Vec<WireView>>, ApiError> {
let prefix = parse_prefix(&q.prefix)?;
let route = state
) -> Result<Json<Vec<LocalRouteView>>, Unavailable> {
let mut routes: Vec<LocalRouteView> = state
.local_rib
.snapshot()
.await
.into_iter()
.find(|r| r.prefix == prefix)
.ok_or_else(|| ApiError { status: StatusCode::NOT_FOUND, message: format!("{prefix} is not announced") })?;
let mut views = vec![];
for session in state.registry.list() {
let Some(update) = build_announce(&route, state.info.local_asn, session.local_addr.ip()) else {
continue;
};
let ctx = ParserContext { four_octet_asn: Some(session.four_octet_asn), address_family: None };
let mut buf = BytesMut::new();
Message::Update(update.clone())
.to_wire(&ctx, &mut buf)
.map_err(|e| ApiError { status: StatusCode::INTERNAL_SERVER_ERROR, message: e.to_string() })?;
views.push(WireView {
peer_addr: session.peer.addr.to_string(),
peer_asn: session.peer.asn,
next_hop: session.local_addr.ip().to_string(),
four_octet_asn: session.four_octet_asn,
hex: buf.iter().map(|b| format!("{b:02x}")).collect(),
length: buf.len(),
summary: summarize(&update.path_attributes),
update,
});
}
views.sort_by(|a, b| a.peer_addr.cmp(&b.peer_addr));
Ok(Json(views))
.await?
.iter()
.map(LocalRouteView::from)
.collect();
routes.sort_by(|a, b| a.prefix.cmp(&b.prefix));
Ok(Json(routes))
}
#[derive(Debug, Clone, Serialize)]
pub struct WellKnownCommunity {
pub name: &'static str,
pub value: String,
}
/// How many routes [`received_routes`] returns unless asked for another
/// number. The table can hold hundreds of millions.
const DEFAULT_ROUTE_LIMIT: usize = 1000;
/// Names the API accepts in place of a numeric standard community.
pub async fn well_known_communities() -> Json<Vec<WellKnownCommunity>> {
Json(
community::WELL_KNOWN
.iter()
.map(|(name, value)| WellKnownCommunity { name, value: community::format_standard(*value) })
.collect(),
)
}
pub async fn received_routes(State(state): State<AppState>) -> Json<Vec<ReceivedRouteView>> {
let mut routes: Vec<ReceivedRouteView> =
state.adj_rib_in.snapshot().await.iter().map(ReceivedRouteView::from).collect();
routes.sort_by(|a, b| (&a.prefix, &a.peer_addr).cmp(&(&b.prefix, &b.peer_addr)));
Json(routes)
}
// ---------------------------------------------------------------------------
// Pings
// ---------------------------------------------------------------------------
/// Flows returned by `/api/pings` unless `?limit=` says otherwise.
const DEFAULT_PING_FLOWS: usize = 100;
/// All pings from one source to one destination.
#[derive(Debug, Clone, Serialize)]
pub struct PingView {
pub src: String,
pub dst: String,
/// The local route the destination belongs to and who announced it;
/// `null` when nobody announces it at the moment.
pub prefix: Option<String>,
pub owner: Option<String>,
/// Interface the most recent packet arrived on.
pub interface: Option<String>,
/// Hop limit left when the most recent packet arrived.
pub hop_limit: Option<u8>,
/// Routers the packet crossed, assuming the sender started at the usual
/// 64, 128 or 255.
pub hops: Option<u8>,
pub ident: u16,
pub seq: u16,
/// Size of the most recent ICMPv6 message.
pub bytes: usize,
/// That message as lowercase hex, from the ICMPv6 type byte on (no IPv6
/// header). Long packets are cut off, so this can be shorter than `bytes`.
pub hex: String,
pub count: u64,
pub first_seen_unix: u64,
pub last_seen_unix: u64,
}
fn estimated_hops(hop_limit: u8) -> u8 {
let initial = [64u8, 128, 255].into_iter().find(|&start| start >= hop_limit).unwrap_or(255);
initial - hop_limit
}
impl From<&PingFlow> for PingView {
fn from(flow: &PingFlow) -> Self {
Self {
src: flow.last.src.to_string(),
dst: flow.last.dst.to_string(),
prefix: flow.route.as_ref().map(|r| r.prefix.to_string()),
owner: flow.route.as_ref().map(|r| r.owner.clone()),
interface: flow.last.interface.clone(),
hop_limit: flow.last.hop_limit,
hops: flow.last.hop_limit.map(estimated_hops),
ident: flow.last.ident,
seq: flow.last.seq,
bytes: flow.last.bytes,
hex: flow.last.packet.iter().map(|b| format!("{b:02x}")).collect(),
count: flow.count,
first_seen_unix: unix_secs(flow.first_seen),
last_seen_unix: unix_secs(flow.last_seen),
}
}
}
#[derive(Debug, Clone, Serialize)]
pub struct PingMonitorView {
/// `listening`, `disabled` (not configured) or `failed`.
pub status: &'static str,
/// Why the monitor failed to start.
pub detail: Option<String>,
/// Pings to destinations inside these prefixes are shown.
pub prefixes: Vec<String>,
/// Flows remembered; `flows` holds the most recently seen of them.
pub total_flows: usize,
pub flows: Vec<PingView>,
}
/// The most routes [`received_routes`] returns, whatever is asked for:
/// each is copied out of the table and written as JSON.
const MAX_ROUTE_LIMIT: usize = 100_000;
#[derive(Debug, Deserialize)]
pub struct PingQuery {
pub limit: Option<usize>,
pub struct RouteQuery {
/// At most this many routes.
limit: Option<usize>,
}
pub async fn pings(State(state): State<AppState>, Query(query): Query<PingQuery>) -> Json<PingMonitorView> {
let (status, detail) = match state.pings.status() {
PingMonitorStatus::Listening => ("listening", None),
PingMonitorStatus::Disabled => ("disabled", None),
PingMonitorStatus::Failed(why) => ("failed", Some(why.clone())),
};
let flows = state.pings.flows().await;
Json(PingMonitorView {
status,
detail,
prefixes: state.pings.prefixes().iter().map(ToString::to_string).collect(),
total_flows: flows.len(),
flows: flows.iter().take(query.limit.unwrap_or(DEFAULT_PING_FLOWS)).map(PingView::from).collect(),
/// Some of the routes learned from peers: all the routes of as many
/// prefixes as fit in the limit. `/api/status` has the totals.
pub async fn received_routes(
State(state): State<AppState>,
Query(query): Query<RouteQuery>,
) -> Result<Json<Vec<ReceivedRouteView>>, StatusCode> {
let limit = query
.limit
.unwrap_or(DEFAULT_ROUTE_LIMIT)
.min(MAX_ROUTE_LIMIT);
let names = PeerNames::of(&state.registry);
// Copying, parsing and sorting many routes takes long enough to keep
// off the threads that sessions run on.
let routes = tokio::task::spawn_blocking(move || {
let mut routes: Vec<ReceivedRouteView> = state
.rib
.some_routes(limit)
.iter()
.filter_map(|route| names.view(route))
.collect();
routes.sort_by(|a, b| (&a.prefix, &a.peer_addr).cmp(&(&b.prefix, &b.peer_addr)));
routes
})
}
#[cfg(test)]
mod ping_tests {
use super::estimated_hops;
#[test]
fn hops_are_counted_from_the_nearest_usual_starting_value() {
assert_eq!(estimated_hops(64), 0);
assert_eq!(estimated_hops(57), 7);
assert_eq!(estimated_hops(118), 10);
assert_eq!(estimated_hops(243), 12);
assert_eq!(estimated_hops(255), 0);
}
.await
// Only if making the views panicked.
.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
Ok(Json(routes))
}
crates/web/src/events.rs
+156 -66
View File
@@ -1,108 +1,198 @@
//! Server-sent event stream merging session, local-route, received-route and
//! ping events into one feed the UI can follow.
//! Server-sent event stream merging session, local-route and best-route
//! changes into one feed the UI can follow.
use std::convert::Infallible;
use std::sync::Arc;
use axum::extract::State;
use axum::response::sse::{Event, KeepAlive, Sse};
use bgp_rib::{LocalRibEvent, RibEvent};
use bgp_server::ping::PingEvent;
use bgp_primitives::ip_prefix::IpPrefix;
use bgp_rib::LocalRibEvent;
use bgp_rib::table::{self, Route};
use bgp_server::session::SessionEvent;
use futures::stream::{Stream, StreamExt};
use serde::Serialize;
use tokio_stream::wrappers::BroadcastStream;
use tokio::sync::mpsc;
use tokio_stream::wrappers::errors::BroadcastStreamRecvError;
use tokio_stream::wrappers::{BroadcastStream, ReceiverStream};
use crate::AppState;
use crate::api::{LocalRouteView, PeerView, PingView, ReceivedRouteView, summarize};
use crate::api::{LocalRouteView, PeerNames, PeerView, ReceivedRouteView};
#[derive(Debug, Serialize)]
#[serde(tag = "type", rename_all = "snake_case")]
pub enum WireEvent {
SessionUp { peer: PeerView },
SessionDown { peer_addr: String, peer_asn: u32 },
SessionUp {
peer: PeerView,
},
SessionDown {
peer_addr: String,
peer_asn: u32,
},
/// The peer asked us to send all our routes again (ROUTE-REFRESH).
RouteRefreshRequested { peer: PeerView },
LocalAnnounced { route: LocalRouteView },
LocalWithdrawn { prefix: String },
ReceivedAnnounced { route: ReceivedRouteView },
ReceivedWithdrawn { peer_addr: String, peer_asn: u32, prefix: String },
/// Somebody is pinging an address in the watched prefixes. `new_burst`
/// marks the start of a run of pings; the events that follow while it
/// continues only update the count.
PingSeen { ping: PingView, new_burst: bool },
/// The client fell behind and missed `missed` events on `stream`; it
/// should refetch the corresponding table.
Lagged { stream: &'static str, missed: u64 },
RouteRefreshRequested {
peer: PeerView,
},
LocalAnnounced {
route: LocalRouteView,
},
LocalWithdrawn {
prefix: String,
},
/// The best route for a prefix is now this one.
BestChanged {
route: ReceivedRouteView,
},
/// A prefix has no route any more.
BestWithdrawn {
prefix: String,
},
/// The client fell behind and missed events on `stream` (`missed` of
/// them, where that is known); it should refetch the corresponding
/// table.
Lagged {
stream: &'static str,
missed: u64,
},
}
impl WireEvent {
fn from_session(ev: Result<SessionEvent, BroadcastStreamRecvError>) -> Self {
match ev {
Ok(SessionEvent::Up(h)) => Self::SessionUp { peer: PeerView::from(&h) },
Ok(SessionEvent::Down(p)) => Self::SessionDown { peer_addr: p.addr.to_string(), peer_asn: p.asn },
Ok(SessionEvent::RefreshRequested(h)) => Self::RouteRefreshRequested { peer: PeerView::from(&h) },
Err(BroadcastStreamRecvError::Lagged(n)) => Self::Lagged { stream: "sessions", missed: n },
Ok(SessionEvent::Up(h)) => Self::SessionUp {
peer: PeerView::from(&h),
},
Ok(SessionEvent::Down(p)) => Self::SessionDown {
peer_addr: p.addr.to_string(),
peer_asn: p.asn,
},
Ok(SessionEvent::RefreshRequested(h)) => Self::RouteRefreshRequested {
peer: PeerView::from(&h),
},
Err(BroadcastStreamRecvError::Lagged(n)) => Self::Lagged {
stream: "sessions",
missed: n,
},
}
}
fn from_local(ev: Result<LocalRibEvent, BroadcastStreamRecvError>) -> Self {
match ev {
Ok(LocalRibEvent::Announced(r)) => Self::LocalAnnounced { route: LocalRouteView::from(&r) },
Ok(LocalRibEvent::Withdrawn { prefix }) => Self::LocalWithdrawn { prefix: prefix.to_string() },
Err(BroadcastStreamRecvError::Lagged(n)) => Self::Lagged { stream: "local_routes", missed: n },
}
}
fn from_ping(ev: Result<PingEvent, BroadcastStreamRecvError>) -> Self {
match ev {
Ok(ev) => Self::PingSeen { ping: PingView::from(&ev.flow), new_burst: ev.new_burst },
Err(BroadcastStreamRecvError::Lagged(n)) => Self::Lagged { stream: "pings", missed: n },
}
}
fn from_received(ev: Result<RibEvent, BroadcastStreamRecvError>) -> Self {
match ev {
Ok(RibEvent::Announced { peer, prefix, entry }) => Self::ReceivedAnnounced {
route: ReceivedRouteView {
peer_addr: peer.addr.to_string(),
peer_asn: peer.asn,
prefix: prefix.to_string(),
received_at_unix: entry
.received_at
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_secs())
.unwrap_or(0),
summary: summarize(&entry.path_attributes),
attributes: entry.path_attributes,
},
Ok(LocalRibEvent::Announced(r)) => Self::LocalAnnounced {
route: LocalRouteView::from(&r),
},
Ok(RibEvent::Withdrawn { peer, prefix }) => Self::ReceivedWithdrawn {
peer_addr: peer.addr.to_string(),
peer_asn: peer.asn,
Ok(LocalRibEvent::Withdrawn { prefix }) => Self::LocalWithdrawn {
prefix: prefix.to_string(),
},
Err(BroadcastStreamRecvError::Lagged(n)) => Self::Lagged { stream: "received_routes", missed: n },
Err(BroadcastStreamRecvError::Lagged(n)) => Self::Lagged {
stream: "local_routes",
missed: n,
},
}
}
}
/// The table's changes of best route as events, from when the client
/// connects. Each client follows the table's journal with a reader of its
/// own, so one that reads slowly holds nobody else up.
fn best_routes(state: &AppState) -> impl Stream<Item = WireEvent> + use<> {
let (events, stream) = mpsc::channel(BEST_ROUTE_BUFFER);
let registry = state.registry.clone();
// Taken before this returns, so that nothing that happens after the
// client is answered is missed.
let mut reader = state.rib.reader_from_now();
tokio::spawn(async move {
// If the reader falls too far behind it is given the whole table
// again. That is not news: the client is told it missed something
// and gets its table from the API.
let mut syncing = false;
loop {
tokio::select! {
() = reader.changed() => {}
() = events.closed() => return,
}
loop {
// Only what is needed is taken while the table is held;
// the views are made afterwards.
let mut changes = Vec::new();
let progress = reader.poll(BEST_ROUTE_BUFFER, |event| match event {
table::Event::Reset => {
syncing = true;
changes.push(Change::Lagged);
}
table::Event::Synced => syncing = false,
table::Event::Best {
prefix,
peer,
path_id,
attrs,
} if !syncing => {
changes.push(Change::Best(Route {
prefix,
peer,
path_id,
attrs: Arc::clone(attrs),
best: true,
}));
}
table::Event::Withdrawn { prefix } if !syncing => {
changes.push(Change::Withdrawn(prefix));
}
_ => {}
});
let names = PeerNames::of(&registry);
for change in changes {
let event = match change {
Change::Best(route) => match names.view(&route) {
Some(route) => WireEvent::BestChanged { route },
None => continue,
},
Change::Withdrawn(prefix) => WireEvent::BestWithdrawn {
prefix: prefix.to_string(),
},
Change::Lagged => WireEvent::Lagged {
stream: "best_routes",
missed: 0,
},
};
if events.send(event).await.is_err() {
return;
}
}
if progress.caught_up {
break;
}
}
}
});
ReceiverStream::new(stream)
}
/// How many best-route events are read from the journal, and queued for a
/// client, at a time.
const BEST_ROUTE_BUFFER: usize = 256;
enum Change {
Best(Route),
Withdrawn(IpPrefix),
/// The client's reader fell too far behind the table.
Lagged,
}
pub async fn stream(
State(state): State<AppState>,
) -> Sse<impl Stream<Item = Result<Event, Infallible>>> {
let sessions = BroadcastStream::new(state.registry.subscribe()).map(WireEvent::from_session);
let local = BroadcastStream::new(state.local_rib.subscribe()).map(WireEvent::from_local);
let received = BroadcastStream::new(state.adj_rib_in.subscribe()).map(WireEvent::from_received);
let received = best_routes(&state);
let pings = BroadcastStream::new(state.pings.subscribe()).map(WireEvent::from_ping);
let routes = futures::stream::select(local, received);
let merged = futures::stream::select(futures::stream::select(sessions, pings), routes).map(|ev| {
Ok(Event::default()
.event("bgp")
.json_data(&ev)
.unwrap_or_else(|e| Event::default().event("error").data(e.to_string())))
});
let merged =
futures::stream::select(sessions, futures::stream::select(local, received)).map(|ev| {
Ok(Event::default()
.event("bgp")
.json_data(&ev)
.unwrap_or_else(|e| Event::default().event("error").data(e.to_string())))
});
Sse::new(merged).keep_alive(KeepAlive::default())
}
crates/web/src/lib.rs
+22 -27
View File
@@ -1,12 +1,11 @@
//! HTTP surface of the daemon: a JSON API for originating routes and
//! inspecting state, a server-sent event stream, and Prometheus metrics.
//! HTTP surface of the daemon: a read-only JSON API for inspecting state, a
//! status page, a server-sent event stream, and Prometheus metrics.
//!
//! Everything here is a thin layer over handles owned by the BGP server
//! ([`SessionRegistry`], [`LocalRib`], [`AdjRibIn`], [`PingMonitor`]); no state lives
//! ([`SessionRegistry`], [`LocalRib`], [`RibTable`]); no routing state lives
//! in this crate.
mod api;
mod community;
mod events;
mod metrics;
@@ -17,19 +16,14 @@ use std::time::{Instant, SystemTime};
use axum::Router;
use axum::http::header;
use axum::response::{Html, IntoResponse, Redirect};
use axum::routing::{delete, get, post};
use bgp_rib::{AdjRibIn, LocalRib};
use bgp_server::ping::PingMonitor;
use axum::routing::get;
use bgp_rib::LocalRib;
use bgp_rib::table::RibTable;
use bgp_server::session::SessionRegistry;
use eyre::{Result, WrapErr};
use tokio::net::TcpListener;
use tracing::info;
pub use api::{
AnnounceRequest, AttrSummary, ConfigView, LocalRouteView, PeerView, PingMonitorView, PingView,
ReceivedRouteView, StatusView,
WellKnownCommunity, WireView, WithdrawRequest,
};
pub use api::{AttrSummary, ConfigView, LocalRouteView, PeerView, ReceivedRouteView, StatusView};
/// Static facts about the router shown in status output.
#[derive(Debug, Clone)]
@@ -85,15 +79,18 @@ pub struct AppState {
pub info: Arc<RouterInfo>,
pub registry: SessionRegistry,
pub local_rib: LocalRib,
pub adj_rib_in: AdjRibIn,
pub pings: PingMonitor,
/// The routing table: routes learned from peers and the best of them.
pub rib: Arc<RibTable>,
}
/// The status page, embedded so the binary is self-contained.
const STATUSZ_HTML: &str = include_str!("../static/statusz.html");
async fn statusz() -> impl IntoResponse {
([(header::CONTENT_TYPE, "text/html; charset=utf-8")], Html(STATUSZ_HTML))
(
[(header::CONTENT_TYPE, "text/html; charset=utf-8")],
Html(STATUSZ_HTML),
)
}
/// Builds the router. Exposed so tests can drive it without a socket.
@@ -104,28 +101,26 @@ pub fn router(state: AppState) -> Router {
.route("/api/status", get(api::status))
.route("/api/config", get(api::config))
.route("/api/peers", get(api::peers))
.route("/api/routes/local", get(api::local_routes).post(api::announce))
.route("/api/routes/local", delete(api::withdraw))
.route("/api/routes/local/withdraw", post(api::withdraw))
.route("/api/routes/local/wire", get(api::wire))
.route("/api/routes/local", get(api::local_routes))
.route("/api/routes/received", get(api::received_routes))
.route("/api/pings", get(api::pings))
.route("/api/communities/well-known", get(api::well_known_communities))
.route("/api/events", get(events::stream))
.route("/metricz", get(metrics::metricz))
.with_state(state)
}
/// Binds `listen` and serves the API until the task is dropped.
pub async fn serve(state: AppState, listen: SocketAddr) -> Result<()> {
let listener = TcpListener::bind(listen)
.await
.wrap_err_with(|| format!("Failed to bind HTTP listener on {listen}"))?;
pub async fn serve(state: AppState, listen: SocketAddr) -> std::io::Result<()> {
let listener = TcpListener::bind(listen).await.map_err(|e| {
std::io::Error::new(
e.kind(),
format!("failed to bind HTTP listener on {listen}: {e}"),
)
})?;
serve_on(state, listener).await
}
/// Serves the API on an already-bound listener (lets tests use port 0).
pub async fn serve_on(state: AppState, listener: TcpListener) -> Result<()> {
pub async fn serve_on(state: AppState, listener: TcpListener) -> std::io::Result<()> {
info!(addr = %listener.local_addr()?, "HTTP API listening");
axum::serve(listener, router(state)).await?;
Ok(())
crates/web/src/metrics.rs
+21 -4
View File
@@ -4,18 +4,35 @@
use axum::extract::State;
use axum::http::{StatusCode, header};
use axum::response::IntoResponse;
use bgp_server::metrics::{ADJ_RIB_IN_ROUTES, LOCAL_ROUTES};
use bgp_server::metrics::{
LOCAL_ROUTES, RIB_ATTRIBUTE_SETS, RIB_JOURNAL_ENTRIES, RIB_PREFIXES, RIB_ROUTES,
};
use prometheus::{Encoder, TextEncoder};
use crate::AppState;
pub async fn metricz(State(state): State<AppState>) -> impl IntoResponse {
LOCAL_ROUTES.set(state.local_rib.snapshot().await.len() as i64);
ADJ_RIB_IN_ROUTES.set(state.adj_rib_in.snapshot().await.len() as i64);
// A scrape that cannot read a table fails. Serving the last known
// size would hide that the daemon is broken from the one system whose
// job is to notice.
let gauge = |count: usize| i64::try_from(count).unwrap_or(i64::MAX);
match state.local_rib.len().await {
Ok(local) => LOCAL_ROUTES.set(gauge(local)),
Err(e) => return (StatusCode::SERVICE_UNAVAILABLE, e.to_string()).into_response(),
}
RIB_ROUTES.set(gauge(state.rib.len()));
RIB_PREFIXES.set(gauge(state.rib.prefix_count()));
RIB_ATTRIBUTE_SETS.set(gauge(state.rib.attrs().len()));
RIB_JOURNAL_ENTRIES.set(gauge(state.rib.journal_len()));
let mut buf = Vec::new();
match TextEncoder::new().encode(&prometheus::gather(), &mut buf) {
Ok(()) => (StatusCode::OK, [(header::CONTENT_TYPE, "text/plain; version=0.0.4")], buf).into_response(),
Ok(()) => (
StatusCode::OK,
[(header::CONTENT_TYPE, "text/plain; version=0.0.4")],
buf,
)
.into_response(),
Err(e) => (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()).into_response(),
}
}
crates/web/static/statusz.html
+63 -425
View File
@@ -6,11 +6,10 @@
<title>Status for ferrix_bgp</title>
<link rel="icon" href="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 16 16'%3E%3Crect width='16' height='16' fill='%23eef'/%3E%3Ctext x='8' y='12' font-size='11' font-family='sans-serif' font-weight='bold' fill='%2300c' text-anchor='middle'%3EB%3C/text%3E%3C/svg%3E">
<!--
Styled after the classic Google-internal /statusz pages as mirrored by
Vitess (go/vt/servenv/status.go) and golang.org/x/net/trace: plain
sans-serif HTML, default links and form controls, dense bordered tables,
and one full-width light blue (#eef) heading bar per section. Deliberately
no dark theme, no framework, no external resources.
A read-only status page in the manner of the classic /statusz pages: plain
sans-serif HTML, default links, dense bordered tables, and one full-width
light blue (#eef) heading bar per section. Deliberately no dark theme, no
framework, no external resources. Everything shown comes from /api/*.
-->
<style>
body { font-family: sans-serif; font-size: 13px; color: #000; background: #fff; margin: 8px; }
@@ -23,25 +22,10 @@ table.data { border-collapse: collapse; margin: 4px 0; }
table.data th, table.data td { border: 1px solid #99a; padding: 2px 6px; text-align: left; vertical-align: top; }
table.data th { background: #dde; font-weight: bold; }
.mono, table.data td.mono { font-family: monospace; }
table.data tr.loop td { background: #ffc; }
table.data tr.mine td.owner { font-weight: bold; }
.ok { color: #080; font-weight: bold; } .bad { color: #c00; font-weight: bold; } .warn { color: #a60; font-weight: bold; }
.dim { color: #666; }
table.form td { padding: 2px 6px 2px 0; }
table.form td:first-child { width: 11em; }
select { max-width: 14em; }
pre.bytes { border: 1px solid #99a; background: #fff; padding: 4px; margin: 4px 0; font-size: 12px; white-space: pre-wrap; word-break: break-all; line-height: 1.6; }
.hh { background: #dde; } .hl { background: #fcc; } .ha { background: #cfc; } .ha2 { background: #ada; } .hn { background: #ffc; }
#event-log { font-family: monospace; font-size: 12px; max-height: 22em; overflow-y: auto; }
#event-log div { white-space: pre-wrap; }
pre.bytes span[data-kind] { cursor: help; }
pre.bytes span[data-kind]:hover, pre.bytes span[data-kind]:focus { outline: 1px solid #00c; }
#tip { position: absolute; z-index: 10; max-width: 46em; background: #fff; border: 1px solid #99a; font-size: 12px; }
#tip .tip-title { background: #eef; font-weight: bold; text-align: center; padding: 1px 6px; border-bottom: 1px solid #99a; }
#tip table { border-collapse: collapse; }
#tip td { padding: 1px 6px; vertical-align: top; }
#tip td:first-child { color: #666; white-space: nowrap; }
#tip td.mono { word-break: break-word; }
details { clear: both; margin: 8px 0 0; }
summary { cursor: pointer; color: #00c; }
summary:hover { text-decoration: underline; }
@@ -61,7 +45,7 @@ Router ID <b id="s-router-id">...</b>, local AS <b id="s-asn">...</b><br>
</div>
<div class="righthand">
Running on <span id="s-hostname">...</span><br>
View: <a href="/metricz">variables</a>, <a href="/api/status">status json</a>, <a href="/api/peers">peers json</a>, <a href="/api/routes/local">local routes json</a>, <a href="/api/routes/received">received routes json</a>, <a href="/api/pings">pings json</a><br>
View: <a href="/metricz">variables</a>, <a href="/api/status">status json</a>, <a href="/api/peers">peers json</a>, <a href="/api/routes/local">originated routes json</a>, <a href="/api/routes/received">received routes json</a><br>
<span id="refresh-ctl"></span><br>
</div>
</div>
@@ -73,90 +57,40 @@ View: <a href="/metricz">variables</a>, <a href="/api/status">status json</a>, <
<h1 id="peers">Peers (<span id="peers-count">...</span>)</h1>
<table class="data">
<thead><tr><th>name</th><th>peer</th><th>AS</th><th>state</th><th>opened by</th><th>local address (next hop)</th><th>established</th><th>uptime</th><th>4-byte AS</th><th>detail</th></tr></thead>
<thead><tr><th>name</th><th>peer</th><th>AS</th><th>state</th><th>opened by</th><th>local address (next hop)</th><th>established</th><th>uptime</th><th>4-byte AS</th><th>address families</th><th>detail</th></tr></thead>
<tbody id="peers-body"></tbody>
</table>
<h1 id="announce">Announce a prefix</h1>
<form id="announce-form" autocomplete="off">
<table class="form">
<tr><td><label for="f-owner">Owner (your name):</label></td><td><input id="f-owner" size="20" maxlength="64" required></td>
<td><label for="f-prefix">Prefix:</label></td><td><input id="f-prefix" size="24" required placeholder="2a0d:d741:1::/48"></td></tr>
<tr><td><label for="f-origin">Origin:</label></td><td><select id="f-origin"><option>IGP</option><option>EGP</option><option>INCOMPLETE</option></select></td>
<td><label for="f-med">MED:</label></td><td><input id="f-med" type="number" min="0" max="4294967295" placeholder="none" style="width:8em"></td></tr>
</table>
<table class="form">
<tr><td><label for="f-communities">Communities:</label></td><td><input id="f-communities" size="50" placeholder="65000:100, NO_EXPORT">
<select id="f-wk" aria-label="Add a well-known community"><option value="">+ well-known...</option></select>
<span class="dim">comma separated, ASN:VALUE or a name</span></td></tr>
<tr><td><label for="f-large">Large communities:</label></td><td><input id="f-large" size="50" placeholder="210036:0:0, 210036:1:42476">
<select id="f-lc" aria-label="Add a large community preset"><option value="">+ presets...</option></select>
<span class="dim">GLOBAL:LOCAL1:LOCAL2</span></td></tr>
<tr><td></td><td><input type="submit" value="Announce"> <input type="button" id="f-clear" value="Clear">
&nbsp; <span id="announce-msg"></span></td></tr>
</table>
</form>
<h1 id="local">Local routes (<span id="local-count">...</span>)</h1>
<h1 id="local">Originated routes (<span id="local-count">...</span>)</h1>
<table class="data">
<thead><tr><th>prefix</th><th>owner</th><th>origin</th><th>AS path</th><th>MED</th><th>communities</th><th>large communities</th><th>age</th><th>actions</th></tr></thead>
<thead><tr><th>prefix</th><th>origin</th><th>AS path</th><th>MED</th><th>communities</th><th>large communities</th><th>since</th></tr></thead>
<tbody id="local-body"></tbody>
</table>
<div id="pings-section" hidden>
<h1 id="pings">Pings to announced addresses (<span id="pings-count">...</span>)</h1>
<div class="dim" id="pings-note"></div>
<h1 id="received">Received routes (<span id="received-count">...</span>)</h1>
<table class="data">
<thead><tr><th>destination</th><th>route</th><th>owner</th><th>from</th><th>arrived on</th><th>hop limit</th><th>pings</th><th>last sequence</th><th>first seen</th><th>last seen</th><th>actions</th></tr></thead>
<tbody id="pings-body"></tbody>
</table>
</div>
<h1 id="received">Received routes, Adj-RIB-In (<span id="received-count">...</span>)</h1>
<table class="data">
<thead><tr><th>prefix</th><th>from peer</th><th>AS path</th><th>next hop</th><th>origin</th><th>MED</th><th>communities</th><th>large communities</th><th>age</th></tr></thead>
<thead><tr><th>prefix</th><th>from peer</th><th>AS path</th><th>next hop</th><th>origin</th><th>MED</th><th>local pref</th><th>communities</th><th>large communities</th><th>best</th></tr></thead>
<tbody id="received-body"></tbody>
</table>
<h1 id="events">Event log</h1>
<div id="log-state" class="dim"></div>
<div id="event-log"></div>
<div id="tip" hidden></div>
<address>ferrix_bgp &middot; <a href="/statusz">/statusz</a> <a href="/metricz">/metricz</a> <a href="/api/events">/api/events</a></address>
<script>
(() => {
// Large-community presets offered next to the field, grouped by who acts
// on them. "PEER" is replaced by an AS number the user is asked for.
// AS210036: docs/lab/prod/bird-ferrix.conf (lab_export_allowed). Every
// session there has a neighbour AS and a location; "never"
// (functions 0 and 2) beats "announce" (1 and 3) beats 0:0.
// SwissIX: https://www.swissix.ch/resources/action-bgp-communities/
const LOCATIONS = { 1: 'SwissIX', 2: 'NL-ix', 3: 'France-IX', 4: 'DE-CIX Frankfurt', 5: 'Free-IX Switzerland', 6: 'Community-IX.ch', 100: 'transit', 200: 'downstream customers' };
const LARGE_PRESET_GROUPS = [
['AS210036', 'AS210036: by neighbour', [
['210036:0:0', 'announce nowhere (then add where to)'],
['210036:1:42476', 'announce to the SwissIX route servers'],
['210036:1:PEER', 'announce to neighbour AS'],
['210036:0:PEER', 'never announce to neighbour AS'],
]],
['AS210036', 'AS210036: by location', Object.entries(LOCATIONS).filter(([id]) => id < 100).map(([id, name]) => [`210036:3:${id}`, `announce at ${name}`])
.concat([['210036:2:100', 'never announce to transit']])],
['SwissIX', 'SwissIX route servers (AS42476)', [
['42476:0:0', 'send to nobody'],
['42476:1:559', 'send to SWITCH (AS559)'],
['42476:1:PEER', 'send to peer AS'],
['42476:0:PEER', 'do not send to peer AS'],
['42476:1:0', 'send to all (default)'],
['42476:901:PEER', 'add NO_EXPORT towards peer AS'],
['42476:902:PEER', 'add NO_ADVERTISE towards peer AS'],
]],
];
// Well-known standard communities (RFC 1997, RFC 7999, RFC 8326), shown by
// name with the number as a tooltip.
const WELL_KNOWN = {
'65535:0': 'GRACEFUL_SHUTDOWN', '65535:666': 'BLACKHOLE', '65535:65281': 'NO_EXPORT',
'65535:65282': 'NO_ADVERTISE', '65535:65283': 'NO_EXPORT_SUBCONFED', '65535:65284': 'NOPEER',
};
const $ = id => document.getElementById(id);
const esc = s => String(s).replace(/[&<>"]/g, c => ({'&':'&amp;','<':'&lt;','>':'&gt;','"':'&quot;'}[c]));
const state = { status: null, peers: [], local: [], received: [], pings: null, wellKnown: {}, openWire: new Set(), openPing: new Set(), wire: new Map() };
const state = { status: null, peers: [], local: [], received: [] };
const DASH = '<span class="dim">-</span>';
// ---- helpers -----------------------------------------------------------
@@ -167,9 +101,9 @@ View: <a href="/metricz">variables</a>, <a href="/api/status">status json</a>, <
};
const now = () => Math.floor(Date.now() / 1000);
// Tables are only rewritten when their markup actually changes, so a
// refresh with unchanged data never touches the DOM (no flicker, and open
// rows, selections and hover state survive). Anything that changes with
// the clock is a placeholder that tick() fills in as plain text.
// refresh with unchanged data never touches the DOM (no flicker, and
// selections survive). Anything that changes with the clock is a
// placeholder that tick() fills in as plain text.
function setHTML(el, html) { if (el._html !== html) { el.innerHTML = html; el._html = html; tick(); } }
const age = unix => `<span data-since="${unix}"></span>`;
const until = unix => `<span data-until="${unix}"></span>`;
@@ -180,16 +114,13 @@ View: <a href="/metricz">variables</a>, <a href="/api/status">status json</a>, <
if (state.status) $('s-uptime').textContent = fmtDur(t - state.status.started_at_unix);
}
const fmtTime = unix => new Date(unix * 1000).toLocaleTimeString();
const commList = (list, wk) => list.length ? list.map(c => {
const name = wk && state.wellKnown[c];
return name ? `<span title="${esc(c)}">${esc(name)}</span>` : esc(c);
}).join(' ') : DASH;
const commList = (list, named) => list.length ? list.map(c =>
named && WELL_KNOWN[c] ? `<span title="${esc(c)}">${WELL_KNOWN[c]}</span>` : esc(c)).join(' ') : DASH;
const asPath = p => p.length ? p.join(' ') : '<span class="dim">(empty)</span>';
const myOwner = () => $('f-owner').value.trim();
async function getJSON(url) {
const r = await fetch(url);
if (!r.ok) throw new Error(`${url}: ${r.status}`);
if (!r.ok) throw new Error(`${url}: ${r.status} ${await r.text()}`);
return r.json();
}
@@ -203,10 +134,19 @@ View: <a href="/metricz">variables</a>, <a href="/api/status">status json</a>, <
}
const STATE_CLASS = { Established: 'ok', Connecting: 'warn', Backoff: 'bad', Idle: 'dim' };
// Where several routes for a prefix are exchanged (ADD-PATH), and which way.
function addPath(p) {
const rx = p.add_path_receive || [], tx = p.add_path_send || [];
if (!rx.length && !tx.length) return '';
const parts = [];
if (rx.length) parts.push(`receive ${esc(rx.join(', '))}`);
if (tx.length) parts.push(`send ${esc(tx.join(', '))}`);
return ` <span class="dim">· add-path: ${parts.join('; ')}</span>`;
}
function renderPeers() {
const b = $('peers-body');
$('peers-count').textContent = `${state.peers.filter(p => p.state === 'Established').length} of ${state.peers.length} established`;
if (!state.peers.length) { setHTML(b, '<tr><td colspan="10" class="dim">No peers configured and none connected.</td></tr>'); return; }
if (!state.peers.length) { setHTML(b, '<tr><td colspan="11" class="dim">No peers configured and none connected.</td></tr>'); return; }
setHTML(b, state.peers.map(p => {
const up = p.state === 'Established';
const detail = p.state === 'Backoff' ? `${esc(p.last_error || '')}; retry in ${until(p.retry_at_unix)}` : '';
@@ -214,266 +154,40 @@ View: <a href="/metricz">variables</a>, <a href="/api/status">status json</a>, <
<td>${p.name ? esc(p.name) : '<span class="dim">(inbound)</span>'}</td><td class="mono">${esc(p.peer_addr)}</td><td>${p.peer_asn}</td>
<td><span class="${STATE_CLASS[p.state] || ''}">${esc(p.state)}</span></td><td>${up ? (p.direction === 'outbound' ? 'us' : 'peer') : DASH}</td>
<td class="mono">${up ? esc(p.local_addr) : DASH}</td><td>${up ? fmtTime(p.established_at_unix) : DASH}</td>
<td>${up ? age(p.established_at_unix) : DASH}</td><td>${up ? (p.four_octet_asn ? 'yes' : 'no') : DASH}</td><td>${detail}</td></tr>`;
<td>${up ? age(p.established_at_unix) : DASH}</td><td>${up ? (p.four_octet_asn ? 'yes' : 'no') : DASH}</td>
<td>${up ? esc((p.address_families || []).join(', ')) + addPath(p) : DASH}</td><td>${detail}</td></tr>`;
}).join(''));
}
function renderLocal() {
const b = $('local-body'); $('local-count').textContent = state.local.length;
if (!state.local.length) { setHTML(b, '<tr><td colspan="9" class="dim">Nothing announced yet.</td></tr>'); return; }
const me = myOwner(), asn = state.status ? state.status.local_asn : '?', stale = [];
setHTML(b, state.local.map(r => {
const mine = r.owner === me, open = state.openWire.has(r.prefix), p = esc(r.prefix);
const withdraw = mine ? `<a href="#" data-act="withdraw" data-prefix="${p}">withdraw</a>`
: '<span class="dim" title="only the owner can withdraw">withdraw</span>';
return `<tr class="${mine ? 'mine' : ''}">
<td class="mono">${p}</td><td class="owner">${esc(r.owner)}</td><td>${esc(r.origin)}</td>
<td class="mono">${Array(1 + r.prepend).fill(asn).join(' ')}</td><td>${r.med ?? DASH}</td>
<td class="mono">${commList(r.communities, true)}</td><td class="mono">${commList(r.large_communities)}</td><td>${age(r.announced_at_unix)}</td>
<td><a href="#" data-act="wire" data-prefix="${p}">${open ? 'hide bytes' : 'bytes'}</a> | ${withdraw}</td></tr>` +
(open ? `<tr><td colspan="9">${wireHTML(r, stale)}</td></tr>` : '');
}).join(''));
stale.forEach(loadWire);
if (!state.local.length) { setHTML(b, '<tr><td colspan="7" class="dim">This router originates no routes.</td></tr>'); return; }
const asn = state.status ? state.status.local_asn : '?';
setHTML(b, state.local.map(r => `<tr>
<td class="mono">${esc(r.prefix)}</td><td>${esc(r.origin)}</td>
<td class="mono">${Array(1 + r.prepend).fill(asn).join(' ')}</td><td>${r.med ?? DASH}</td>
<td class="mono">${commList(r.communities, true)}</td><td class="mono">${commList(r.large_communities)}</td>
<td>${age(r.announced_at_unix)}</td></tr>`).join(''));
}
function renderReceived() {
const b = $('received-body'); $('received-count').textContent = state.received.length;
if (!state.received.length) { setHTML(b, '<tr><td colspan="9" class="dim">No routes received from peers.</td></tr>'); return; }
const localPrefixes = new Set(state.local.map(r => r.prefix));
setHTML(b, state.received.map(r => { const s = r.summary; return `<tr class="${localPrefixes.has(r.prefix) ? 'loop' : ''}">
<td class="mono">${esc(r.prefix)}</td><td class="mono">${esc(r.peer_addr)} <span class="dim">AS${r.peer_asn}</span></td><td class="mono">${asPath(s.as_path)}</td>
<td class="mono">${s.next_hop ? esc(s.next_hop) : DASH}</td><td>${s.origin ? esc(s.origin) : DASH}</td><td>${s.med ?? DASH}</td>
<td class="mono">${commList(s.communities, true)}</td><td class="mono">${commList(s.large_communities)}</td><td>${age(r.received_at_unix)}</td></tr>`; }).join(''));
}
// Pings are shown per source and destination pair. The router answers
// them itself; this table is a copy of what arrived.
function renderPings() {
const p = state.pings; if (!p || p.status === 'disabled') { $('pings-section').hidden = true; return; }
$('pings-section').hidden = false;
if (p.status === 'failed') {
$('pings-count').textContent = 'off';
$('pings-note').innerHTML = `<span class="bad">The ping monitor could not start:</span> ${esc(p.detail || 'unknown error')}`;
setHTML($('pings-body'), ''); return;
}
const more = p.total_flows > p.flows.length ? `, showing the ${p.flows.length} most recent` : '';
$('pings-count').textContent = `${p.total_flows} flow${p.total_flows === 1 ? '' : 's'}${more}`;
$('pings-note').innerHTML = '';
if (!p.flows.length) { setHTML($('pings-body'), '<tr><td colspan="11" class="dim">No pings seen yet.</td></tr>'); return; }
const me = myOwner();
setHTML($('pings-body'), p.flows.map(f => { const key = `${f.src}|${f.dst}`, open = state.openPing.has(key); return `<tr class="${f.owner && f.owner === me ? 'mine' : ''}">
<td class="mono">${esc(f.dst)}</td><td class="mono">${f.prefix ? esc(f.prefix) : '<span class="dim">not announced</span>'}</td><td class="owner">${f.owner ? esc(f.owner) : DASH}</td>
<td class="mono">${esc(f.src)}</td><td class="mono">${f.interface ? esc(f.interface) : DASH}</td>
<td>${f.hop_limit == null ? DASH : `${f.hop_limit} <span class="dim">(${f.hops} hop${f.hops === 1 ? '' : 's'} away)</span>`}</td>
<td>${f.count}</td><td>${f.seq}</td><td>${fmtTime(f.first_seen_unix)}</td><td>${age(f.last_seen_unix)} ago</td>
<td><a href="#" data-act="ping-bytes" data-key="${esc(key)}">${open ? 'hide bytes' : 'bytes'}</a></td></tr>` +
(open ? `<tr><td colspan="11">${pingBytesHTML(f)}</td></tr>` : ''); }).join(''));
}
// The last echo request of a flow, split into the fields of RFC 4443. It
// starts at the ICMPv6 header: the socket we listen on is never given the
// IPv6 header, whose interesting fields are the columns of the table.
function pingBytesHTML(f) {
const bytes = f.hex.match(/../g) || [], segs = [];
const take = (n, cls, kind) => { if (bytes.length) segs.push({ cls, kind, hex: bytes.splice(0, n).join(' ') }); };
take(1, 'hh', 'icmp-type'); take(1, 'hh', 'icmp-code'); take(2, 'hl', 'icmp-checksum');
take(2, 'ha', 'icmp-ident'); take(2, 'ha2', 'icmp-seq'); take(bytes.length, 'hn', 'icmp-payload');
const kept = f.hex.length / 2;
return `Last ICMPv6 message from <b>${esc(f.src)}</b>, ${f.bytes} bytes${kept < f.bytes ? ` (first ${kept} shown)` : ''}, IPv6 header not included &nbsp;
<span class="hh">&nbsp;type, code&nbsp;</span> <span class="hl">&nbsp;checksum&nbsp;</span> <span class="ha">&nbsp;identifier, sequence&nbsp;</span> <span class="hn">&nbsp;payload&nbsp;</span>
<span class="dim">(hover a segment to decode it)</span>
<pre class="bytes">${segs.map(x => `<span class="${x.cls}" tabindex="0" data-kind="${x.kind}" data-hex="${x.hex}">${x.hex}</span>`).join(' ')}</pre>`;
}
function decodePing(kind, b) {
switch (kind) {
case 'icmp-type': return { title: 'ICMPv6: type', rows: [['value', `${b[0]} = ${{128: 'Echo Request', 129: 'Echo Reply'}[b[0]] || 'other'}`], ['meaning', 'The router answers with type 129, Echo Reply, carrying the same identifier, sequence number and payload.']] };
case 'icmp-code': return { title: 'ICMPv6: code', rows: [['value', `${b[0]}`], ['meaning', 'Always 0 for echo messages.']] };
case 'icmp-checksum': return { title: 'ICMPv6: checksum', rows: [['value', `0x${u(b, 0, 2).toString(16).padStart(4, '0')}`], ['covers', 'This message and a pseudo-header holding the IPv6 source and destination addresses, so it differs between two otherwise equal pings to different addresses.']] };
case 'icmp-ident': return { title: 'Echo: identifier', rows: [['value', `${u(b, 0, 2)}`], ['meaning', 'Chosen by the sender so that several ping programs on one host can tell their replies apart.']] };
case 'icmp-seq': return { title: 'Echo: sequence number', rows: [['value', `${u(b, 0, 2)}`], ['meaning', 'Counts up with every request; a gap in the replies is a lost packet.']] };
default: {
const rows = [['length', `${b.length} bytes shown`], ['as text', b.map(x => x >= 32 && x < 127 ? String.fromCharCode(x) : '.').join('')]];
// ping puts the time of sending first so that the reply carries it
// back. Only claim that when the value is in fact close to now.
const near = secs => Math.abs(secs - now()) < 86400;
const le = b.length >= 16 && !b[4] && !b[5] && !b[6] && !b[7] ? b[0] + b[1] * 256 + b[2] * 65536 + b[3] * 16777216 : null;
const be = b.length >= 8 ? u(b, 0, 4) : null;
if (le != null && near(le)) rows.push(['starts with', `the sender's clock when it sent this: ${new Date(le * 1000).toLocaleTimeString()} (a 16-byte timeval, as Linux ping writes it)`]);
else if (be != null && near(be)) rows.push(['starts with', `the sender's clock when it sent this: ${new Date(be * 1000).toLocaleTimeString()} (an 8-byte timestamp, as BSD and macOS ping6 write it)`]);
rows.push(['meaning', 'Anything the sender likes; the reply must return it unchanged.']);
return { title: 'Echo: payload', rows };
}
}
}
// A ping event carries the whole row, so the table follows the stream
// without asking the server again.
function pingSeen(f) {
const p = state.pings; if (!p || p.status !== 'listening') return;
const at = p.flows.findIndex(x => x.src === f.src && x.dst === f.dst);
if (at >= 0) p.flows.splice(at, 1); else p.total_flows++;
p.flows.unshift(f); p.flows.length = Math.min(p.flows.length, 100);
renderPings();
}
// ---- wire view: split the UPDATE bytes into labelled fields ------------
const ATTR_NAMES = {1:'ORIGIN',2:'AS_PATH',3:'NEXT_HOP',4:'MED',5:'LOCAL_PREF',6:'ATOMIC_AGGREGATE',7:'AGGREGATOR',8:'COMMUNITIES',14:'MP_REACH_NLRI',15:'MP_UNREACH_NLRI',16:'EXT_COMMUNITIES',32:'LARGE_COMMUNITIES'};
function segmentUpdate(hex) {
const bytes = hex.match(/../g) || [], segs = [];
const take = (n, cls, kind) => { segs.push({ cls, kind, hex: bytes.splice(0, n).join(' ') }); };
take(16, 'hh', 'marker'); take(2, 'hh', 'length'); take(1, 'hh', 'type');
const wlen = parseInt(hex.substr(38, 4), 16); take(2, 'hl', 'wlen'); if (wlen) take(wlen, 'hn', 'withdrawn');
const alen = parseInt(bytes.slice(0, 2).join(''), 16); take(2, 'hl', 'alen');
let rem = alen, i = 0;
while (rem > 0) {
const ext = parseInt(bytes[0], 16) & 0x10;
const hdr = ext ? 4 : 3, len = ext ? parseInt(bytes[2] + bytes[3], 16) : parseInt(bytes[2], 16);
take(hdr + len, i++ % 2 ? 'ha2' : 'ha', 'attr'); rem -= hdr + len;
}
if (bytes.length) take(bytes.length, 'hn', 'nlri');
return segs;
}
// ---- decoding a hovered segment into something readable ----------------
const u = (b, o, n) => { let v = 0; for (let k = 0; k < n; k++) v = v * 256 + b[o + k]; return v; };
const fmtV4 = b => b.slice(0, 4).join('.');
function fmtV6(b) {
const g = []; for (let k = 0; k < 16; k += 2) g.push(((b[k] << 8) | b[k + 1]).toString(16));
let best = -1, bestLen = 0;
for (let k = 0; k < 8; k++) { let n = 0; while (k + n < 8 && g[k + n] === '0') n++; if (n > bestLen && n > 1) { best = k; bestLen = n; } }
if (best < 0) return g.join(':');
return g.slice(0, best).join(':') + '::' + g.slice(best + bestLen).join(':');
}
function prefixList(b, afi) {
const out = []; let o = 0;
while (o < b.length) {
const len = b[o++], n = Math.ceil(len / 8), raw = b.slice(o, o + n); o += n;
const full = raw.concat(Array((afi === 2 ? 16 : 4) - raw.length).fill(0));
out.push((afi === 2 ? fmtV6(full) : fmtV4(full)) + '/' + len);
}
return out;
}
function largeLabel(c) {
// Our own scheme is decoded from its structure, so any value reads well,
// not only the ones in the menu.
const own = c.match(/^210036:([0-3]):(\d+)$/);
if (own) {
const [f, t] = [Number(own[1]), Number(own[2])], where = LOCATIONS[t] || `location ${t}`;
if (f === 0 && t === 0) return 'AS210036: announce nowhere, unless another community says where';
if (f === 0) return `AS210036: never announce to AS${t}`;
if (f === 1) return `AS210036: announce to AS${t}${t === 42476 ? ' (the SwissIX route servers)' : ''}`;
return `AS210036: ${f === 2 ? 'never announce' : 'announce'} ${t >= 100 ? 'to' : 'at'} ${where}`;
}
// Exact presets first, so 42476:1:559 reads "SWITCH" rather than "AS559".
for (const pass of [false, true]) for (const [who, , presets] of LARGE_PRESET_GROUPS) for (const [pattern, label] of presets) {
if (pattern.includes('PEER') !== pass) continue;
const m = c.match(new RegExp('^' + pattern.replace('PEER', '(\\d+)') + '$'));
if (m) return `${who}: ${label.replace(/(peer|neighbour) AS/, 'AS' + (m[1] || ''))}`;
}
return null;
}
const AFI = { 1: 'IPv4', 2: 'IPv6' }, SAFI = { 1: 'unicast', 2: 'multicast' };
function decodeAttr(b, w4) {
const flags = b[0], type = b[1], ext = flags & 0x10, hdr = ext ? 4 : 3, len = ext ? u(b, 2, 2) : b[2], v = b.slice(hdr);
const name = ATTR_NAMES[type] || `attribute ${type}`;
const rows = [
['flags', `0x${flags.toString(16)}: ${flags & 0x80 ? 'optional' : 'well-known'}, ${flags & 0x40 ? 'transitive' : 'non-transitive'}${flags & 0x20 ? ', partial' : ''}${ext ? ', 2-byte length' : ''}`],
['type code', `${type}`], ['value length', `${len} bytes`],
];
switch (type) {
case 1: rows.push(['origin', ['IGP (originated inside the AS)', 'EGP', 'INCOMPLETE (learned some other way)'][v[0]] || `unknown ${v[0]}`]); break;
case 2: {
const w = w4 ? 4 : 2; let o = 0, hops = 0;
while (o + 2 <= v.length) {
const st = v[o], n = v[o + 1], asns = []; o += 2;
for (let k = 0; k < n; k++, o += w) asns.push(u(v, o, w));
rows.push([st === 2 ? 'AS_SEQUENCE' : st === 1 ? 'AS_SET' : `segment ${st}`, asns.join(' ')]);
hops += st === 2 ? n : 1;
}
rows.push(['path length', `${hops} (what best-path selection compares)`], ['AS number width', w4 ? '4 bytes (RFC 6793)' : '2 bytes']);
break;
}
case 3: rows.push(['next hop', fmtV4(v)]); break;
case 4: rows.push(['MED', `${u(v, 0, 4)} (lower is preferred, compared between routes from the same neighbour AS)`]); break;
case 5: rows.push(['local preference', `${u(v, 0, 4)} (higher is preferred)`]); break;
case 8: for (let o = 0; o + 4 <= v.length; o += 4) { const c = `${u(v, o, 2)}:${u(v, o + 2, 2)}`; rows.push(['community', c + (state.wellKnown[c] ? ` = ${state.wellKnown[c]}` : '')]); } break;
case 32: for (let o = 0; o + 12 <= v.length; o += 12) { const c = `${u(v, o, 4)}:${u(v, o + 4, 4)}:${u(v, o + 8, 4)}`, l = largeLabel(c); rows.push(['large community', c + (l ? ` = ${l}` : '')]); } break;
case 14: {
const afi = u(v, 0, 2), nhLen = v[3], nh = v.slice(4, 4 + nhLen);
rows.push(['address family', `${AFI[afi] || afi} ${SAFI[v[2]] || v[2]}`]);
if (nhLen === 4) rows.push(['next hop', fmtV4(nh)]);
else if (nhLen === 16) rows.push(['next hop', fmtV6(nh)]);
else if (nhLen === 32) rows.push(['next hop', fmtV6(nh.slice(0, 16))], ['link-local next hop', fmtV6(nh.slice(16))]);
prefixList(v.slice(5 + nhLen), afi).forEach(p => rows.push(['announced prefix', p]));
break;
}
case 15: { const afi = u(v, 0, 2); rows.push(['address family', `${AFI[afi] || afi} ${SAFI[v[2]] || v[2]}`]); prefixList(v.slice(3), afi).forEach(p => rows.push(['withdrawn prefix', p])); break; }
default: rows.push(['value', v.map(x => x.toString(16).padStart(2, '0')).join(' ')]);
}
return { title: `Path attribute: ${name}`, rows };
}
function decodeSegment(kind, b, w4) {
switch (kind) {
case 'marker': return { title: 'Header: marker', rows: [['value', '16 bytes of 0xff'], ['meaning', 'Fixed filler. It once carried authentication data; now it only helps a receiver find message boundaries.']] };
case 'length': return { title: 'Header: message length', rows: [['value', `${u(b, 0, 2)} bytes, including this 19-byte header`], ['limits', '19 to 4096 bytes']] };
case 'type': return { title: 'Header: message type', rows: [['value', `${b[0]} = ${['', 'OPEN', 'UPDATE', 'NOTIFICATION', 'KEEPALIVE', 'ROUTE-REFRESH'][b[0]] || 'unknown'}`]] };
case 'wlen': return { title: 'Withdrawn routes length', rows: [['value', `${u(b, 0, 2)} bytes`], ['meaning', u(b, 0, 2) ? 'IPv4 prefixes being withdrawn follow.' : 'No IPv4 prefixes are withdrawn by this message.']] };
case 'withdrawn': return { title: 'Withdrawn routes', rows: prefixList(b, 1).map(p => ['withdrawn prefix', p]) };
case 'alen': return { title: 'Total path attribute length', rows: [['value', `${u(b, 0, 2)} bytes`], ['meaning', 'The path attributes follow; whatever remains after them is IPv4 NLRI.']] };
case 'nlri': return { title: 'NLRI: announced IPv4 prefixes', rows: prefixList(b, 1).map(p => ['announced prefix', p]).concat([['encoding', 'one length byte, then only as many address bytes as the length needs']]) };
default: return decodeAttr(b, w4);
}
}
function showTip(el) {
const b = el.dataset.hex.split(' ').map(x => parseInt(x, 16));
const kind = el.dataset.kind, tip = $('tip');
const d = kind.startsWith('icmp-') ? decodePing(kind, b) : decodeSegment(kind, b, el.dataset.w4 === '1');
tip.innerHTML = `<div class="tip-title">${esc(d.title)}</div><table>${d.rows.map(([k, v]) => `<tr><td>${esc(k)}</td><td class="mono">${esc(v)}</td></tr>`).join('')}</table>`;
tip.hidden = false;
const rects = el.getClientRects(), first = rects[0], last = rects[rects.length - 1];
const left = Math.max(8, Math.min(first.left, window.innerWidth - tip.offsetWidth - 8));
tip.style.left = `${left + window.scrollX}px`; tip.style.top = `${last.bottom + window.scrollY + 4}px`;
}
const hideTip = () => { $('tip').hidden = true; };
const segOf = e => e.target.closest && e.target.closest('pre.bytes span[data-kind]');
document.addEventListener('mouseover', e => { const el = segOf(e); if (el) showTip(el); });
document.addEventListener('mouseout', e => { if (segOf(e)) hideTip(); });
document.addEventListener('focusin', e => { const el = segOf(e); if (el) showTip(el); });
document.addEventListener('focusout', e => { if (segOf(e)) hideTip(); });
// The bytes for a route depend only on the route itself and on which
// sessions are up, so they are fetched once per such combination and kept.
const wireSig = r => JSON.stringify(r) + '|' + state.peers.filter(p => p.state === 'Established').map(p => p.peer_addr + p.local_addr).join(',');
function wireHTML(route, stale) {
const cached = state.wire.get(route.prefix), sig = wireSig(route);
if (cached && cached.sig === sig) return cached.html;
stale.push(route);
// Keep showing the previous bytes while the new ones load.
return cached ? cached.html : '<span class="dim">loading...</span>';
}
async function loadWire(route) {
const sig = wireSig(route);
const pending = state.wire.get(route.prefix);
if (pending && pending.loading === sig) return;
state.wire.set(route.prefix, { ...(pending || { html: '<span class="dim">loading...</span>' }), loading: sig });
let html;
try {
const views = await getJSON(`/api/routes/local/wire?prefix=${encodeURIComponent(route.prefix)}`);
html = !views.length ? '<span class="dim">No established session to send this on.</span>' :
views.map(v => `UPDATE as sent to <b>${esc(v.peer_addr)}</b> AS${v.peer_asn}, ${v.length} bytes, next hop ${esc(v.next_hop)} &nbsp;
<span class="hh">&nbsp;header&nbsp;</span> <span class="hl">&nbsp;lengths&nbsp;</span> <span class="ha">&nbsp;path attributes&nbsp;</span> <span class="hn">&nbsp;NLRI&nbsp;</span>
<span class="dim">(hover a segment to decode it)</span>
<pre class="bytes">${segmentUpdate(v.hex).map(s => `<span class="${s.cls}" tabindex="0" data-kind="${s.kind}" data-w4="${v.four_octet_asn ? 1 : 0}" data-hex="${s.hex}">${s.hex}</span>`).join(' ')}</pre>`).join('');
} catch (e) { html = `<span class="bad">${esc(e.message)}</span>`; }
state.wire.set(route.prefix, { sig, html });
renderLocal();
const b = $('received-body'), st = state.status;
$('received-count').textContent = st && st.received_routes > state.received.length
? `showing ${state.received.length} of ${st.received_routes}, for ${st.received_prefixes} prefixes`
: `${state.received.length}${st ? `, for ${st.received_prefixes} prefixes` : ''}`;
if (!state.received.length) { setHTML(b, '<tr><td colspan="10" class="dim">No routes received from peers.</td></tr>'); return; }
setHTML(b, state.received.map(r => { const s = r.summary; return `<tr>
<td class="mono">${esc(r.prefix)}</td><td class="mono">${esc(r.peer_addr)} <span class="dim">AS${r.peer_asn}${r.path_id ? ` path ${r.path_id}` : ''}</span></td><td class="mono">${asPath(s.as_path)}</td>
<td class="mono">${s.next_hop ? esc(s.next_hop) : DASH}</td><td>${s.origin ? esc(s.origin) : DASH}</td><td>${s.med ?? DASH}</td><td>${s.local_pref ?? DASH}</td>
<td class="mono">${commList(s.communities, true)}</td><td class="mono">${commList(s.large_communities)}</td><td>${r.best ? '<span class="ok">best</span>' : ''}</td></tr>`; }).join(''));
}
// ---- data loading ------------------------------------------------------
async function loadStatus() { state.status = await getJSON('/api/status'); renderStatus(); renderLocal(); }
async function loadPeers() { state.peers = await getJSON('/api/peers'); renderPeers(); renderLocal(); }
async function loadLocal() { state.local = await getJSON('/api/routes/local'); renderLocal(); renderReceived(); }
async function loadStatus() { state.status = await getJSON('/api/status'); renderStatus(); renderLocal(); renderReceived(); }
async function loadPeers() { state.peers = await getJSON('/api/peers'); renderPeers(); }
async function loadLocal() { state.local = await getJSON('/api/routes/local'); renderLocal(); }
async function loadReceived() { state.received = await getJSON('/api/routes/received'); renderReceived(); }
async function loadPings() { state.pings = await getJSON('/api/pings'); renderPings(); }
async function loadAll() { await Promise.all([loadStatus(), loadPeers(), loadLocal(), loadReceived(), loadPings()]); }
async function loadAll() { await Promise.all([loadStatus(), loadPeers(), loadLocal(), loadReceived()]); }
// ---- events ------------------------------------------------------------
function logEvent(html) {
@@ -481,21 +195,19 @@ View: <a href="/metricz">variables</a>, <a href="/api/status">status json</a>, <
d.innerHTML = `${new Date().toLocaleTimeString()} ${html}`;
box.prepend(d); while (box.children.length > 200) box.lastChild.remove();
}
const names = list => list.map(c => state.wellKnown[c] || c).map(esc).join(' ');
const names = list => list.map(c => WELL_KNOWN[c] || c).map(esc).join(' ');
function describe(ev) {
switch (ev.type) {
case 'session_up': return `<span class="ok">session UP</span> with ${esc(ev.peer.name || ev.peer.peer_addr)} AS${ev.peer.peer_asn} (opened by ${ev.peer.direction === 'outbound' ? 'us' : 'peer'})`;
case 'route_refresh_requested': return `${esc(ev.peer.name || ev.peer.peer_addr)} AS${ev.peer.peer_asn} asked for a route refresh; re-sent all local routes`;
case 'route_refresh_requested': return `${esc(ev.peer.name || ev.peer.peer_addr)} AS${ev.peer.peer_asn} asked for a route refresh; re-sent all originated routes`;
case 'session_down': return `<span class="bad">session DOWN</span> with ${esc(ev.peer_addr)} AS${ev.peer_asn}`;
case 'local_announced': { const r = ev.route; return `announced ${esc(r.prefix)} by ${esc(r.owner)}` +
case 'local_announced': { const r = ev.route; return `originating ${esc(r.prefix)}` +
(r.communities.length ? ` [${names(r.communities)}]` : '') + (r.large_communities.length ? ` [${names(r.large_communities)}]` : '') +
(r.prepend ? ` prepend ${r.prepend}` : '') + (r.med != null ? ` med ${r.med}` : ''); }
case 'local_withdrawn': return `<span class="warn">withdrew</span> ${esc(ev.prefix)}`;
case 'received_announced': return `received ${esc(ev.route.prefix)} from AS${ev.route.peer_asn} path [${ev.route.summary.as_path.join(' ')}]`;
case 'received_withdrawn': return `peer AS${ev.peer_asn} <span class="warn">withdrew</span> ${esc(ev.prefix)}`;
case 'ping_seen': { const f = ev.ping; return `ping to ${esc(f.dst)}${f.owner ? ` (${esc(f.owner)})` : ''} from ${esc(f.src)}` +
(f.interface ? ` via ${esc(f.interface)}` : '') + (f.hop_limit != null ? `, hop limit ${f.hop_limit}` : ''); }
case 'lagged': return `<span class="bad">missed ${ev.missed} events</span> on ${esc(ev.stream)}; reloading`;
case 'local_withdrawn': return `<span class="warn">no longer originating</span> ${esc(ev.prefix)}`;
case 'best_changed': return `best route for ${esc(ev.route.prefix)} is now from AS${ev.route.peer_asn} path [${ev.route.summary.as_path.join(' ')}]`;
case 'best_withdrawn': return `${esc(ev.prefix)} has <span class="warn">no route</span> any more`;
case 'lagged': return `<span class="bad">missed ${ev.missed} events</span> on ${esc(ev.stream)}`;
default: return esc(JSON.stringify(ev));
}
}
@@ -506,14 +218,7 @@ View: <a href="/metricz">variables</a>, <a href="/api/status">status json</a>, <
es.onerror = () => { st.textContent = 'Reconnecting to the event stream...'; };
// Events only feed the log. How often the tables reload is decided by
// ?refresh=, so the page never changes faster than the reader asked for.
// Pings are the exception: the reader causes them from a terminal, not
// from this page, so their table follows the stream. A steady ping is
// logged once, when it starts.
es.addEventListener('bgp', e => {
const ev = JSON.parse(e.data);
if (ev.type === 'ping_seen') { pingSeen(ev.ping); if (!ev.new_burst) return; }
logEvent(describe(ev));
});
es.addEventListener('bgp', e => logEvent(describe(JSON.parse(e.data))));
}
// ---- auto-refresh, opt-in through ?refresh=<seconds> --------------------
@@ -532,74 +237,7 @@ View: <a href="/metricz">variables</a>, <a href="/api/status">status json</a>, <
$('reload-now').addEventListener('click', e => { e.preventDefault(); loadAll().catch(console.error); });
}
// ---- form --------------------------------------------------------------
// The reader's own action is not "auto" refresh: show its effect at once,
// and again shortly after for whatever the network sends back.
function reloadAfterAction() { loadAll().catch(console.error); setTimeout(() => loadAll().catch(console.error), 1500); }
const splitList = s => s.split(',').map(x => x.trim()).filter(Boolean);
function setMsg(text, cls) { const m = $('announce-msg'); m.className = cls || ''; m.textContent = text; }
function appendTo(id, value) {
const cur = splitList($(id).value); if (!cur.includes(value)) cur.push(value);
$(id).value = cur.join(', ');
}
$('announce-form').addEventListener('submit', async e => {
e.preventDefault();
const body = {
prefix: $('f-prefix').value.trim(), owner: myOwner(), origin: $('f-origin').value,
communities: splitList($('f-communities').value),
large_communities: splitList($('f-large').value),
};
if ($('f-med').value !== '') body.med = Number($('f-med').value);
try { localStorage.setItem('owner', body.owner); } catch (_) {}
setMsg('announcing...', 'dim');
const r = await fetch('/api/routes/local', { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify(body) });
const j = await r.json().catch(() => ({}));
const up = state.peers.filter(p => p.state === 'Established').length;
if (r.ok) { setMsg(`announced ${j.prefix} to ${up} peer(s)`, 'ok'); reloadAfterAction(); }
else setMsg(`${r.status}: ${j.error || 'request failed'}`, 'bad');
});
$('f-clear').addEventListener('click', () => { ['f-prefix','f-communities','f-large','f-med'].forEach(id => $(id).value = ''); $('f-origin').value = 'IGP'; setMsg(''); });
$('f-wk').addEventListener('change', e => { if (e.target.value) appendTo('f-communities', e.target.value); e.target.value = ''; });
$('f-lc').addEventListener('change', e => {
let v = e.target.value; e.target.value = '';
if (!v) return;
if (v.includes('PEER')) {
const peer = (prompt('AS number of the peer or neighbour?') || '').trim().replace(/^AS/i, '');
if (!/^\d+$/.test(peer)) { if (peer) setMsg(`"${peer}" is not an AS number`, 'bad'); return; }
v = v.replace('PEER', peer);
}
appendTo('f-large', v);
});
$('f-owner').addEventListener('input', () => { renderLocal(); renderPings(); });
$('pings-body').addEventListener('click', e => {
const link = e.target.closest('a[data-act="ping-bytes"]'); if (!link) return;
e.preventDefault();
const key = link.dataset.key;
state.openPing.has(key) ? state.openPing.delete(key) : state.openPing.add(key);
renderPings();
});
$('local-body').addEventListener('click', async e => {
const link = e.target.closest('a[data-act]'); if (!link) return;
e.preventDefault();
const prefix = link.dataset.prefix;
if (link.dataset.act === 'wire') { state.openWire.has(prefix) ? state.openWire.delete(prefix) : state.openWire.add(prefix); renderLocal(); return; }
if (!confirm(`Withdraw ${prefix}?`)) return;
const r = await fetch('/api/routes/local', { method: 'DELETE', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ prefix, owner: myOwner() }) });
if (!r.ok) { const j = await r.json().catch(() => ({})); setMsg(`${r.status}: ${j.error || 'withdraw failed'}`, 'bad'); }
else { setMsg(`withdrew ${prefix}`, 'warn'); state.openWire.delete(prefix); state.wire.delete(prefix); reloadAfterAction(); }
});
// ---- boot --------------------------------------------------------------
try { $('f-owner').value = localStorage.getItem('owner') || ''; } catch (_) {}
LARGE_PRESET_GROUPS.forEach(([, title, presets]) => {
const g = document.createElement('optgroup'); g.label = title;
presets.forEach(([value, label]) => { const o = document.createElement('option'); o.value = value; o.textContent = `${value.replace('PEER', '<as>')} ${label}`; g.appendChild(o); });
$('f-lc').appendChild(g);
});
getJSON('/api/communities/well-known').then(list => {
list.forEach(c => { state.wellKnown[c.value] = c.name; const o = document.createElement('option'); o.value = c.name; o.textContent = `${c.name} (${c.value})`; $('f-wk').appendChild(o); });
renderLocal(); renderReceived();
}).catch(console.error);
getJSON('/api/config').then(c => {
if (!c.text) return;
$('config-path').textContent = c.path ? `(${c.path}, as read at startup)` : '';
docs/lab/HANDOUT.md
-62
View File
@@ -1,62 +0,0 @@
# Announcing a prefix with BGP
You have a BGP router. It speaks to two neighbouring autonomous systems, and
whatever you announce will travel around a loop and come back to you, so you
can see exactly what the network did with it.
Open the router's status page (your instructor will give you the address,
usually `http://<host>:8080/statusz`). Everything on the page is live.
**Your prefix:** `10.100.<N>.0/24` where `N` is the number you were given.
Use your name as *owner*: only you can withdraw what you announce.
## The topology
```
you ──announce──▶ AS65000 (ferrix) ──▶ AS65001 (r1) ──▶ AS65002 (r2) ──▶ back to AS65000
```
Everything under **Received routes** came in from a neighbour. A yellow row
means it is one of the prefixes announced locally: it went around the loop.
## Exercises
1. **Plain announcement.** Announce your prefix with no options. Watch the
event log, then find your prefix in *Received routes*. What is its AS
path, and why is it that long? Which ASN appears first, and what does that
tell you about the direction the route travelled?
2. **Read the bytes.** Click *bytes* on your route under *Local routes*.
Hover the coloured segments. Find the 16 bytes of marker, the length, and
the AS_PATH attribute. How many bytes does your ASN take up in the path,
and why?
3. **NO_EXPORT.** Announce with the community `NO_EXPORT` (use the
well-known picker). Wait ten seconds. Does your prefix come back? Look at
the topology and explain which router stopped it and why it still
accepted it.
4. **Asking a neighbour for help.** Announce with the community
`65001:100`. r1 has been configured to act on that. What changed in the AS
path this time, and who changed it? This is how real operators ask an
upstream to prepend on their behalf; look up your ISP's "BGP communities"
page later.
5. **Blackholing.** Announce with `BLACKHOLE`. What happens? When would an
operator want this?
6. **Withdraw.** Withdraw your prefix and watch the event log. How many
messages did it take, and what did the withdrawal look like on the wire
compared to the announcement? (Click *bytes* before withdrawing to compare.)
## Things to notice
- Your router never sent anyone else's routes onward. Only the two FRR
routers re-advertise. What is a router that re-advertises called, and what
would happen to the AS path if ferrix did it too?
- The AS path is BGP's loop prevention. Your prefix came back with your own
ASN in the path and the router accepted it anyway. What should it have
done, and why is this lab allowed to cheat?
- Communities are just 32-bit tags with no built-in meaning except a few
well-known ones. Everything r1 did with `65001:100` was policy someone
wrote. Whose job is it to document what a community means?
docs/lab/README.md
-81
View File
@@ -1,81 +0,0 @@
# BGP teaching lab
A small topology in which a prefix announced from the ferrix_bgp web UI is
carried around a loop of two routers and arrives back, so students can watch
what their path attributes and communities do.
```
announce re-advertise
┌─────────┐ ──────▶ ┌────────┐ ──────▶ ┌────────┐
│ ferrix │ │ r1 │ │ r2 │
│ AS65000 │ │ AS65001│ │ AS65002│
│ web UI │ ◀──────────────────────────────────────┘
└─────────┘ comes back with AS path 65002 65001 65000
```
- **ferrix** (AS65000, `10.200.0.1`) dials r1 and r2 and announces whatever
students enter on `http://localhost:8080/statusz`. It does not forward
routes it learns, so the only routes it sends are the students'.
- **r1** (AS65001, FRR) applies policy to routes from ferrix based on the
communities they carry, then passes them to r2. It originates
`192.0.2.0/24`.
- **r2** (AS65002, FRR) refuses routes directly from ferrix, so it only learns
student prefixes via r1, and re-advertises everything to ferrix. It
originates `198.51.100.0/24`.
## Running it
From the repository root:
```
docker compose -f docs/lab/docker-compose.yml up --build
```
Then open <http://localhost:8080/statusz>. Both peers should show
`Established` within a few seconds and the received table should already
list the two prefixes r1 and r2 originate.
Useful on the routers:
```
docker exec r1 vtysh -c 'show bgp ipv4 unicast'
docker exec r2 vtysh -c 'show bgp ipv4 unicast 10.100.1.0/24'
docker exec r1 vtysh -c 'show bgp neighbors 10.200.0.1 advertised-routes'
```
## Policy on r1
| Students set | r1 does | Visible effect in ferrix |
|---------------------------|---------------------------------------------|-------------------------------------------------------|
| nothing | forwards to r2 | prefix returns, AS path `65002 65001 65000` |
| `NO_EXPORT` | keeps it, does not send to r2 (RFC 1997) | prefix does not return |
| `65001:100` | prepends `65001` three times | returns with path `65002 65001 65001 65001 65001 65000` |
| `BLACKHOLE` (65535:666) | drops it on import | prefix does not return |
## Without containers
`fake_peer` in `crates/integration-tests/examples` is enough for the core
loop on one machine: it echoes every prefix back with its own ASN prepended
and honours `NO_EXPORT`. See `bin/bgp/README.md`.
## Caveats
These container files were written without a container runtime to hand, so
they have not been executed. Things most likely to need a touch:
- the FRR image tag, and FRR's route-map / community-list syntax if your
version differs from 10.x;
- `no bgp ebgp-requires-policy` is required on FRR 7.4+ or eBGP routes are
silently dropped; it is present in both configs;
- ferrix binds port 179 inside its container, which requires no special
capability there. If you run it on the host instead, either run as root,
grant `CAP_NET_BIND_SERVICE`, or change `listen_endpoints` and the FRR
neighbor port.
- Both ferrix and the FRR routers dial each other, so connection collisions
(RFC 4271 §6.8) are normal at start-up: one of the two TCP connections is
closed with a Cease / Connection Collision Resolution NOTIFICATION and the
other carries the session. ferrix has the lowest BGP Identifier here, so
the connection the FRR router opened is the one that survives; the
"opened by" column on the status page shows which side that was.
- ferrix does not check AS paths for its own ASN. That is what lets the
looped route be accepted; a production BGP speaker would reject it.
docs/lab/docker-compose.yml
-53
View File
@@ -1,53 +0,0 @@
# Teaching lab: ferrix_bgp plus two FRR routers forming a loop, so a prefix
# announced from the web UI travels ferrix -> r1 -> r2 -> ferrix and shows up
# in the "received routes" table with the AS path it collected on the way.
#
# AS65000 ferrix 10.200.0.1 web UI on http://localhost:8080/statusz
# AS65001 r1 10.200.0.2 applies community-driven policy
# AS65002 r2 10.200.0.3 learns only via r1, re-advertises to ferrix
#
# Usage (from the repository root):
# docker compose -f docs/lab/docker-compose.yml up --build
#
# NOTE: this topology has not been run by the author of this file on the
# machine it was written on (no container runtime was available). Expect to
# adjust image tags and FRR config syntax for the FRR version you pull.
services:
ferrix:
build:
context: ../..
container_name: ferrix
volumes:
- ./ferrix.textpb:/etc/ferrix/server.textpb:ro
ports:
- "8080:8080"
networks:
lab:
ipv4_address: 10.200.0.1
r1:
image: quay.io/frrouting/frr:10.2.1
container_name: r1
privileged: true
volumes:
- ./r1:/etc/frr:ro
networks:
lab:
ipv4_address: 10.200.0.2
r2:
image: quay.io/frrouting/frr:10.2.1
container_name: r2
privileged: true
volumes:
- ./r2:/etc/frr:ro
networks:
lab:
ipv4_address: 10.200.0.3
networks:
lab:
driver: bridge
ipam:
config:
- subnet: 10.200.0.0/24
docs/lab/ferrix.textpb
-30
View File
@@ -1,30 +0,0 @@
router_id: "10.200.0.1"
local_asn: 65000
listen_endpoints { address: "0.0.0.0" port: 179 }
peers {
name: "r1"
endpoint { address: "10.200.0.2" port: 179 }
remote_asn: 65001
hold_time_seconds: 30
}
peers {
name: "r2"
endpoint { address: "10.200.0.3" port: 179 }
remote_asn: 65002
hold_time_seconds: 30
}
connect_retry_seconds: 5
# If r1 or r2 is restarted it will dial us again while we still believe the
# old session is up. Let that new connection replace the stale one straight
# away instead of refusing it until the hold timer (30 s here) expires.
collision_detect_established_state: true
# Students may only announce out of 10.100.0.0/16; one /24 each is plenty.
local_route_policy {
allowed_prefixes: "10.100.0.0/16"
max_routes_per_owner: 4
}
http_endpoint { address: "0.0.0.0" port: 8080 }
docs/lab/handout/.gitignore
-8
View File
@@ -1,8 +0,0 @@
*.aux
*.log
*.out
*.fls
*.fdb_latexmk
*.synctex.gz
build.log
class.pdf
docs/lab/handout/build-class.py
-63
View File
@@ -1,63 +0,0 @@
#!/usr/bin/env python3
"""One exercise sheet per student, with their prefix filled in, in one PDF.
python3 build-class.py 100 # students 1..100 -> class.pdf
python3 build-class.py 100 -o out.pdf
python3 build-class.py 5 12 -o few.pdf # students 5..12
Student n gets 2a0d:d741:<n in hex>::/48; the lab /40 allows 1..255. Each
sheet is compiled separately and must come out to exactly two pages, so
the result prints double-sided with one student per leaf. Needs pdflatex
and pdfunite (poppler).
"""
import argparse
import os
import shutil
import subprocess
import sys
import tempfile
HERE = os.path.dirname(os.path.abspath(__file__))
def page_count(pdf):
out = subprocess.run(["pdfinfo", pdf], capture_output=True, text=True, check=True).stdout
return int(next(line.split()[-1] for line in out.splitlines() if line.startswith("Pages:")))
def main():
ap = argparse.ArgumentParser(description=__doc__.split("\n")[0])
ap.add_argument("first", type=int, help="first student number, or the count if it is the only number")
ap.add_argument("last", type=int, nargs="?")
ap.add_argument("-o", "--output", default=os.path.join(HERE, "class.pdf"))
args = ap.parse_args()
first, last = (1, args.first) if args.last is None else (args.first, args.last)
if not 1 <= first <= last <= 255:
sys.exit("student numbers must be within 1..255: the lab /40 has 255 usable /48s")
with tempfile.TemporaryDirectory() as tmp:
for name in os.listdir(HERE):
if name.endswith((".tex", ".cls", ".sty", ".bst", ".bib")):
shutil.copy(os.path.join(HERE, name), tmp)
parts = []
for n in range(first, last + 1):
job = f"student-{n:03d}"
tex = f"\\def\\studentnum{{{n}}}\\def\\studenthex{{{n:x}}}\\input{{handout.tex}}"
for _ in range(2): # twice, for the figure and table references
r = subprocess.run(["pdflatex", "-interaction=nonstopmode", "-halt-on-error", "-jobname", job, tex],
cwd=tmp, capture_output=True, text=True)
if r.returncode != 0:
sys.exit(f"student {n}: pdflatex failed:\n{r.stdout[-2000:]}")
pdf = os.path.join(tmp, job + ".pdf")
pages = page_count(pdf)
if pages != 2:
sys.exit(f"student {n}: sheet is {pages} pages, not 2; double-sided printing would misalign")
parts.append(pdf)
print(f"\rstudent {n}: 2a0d:d741:{n:x}::/48", end="", flush=True)
print()
subprocess.run(["pdfunite", *parts, args.output], check=True)
print(f"wrote {args.output}: students {first}..{last}, {page_count(args.output)} pages")
if __name__ == "__main__":
main()
docs/lab/handout/handout.pdf
BIN
View File
Binary file not shown.
docs/lab/handout/handout.tex
-195
View File
@@ -1,195 +0,0 @@
% Exercise sheet for the "Hands-on BGP" lecture: two A4 pages, one sheet.
% Springer LNCS class (llncs.cls ships with TeX Live). Build: latexmk -pdf handout.tex
\documentclass[a4paper]{llncs}
\usepackage[a4paper,margin=1in]{geometry}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{microtype}
\usepackage{booktabs}
\usepackage{amssymb}
\usepackage{listings}
\usepackage{tikz}
\usepackage[bitstream-charter]{mathdesign}
\usetikzlibrary{positioning,arrows.meta}
\PassOptionsToPackage{hyphens}{url}
\usepackage[hidelinks]{hyperref}
\renewcommand\UrlFont{\ttfamily\small}
\pagestyle{empty}
% Floats sit in the text here, so they need less air than in a paper.
\setlength{\intextsep}{7pt}
\setlength{\abovecaptionskip}{4pt}
\setlength{\belowcaptionskip}{0pt}
% ---- things to change per class -------------------------------------------
\newcommand{\laburl}{https://bgplab.rayhaan.net/}
% build-class.py compiles one copy per student with \studentnum and
% \studenthex defined (e.g. 26 and 1a); without them this is the blank sheet.
\providecommand{\studentnum}{}
\providecommand{\studenthex}{}
\ifx\studentnum\empty
\newcommand{\myprefix}{\texttt{2a0d:d741:}\underline{\texttt{\,n\,}}\texttt{::/48}}
\newcommand{\nhex}{<n>}
\else
\newcommand{\myprefix}{\texttt{2a0d:d741:\studenthex::/48}}
\newcommand{\nhex}{\studenthex}
\fi
% ---- exercise machinery ------------------------------------------------------
\lstset{basicstyle=\ttfamily\small,columns=fullflexible,keepspaces=true,
aboveskip=0.5em,belowskip=0.3em}
\newcounter{task}[section]
\renewcommand{\thetask}{\thesection.\arabic{task}}
\newcommand{\task}[1]{\refstepcounter{task}\smallskip\noindent
\textbf{Task \thetask}\enspace\textit{#1}\enspace\ignorespaces}
\newcommand{\answer}[1][1]{\par\nobreak
\foreach \i in {1,...,#1}{\vspace{1.0em}\noindent\makebox[\linewidth]{\dotfill}\par}\smallskip}
% A labelled answer box: the label on its own line, then #1 dotted lines to
% write on (default 2). Used side by side by \fillintwo and \fillinthree.
\newcommand{\writelines}[1]{\foreach \i in {1,...,#1}{\vspace{1.0em}\makebox[\linewidth]{\dotfill}\par}}
\newcommand{\fillbox}[3][2]{\parbox[t]{#2}{#3\par\nopagebreak\writelines{#1}}}
\newcommand{\fillintwo}[3][2]{\par\nopagebreak\smallskip\noindent
\fillbox[#1]{0.485\linewidth}{#2}\hfill\fillbox[#1]{0.485\linewidth}{#3}\par\vspace{0.4em}}
\newcommand{\fillinthree}[4][2]{\par\nopagebreak\smallskip\noindent
\fillbox[#1]{0.32\linewidth}{#2}\hfill\fillbox[#1]{0.32\linewidth}{#3}\hfill\fillbox[#1]{0.32\linewidth}{#4}\par\vspace{0.4em}}
\newcommand{\cbox}{$\square$\;}
\newcommand{\reach}{\par\nopagebreak\smallskip\noindent
Reachable from:\hspace{1.2em}\cbox laptop on the university network\hspace{2em}\cbox phone on mobile data\par\smallskip}
\begin{document}
\title{Hands on BGP: Announcing and Steering an IPv6 Prefix}
\author{Rayhaan Jaufeerally}
\institute{Network Security, ETHZ\\ Lab page: \url{\laburl} (username ``netsec'', password ``perrig'')}
\maketitle
\medskip\noindent
\ifx\studentnum\empty
\parbox[t]{0.44\linewidth}{Name:~\dotfill}\hfill
\parbox[t]{0.27\linewidth}{Prefix: \myprefix}
\smallskip\noindent
Your prefix is \myprefix, where \texttt{n} is your number \emph{in
hexadecimal}.\par\noindent
Student 10 has \texttt{2a0d:d741:a::/48} and student 26 has
\texttt{2a0d:d741:1a::/48}.
\else
\parbox[t]{0.44\linewidth}{Name:~\dotfill}\hfill
\parbox[t]{0.5\linewidth}{Student \studentnum\quad Prefix: \myprefix}
\smallskip\noindent
Your prefix is \myprefix: student number \studentnum, written in
hexadecimal. Make sure to use this prefix in the exercise.
\fi
% ==============================================================================
\section{Who Announces the Address You Are Using?}
\task{Find your IPv6 address.}
Open \texttt{https://bgp.tools} in a browser: the top of the page shows the address
you are connecting from, the prefix it belongs to, and the network that
announces that prefix.
\fillinthree{Your IPv6 address:}{its prefix:}{announced by AS:}
\task{Ask the registry.}
The RIPE database records who holds address space
(\texttt{inet6num}, or \texttt{inetnum} for IPv4) and who may announce it
(\texttt{route6}, \texttt{route}). Look up your IPv6 address, then your lab prefix:
\begin{lstlisting}[escapeinside={(*}{*)}]
whois <your IPv6 address> whois 2a0d:d741:(*\nhex*)::/48
\end{lstlisting}
\fillintwo{Your IPv6 address: held by (\texttt{netname})}{\texttt{origin} AS:}
\fillintwo{Your lab prefix: \texttt{origin} AS}{is it announced right now?}
% ==============================================================================
\section{Announcing Your Prefix}
\begin{figure}[h]
\centering
\begin{tikzpicture}[font=\footnotesize, node distance=5mm and 11mm,
box/.style={draw, rounded corners=1.5pt, minimum height=7.5mm, inner xsep=4pt, align=center},
bgp/.style={-{Stealth[length=2mm]}, dashed}, web/.style={-{Stealth[length=2mm]}}]
\node[box] (you) {you\\[-1pt]\scriptsize browser};
\node[box, right=of you] (ferrix) {lab daemon\\[-1pt]\scriptsize AS4200000001};
\node[box, right=of ferrix] (pr) {\texttt{pr01.rue}\\[-1pt]\scriptsize AS210036};
\node[box, right=of pr] (rs) {SwissIX route servers\\[-1pt]\scriptsize AS42476};
\node[box, right=of rs] (sw) {SWITCH (ETH)\\[-1pt]\scriptsize AS559};
\node[box, below=4mm of rs] (other) {other exchanges, transit};
\draw[web] (you) -- node[above, inner sep=1.5pt]{\tiny HTTPS} (ferrix);
\draw[bgp] (ferrix) -- node[above, inner sep=1.5pt]{\tiny BGP} (pr);
\draw[bgp] (pr) -- (rs);
\draw[bgp] (rs) -- (sw);
\draw[bgp] (pr) |- (other);
\end{tikzpicture}
\caption{The lab. Dashed arrows are BGP sessions; everything right of
\texttt{pr01.rue} is the real Internet.}
\label{fig:lab}
\end{figure}
\task{Announce.}
Open \url{\laburl} (username ``netsec'' password ``perrig''). Enter your name as
\emph{owner} and your prefix, and press \emph{Announce}.
\task{Read what you sent.}
Click \emph{bytes} next to your route and hover over the coloured segments.
\fillintwo{AS number(s) in \texttt{AS\_PATH}:}{attribute carrying your prefix:}
\task{See it from outside.}
Wait a moment, then look your prefix up on \texttt{https://bgp.tools/super-lg}.
\fillintwo{Origin AS the world sees:}{an AS path you found:}
\noindent The lab daemon's AS4200000001 is missing from that path. Where did it go, and why?
\answer[2]
\task{Reach it.}
The router answers for every address in the lab space, so your prefix is
reachable exactly when your announcement got out:\quad
\texttt{ping -6 2a0d:d741:\nhex::1}
\reach
% ==============================================================================
\section{Steering Your Prefix with Large Communities}
A Large Community is a tag added to a route. It can have any meaning that the network
which issued it decides. In BGP large communities, the first part is the AS number
and the subsequent two integers are up to the interpretation of that network.
\begin{table}[h]
\centering
\begin{tabular}{@{}l@{\hspace{1.6em}}l@{\hspace{1.6em}}l@{}}
\toprule
Large community & Read by & Effect \\
\midrule
\texttt{210036:0:0} & \texttt{pr01.rue} & announce nowhere \ldots \\
\texttt{210036:1:42476} & \texttt{pr01.rue} & \ldots except to AS42476, the SwissIX route servers \\
\midrule
\texttt{42476:0:0} & SwissIX & send to no member \ldots \\
\texttt{42476:1:559} & SwissIX & \ldots except AS559, SWITCH, the network ETH is part of \\
\bottomrule
\end{tabular}
\end{table}
\task{One exchange only.}
Announce your prefix again with \texttt{210036:0:0} and
\texttt{210036:1:42476}; announcing a prefix you own replaces it. Before you
look: who do you expect to still see it?
\answer[2]
\noindent Wait a moment, then check on \texttt{https://bgp.tools/super-lg} and with \texttt{ping}.
\reach
\task{To SWITCH only.}
Add \texttt{42476:0:0} and \texttt{42476:1:559} to the two you already have and
announce again. Our router now tells only the route servers, and they tell
only SWITCH.
\reach
\noindent Is your prefix still visible on \texttt{https://bgp.tools/super-lg}? Explain what you see.
\answer[2]
\task{Use of large communities.}
You steered routing in two networks you do not operate, using nothing but
numbers attached to a route. What stops you from attaching a community that
one of \emph{their} upstreams acts on?
\answer[2]
\task{Clean up.} Withdraw your prefix and see how long it takes to go away.
\end{document}
docs/lab/prod/bird-ferrix.conf
-172
View File
@@ -1,172 +0,0 @@
# BIRD 2.x / 3.x: peer the production router (AS210036) with ferrix_bgp over
# WireGuard. ferrix is announce-only: we accept the lab /48s from it and send
# it nothing.
#
# Include from bird.conf: include "/etc/bird/bird-ferrix.conf";
# Check and apply: birdc configure check && birdc configure
# Inspect: birdc show protocols all ferrix
# birdc show route protocol ferrix all
define MY_ASN = 210036;
define FERRIX_ASN = 4200000001;
define LAB_PREFIXES = [ 2a0d:d741::/40{48,48} ];
filter ferrix_import
{
# Only the students' /48s, nothing shorter or longer, nothing else.
if !(net ~ LAB_PREFIXES) then reject;
# ferrix allows at most 16 prepends on top of its own ASN.
if bgp_path.len > 17 then reject;
if bgp_path.first != FERRIX_ASN then reject;
# Students type communities freely into a web form with no login, so
# only namespaces we have chosen to expose get through.
#
# Standard: just the two well-known ones BIRD itself honours, so
# NO_EXPORT / NO_ADVERTISE visibly work. Notably BLACKHOLE (65535:666),
# which SwissIX acts on, stays scrubbed.
bgp_community.filter([ (65535, 65281), (65535, 65282) ]);
# Large: the SwissIX route-server action communities, all of which live
# under their route-server ASN 42476
# (https://www.swissix.ch/resources/action-bgp-communities/):
# 42476:0:<peer-as> do not redistribute to peer 42476:0:0 to nobody
# 42476:1:<peer-as> redistribute to peer 42476:1:0 to all
# 42476:101|102|103:<peer-as or 0> prepend 1x / 2x / 3x
# 42476:901|902:<peer-as> add NO_EXPORT / NO_ADVERTISE
# Everything else is dropped, including the rest of our own
# (210036, *, *) space, so students cannot trigger anything else this
# network acts on. Add further namespaces here deliberately.
#
# And our own export-control communities, functions 0 to 3, evaluated by
# lab_export_allowed() below.
bgp_large_community.filter([ (42476, *, *), (MY_ASN, 0..3, *) ]);
bgp_ext_community.empty;
# Announce-only: there is nothing behind these prefixes. Do not forward
# their traffic into the tunnel; answer with "unreachable" instead.
dest = RTD_UNREACHABLE;
accept;
}
protocol bgp ferrix
{
description "ferrix_bgp teaching lab (announce only)";
local 2a0d:d747:1::1 as 210036;
neighbor 2a0d:d747:1:: as FERRIX_ASN;
direct;
# ferrix dials us. Staying passive avoids log noise from failed attempts
# towards a laptop that is asleep or firewalled.
passive on;
# ferrix supports none of these; leave them off.
# password "..."; (TCP MD5)
# ttl security on; (GTSM)
# bfd on;
ipv6 {
import filter ferrix_import;
# 128 students, one /48 each. Extra routes are ignored, the session
# stays up.
import limit 130 action block;
# Send nothing: ferrix keeps received routes in memory and renders
# every one of them on its status page.
export none;
};
}
# ---------------------------------------------------------------------------
# Export control for lab routes.
#
# BIRD export filters cannot see which neighbour they are exporting to, and
# filters take no arguments; functions do. So every eBGP session states who it
# talks to and where, by calling the shared export function with its neighbour
# AS and a location:
#
# ipv6 { export where peers_out(42476, AT_SWISSIX); ... };
#
# Students steer their prefix with large communities, in this precedence:
#
# 1. 210036:0:<as> never announce to that neighbour AS
# 210036:2:<location> never announce at that location
# 2. 210036:1:<as> announce to that neighbour AS
# 210036:3:<location> announce at that location
# 3. 210036:0:0 announce nowhere (unless rule 2 says otherwise)
# 4. nothing announce everywhere <- default
#
# A "never" always wins, so the two dimensions combine. The shape and
# precedence match the SwissIX route-server communities on purpose, so one
# idea is applied at two hops.
#
# only the SwissIX route servers: 210036:0:0 210036:1:42476
# ...and there only SWITCH: add 42476:0:0 42476:1:559
# every session at SwissIX: 210036:0:0 210036:3:1
# Cloudflare, but not at DE-CIX: 210036:0:0 210036:1:13335 210036:2:4
# ---------------------------------------------------------------------------
define AT_SWISSIX = 1;
define AT_NLIX = 2;
define AT_FRANCEIX = 3;
define AT_DECIX = 4;
define AT_FREEIX = 5;
define AT_COMMUNITYIX = 6;
define AT_TRANSIT = 100;
define AT_DOWNSTREAM = 200;
function lab_export_allowed(int peer_as; int location)
{
if (MY_ASN, 0, peer_as) ~ bgp_large_community then return false;
if (MY_ASN, 2, location) ~ bgp_large_community then return false;
if (MY_ASN, 1, peer_as) ~ bgp_large_community then return true;
if (MY_ASN, 3, location) ~ bgp_large_community then return true;
if (MY_ASN, 0, 0) ~ bgp_large_community then return false;
return true;
}
# Decides whether a lab route goes out on a session and, if so, makes it
# presentable: the origin must appear as AS210036 (matching the route6
# objects) and our action communities must not leak.
function lab_route_out(int peer_as; int location)
{
if !lab_export_allowed(peer_as, location) then return false;
bgp_path.delete([ 64512..65534, 4200000000..4294967294 ]);
bgp_large_community.delete([ (MY_ASN, 0..3, *) ]);
return true;
}
# The shared export function then ends its list of accepted routes with
#
# if proto = "ferrix" then return lab_route_out(peer_as, location);
# return false;
#
# and returns true for everything else it accepts. Sessions that export
# everything (downstream customers) need the same gate, or they receive lab
# routes regardless of what the student asked for:
#
# export filter {
# if proto = "ferrix" then {
# if lab_route_out(210400, AT_DOWNSTREAM) then accept;
# reject;
# }
# accept;
# };
#
# The ferrix branch only ever applies to routes learned from ferrix, so these
# communities cannot be used by anyone else to steer other exports.
#
# If this BIRD rejects "int peer_as; int location", it wants a comma there.
#
# Check what a session would send, without waiting for a looking glass:
# birdc show route 2a0d:d741:1::/48 export <protocol-name> all
#
# The status page no longer offers AS-path prepending or the SwissIX prepend
# communities (42476:101..103:*), to keep the exercise focused. The import
# filter still admits the whole 42476 namespace, so a student who types one
# by hand can use it; narrow the filter if that is not wanted.
#
# The session towards the SwissIX route servers must send large communities
# unmodified apart from the delete above; check nothing else clears them.
# ---------------------------------------------------------------------------
docs/lab/prod/ferrix.textpb
-20
View File
@@ -1,20 +0,0 @@
router_id: "10.255.0.2" # must differ from the production router's
local_asn: 4200000001 # private asn used for internal peering.
listen_endpoints { address: "2a0d:d747:1::" port: 9179 }
peers {
name: "pr01.rue.rayhaan.net"
endpoint { address: "2a0d:d747:1::1" port: 179 }
remote_asn: 210036
hold_time_seconds: 90
}
connect_retry_seconds: 5
collision_detect_established_state: true
local_route_policy {
allowed_prefixes: "2a0d:d741::/40"
max_routes_per_owner: 1
}
http_endpoint { address: "127.0.0.1" port: 8080 }
Loaded 100 of 177 files, more files were not shown because too many files have changed in this diff. Show more