rayhaan/ferrix
0
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

merge into: rayhaan/ferrix:main
rayhaan/ferrix:main rayhaan/ferrix:speaker-cleanup rayhaan/ferrix:netsec_2026
rayhaan/ferrix:v0.2.0
..
pull from: rayhaan/ferrix:speaker-cleanup
rayhaan/ferrix:main rayhaan/ferrix:speaker-cleanup rayhaan/ferrix:netsec_2026
rayhaan/ferrix:v0.2.0
65 Commits
main .. speaker-cleanup
Author SHA1 Message Date
Rayhaan JaufeerallyandClaude Fable 5.1 9caaf5018e Review pass on the server: bugs fixed, session setup simplified
ci / check (push) Canceled after 0s
Details
ci / fuzz (push) Canceled after 0s
Details 
A review of the session driver, the server loop, the dialer, the
advertiser and the HTTP API, with the fixes.

Bugs:

- A peer that stopped reading held its session, and our routes, for as
  long as TCP kept retrying: a blocked write was awaited with no timer
  running. Writes now give up after the Send Hold Time of RFC 9687.
- Routes we originate were sent to a peer in our own AS with our AS
  number on the path and no LOCAL_PREF, which such a peer refuses. They
  now go with an empty path and a LOCAL_PREF.
- Collision detection knew a peer by its BGP Identifier alone. A
  neighbour with one session over IPv4 and one over IPv6 had the second
  closed as a collision, and any host that could connect could displace
  a peer's connection by claiming its identifier. A peer is now its
  address and identifier together.
- The dialer reset its backoff whenever TCP connected, so a peer that
  accepts the connection and refuses the session was dialled every few
  seconds for ever. It resets only after a session was established.
- A failure on our side (routes that cannot be stored) dropped the
  connection without a NOTIFICATION. It goes through the state machine
  and the peer is sent Cease. A queued message that cannot be encoded
  is logged and skipped instead of ending the session.
- The accept loop retried at once after an error, spinning when the
  process was out of file descriptors.
- Attribute sets no route used any more were dropped only when a peer
  went away. A task now does it every minute.
- The advertiser subscribed to session events inside its task, after the
  dialers were started, and could miss the first session.
- Shutdown waited without limit on a session whose queue was full.
- /api/routes/received took any limit and built the reply on a runtime
  thread. The limit is capped and the work is done off those threads.
- The default log level was TRACE, which logs every UPDATE.
- IPv6 listeners are IPv6-only: listening on 0.0.0.0 and [::] fails on
  systems where the second would take IPv4 too, and IPv4 peers arriving
  under mapped addresses matched no configured peer.
- A connection closed after a NOTIFICATION is kept for a moment so that
  the peer can read it.
- The configuration accepted two peers with one name or one address,
  and a router id of 0.0.0.0.
- An HTTP address that cannot be bound now stops the daemon at startup.

Simplifications:

- SessionConfig mixed what a server's sessions share with what is
  configured for one peer, and was cloned and patched field by field in
  two places. It is ServerContext, shared, and PeerSettings, made from a
  PeerConfig in one place.
- The attributes of an UPDATE as they were on the wire travelled from
  the reader to the table through a field of the session, around the
  state machine. They go through it, with the message.
- run_session returns how the session ended and whether it had been
  established; Ok no longer reads as "closed by peer".
- Session is not generic over its reader.
- BgpServerBuilder had nothing to build: BgpServer::bind takes the
  configuration.
- One shutdown path, close_sessions, for the signal handler and for a
  failed server.
- The advertiser's three copies of "send every route" are one.
- PeerId moved from the rib crate, which did not use it, to the server.
- Unused API removed or made private to the crate.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-02 10:46:09 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 a8aba52562 Review pass on the codec and the table: bugs fixed, encoders simplified 
A review of the wire codec and the routing table, with the fixes.

Bugs in the codec:

- A NOTIFICATION with a code or subcode we have no name for (subcode 0,
  "unspecific", is common) failed to decode and was answered as a bad
  message. Every pair now reads, as BgpError::Other.
- Optional parameters in the extended form of RFC 9072 were read and
  written with one-octet parameter lengths; they are two.
- An ORF capability listing more than one address family made the whole
  OPEN malformed.
- Bytes after an OPEN's optional parameters were accepted.
- Extended communities were split into fields by rules that were wrong
  for the common types. They are kept as their eight octets.
- Several length fields were computed in u16 and could wrap on encode.

Simplifications in the codec:

- Lengths are filled in after the value is written, for messages,
  attributes, capabilities and the UPDATE's two fields. The parallel
  wire_len functions, which had to be kept in step by hand, are gone.
  Message::to_wire no longer encodes into a second buffer.
- Type codes for messages, attributes and capabilities are each defined
  once. The strum discriminant enums and their hand-written conversions
  are removed.
- One function splits an attribute off a buffer; the UPDATE decoder, the
  plain attribute parser and the table's key reader all use it.
- The UPDATE decoder's 185-line loop is an AttributeReader with one
  method that decides what becomes of an attribute. Duplicate detection
  is a bitset, not a HashSet allocated per UPDATE.
- ParserContext::four_octet_asn is a bool. The "context not set" error
  path, which treated a programming error as a malformed route, is gone.
- Codec functions take a context only if they read it, and return a
  Result only if they can fail.
- MP_REACH_NLRI and MP_UNREACH_NLRI hold an AfiSafi.
- AddressFamilyId is IpVersion, which is what it was.
- Deserialize is no longer derived on codec types: nothing used it, and
  it bypassed IpPrefix's invariants. IpPrefix::to_wire, which was
  broken and unused, is removed.

Bugs in the table:

- The index of a row's best route was sixteen bits. A peer using
  ADD-PATH could send more routes for a prefix than that counts, and
  the index wrapped. It is 32 bits, at no cost in row size.
- Shards and attribute-store stripes were picked from the top bits of
  the hash, which are the bits hashbrown keeps as the tag it compares
  first. With 64 shards each shard's table had 2 of 128 tag values.
  They are now picked from bits the table does not use.
- remove_peer held the peer list's lock for the whole scan of the
  table, so no session could establish while a peer was removed.
- When more prefixes went at once than were left, the forced trim
  dropped a fixed few entries, reset every reader, and re-ran the tidy
  on every later write. It drops the older half.
- Readers were woken by every change, including a repeated announcement
  and changes their journal does not record.
- Reader::poll with a budget of zero never made progress.

Simplifications in the table:

- The two journals of a shard are one Journals value written through
  three calls; the five places that kept them in step by hand are gone.
  Whether an entry is still live is decided in one place.
- The shortcuts of route selection moved to select.rs, next to the
  invariant they depend on, which is now written down.
- announce and withdraw take routes with or without path identifiers;
  the _paths variants are removed.
- The selection key is read off the wire form only; the second
  implementation over parsed attributes is removed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-02 10:32:27 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 d49bab58a5 Add ADD-PATH (RFC 7911): several routes for a prefix from one peer 
A peer can now send several routes for one prefix, told apart by a path
identifier, and each is a route of its own in the table.

Wire:
- The ADD-PATH capability lists several address families. It was read as
  exactly one, so the OPEN of a router offering it for IPv4 and IPv6 was
  malformed and the session never came up.
- Prefixes in UPDATEs are `Nlri`: a prefix and a path identifier. The
  identifier is on the wire, in the body and in MP_REACH/MP_UNREACH, for
  the families the session uses ADD-PATH for in that direction. Sending
  and receiving are settled separately, so a session has one codec context
  for each.

Negotiation:
- `add_path` on a peer in the configuration: off (the default), receive,
  send or both, offered for every family offered to the peer.
- In use for a family and a direction when one side offered to send and
  the other to receive. Shown on /statusz and in /api/peers.

Table:
- A route is known by its peer and path identifier together. Identifiers
  are stored only in rows that have a non-zero one, so routes of peers
  that send one route each cost nothing more.
- Equal routes from one peer are chosen between by the lower identifier.
- /api/routes/received gives each route's path_id.

Journal:
- A second journal in each shard records every change to any route, for
  followers that want all the routes of a prefix and not only the best:
  RibTable::reader_of_all_routes and Event::Routes. This is what sending
  several routes to a peer will read. It costs nothing while nobody
  follows it.

Sending: prefixes go out with a path identifier where that was agreed, but
there is still one route for each prefix to send: only originated routes
are announced.

Tested with the table and journal model tests extended to path identifiers
and to followers of all routes, the codec property tests and fuzz targets
extended to ADD-PATH sessions, and end to end in both directions and for
the case where only one side offers it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-02 00:57:15 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 31aa3a4563 Use the sharded table in the server: sessions choose best routes 
The server now keeps what it learns in table::RibTable. The Adj-RIB-In
actor is gone, and with it the queue every UPDATE from every session went
through.

- A session takes a number in the table when it is established and writes
  its UPDATEs into the table itself. Its routes are removed when it ends,
  on a blocking thread since that looks at every prefix; a session torn
  down without cleaning up still has them removed.
- For every prefix the best route is chosen (RFC 4271 §9.1.2.2) as routes
  come and go. Nothing acts on the choice yet beyond showing it.
- The UPDATE decoder hands over the bytes of the attributes it kept, so the
  table finds the attribute set from them without writing the attributes
  out again. On a session with two-octet AS numbers the bytes are not in
  the table's form, and the parsed attributes are used.
- /api/routes/received marks the best route of each prefix and returns at
  most `limit` routes (default 1000): the table can hold hundreds of
  millions. /api/status has the totals, and the status page says how many
  it is showing. The time a route was received is no longer kept or shown.
- The event stream reports changes of best route (best_changed,
  best_withdrawn) in place of every route received. Each client follows
  the table's journal with a reader of its own, from the moment it
  connects: table::RibTable::reader_from_now.
- Metrics: bgp_rib_routes, bgp_rib_prefixes, bgp_rib_attribute_sets and
  bgp_rib_journal_entries replace bgp_adj_rib_in_routes.
- A line is no longer logged at INFO for every UPDATE sent or received; a
  full table is a million of them. They are at DEBUG.
- A table with no room for another peer ends the session with Cease / Out
  of Resources.

Four peers sending a synthetic full table each over TCP, 4.9 million
routes: 0.80 s, against 4.76 s before. One peer: 0.45 s against 0.69 s.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-02 00:31:28 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 3b20576971 Add the journal: following the table's best routes at one's own pace 
Whatever acts on the best routes (export to peers, the forwarding table,
the status page) reads their changes from a per-shard journal through a
table::Reader. Nothing uses it yet.

- The journal records that a prefix's best route changed, not what to. A
  reader that comes to an entry is given the route as it is then. A prefix
  that changes a thousand times between reads is one event, and the
  journal never holds more live entries than the table has prefixes.
- Each row keeps the sequence number of its latest change; an entry with
  another number is dead and is dropped when the journal is tidied.
- Entries go once every reader has passed them. A reader that does not
  read cannot hold them back for ever: past a limit the oldest half is
  dropped, and that reader is given the whole table again (Reset, every
  best route, Synced). A new reader starts the same way.
- With no reader nothing is recorded.
- Reader::changed waits for there to be something to read.

Tested against a mirror kept from the events: after any interleaving of
announcements, withdrawals, session flaps, reads with small budgets and
readers coming and going, with journals small enough to be tidied and
overrun constantly, a reader that has caught up holds exactly the table's
best routes. Also with writers on four threads and a reader on a fifth.
The property test found one fault (a pass that resumed part-way claimed
to have caught up) and catches three more put in on purpose.

On the AMS-IX dump, 12 threads: loading takes 2.29 s with a follower
reading throughout against 2.17 s with none, the journal peaking at
170,000 entries for 1.39 million prefixes. With a follower that does not
read, the journal ends at 1.74 million entries, and the follower then
catches up in 0.08 s.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-02 00:14:20 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 eab891567b Add the sharded routing table: Adj-RIBs-In and Loc-RIB in one 
bgp_rib::table::RibTable holds every route from every peer and the best
one for each prefix. It is not used by the server yet; this adds it with
its tests and benchmarks.

- Organised by prefix. A row holds the prefix's routes from all peers in
  one array; choosing again after a change reads that row and nothing else.
- A route is sixteen bytes: a pointer to its attribute set and one word
  holding the peer and the first three steps of selection (LOCAL_PREF, AS
  path length, ORIGIN) as a single number. That number alone settles most
  changes; a tie goes on to MED, compared only within a neighbouring AS
  and in a way that does not depend on arrival order, and the tie-breaks.
- Attribute sets are stored once, in wire form, with what selection needs
  from them worked out when they are stored.
- Sharded by hash of the prefix, a lock per shard. A session applies its
  own UPDATEs; there is no queue or owning thread in front of the table.
- Readers take one shard at a time for reading and look at rows in place.

Route selection is checked against a plain implementation with no
shortcuts by a property test over random announcements, withdrawals and
session flaps, for both the chosen route and the count of changes.

On a RIPE RIS dump from AMS-IX (148 peers, 39.3 million routes), against
the Adj-RIB-In actor, which does no selection:

                          actor       table
  memory                  11.0 GiB    1.8 GiB   (279 -> 48 bytes a route)
  load, 1 thread          10.2 s      12.3 s
  load, 12 threads        (no gain)   2.1 s     (19 million routes a second)
  read every route        1.6 s, copying 8 GiB, table stopped
                                      0.2 s, in place, table running

The benchmark crate counts memory only with --features count-memory: the
counters are shared between threads and distort the timings.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-02 00:01:50 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 c1cf15c792 Benchmark on a real table: read MRT dumps from route collectors 
crates/bench reads TABLE_DUMP_V2 dumps (RFC 6396) as published by RIPE RIS
and RouteViews, and `--bench real_table` runs the Adj-RIB-In on one. The
dumps are not checked in; crates/bench/data/README.md says how to fetch
one. Without a dump the benchmark says so and does nothing.

It also prints what in the table decides the cost of a RIB: routes per
prefix, distinct attribute sets, how many routes share a set, and how much
the sets are shared between peers.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 23:43:22 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 7e488e31aa Make IpPrefix a fixed-size value with no allocation behind it 
A prefix was a Vec of its bytes: a heap allocation for every prefix parsed,
stored or copied, and a variable-length key to hash. It is now twenty
bytes, Copy, with private fields and one representation; `bytes()` gives
the wire form.

On the full-table benchmark this alone takes decoding from 9.3 to 10.7
million routes a second and inserting into the Adj-RIB-In from 2.9 to 3.6.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 23:38:39 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 64df419b3a Add a benchmark of taking in the Internet routing table 
crates/bench makes a synthetic full table (a million IPv4 and 230,000 IPv6
prefixes, announced by several peers in UPDATEs shaped like a router's)
and times the steps of taking it in: decoding, the Adj-RIB-In, snapshot,
peer removal, and the whole path over TCP. It also counts the memory the
table holds, and fills a plain packed hash map with the same routes as a
point of comparison.

Run with `cargo bench -p bgp-bench`.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 23:30:29 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 a772322d9e Unwind panics safely: say goodbye to the peer, keep the other sessions 
A panic is a bug, and it should cost as little as it can. The server
already closed every session with a Cease and exited when a RIB or a
background task was lost. This fills in the rest:

- A panic inside a session is caught. The peer is sent Cease /
  Administrative Reset, the session's routes are removed in the usual
  order, and the dialer reconnects. Before, the peer was left with a TCP
  connection closing under it and no reason.
- The tables every session shares (the session registry and the collision
  table) no longer refuse to be locked after a panic. Lock poisoning would
  have made each remaining session panic in turn the next time it touched
  one, so one bug in one session took all of them down.
- The server's watcher reports a task that panicked with the panic's
  message, and is now tested for both a panicking task and a stopped RIB.
- The daemon sends panics to its log, with their location, instead of
  leaving them to the default report on stderr.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 23:24:00 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 41090966e4 Stop treating a dead RIB as an empty one 
The RIB handles swallowed every failure of their actor: an UPDATE applied
to a stopped Adj-RIB-In vanished, and a snapshot of it came back empty.
Sessions stayed up, peers' routes went nowhere, and the status page showed
a healthy router with no routes. An actor stops only by panicking, so this
is the state the server would be in after a bug.

- Every operation on AdjRibIn and LocalRib returns Result<_, RibStopped>.
  LocalRibError is replaced by it.
- The server waits on both actors, the advertiser and the dialers. If one
  stops it sends every peer Cease / Administrative Shutdown and returns an
  error, so the process exits and can be restarted. Before, the tasks'
  handles were held and never looked at.
- A session that cannot store an UPDATE ends with the same Cease.
- The HTTP API answers 503 with the reason for anything that needs a table
  it cannot read, including /metricz: a scrape that fails is noticed, last
  known sizes are not.

Also:
- The routes of a session are now removed even if its task is torn down
  without running its cleanup (a panic), instead of staying in the table
  as routes of a peer that is gone.
- Table sizes for the status page and metrics are asked for as a count
  instead of copying the whole table to take its length.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 23:18:59 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 6197a7ad74 Handle errors in UPDATE messages as RFC 7606 says, not by resetting 
Any malformed UPDATE used to reset the session, withdrawing every route
learned over it. One bad attribute, typically originated ASes away and
passed on by a peer that did not look at it, could take down a session
carrying a full table.

UpdateMessage::decode now applies RFC 7606 and returns the UPDATE as it is
to be acted on, plus a report of what was wrong:

- Treat-as-withdraw: a malformed or misflagged attribute, a missing
  mandatory one, or an attribute that overruns the attribute list. The
  routes the UPDATE announced (in the body and in MP_REACH_NLRI) are
  withdrawn.
- Attribute discard: malformed ATOMIC_AGGREGATE, AGGREGATOR, AS4_PATH and
  AS4_AGGREGATOR; an attribute sent twice (the first is kept); LOCAL_PREF,
  ORIGINATOR_ID and CLUSTER_LIST from an external peer; AS4_PATH between
  four-octet speakers.
- Session reset remains where the prefixes are in doubt: the lengths that
  frame the message, an unreadable prefix, a malformed or repeated
  MP_REACH_NLRI or MP_UNREACH_NLRI, an unrecognised well-known attribute.
  Each now sends its own subcode instead of always Malformed Attribute
  List.

The session logs each handled error and counts it in
bgp_update_errors_total{handling}. The parser context says whether the
peer is internal (replacing a field nothing read).

Stricter than before, because the checks did not exist:
- An announcement must carry ORIGIN and AS_PATH, NEXT_HOP for routes in the
  body, and LOCAL_PREF from an internal peer.
- An AS_PATH segment must be an AS_SET or AS_SEQUENCE and not empty.
- An attribute value must be used up entirely by its parser; a NEXT_HOP of
  six octets, say, was accepted before.

Bugs found on the way: we sent ATOMIC_AGGREGATE flagged optional and
AGGREGATOR flagged non-transitive. The flags now come from one table used
for both writing and checking.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 23:12:33 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 39632e3c38 Property tests and fuzz targets for the parsers, and the bugs they found 
Property tests (run by `cargo test`):
- crates/packet/tests/properties.rs: every kind of message, generated,
  must be written, be as long as wire_len says, and read back equal; and
  noise, noise behind a valid header, and valid messages with bytes damaged
  must never do more than return an error. Anything the decoder accepts
  must be showable and survive being written and read again.
- IpPrefix: text round trip, noise as text, `contains` against a bitwise
  model.
- The session's framing: a stream of messages yields the same messages
  however it is cut into reads.

Fuzz targets (fuzz/, cargo-fuzz on nightly, a minute each in CI): raw
messages, message bodies behind a valid header, prefix text, the
configuration file.

Bugs found, each with a named regression test:
- A prefix longer than its address family (e.g. an IPv4 NLRI of length 99)
  was accepted, and showing it panicked. The session logs every route, so
  one UPDATE from a peer ended the session task. Such a prefix is now
  malformed, and showing a prefix cannot fail.
- Bits past a prefix's length were kept, so the same prefix sent with stray
  bits was a different key in the RIB. Prefixes now have one form.
- An opaque extended community was read as seven octets instead of eight,
  putting every community after it out of step.
- AGGREGATOR could be neither read from nor written to a peer with
  two-octet AS numbers.
- The Outbound Route Filter and Long-Lived Graceful Restart capabilities
  were written differently from how they are read.
- A capability value or unknown attribute too long for its length field
  was written with the length wrapped around; it is now an error, or for
  the attribute the two-octet length.

Also: ROUTE-REFRESH carries its AFI and SAFI instead of being modelled as
an empty message, and one with a body of the wrong length is refused.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 22:54:37 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 b0762874e0 Make address families open values; keep "supported" a separate list 
AFI and SAFI were closed enums, so a family without a variant could not be
represented at all, and the set of negotiated families was a struct with
one bool per family. Supporting another family meant touching both.

- `Afi`, `Safi` and `AfiSafi` in bgp-primitives are newtypes over the wire
  numbers, with constants and names for the common families. Capabilities
  and MP_REACH/MP_UNREACH use them, so any family a peer names is read as
  what it is. The special case that turned a capability for an unlisted
  family into an unknown capability is gone.
- `FamilySet` is a set of `AfiSafi`. What we can exchange routes for is the
  explicit `family::SUPPORTED` list; only those are offered in our OPEN.
- `AddressFamilyId` stays as the IP version of a prefix.
- NLRI is still a list of IP prefixes. MP_REACH/MP_UNREACH for a family
  whose NLRI is something else (labelled, VPN, EVPN, ...) is refused
  rather than misread as prefixes, which is what happened before for the
  SAFIs the old enum listed.

Bug found on the way: the reserved octet in MP_REACH_NLRI was skipped only
for IPv6, so an IPv4 MP_REACH_NLRI was parsed with an extra 0.0.0.0/0
route that the peer never sent.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 22:38:40 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 22d0099a46 Negotiate address families and AS number width instead of assuming them 
The OPEN used to offer IPv4 and IPv6 unicast unconditionally, and what the
peer offered was ignored apart from its four-octet AS number.

Address families (RFC 4760):
- A peer's `address_families` in the configuration says which to offer it;
  all of them by default.
- A session carries the families both OPENs offered. A peer that sends no
  Multiprotocol capability is a plain RFC 4271 speaker: IPv4 unicast.
- Routes of a family that was not negotiated are not sent, and are ignored
  when received; the rest of the UPDATE is applied and the session stays up.
- A session with no family in common is refused with Unsupported
  Capability, naming the capabilities we needed (RFC 5492).
- The negotiated families are shown on /statusz and in /api/peers.

Bugs found on the way:
- A capability naming an address family we have no constant for (EVPN,
  VPNv4, ...) failed the whole OPEN, so such a peer could never establish.
  It is now kept as an unknown capability and ignored.
- An UPDATE could not be encoded for a peer without four-octet AS support,
  which ended the session on the first route. AS_PATH is now written two
  octets wide with AS_TRANS where needed, and AS4_PATH carries the real
  numbers (RFC 6793).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 22:22:54 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 a4a180e948 Make the session state machine explicit and free of I/O 
A session was one function that walked OpenSent, OpenConfirm and
Established with the socket, the timers, collision detection, the registry
and the RIB all in the same loop. There was no state or event type, and no
way to test a transition without a TCP connection.

It is now two pieces:

- fsm.rs is the state machine of RFC 4271 section 8 for a connection. It
  takes an Event and returns the Actions to carry out. It owns no socket,
  timer or task. What the OPENs negotiated lives in the states that have
  it, so there is nothing to unwrap. Thirteen tests cover the handshake,
  hold-time negotiation, refused OPENs, messages out of turn, timer expiry,
  collisions and shutdown by feeding it events.
- connection.rs is the driver: it turns the socket, the timers and the
  outbound queue into events and carries the actions out.

Behaviour changes:

- A message that arrived in pieces was lost if a timer fired between the
  pieces: read_exact was used inside select!, which discards what it has
  read when another branch wins. Messages are now framed from a buffer
  that survives cancellation.
- A NOTIFICATION from the peer during the handshake is no longer answered
  with an FSM error; RFC 4271 says to close without replying.
- A session the peer ended is reported as 'peer sent NOTIFICATION: ...'
  with its error, where the dialer used to record 'session closed by peer'.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 21:59:55 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 16a4e3b640 Typed errors in the library crates; send the NOTIFICATION RFC 4271 assigns 
The library crates returned eyre reports, so a failure was a string and the
session code could not tell a truncated UPDATE from a bad header. Whatever
went wrong, the peer got an FSM error or nothing.

Each crate now has its own error types:

- bgp-primitives: PrefixError, UnknownAfi and UnknownSafi, and EncodeError
  for every to_wire.
- bgp-packet: Message::decode returns a DecodeError naming the NOTIFICATION
  code, subcode and data for the failure (header errors per section 6.1,
  OPEN per 6.2, UPDATE per 6.3). InvalidValue and CommunityError replace
  the remaining strings. from_wire is now a thin wrapper around decode.
- bgp-server: SessionError, ServerError and ConfigError. Every abnormal end
  of a session goes through one function that sends the NOTIFICATION the
  error maps to. The message length is checked against the 4096-octet
  maximum before the body is read.
- bgp-web: serve returns io::Error.

eyre remains in the binary and in tests.

Bugs found and fixed on the way:

- An OPEN with no optional parameters, which is valid, failed to parse.
- An unsupported BGP version was reported as an unknown optional parameter.
  It now gets Unsupported Version Number with the version we speak as data.
- A SAFI above 255 was truncated to its low octet and could match a known
  one.
- Prefix text with a length beyond the address size (10.0.0.0/99) was
  accepted.
- Message::to_wire truncated the body length to 16 bits before checking it.
- A peer hold time above 65535 seconds in the configuration wrapped.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 21:18:35 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 666471e38d Share path attributes between Adj-RIB-In entries 
Each received prefix kept its own copy of the UPDATE's whole attribute list,
and for multiprotocol routes that list included MP_REACH_NLRI with every
other prefix from the same UPDATE: n prefixes stored n copies of n prefixes.

An UPDATE is now turned into a ReceivedUpdate once. Its prefixes share one
attribute list, which leaves out MP_REACH_NLRI and MP_UNREACH_NLRI; the next
hop from MP_REACH_NLRI is kept on the entry. The RIB takes one command per
UPDATE instead of one per prefix, and its table is keyed by peer so that
dropping a peer's routes no longer scans every route.

The status API shows the multiprotocol next hop as an address instead of
its Debug form.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 21:08:14 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 2de0a14712 Add the Apache 2.0 licence text and CI 
Cargo.toml has declared Apache-2.0 without the licence being in the tree.

CI checks formatting, clippy and the tests on every push, with warnings as
errors. The workflow lives in .github/workflows, which Gitea Actions also
reads.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 21:01:26 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 b7c7514d3e Fix all clippy warnings and enforce lints across the workspace 
Lints are declared once under [workspace.lints] and every crate opts in:
clippy's default set, plus dbg!, todo! and unimplemented!, and unsafe code
is forbidden. Every crate now inherits its version, licence, authors and
repository from the workspace, so they no longer disagree (four were at
0.0.0).

The poptrie module is unfinished and unused; it says so and expects the
dead-code lint instead of warning on every build. Its construction method
is renamed from new, which took &mut self, to build.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 21:01:26 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 126368852a Parse the Multiple Labels capability instead of panicking 
The capability parser had a todo!() for capability code 8 (RFC 8277), and
the encoder two more. A peer advertising it took the session task down.
It is now parsed into its (AFI, SAFI, count) entries and round-trips.

Also carries clippy's mechanical fixes for this file.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 21:01:26 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 2ab6c1c38b Format the workspace with rustfmt 
Default settings. No change in behaviour; the tree was last formatted
before the lecture work and had drifted to a mix of line widths.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 20:58:12 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 d8e8924b4e Point the repository URL at ferrix 
The repository was renamed. Also drops workspace.package.name, which Cargo
ignores (a package name cannot be inherited) and warned about on every build.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 20:53:26 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 b7e5084f83 Tidy the remaining top-level files after the lecture 
The preview launcher pointed at the removed slides and editor state does
not belong in the tree. The Dockerfile and README no longer refer to the
lab.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 20:51:32 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 22b9eeacdc Count multiprotocol prefixes in the UPDATE log lines 
The 'Sending UPDATE' and 'Received UPDATE' lines counted only the IPv4
fields of the message body, so every IPv6 UPDATE logged announced=0.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 20:51:15 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 b800e3a322 Originate routes from the configuration; make the HTTP API read-only 
Routes were announced and withdrawn through an HTTP form, each tagged with a
free-text owner and checked against an allow-list and a per-owner limit.
That was the lab's interface, not a router's.

Routes now come from 'originate' entries in the configuration file and are
in the local RIB before any session starts. The local RIB holds one route
per prefix with no notion of owners or policy. local_route_policy is gone
from the configuration.

The HTTP API keeps its GET endpoints (status, config, peers, originated and
received routes, events, metrics). The announce, withdraw, wire-bytes and
well-known-community endpoints are removed, and the status page loses the
announce form, the community presets and the byte decoder. Community text
parsing moves from the web crate to bgp-packet so the configuration can use
it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 20:51:15 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 c37cfd029a Remove the lecture material and the ping monitor 
The slides, handout, lab router configs, RIPE objects and load test were
for the Network Security 2026 lecture and are kept on the netsec-2026 tag.

The ping monitor sniffed ICMPv6 echo requests so students could watch their
prefix being used. It has no place in a BGP speaker: it needs CAP_NET_RAW,
adds the only unsafe code in the tree and a libc dependency.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 20:45:24 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 b1b72fed39 lab: multi-line answer boxes, one sheet per student 
The fill-in fields put the label on its own line with dotted lines to write
on. handout.tex takes \studentnum and \studenthex; build-class.py compiles
one copy per student with the prefix filled in, checks each is two pages
and joins them for double-sided printing.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 20:40:55 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 2910a2a800 slides: light theme, final edits, single-file bundle 
As presented. The palette is light on request of the reviewer; code blocks
and byte dumps keep a dark ground so the byte colours still work. The OPEN
walk lights the hovered bytes as an inverted block and its table cells are
dark text on a tint of their group colour.

build.py inlines reveal.js, the stylesheets, scripts and Markdown sections
into slides/dist/hands-on-bgp.html for uploading.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 20:40:55 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 8c6e8a80e1 chore: ignore .DS_Store and generated lecture output 
slides/dist/ (the single-file bundle) and docs/lab/handout/class.pdf (one
sheet per student) are build products.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-10-01 20:40:55 +02:00
Rayhaan Jaufeerally 597941a70a handout finalized 2026-09-24 10:33:45 +02:00
Rayhaan Jaufeerally b607058650 Ferrix web UI working for demo and slides mostly done 2026-09-23 15:38:06 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 f3bb8cc89e lab: exercise sheet cut to two pages, three simple steps 
One double-sided A4 sheet with 1in margins, still in the LNCS look.
1. Who announces the address you are using: find your current address and
   its prefix and origin AS, then whois it and your lab prefix.
2. Announce your prefix, read the bytes, find it from outside, ping it.
3. Announce with communities: only the SwissIX route servers, then only
   SWITCH, with reachability checkboxes, one discussion question, clean up.
Sections 1 and 2 fill page one; section 3 has page two. Dropped the ETH
lookup, the long registry comparison, NO_EXPORT, the byte-decoding task,
the reference appendix and most of the bibliography.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-20 22:07:23 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 83ae367b1c lab: printable exercise sheet for the Hands-on BGP lecture 
docs/lab/handout/handout.tex, typeset with Springer's LNCS class (title
block, abstract and keywords, figure, booktabs tables, RFC references), six
A4 pages so it prints double-sided on three sheets. The built PDF is
committed alongside for printing.

Three sections students work through with the slides:
1. Who holds which addresses: whois their own /48 and ethz.ch's prefix in
   the RIPE database, then compare with what is actually routed.
2. Announcing the prefix on the lab page, reading the UPDATE bytes, finding
   it from outside, and reaching it.
3. Steering with communities until the prefix is reachable only from the
   university network (AS559 via the SwissIX route servers), with
   reachability checkboxes for laptop and phone at each step.
Plus a one-page reference appendix: IPv6 notation and hex, the UPDATE
layout, path attributes, glossary.

The lab host, the lab address space and the prefix pattern are macros at
the top of the file. Expected answers were checked against the RIPE
database and RIPEstat (ETH: 2001:67c:10ec::/48, ASSIGNED PI, origin AS559;
lab: 2a0d:d741::/40, AGGREGATED-BY-LIR, route6 origin AS210036).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-20 22:02:32 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 9d6996c6c1 slides: no step-by-step reveals; UPDATE example is 2a0d:d740::/48 at SwissIX 
- Removed every fragment and stepped code highlight: a slide now shows
  everything at once. Two code blocks keep a static highlight on one line.
- Dropped "What stands between you and an outage".
- The UPDATE walked through in section 3 is now 2a0d:d740::/48 as AS210036
  announces it to the SwissIX route servers: next hop 2001:7f8:24::c9
  (pr01's port on the peering LAN), AS path 210036, MED 0 and 210036:10:2,
  matching what pr01's export filter puts on routes to peers. Encoded with
  the packet crate and parsed back to the identical message; the slide
  generator asserted every quoted number, including the next hop, against
  the bytes. Adds a MULTI_EXIT_DISC slide to the walkthrough.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-20 21:48:12 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 1399cbd39f slides: "Hands-on BGP", restructured into five sections 
- Title is now "Hands-on BGP"; the AS-path diagram is gone from the cover.
- Sections: IPv6 addresses and prefixes; where AS numbers and prefixes
  come from (IANA, RIRs, the RIPE database, IRR vs RPKI); BGP and the
  bytes of an UPDATE; IP transit vs Internet exchanges (the peering LAN as
  an Ethernet segment, route servers); the lab (ferrix, pr01.rue, SwissIX,
  NL-ix, France-IX, DE-CIX, transit).
- The UPDATE walked through in section 3 is a real 94-byte capture from
  ferrix for 2a0d:d741:1::/48. Vertical slides light up one part at a
  time with its decode; the generator asserted every quoted number
  against the capture. Session counts come from the tracked bird.conf,
  the route6 object from the RPSL file, SwissIX's LAN from PeeringDB.
- New components: focusable bytes, decode table, prefix bits, and SVG
  diagrams drawn in the theme's colours.
- Opening index.html straight from disk used to give a title and empty
  panels, because browsers block the module and the section files there.
  The page now explains that, and also reports a failed reveal.js load.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-20 21:41:31 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 106edd1df0 slides: reveal.js deck template for the BGP lecture 
Structured like Matt Godbolt's newer talks (hrt-theme.css in
what-happens-beneath-accu-2026): reveal.js with no stock theme but a custom
dark one, every slide on a rounded panel, IBM Plex Sans and Mono, monokai
code with glowing stepped lines, 16:9 with fade transitions, and the
content in one Markdown file per section.

Made our own: the palette is a handful of variables (with ?accent= presets
for comparing), and his ring-and-thread corner motif became a chain of
nodes and links, an AS path, which also opens the title slide. A wordmark
sits where his decks carry the employer logo.

Also: a small `bird` syntax highlighter for router config on slides, a
bytes component in the four groups of the /statusz bytes view, columns,
boxes, and a `bare` slide class. reveal.js 6 loads from a vendored copy
when slides/vendor-reveal.sh has been run (works offline, ignored by git)
and from the CDN otherwise. Section files carry a starter outline for the
lab that doubles as a tour of the slide types.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-20 21:30:41 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 e35507d082 lab: track the applied router config and the RIPE objects 
- docs/lab/prod/pr01/bird.conf is now the config generated for per-session
  export control (every eBGP session passes its neighbour AS and location
  to the shared export function; customer sessions go through the same
  gate), with the ferrix stanza in its same-host form (::1, multihop).
- docs/lab/ripe/lab-objects.rpsl: the inet6num and the 128 route6 objects
  for the student /48s. Public registry data, no credentials.
- Ignore Python bytecode caches.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-20 21:21:25 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 c60bd8baf5 docs: use built-in Apache error documents so the login covers every path 
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-20 21:13:56 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 36b73844af docs: keep Apache's /error/ pages out of the proxy so 401 prompts for a password 
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-20 21:09:37 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 9d644a212a tools: concurrent load test for the announce API 
tools/loadtest.py releases N simulated students through a barrier so they
announce at the same instant, then verifies behaviour rather than only
status codes: the table holds every route under the right owner, each
route became an UPDATE per established session, the event stream lost no
event, concurrent takeover attempts are all refused, withdrawal leaves
nothing behind, and the daemon's own counters show the matching number of
UPDATEs sent for announcements and withdrawals. Standard library only;
supports basic auth and repeated rounds.

Safe by default against a production router: every route carries
210036:0:0 (announce nowhere), leftovers from earlier runs are removed,
prefixes held by real owners are skipped, and everything is withdrawn even
if the run fails.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-20 19:49:19 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 1ea2a12bd6 docs: tested Apache reverse proxy with basic auth for the status page 
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-20 19:32:50 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 42d79354e3 docs: peer with BIRD over ::1 on the router; explain the multihop requirement 
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-20 19:29:31 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 fd496d26bf feature: run on the router itself: protoc-free build, per-peer source address 
Preparing to run ferrix next to BIRD on the production router.

- build: the config proto is now parsed in pure Rust, so building needs no
  `protoc` binary. It previously worked only where one happened to be
  installed; the Dockerfile in this repo could not have built.
- peers gain `local_address`, the source IP for connections we open (and
  therefore the next hop we advertise). When both speakers share a host
  the OS would otherwise pick an arbitrary local address and the peer,
  which identifies us by address, would refuse the session. Address-family
  mismatches are rejected at config load; an address the host lacks shows
  up as the peer's backoff reason.
- docs/lab/prod/on-router: static musl cross-build instructions, config,
  hardened systemd unit (unprivileged, SIGTERM gives BIRD a Cease),
  loopback addresses for networkd, the BIRD stanza for a same-host
  multihop session, and reverse-proxy examples with TLS and a password,
  since the API has no authentication.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-20 18:15:09 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 af76ecb345 fix: leave sessions politely on shutdown; answer ROUTE-REFRESH 
Against a production BIRD router, restarting ferrix was followed by
several refused connection attempts before the session came back.

- The daemon had no signal handling: Ctrl-C killed the process and the
  peer only ever saw a dead TCP connection. On SIGINT or SIGTERM it now
  sends Cease / Administrative Shutdown (RFC 4486) to every established
  peer, waits up to two seconds for the sessions to close, and stops the
  dialers from reconnecting meanwhile. A NOTIFICATION queued for a session
  ends that session once sent.
- ROUTE-REFRESH was parsed and ignored. A peer sends it when its import
  policy changes (e.g. after `birdc configure`), expecting our routes
  again so the new policy can be applied. The session now reports the
  request and the advertiser re-sends every local route to that peer. The
  request also appears in the event stream and the status page log.

Tests: a refresh request in real wire format re-sends all routes and keeps
the session up; shutdown delivers the Cease to every peer and closes the
sessions. Verified with the real binary: SIGINT, peer logs the Cease.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-20 17:51:04 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 57099da73d feature: export control by neighbour AS and by location 
The production router has the same neighbour AS at several exchanges, so
per-AS control alone cannot express "only at SwissIX". Every session now
has two coordinates, neighbour AS and location, and the scheme gains two
functions:
  210036:0:<as> / 210036:2:<location>   never announce      (wins)
  210036:1:<as> / 210036:3:<location>   announce
  210036:0:0                            announce nowhere
  nothing                               announce everywhere
docs/lab/prod/bird-ferrix.conf documents the functions, the per-session
call (export where peers_out(<as>, <location>)) and the gate that sessions
exporting everything need as well.

Status page: location presets, and AS210036 communities are decoded from
their structure so any value reads well in the bytes hover.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-20 17:34:57 +02:00
Rayhaan Jaufeerally 98f25a5847 Add existing BIRD config 2026-09-20 17:29:01 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 77c406e7d1 docs: wire lab export control into a shared BIRD filter via a function and wrapper filters 
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-20 17:21:36 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 b28f59f30d feature: AS210036 export-control large communities for lab routes 
BIRD side (docs/lab/prod/bird-ferrix.conf): the ferrix import filter now
admits 210036:0:* and 210036:1:*, and lab_export_allowed(peer_as) decides
per eBGP session whether a lab route is announced, with the same shape and
precedence as the SwissIX route-server communities:
  1. 210036:0:<peer-as>  never   2. 210036:1:<peer-as>  announce
  3. 210036:0:0          nobody  4. default             everyone
The per-session export block applies only to routes learned from ferrix,
strips the private ASN and removes the action communities before sending.

Status page: the large-community picker is grouped into "AS210036" and
"SwissIX route servers" presets, including "announce to the SwissIX route
servers" and "send to SWITCH (AS559)"; the bytes hover names both families.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-20 17:02:42 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 eb77090d01 feature: config zippy on statusz; drop prepend options from the UI 
- The status page has a collapsible "Configuration" section showing the
  config file as read at startup, with its path. Served by the new
  GET /api/config; RouterInfo::with_config carries the text.
- Removed the AS-path prepends field and the SwissIX prepend presets
  (42476:101..103) from the form to keep the exercise focused. The API
  still accepts `prepend`.
- Docs follow: the handout loses its prepending exercise, the lab README
  its prepend row, and the BIRD config notes that the 42476 namespace is
  still admitted for hand-typed values.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-19 22:46:07 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 3f04152f19 docs: simplify prod ferrix config, name the peer pr01.rue.rayhaan.net 
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-19 22:44:46 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 dd838eec9a feature: decoded hover for UPDATE bytes; opt-in auto-refresh 
- Hovering (or focusing) a byte segment opens a popup that decodes it:
  header fields, the two length fields, and each path attribute with its
  flags spelled out and its value in readable form. AS_PATH shows segment
  type, ASNs, path length and ASN width; COMMUNITIES shows well-known
  names; LARGE_COMMUNITIES shows the SwissIX meaning of known values;
  MP_REACH/UNREACH show address family, next hop and prefixes; NLRI and
  withdrawn routes list prefixes. Decoding happens on hover from the bytes
  themselves. /api/routes/local/wire gains four_octet_asn so the AS_PATH
  width is known.
- The page is now a snapshot by default. ?refresh=<seconds> (1..3600)
  turns on reloading of all tables at that interval and connects the
  event log; the header offers 2s / 5s / 30s, "turn off" and "reload now".
  Events only feed the log, so the tables never change faster than asked.
  The reader's own announce / withdraw still shows its effect at once.
- Removed the "tables refresh on every event" footer text.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-19 22:20:37 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 20cad53b24 fix: statusz tables no longer flicker on refresh 
Every poll and event rebuilt each table from scratch, and an open bytes
view collapsed to "loading..." and was refetched every five seconds.

- Tables are rewritten only when their markup changes. Clock-dependent
  text (ages, retry countdowns, uptime) is rendered as placeholders and
  ticked in place once a second, so unchanged data means no DOM writes.
- The bytes view is cached per route and set of established sessions, and
  the previous bytes stay visible while new ones load.

Measured with a MutationObserver: zero table rebuilds across 13 s of
polling with a bytes view open; one announcement redraws only the local
and received tables once each, with no "loading..." flash.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-19 17:10:42 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 cc022c3b7b feature: restyle statusz after the classic Google status pages 
Modelled on Vitess's servenv status page and golang.org/x/net/trace, both
written by Google engineers to mirror the internal pages: plain sans-serif
HTML, default links and native form controls, dense bordered tables, and a
full-width centred light blue (#eef) heading bar per section. Header shows
"Status for ferrix_bgp" with start time, uptime, version, router ID and AS
on the left and host, "View:" links and live-update state on the right.
The dark theme is dropped on purpose.

Behaviour is unchanged (live updates, owner-gated withdraw, per-session
UPDATE bytes, looped-route highlight, peers polling). Additions:
- a SwissIX large-community picker next to the field; presets with a peer
  AS ask for the number
- event log lines include prepend and MED
- /api/status gains hostname and started_at_unix (RouterInfo::new)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-19 17:06:00 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 d23a578d72 docs: listen on 9179 in the prod ferrix config (179 needs root on macOS) 
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-19 16:08:22 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 22384d0b07 docs: use the real WireGuard tunnel addresses in the prod peering configs 
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-19 16:05:44 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 e41776f253 docs: allow SwissIX large action communities through the ferrix import filter 
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-19 16:00:42 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 160e28690a fix: capability list parsing; advertise IPv6 unicast; BIRD peering config 
Preparing to peer with a production BIRD router turned up two problems.

- packet: most Capability::from_wire arms returned the per-capability
  buffer (always empty by then) instead of the bytes following the
  capability. The list parser then saw an empty remainder and silently
  dropped every later capability. Any OPEN listing extended message, role,
  graceful restart, add-path, enhanced route refresh, LLGR, extended next
  hop or an unknown capability before the 4-octet ASN capability lost the
  ASN, so the peer appeared as AS_TRANS (23456) and a configured
  remote_asn was refused with Bad Peer AS. BIRD and FRR both do this.
  Single-capability round-trip tests could not catch it; new tests parse
  every capability with another following it, and a full BIRD capability
  list.
- server: advertise MultiProtocol IPv6 unicast alongside IPv4. Without it
  a peer will not accept the IPv6 routes we originate via MP_REACH_NLRI.
- Interop test: byte-for-byte BIRD 2 OPEN is accepted with the right ASN,
  and an IPv6 End-of-RIB neither drops the session nor creates a route.
- docs/lab/prod: BIRD config for an announce-only ferrix neighbour
  (strict /48 import filter, community scrubbing, unreachable next hop,
  import limit, export none, passive) and the matching ferrix config.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-19 15:57:23 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 87f96f9389 feature: RFC 4271 §6.8 connection collision detection 
When both ends dial each other, exactly one TCP connection must survive
and both ends must pick the same one. Previously ferrix relied on the
other router to sort this out and could briefly hold two sessions.

- collision: a shared map keyed by the peer's BGP Identifier holds the one
  connection currently preferred for that peer plus a kill signal. Each
  connection calls admit() on receiving the peer's OPEN. The connection
  initiated by the higher BGP Identifier wins; the loser gets Cease /
  Connection Collision Resolution, whether it is the new connection or an
  existing one in OpenConfirm running in another task.
- A collision with an Established session closes the new connection (RFC
  default). The optional CollisionDetectEstablishedState is available as
  collision_detect_established_state: a restarted peer's new connection
  then replaces the stale session at once instead of being refused until
  the hold timer expires. Enabled in the lab config.
- Two connections opened by the same side: the newer wins.
- OPEN validation rejects a peer using our own BGP Identifier or 0.0.0.0
  with Bad BGP Identifier; collision resolution needs distinct identifiers.
- OpenConfirm no longer times out instantly when the negotiated hold time
  is zero.
- Sessions record which side opened them; shown as "opened by" on the
  status page and as `direction` in /api/peers. New counter
  bgp_connection_collisions_total{closed}.
- Tests: unit tests for the rule (including that both ends agree), and
  integration tests for simultaneous open in both identifier orders and
  both arrival orders, Established default, restart replacement with route
  flush, distinct identifiers, slot release, and Bad BGP Identifier.
  Existing fan-out test now gives its two peers distinct identifiers.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-19 15:05:13 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 a137160b48 feature: lab topology, student handout; drop peer routes on session end 
Phase 5 of the teaching demo plus two fixes found while exercising the
dialer on the status page.

- Adj-RIB-In: routes learned from a peer are removed when its session ends
  (RFC 4271 §8.2.2), emitting a Withdrawn event per prefix before the
  session Down event. Previously stale routes from dead sessions stayed in
  the received table.
- statusz: poll the peers table every 5 s so dial state (Connecting /
  Backoff and its retry countdown) stays current; those changes carry no
  event.
- docs/lab: docker-compose topology with ferrix (AS65000) and two FRR
  routers. r2 accepts nothing directly from ferrix so student prefixes
  only reach it via r1, then come back with AS path 65002 65001 65000. r1
  applies community-driven policy: NO_EXPORT stops forwarding, 65001:100
  prepends, BLACKHOLE drops. Dockerfile for the daemon.
- docs/lab/HANDOUT.md: student exercises walking through announcement,
  reading the bytes, prepending, NO_EXPORT, action communities,
  blackholing and withdrawal.

The container files have not been executed (no runtime available here);
docs/lab/README.md lists what is most likely to need adjusting.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-16 23:26:10 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 e2de44ba78 feature: active dialer for configured peers 
Phase 4 of the teaching demo: the daemon now brings up its own sessions.

- New dialer task per configured peer: connect with a timeout, run the
  same FSM as the passive side, back off exponentially (from
  connect_retry_seconds, capped at two minutes) and retry forever. Stays
  idle while a session with the same remote IP and ASN already exists.
- The peer's OPEN must carry the configured ASN, otherwise the session is
  refused with a Bad Peer AS notification. Inbound connections from a
  configured peer's address get the same check plus the configured name
  and hold time.
- SessionRegistry tracks configured peers and their dial state
  (Idle / Connecting / Backoff with last error and retry time).
- /api/peers merges configured peers with live sessions so the page shows
  peers that are down, why, and when the next attempt is; unconfigured
  inbound sessions are listed after them.
- fake_peer gains --listen so it can stand in for a router the daemon
  dials.
- Integration tests: dial and establish, reconnect after the peer closes,
  wrong ASN refused, inbound from a configured address checked and named.

Not implemented: RFC 4271 §6.8 collision detection; a real router on the
other end resolves the rare race by closing one connection.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-16 23:22:08 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 fb48fc0a85 feature: statusz page and fake_peer example 
Phase 3 of the teaching demo.

- crates/web serves an embedded, dependency-free status page at /statusz
  (/ redirects there). Sections: router header with live indicator, peers,
  announce form (owner remembered locally, well-known community picker,
  prepend / MED / large communities), local routes with per-session UPDATE
  bytes segmented into header / lengths / attributes / NLRI, received
  routes with rows highlighted when a locally announced prefix has come
  back, and a live event log. Tables refresh on every SSE event. Light
  and dark themes.
- fake_peer example in crates/integration-tests: connects, announces,
  prints received UPDATEs and with --echo re-announces everything back
  with its ASN prepended (dropping NO_EXPORT), so one daemon plus one
  process demonstrates the full loop without lab routers.
- README for the binary documents the endpoints and the demo setup.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-16 23:17:27 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 c3f7713965 feature: HTTP API, event stream and Prometheus metrics (bgp-web) 
Phase 2 of the teaching demo: everything the status page will need is now
reachable over HTTP.

- New crate crates/web (axum). JSON endpoints for status, peers, local
  routes (announce / withdraw), received routes, the exact UPDATE bytes a
  local route produces per session, and well-known community names.
  Communities are accepted as ASN:VALUE or by name (NO_EXPORT, BLACKHOLE,
  ...); large communities as A:B:C. Policy errors map to 403/409/422.
- GET /api/events is a server-sent event stream merging session up/down,
  local route and Adj-RIB-In changes, with lag notifications.
- GET /metricz serves Prometheus metrics; the server crate now counts
  messages sent/received by type and sessions established.
- The runnable main moves from crates/server to bin/bgp so it can depend
  on both server and web without a cycle. `cargo run -- config.textpb`.
- Config gains http_endpoint; unset disables the API.
- Integration tests drive the API against a live server with a real BGP
  peer, including verifying the wire view matches the bytes the peer got.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-16 23:10:16 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 73e9fbf0ad fix: sample config used bracketed IPv6 address which the endpoint parser rejects 
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-16 22:57:02 +02:00
Rayhaan JaufeerallyandClaude Fable 5.1 ad07233196 feature: locally originated routes, session registry, outbound UPDATEs 
Phase 1 of the BGP teaching demo: the daemon can now originate routes and
advertise them to every established peer.

- rib: add LocalRib actor holding locally originated routes with owner,
  origin, AS-path prepend, MED, standard and large communities. Enforces a
  LocalRibPolicy (prefix allowlist, per-owner limit, owner match on
  replace/withdraw) and emits LocalRibEvents.
- server: add SessionRegistry with per-session outbound queues; sessions
  register on Established and deregister via a drop guard. Emits
  SessionEvent::{Up,Down}.
- server: advertiser task fans LocalRib changes out to all sessions and
  replays the table to new sessions. UPDATE builders handle IPv4 (legacy
  NLRI) and IPv6 (MP_REACH/MP_UNREACH).
- fsm: merge the timer and timer-less established loops into one select!
  with an outbound branch; reset the hold timer on any received message
  per RFC 4271 §8.2.2 (previously only on KEEPALIVE).
- config: new local_route_policy proto message.
- packet: fix Communitites typo, expose CommunitiesPathAttribute inner vec.
- primitives: IpPrefix::contains.
- integration tests: announce/withdraw/replay/fan-out/policy coverage.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-16 22:56:29 +02:00
202 changed files with 3490 additions and 45907 deletions
Expand all files Collapse all files

No files matched your search

.dockerignore
-5
View File
@@ -1,7 +1,2 @@
target
.git
.claude
fuzz/target
fuzz/corpus
fuzz/artifacts
crates/bench/data
.github/workflows/ci.yml
+3 -52
View File
@@ -23,58 +23,9 @@ jobs:
- name: Format
run: cargo fmt --all -- --check
- name: Lint
run: cargo clippy --locked --workspace --all-targets -- -D warnings
# The memory-counting allocator of the benchmarks is behind a feature
# that the workspace build does not turn on.
- name: Lint the benchmarks with memory counting
run: cargo clippy --locked -p bgp-bench --all-targets --features count-memory -- -D warnings
run: cargo clippy --workspace --all-targets -- -D warnings
- name: Test
run: cargo test --locked --workspace
# A link in the documentation to something that is not there.
- name: Documentation
env:
RUSTDOCFLAGS: -D warnings
run: cargo doc --locked --workspace --no-deps --document-private-items
# The examples operators start from, here and in the documentation,
# through the daemon itself.
- name: Example configurations
run: |
for config in testdata/server.textpb docs/examples/*.textpb; do
cargo run --locked -p bgp -- --check "$config"
done
# The tests that change a kernel's routes, in a network namespace of
# their own; and two daemons that route for each other.
- name: Kernel tests
run: scripts/kernel-tests.sh
# The documentation site. Zola fails the build on a link to a page or an
# anchor that is not there.
docs:
runs-on: ubuntu-latest
env:
ZOLA_VERSION: "0.22.1"
steps:
- uses: actions/checkout@v4
- name: Install Zola
run: |
curl -sSfL "https://github.com/getzola/zola/releases/download/v${ZOLA_VERSION}/zola-v${ZOLA_VERSION}-x86_64-unknown-linux-gnu.tar.gz" \
| sudo tar -xz -C /usr/local/bin zola
- name: Build
run: zola --root docs build
# The daemon against FRR, BIRD and Routinator; see
# docs/content/docs/development/testing.md.
interop:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- name: Install FRR, BIRD and Routinator
run: scripts/interop/install.sh
- name: Build
run: cargo build --release --locked -p bgp
- name: Interoperability lab
run: scripts/interop/lab.sh
run: cargo test --workspace
# A short run of each fuzz target: enough to notice a parser change that
# breaks on input the seeds are close to. Long runs are done by hand; see
@@ -93,7 +44,7 @@ jobs:
env:
RUSTFLAGS: ""
run: |
for target in message message_body attrs prefix_text config rtr netlink; do
for target in message message_body prefix_text config; do
mkdir -p "corpus/$target"
cargo fuzz run "$target" "corpus/$target" "seeds/$target" -- -max_total_time=60
done
.gitignore
+1 -2
View File
@@ -1,5 +1,4 @@
/target
__pycache__/
.DS_Store
.claude/
/docs/public
/docs/static/giallo.css
Cargo.lock
Generated
+13 -151
View File
@@ -157,22 +157,20 @@ checksum = "ac07cdecf99051d9a5238b80f35af32cdeba5b336e55d957b318b50137e18da5"
[[package]]
name = "bgp"
version = "0.2.6"
version = "0.0.2"
dependencies = [
"bgp-server",
"bgp-web",
"clap",
"eyre",
"protobuf",
"tikv-jemallocator",
"tokio",
"tracing",
"tracing-subscriber",
]
[[package]]
name = "bgp-bench"
version = "0.2.6"
version = "0.0.2"
dependencies = [
"bgp-integration-tests",
"bgp-packet",
@@ -182,38 +180,18 @@ dependencies = [
"bytes",
"flate2",
"stats_alloc",
"tikv-jemallocator",
"tokio",
"tracing",
"tracing-subscriber",
]
[[package]]
name = "bgp-fib"
version = "0.2.6"
dependencies = [
"bgp-packet",
"bgp-primitives",
"bgp-rib",
"bytes",
"foldhash",
"netlink-sys",
"nix",
"prometheus",
"proptest",
"thiserror 2.0.12",
"tokio",
"tracing",
]
[[package]]
name = "bgp-integration-tests"
version = "0.2.6"
version = "0.0.2"
dependencies = [
"bgp-fib",
"bgp-packet",
"bgp-primitives",
"bgp-rib",
"bgp-rpki",
"bgp-server",
"bgp-web",
"bytes",
@@ -226,7 +204,7 @@ dependencies = [
[[package]]
name = "bgp-packet"
version = "0.2.6"
version = "0.0.2"
dependencies = [
"bgp-primitives",
"bitfield",
@@ -242,19 +220,21 @@ dependencies = [
[[package]]
name = "bgp-primitives"
version = "0.2.6"
version = "0.0.2"
dependencies = [
"bitfield",
"bytes",
"nom",
"proptest",
"serde",
"serde_repr",
"strum",
"thiserror 2.0.12",
]
[[package]]
name = "bgp-rib"
version = "0.2.6"
version = "0.0.2"
dependencies = [
"bgp-packet",
"bgp-primitives",
@@ -265,34 +245,19 @@ dependencies = [
"serde",
"thiserror 2.0.12",
"tokio",
]
[[package]]
name = "bgp-rpki"
version = "0.2.6"
dependencies = [
"bgp-primitives",
"bytes",
"foldhash",
"proptest",
"thiserror 2.0.12",
"tokio",
"tracing",
]
[[package]]
name = "bgp-server"
version = "0.2.6"
version = "0.0.2"
dependencies = [
"bgp-fib",
"bgp-packet",
"bgp-primitives",
"bgp-rib",
"bgp-rpki",
"bytes",
"foldhash",
"clap",
"futures",
"nix",
"prometheus",
"proptest",
"protobuf",
@@ -301,24 +266,21 @@ dependencies = [
"thiserror 2.0.12",
"tokio",
"tracing",
"tracing-subscriber",
]
[[package]]
name = "bgp-web"
version = "0.2.6"
version = "0.0.2"
dependencies = [
"axum",
"bgp-fib",
"bgp-packet",
"bgp-primitives",
"bgp-rib",
"bgp-rpki",
"bgp-server",
"futures",
"nix",
"prometheus",
"serde",
"tikv-jemalloc-ctl",
"tokio",
"tokio-stream",
"tracing",
@@ -377,28 +339,12 @@ version = "1.10.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d71b6127be86fdcfddb610f7182ac57211d4b18a3e9c82eb2d17662f2227ad6a"
[[package]]
name = "cc"
version = "1.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f74872d07caf508b30a21f6836e7d7016a2eaf7d9ff4f48deaa58cd8a0407630"
dependencies = [
"find-msvc-tools",
"shlex",
]
[[package]]
name = "cfg-if"
version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "baf1de4339761588bc0619e3cbc0120ee582ebb74b53b4efbf79117bd2da40fd"
[[package]]
name = "cfg_aliases"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f079e83a288787bcd14a6aea84cee5c87a67c5a3e660c30f557a3d24761b3527"
[[package]]
name = "clap"
version = "4.5.53"
@@ -503,12 +449,6 @@ version = "2.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "37909eebbb50d72f9059c3b6d82c0463f2ff062c9e95845c43a6c9c0355411be"
[[package]]
name = "find-msvc-tools"
version = "0.1.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "aedcfb3409746eddb02b9e19ebda1c3394f759a152e48ee875a0844d1b955484"
[[package]]
name = "flate2"
version = "1.1.10"
@@ -978,15 +918,6 @@ version = "2.7.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "78ca9ab1a0babb1e7d5695e3530886289c18cf2f87ec19a575a0abdce112e3a3"
[[package]]
name = "memoffset"
version = "0.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "488016bfae457b036d996092f6cb448677611ce4449e970ceaf42695203f218a"
dependencies = [
"autocfg",
]
[[package]]
name = "mime"
version = "0.3.17"
@@ -1014,32 +945,6 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "netlink-sys"
version = "0.8.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cd6c30ed10fa69cc491d491b85cc971f6bdeb8e7367b7cde2ee6cc878d583fae"
dependencies = [
"bytes",
"futures-util",
"libc",
"log",
"tokio",
]
[[package]]
name = "nix"
version = "0.31.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cf20d2fde8ff38632c426f1165ed7436270b44f199fc55284c38276f9db47c3d"
dependencies = [
"bitflags",
"cfg-if",
"cfg_aliases",
"libc",
"memoffset",
]
[[package]]
name = "nom"
version = "8.0.0"
@@ -1102,12 +1007,6 @@ dependencies = [
"windows-targets",
]
[[package]]
name = "paste"
version = "1.0.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "57c0d7b74b563b49d38dae00a0c37d4d6de9b432382b2892f0574ddcae73fd0a"
[[package]]
name = "percent-encoding"
version = "2.3.2"
@@ -1502,12 +1401,6 @@ dependencies = [
"lazy_static",
]
[[package]]
name = "shlex"
version = "2.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8fadd59c855ef2080decdef8ff161eb6661b86933c9d82e5ba29dc602a55aba"
[[package]]
name = "signal-hook-registry"
version = "1.4.2"
@@ -1689,37 +1582,6 @@ dependencies = [
"cfg-if",
]
[[package]]
name = "tikv-jemalloc-ctl"
version = "0.7.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3a184c43b8ab2f41df2733b55556e3f5f632f4aeaa205b1bb018f574b7f5f142"
dependencies = [
"libc",
"paste",
"tikv-jemalloc-sys",
]
[[package]]
name = "tikv-jemalloc-sys"
version = "0.7.1+5.3.1-0-g81034ce1f1373e37dc865038e1bc8eeecf559ce8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1a2825c78386b4ae0314074867860ba9577875de945f05992c38815cbec327f0"
dependencies = [
"cc",
"libc",
]
[[package]]
name = "tikv-jemallocator"
version = "0.7.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "249f09e49ab1609436f34c776e84231bead18d6a955f119f939bdc1d847561bd"
dependencies = [
"libc",
"tikv-jemalloc-sys",
]
[[package]]
name = "tinystr"
version = "0.8.4"
Cargo.toml
+1 -15
View File
@@ -12,7 +12,7 @@ edition = "2024"
homepage = "https://rayhaan.ch"
license = "Apache-2.0"
repository = "https://git.rayhaan.net/rayhaan/ferrix"
version = "0.2.6"
version = "0.0.2"
# Every crate opts in with `[lints] workspace = true`. CI turns warnings into
# errors, so a lint added here is enforced.
@@ -27,11 +27,9 @@ unimplemented = "warn"
[workspace.dependencies]
# --- Our crates ---
bgp-fib = { path = "crates/fib" }
bgp-packet = { path = "crates/packet" }
bgp-primitives = { path = "crates/primitives" }
bgp-rib = { path = "crates/rib" }
bgp-rpki = { path = "crates/rpki" }
bgp-server = { path = "crates/server" }
bgp-web = { path = "crates/web" }
bgp-integration-tests = { path = "crates/integration-tests" }
@@ -43,7 +41,6 @@ bytes = "1.10.1"
eyre = "0.6.12"
nom = "8.0.0"
serde = { version = "1.0.219", features = ["derive"] }
nix = { version = "0.31", default-features = false, features = ["net"] }
socket2 = "0.6"
serde_json = "1.0.140"
serde_repr = "0.1.20"
@@ -63,14 +60,3 @@ hashbrown = "0.17"
axum = "0.8.9"
tokio-stream = { version = "0.1.19", features = ["sync"] }
reqwest = { version = "0.13.5", default-features = false, features = ["json"] }
# The daemon's memory allocator; see bin/bgp/src/main.rs. With its
# statistics, which the status page and the metrics show.
tikv-jemallocator = { version = "0.7.0", features = ["stats"] }
tikv-jemalloc-ctl = { version = "0.7.0", features = ["stats"] }
# `cargo build --profile profiling`: the release build, with the line tables
# a profiler needs to name functions. See scripts/profile/.
[profile.profiling]
inherits = "release"
debug = "line-tables-only"
strip = "none"
Dockerfile
+1 -1
View File
@@ -3,7 +3,7 @@
FROM rust:1.92-bookworm AS build
WORKDIR /src
COPY . .
RUN cargo build --release --locked -p bgp
RUN cargo build --release -p bgp
FROM debian:bookworm-slim
COPY --from=build /src/target/release/bgp /usr/local/bin/bgp
README.md
+30 -166
View File
@@ -1,193 +1,57 @@
# Ferrix BGP
# BGP
Ferrix is a BGP daemon for Linux routers, written in Rust. It exchanges
routes with neighbors, filters them by policy and the RPKI, selects the
best path for each prefix, installs it in the kernel, and passes routes on
as BGP roles and communities permit. It aims to be small, safe, observable,
and configured from one file that can be read by someone who did not write
it.
The `bgp` crate is an implementation of the Border Gateway Protocol, which is used to exchange routing and control plane information between Autonomous Systems.
## Status
The aim of this project is to provide a modern, high performance, secure, programmatic and customizable software package that can be useful to network operators, researchers, and people interested in Internet.
Ferrix routes the author's network, AS210036: three routers carrying full
tables from transit providers, route servers at several Internet exchanges,
and bilateral peers, with routes reflected between the routers over iBGP.
It is young software all the same. Read the [limitations](#not-supported)
before relying on it, and treat version `0.x` as meaning that configuration
and APIs may still change between releases.
## Current status.
What it does:
The project is currently being rewritten from scratch to be more modular and maintainable, as well as to fix deeper issues with the previous implementation.
- BGP-4 over IPv4 and IPv6, with four-octet AS numbers, multiprotocol
reach for both families, IPv4 routes with IPv6 next hops (RFC 8950),
route refresh, graceful restart and ADD-PATH receive.
- Import policy per neighbor: prefix limits, prefix and origin AS lists,
bogon checks, AS path checks, and communities added on the way in.
- Route origin validation and ASPA path verification against an RTR cache.
- BGP roles (RFC 9234) and the Only to Customer attribute, with export
limits as a second line of defence against route leaks.
- Best path selection and installation of the result in the Linux kernel
over netlink, in a routing table of your choice.
- Route reflection between routers of the same AS, and passive neighbors.
- A status page, a JSON API, server-sent events, and Prometheus metrics.
- Graceful restart of the daemon itself for upgrades, with routes left in
the kernel while it is away.
⚠️ Warning: This is currently a work in progress and not ready for use.
### Not supported
### 🛣️ Roadmap to v1.0
- Per-client best path selection for route servers, and ADD-PATH for
advertised routes
- TCP-AO, TCP MD5, and GTSM
- Long-Lived Graceful Restart, BFD, and configuration changes without a
restart
- Address families other than IPv4 and IPv6 unicast
We aim for the 1.0 release to be functional and stable to operate peering sessions, and to install routes in the Linux Kernel.
## Building
You need a stable Rust toolchain from [rustup.rs](https://rustup.rs), and a
C compiler and `make` for the jemalloc allocator the daemon uses (for
example `build-essential` on Debian and Ubuntu, `base-devel` on Arch Linux,
or the Xcode command line tools on macOS). The Protocol Buffers compiler is
not needed: the configuration schema is compiled in pure Rust.
```bash
git clone https://git.rayhaan.net/rayhaan/ferrix.git
cd ferrix
cargo build --release --locked -p bgp
```
The binary is `target/release/bgp`. `--locked` builds with the dependency
versions recorded in `Cargo.lock`, which are the versions CI tested.
To try it without touching any router, run the quick-start configuration.
It listens on loopback, needs no privileges, and serves the status page at
<http://127.0.0.1:8080>; the [quick start
guide](docs/content/docs/getting-started/quick-start.md) adds a test peer
that announces routes to it.
```bash
target/release/bgp --check docs/examples/quick-start.textpb
target/release/bgp docs/examples/quick-start.textpb
```
Other ways to build:
- **A static Linux executable** that runs on any x86_64 distribution with
nothing else installed: `scripts/build-static.sh`. It needs the
`x86_64-unknown-linux-musl` target and a musl C compiler; the script's
header says how to install them, including on macOS.
- **A container image** with the provided `Dockerfile`:
`docker build -t ferrix .`
Kernel route installation needs Linux and `CAP_NET_ADMIN`; everywhere else
Ferrix is a BGP speaker only. The [installation
guide](docs/content/docs/getting-started/installation.md) and the
[deployment guide](docs/content/docs/operations/deployment.md) cover
privileges, the systemd unit, and upgrading without withdrawing routes.
## Documentation
The documentation is a website built from [`docs/`](docs/) with
[Zola](https://www.getzola.org): `zola --root docs serve`, then open
<http://127.0.0.1:1111>. The sources are Markdown with Zola front matter,
shortcodes and internal links, so the forge shows them with those parts
unrendered; [`docs/README.md`](docs/README.md) explains what each is. The
pages:
- [Overview](docs/content/docs/getting-started/introduction.md),
[installation](docs/content/docs/getting-started/installation.md) and
[quick start](docs/content/docs/getting-started/quick-start.md)
- User guide: [BGP neighbors](docs/content/docs/guides/peers.md),
[import policy](docs/content/docs/guides/import-policy.md),
[BGP roles](docs/content/docs/guides/roles.md),
[RPKI](docs/content/docs/guides/rpki.md),
[kernel route installation](docs/content/docs/guides/kernel.md),
[route advertisement](docs/content/docs/guides/passing-routes-on.md),
[graceful restart](docs/content/docs/guides/graceful-restart.md),
[configuration examples](docs/content/docs/guides/examples.md)
- [Deployment](docs/content/docs/operations/deployment.md) and
[monitoring](docs/content/docs/operations/monitoring.md)
- Reference: [configuration](docs/content/docs/reference/configuration.md),
[command line](docs/content/docs/reference/command-line.md),
[HTTP API](docs/content/docs/reference/http-api.md),
[metrics](docs/content/docs/reference/metrics.md),
[supported standards](docs/content/docs/reference/rfcs.md)
- [Architecture](docs/content/docs/development/architecture.md),
[testing](docs/content/docs/development/testing.md) and
[contributing](docs/content/docs/development/contributing.md)
Complete configurations for an edge router, an exchange router and a
transit customer are in [`docs/examples/`](docs/examples/); CI checks that
each of them loads.
## Security
- The status page and HTTP API are not authenticated and show the whole
configuration and routing table. Bind `http_endpoint` to a loopback
address, or put it behind a proxy that authenticates.
- Ferrix does not implement TCP-MD5 or TCP-AO. Filter TCP port 179 in the
host firewall so that only configured neighbors can connect.
- Everything read from the network, the RPKI cache, and the kernel is
parsed without panicking; each parser has property tests and a fuzz
target. `unsafe` code is forbidden throughout.
If you find a vulnerability, please contact the maintainer directly rather
than opening a public issue.
## Layout
| Crate | What is in it |
| -------------------------- | -------------------------------------------------------------------------- |
| `crates/primitives` | Prefixes, address families, parsing helpers |
| `crates/packet` | The wire format: messages, attributes, capabilities |
| `crates/rib` | The routing table, best route selection, the journal of changes |
| `crates/rpki` | The RTR client; origin validation and ASPA path verification |
| `crates/fib` | The best routes in the Linux kernel, over netlink |
| `crates/server` | Sessions, the state machine, policy, configuration |
| `crates/web` | The status page, HTTP API and metrics |
| `crates/integration-tests` | A test server and test peers, and the tests that run the daemon end to end |
| `crates/bench` | Benchmarks at the scale of the Internet routing table |
| `bin/bgp` | The daemon |
- ☐ Implement basic parsers for the fundamental types required
- ✅ Path Attributes from RFC4271
- ✅ Open message
- ✅ Capabilities from RFC4271
- ✅ Update message
- ☐ Notification message
- ✅ Implement scaffolding required to start and configure a server
- ✅ Implement connection logic to establish sessions with peers (passive and active)
- ✅ Routes originated from the configuration, with communities, announced to all peers
- ✅ Read-only HTTP API, status page and Prometheus metrics
- ☐ Implement an efficient data structure for storing prefixes (e.g. Poptrie)
- ☐ Filtering for peer RIB in / out
- ☐ Main / auxiliary RIBs for exchanging routes between peers
## Development
CI runs these on every push, and fails on any warning:
CI runs these three on every push, and fails on any warning:
```bash
```
cargo fmt --all -- --check
cargo clippy --locked --workspace --all-targets -- -D warnings
cargo test --locked --workspace
RUSTDOCFLAGS="-D warnings" cargo doc --locked --workspace --no-deps --document-private-items
zola --root docs build
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace
```
It also runs the tests that need a real Linux kernel
(`scripts/kernel-tests.sh`) and a lab that peers the daemon with FRR, BIRD
and Routinator (`scripts/interop/lab.sh`); `scripts/linux-vm.sh` runs
either from macOS in a Lima VM.
Lints are configured once, under `[workspace.lints]` in the root `Cargo.toml`.
`unsafe` code is forbidden throughout.
`cargo test` includes property tests: generated messages must survive being
written and read back, and no input, however damaged, may make a parser
panic. They take about a second; for a longer hunt:
```bash
```
PROPTEST_CASES=1000000 cargo test --release -p bgp-packet --test properties
```
Coverage-guided fuzz targets for the same parsers are in `fuzz/`; see
[`fuzz/README.md`](fuzz/README.md). CI runs each of them for a minute.
### Development with AI agents
Much of Ferrix is written with an AI coding agent (Claude Code) working
alongside the maintainer, who directs the work, reviews every change, and
runs the result on real routers. Nothing is merged on the agent's say-so:
every change passes the checks above, the kernel tests and the
interoperability lab, and the changes that matter are tried on the network
first. Commits made with an agent carry a `Co-Authored-By` trailer that
says so. If you contribute, you are welcome to work the same way, or not;
what is reviewed is the change, not how it was produced.
`fuzz/README.md`. CI runs each of them for a minute.
## License
Apache License 2.0; see [`LICENSE`](LICENSE).
Apache License 2.0; see `LICENSE`.
bin/bgp/Cargo.toml
-17
View File
@@ -16,23 +16,6 @@ eyre.workspace = true
protobuf.workspace = true
tokio.workspace = true
tracing.workspace = true
tracing-subscriber.workspace = true
# jemalloc does not build with the MSVC toolchain; there the system
# allocator is used.
[target.'cfg(not(target_env = "msvc"))'.dependencies]
tikv-jemallocator.workspace = true
# jemalloc's background thread returns freed memory to the kernel on time
# (see the memory figures on the status page); jemalloc has it on Linux
# only, and warns at every start elsewhere if asked for it.
[target.'cfg(target_os = "linux")'.dependencies]
tikv-jemallocator = { workspace = true, features = ["background_threads"] }
[features]
# jemalloc's heap profiler, for scripts/profile/full-table.sh with
# HEAP=1: not for a production build, where it costs memory and time.
heap-profiling = ["tikv-jemallocator/profiling"]
[lints]
workspace = true
bin/bgp/README.md
+51 -11
View File
@@ -1,17 +1,57 @@
# bgp
# BGP
The daemon: the BGP speaker, the RTR client, the kernel route installer and
the status page, in one binary.
This crate provides the `bgp` binary, which runs the Border Gateway Protocol
daemon together with its read-only HTTP status interface.
```
cargo run -p bgp -- --check docs/examples/quick-start.textpb
cargo run -p bgp -- docs/examples/quick-start.textpb
cargo run -- path/to/config.textpb
```
How to configure, run and operate it is in the documentation, which is
built from `docs/` (`zola --root docs serve`):
The configuration format is defined in `crates/server/proto/config.proto`;
`testdata/server.textpb` is a worked example. The daemon is configured
entirely through that file: peers, and the routes it originates
(`originate { prefix: ... }`). There is no way to change either at runtime.
- [Quick start](../../docs/content/docs/getting-started/quick-start.md)
- [Configuration reference](../../docs/content/docs/reference/configuration.md)
- [Deployment](../../docs/content/docs/operations/deployment.md) and
[monitoring](../../docs/content/docs/operations/monitoring.md)
When `http_endpoint` is set the binary also serves:
| Path | Purpose |
|---|---|
| `GET /statusz` | Status page: configuration, peers and session state, originated and received routes, live event log |
| `GET /api/status` | Router ID, ASN, uptime and table sizes |
| `GET /api/config` | The configuration file as read at startup |
| `GET /api/peers` | Configured peers and established sessions |
| `GET /api/routes/local` | Originated routes |
| `GET /api/routes/received` | Adj-RIB-In contents |
| `GET /api/events` | Server-sent events for session and route changes |
| `GET /metricz` | Prometheus metrics |
Everything under it is read-only and unauthenticated. Bind it to loopback or
put it behind a reverse proxy.
## What it does today
It establishes sessions (both dialling configured peers and accepting
connections, with RFC 4271 collision detection), announces its originated
routes to every peer with itself as next hop, and records what peers send in
an Adj-RIB-In. It does not select best paths, re-advertise received routes or
program a forwarding table, and it has no import or export policy.
## Trying it without a router
`fake_peer` is a minimal BGP speaker that connects to the daemon, announces a
prefix, prints every UPDATE it receives and, with `--echo`, re-announces each
received prefix back with its own ASN prepended (dropping NO_EXPORT routes).
That makes originated prefixes show up in the received table as if they had
travelled through a neighbouring AS:
```
cargo run -- testdata/server.textpb
cargo run -p bgp-integration-tests --example fake_peer -- \
--server 127.0.0.1:179 --asn 65001 --announce 203.0.113.0/24 --echo
```
Please refer to the other implementation crates in this repository where most
of the actual implementation lives.
Warning: This is still very much a work in progress and is not yet ready for
production use.
bin/bgp/src/logging.rs
-36
View File
@@ -1,36 +0,0 @@
//! What the daemon logs, and how much.
use std::io::IsTerminal;
use clap::Args;
use tracing::Level;
use tracing_subscriber::EnvFilter;
use tracing_subscriber::filter::ParseError;
/// What the level given on the command line is for: this program and the
/// crates it is made of, which all begin with `bgp`. Everything else (the
/// HTTP server, say) stays at what `RUST_LOG` says, or silent.
const OURS: &str = "bgp";
#[derive(Debug, Args)]
pub struct LoggingArgs {
/// Set the minimum log level. At `debug` and below there is a line for
/// every UPDATE and every route announced.
#[arg(short, long, global = true, default_value_t = Level::INFO, help_heading = "Logging")]
level: Level,
}
impl LoggingArgs {
pub fn install(&self) -> Result<(), ParseError> {
let filter = EnvFilter::builder()
.from_env_lossy()
.add_directive(format!("{OURS}={}", self.level).parse()?);
// Colours for a person at a terminal; plain text for the journal
// or a file, where escape codes are noise.
tracing_subscriber::fmt()
.with_env_filter(filter)
.with_ansi(std::io::stdout().is_terminal())
.init();
Ok(())
}
}
bin/bgp/src/main.rs
+21 -98
View File
@@ -1,68 +1,34 @@
use std::path::PathBuf;
use std::process::ExitCode;
use std::sync::Arc;
use bgp_server::config::Config;
use bgp_server::proto::config::BgpConfig;
use bgp_server::server::{BgpServer, leave_for_restart, shut_down};
use bgp_web::{AppState, CommunityName, RouterInfo};
use bgp_server::server::{BgpServer, close_sessions};
use bgp_server::tracing::TracingArgs;
use bgp_web::{AppState, RouterInfo};
use clap::Parser;
use eyre::{Context, Result};
use tracing::{error, info};
use crate::logging::LoggingArgs;
mod logging;
// jemalloc, rather than the C library's allocator. Taking in a full table
// allocates from many threads at once, which glibc's allocator handles
// less well and musl's, in the static build, much worse.
#[cfg(not(target_env = "msvc"))]
#[global_allocator]
static ALLOCATOR: tikv_jemallocator::Jemalloc = tikv_jemallocator::Jemalloc;
/// The exit status after leaving to be started again (SIGUSR1):
/// EX_TEMPFAIL, on which the systemd unit starts the daemon again
/// (`RestartForceExitStatus=75`).
const RESTART_EXIT_STATUS: u8 = 75;
/// A BGP speaker.
#[derive(Parser, Debug)]
#[command(version)]
struct ServerArgs {
/// Path to the textproto configuration file.
config: PathBuf,
/// Read the configuration, say whether it is in order, and stop:
/// nothing is listened on and no route is touched.
#[arg(long)]
check: bool,
config: String,
#[command(flatten)]
logging: LoggingArgs,
tracing: TracingArgs,
}
#[tokio::main]
async fn main() -> Result<ExitCode> {
async fn main() -> Result<()> {
let args = ServerArgs::parse();
args.logging.install()?;
args.tracing.install()?;
log_panics();
let config_string = std::fs::read_to_string(&args.config)
.wrap_err_with(|| format!("Failed to read {}", args.config.display()))?;
let config_string =
std::fs::read_to_string(&args.config).wrap_err("Failed to open config file")?;
let proto_config = protobuf::text_format::parse_from_str::<BgpConfig>(&config_string)
.wrap_err("Failed to parse config file")?;
let config = Config::from_proto(proto_config).wrap_err("Invalid configuration")?;
if args.check {
println!(
"{} is in order: AS{}, peers: {}, routes originated: {}",
args.config.display(),
config.local_asn,
config.peers.len(),
config.originate.len()
);
return Ok(ExitCode::SUCCESS);
}
let server = BgpServer::bind(config).await?;
@@ -79,27 +45,11 @@ async fn main() -> Result<ExitCode> {
server.config().local_asn,
env!("CARGO_PKG_VERSION"),
)
.with_config(args.config.to_string_lossy(), &config_string)
.with_control_communities(server.config().control.is_some())
.with_community_names(
server
.config()
.community_names
.iter()
.map(|(community, name)| CommunityName {
community: community.clone(),
name: name.clone(),
})
.collect(),
),
.with_config(&args.config, &config_string),
),
registry: server.registry.clone(),
local_rib: server.local_rib.clone(),
rib: Arc::clone(&server.rib),
rejections: server.rejections.clone(),
rpki: server.rpki.clone(),
kernel: server.kernel.clone(),
restarts: Arc::clone(&server.restarts),
};
tokio::spawn(async move {
if let Err(e) = bgp_web::serve_on(state, listener).await {
@@ -111,30 +61,13 @@ async fn main() -> Result<ExitCode> {
// Leave politely: tell every peer we are going (Cease / Administrative
// Shutdown) rather than letting them find a dead TCP connection, which
// makes them slower to take us back.
// And take our routes out of the kernel: with us gone nothing would
// keep them true.
// Unless we are to be back at once (SIGUSR1, for an upgrade): then the
// routes stay, and peers that agreed to graceful restart keep ours.
let (registry, kernel) = (server.registry.clone(), server.kernel.clone());
let registry = server.registry.clone();
tokio::select! {
result = server.run() => {
result?;
Ok(ExitCode::SUCCESS)
}
signal = signalled() => match signal {
Signal::Stop(name) => {
info!(signal = name, "Shutting down, closing BGP sessions");
shut_down(&registry, kernel.as_ref()).await;
Ok(ExitCode::SUCCESS)
}
Signal::Restart => {
info!(
signal = "SIGUSR1",
"Leaving to be started again: routes stay in the kernel, and peers keep ours"
);
leave_for_restart(&registry, kernel.as_ref()).await;
Ok(ExitCode::from(RESTART_EXIT_STATUS))
}
result = server.run() => Ok(result?),
signal = shutdown_signal() => {
info!(signal, "Shutting down, closing BGP sessions");
close_sessions(&registry).await;
Ok(())
}
}
}
@@ -161,30 +94,20 @@ fn log_panics() {
}));
}
/// What a signal asks of the daemon.
enum Signal {
/// To stop: SIGINT or SIGTERM, by name.
Stop(&'static str),
/// To leave, to be started again at once: SIGUSR1.
Restart,
}
/// Resolves with the first signal received that ends the daemon.
async fn signalled() -> Signal {
/// Resolves with the name of the first termination signal received.
async fn shutdown_signal() -> &'static str {
#[cfg(unix)]
{
use tokio::signal::unix::{SignalKind, signal};
let mut term = signal(SignalKind::terminate()).expect("install SIGTERM handler");
let mut restart = signal(SignalKind::user_defined1()).expect("install SIGUSR1 handler");
tokio::select! {
_ = tokio::signal::ctrl_c() => Signal::Stop("SIGINT"),
_ = term.recv() => Signal::Stop("SIGTERM"),
_ = restart.recv() => Signal::Restart,
_ = tokio::signal::ctrl_c() => "SIGINT",
_ = term.recv() => "SIGTERM",
}
}
#[cfg(not(unix))]
{
let _ = tokio::signal::ctrl_c().await;
Signal::Stop("ctrl-c")
"ctrl-c"
}
}
crates/bench/Cargo.toml
+1 -11
View File
@@ -16,14 +16,12 @@ bytes.workspace = true
flate2 = "1"
stats_alloc = { version = "0.1.10", optional = true }
[target.'cfg(not(target_env = "msvc"))'.dependencies]
tikv-jemallocator.workspace = true
[dev-dependencies]
bgp-integration-tests.workspace = true
bgp-rib.workspace = true
bgp-server.workspace = true
tokio.workspace = true
tracing.workspace = true
tracing-subscriber.workspace = true
[features]
@@ -40,13 +38,5 @@ harness = false
name = "real_table"
harness = false
[[bench]]
name = "transit"
harness = false
[lints]
workspace = true
[[bench]]
name = "mesh"
harness = false
crates/bench/benches/full_table.rs
+27 -14
View File
@@ -3,8 +3,7 @@
//!
//! Run with `cargo bench -p bgp-bench --bench full_table`. Set
//! `BENCH_PEERS` (default 4) and `BENCH_SCALE_DOWN` (default 1; 10 makes the
//! table ten times smaller) to change the size. `BENCH_PRESIZE=0` leaves
//! the table to grow as it fills, as it does without `expected_routes`. The table is synthetic (see
//! table ten times smaller) to change the size. The table is synthetic (see
//! the crate documentation); `real_table` runs on a real one.
//!
//! This is a stopwatch around whole operations at full size, not a
@@ -18,8 +17,7 @@ use std::sync::atomic::{AtomicUsize, Ordering};
use std::time::{Duration, Instant};
use bgp_bench::memory::{Region, describe};
use bgp_bench::report::{mib, per_second, row};
use bgp_bench::{Prefixes, Shape, announcements, encoded, env, wire_context};
use bgp_bench::{Prefixes, Shape, announcements, encoded, wire_context};
use bgp_integration_tests::test_peer::TestPeer;
use bgp_integration_tests::test_server::TestServer;
use bgp_packet::message::{Message, PathAttribute};
@@ -27,6 +25,30 @@ use bgp_primitives::ip_prefix::IpPrefix;
use bgp_rib::table::{PeerIndex, PeerInfo, RibTable};
use bytes::BytesMut;
fn env(name: &str, default: usize) -> usize {
std::env::var(name)
.ok()
.and_then(|v| v.parse().ok())
.unwrap_or(default)
}
fn mib(bytes: usize) -> f64 {
bytes as f64 / (1024.0 * 1024.0)
}
fn per_second(count: usize, elapsed: Duration) -> String {
let rate = count as f64 / elapsed.as_secs_f64();
if rate >= 1e6 {
format!("{:.2} M/s", rate / 1e6)
} else {
format!("{:.0} k/s", rate / 1e3)
}
}
fn row(what: &str, elapsed: Duration, detail: String) {
println!("{what:<44} {:>9.3} s {detail}", elapsed.as_secs_f64());
}
fn main() {
let peers = env("BENCH_PEERS", 4) as u32;
let shape = Shape::INTERNET.scaled_down(env("BENCH_SCALE_DOWN", 1));
@@ -83,9 +105,7 @@ fn main() {
.build()
.expect("a runtime");
runtime.block_on(async {
// An AS number that is in none of the generated paths: a route
// with the server's own AS in its path is refused as a loop.
let server = TestServer::start(4_200_000_000).await.expect("a server");
let server = TestServer::start(65000).await.expect("a server");
let mut senders = Vec::new();
for (index, messages) in wire.iter().enumerate() {
let peer = index as u32 + 1;
@@ -94,8 +114,6 @@ fn main() {
.handshake(64512 + peer, Ipv4Addr::new(10, 0, 0, peer as u8), 90)
.await
.expect("a session");
// The same bytes `table_stream` would make, from the messages
// already encoded for the decoding above.
let stream: Vec<u8> = messages.iter().flat_map(|m| m.iter().copied()).collect();
senders.push((session, stream));
}
@@ -135,9 +153,6 @@ fn main() {
fn into_table(wire: &[Vec<BytesMut>], routes: usize, threads: usize) {
let region = Region::begin();
let table = RibTable::new(threads * 16);
if env("BENCH_PRESIZE", 1) == 1 {
table.expect_prefixes(routes / wire.len());
}
let peers: Vec<PeerIndex> = (0..wire.len())
.map(|peer| {
table
@@ -145,9 +160,7 @@ fn into_table(wire: &[Vec<BytesMut>], routes: usize, threads: usize) {
asn: 64512 + peer as u32 + 1,
router_id: Ipv4Addr::from(peer as u32 + 1),
address: Ipv4Addr::new(192, 0, 2, peer as u8 + 1).into(),
interface: 0,
ibgp: false,
reflector_client: false,
})
.expect("room for the peer")
})
crates/bench/benches/mesh.rs
-659
View File
@@ -1,659 +0,0 @@
//! Two routers of one AS, each taking in full tables from its own peers
//! and sending the other every path (ADD-PATH both ways, `all_paths`), as
//! a mesh of routers that each hold the network's whole view do. How
//! long until each router has the other's routes is what this measures:
//! the exporter reading the table's journal while sessions fill the
//! table, which nothing else exercises at this size.
//!
//! Then a new peer takes the first router's table, as a collector or a
//! router coming back does: how soon its first UPDATE arrives and how
//! long the table takes are measured too. It is a peer of the routers'
//! AS over IPv4, so what it gets is the first router's IPv4 routes from
//! its own peers: the other router's are not reflected to it, and IPv6
//! routes need an IPv6 session.
//!
//! Run with `cargo bench -p bgp-bench --bench mesh`. `BENCH_PEERS`
//! (default 4) feed the first router and `BENCH_PEERS_B` (default 2) the
//! second; `BENCH_SCALE_DOWN` shrinks the table. The routers peer over
//! IPv6 loopback and the feeders come over IPv4 loopback; the new peer
//! comes from 127.0.0.2, which Linux has and macOS does not, so that
//! part is skipped on macOS. With `BENCH_RECEIVER_EARLY=1` the new peer
//! comes as the feeders start instead, so that its exporter goes
//! through the table while the table is being filled, as a router's
//! do after a restart. `BENCH_COLLECTORS` (default 0) more peers of the
//! first router, from 127.0.1.0/24, are peers in the sense of RFC 9234
//! told of learned routes, and the feeders' routes are marked as having
//! come across from a peer, so none of them goes: as many exporters
//! following the table while it fills and passing nearly all of it
//! over, as a router with many peers at exchanges has.
use std::net::{Ipv4Addr, Ipv6Addr, SocketAddr};
use std::sync::Arc;
use std::sync::atomic::{AtomicU64, Ordering};
use std::time::{Duration, Instant};
use bgp_bench::report::row;
use bgp_bench::{Prefixes, Shape, announcements, encoded, env, table_stream};
use bgp_integration_tests::peer_config;
use bgp_integration_tests::test_peer::{TestPeer, collector_capabilities};
use bgp_integration_tests::test_server::TestServer;
use bgp_packet::attributes::PathAttribute;
use bgp_packet::message::msg_type;
use bgp_packet::open::SendReceive;
use bgp_server::config::PeerConfig;
use bgp_server::family::SUPPORTED;
use bgp_server::policy::{ExportPolicy, ImportPolicy, PeerRole};
use bgp_server::session::SessionRegistry;
use tokio::io::{AsyncReadExt, AsyncWriteExt, BufReader};
use tokio::net::tcp::OwnedWriteHalf;
use tokio::task::JoinHandle;
/// An AS number that is in none of the synthetic paths.
const ASN: u32 = 4_200_000_000;
/// Where the peer that takes the whole table connects from.
const RECEIVER: Ipv4Addr = Ipv4Addr::new(127, 0, 0, 2);
/// Where the collectors connect from: this and the addresses after it.
fn collector_address(n: u32) -> Ipv4Addr {
Ipv4Addr::new(127, 0, 1, n as u8)
}
/// The collectors' AS numbers start here.
const COLLECTOR_ASN: u32 = 65000;
/// How often the tables are looked at while they fill.
const POLL: Duration = Duration::from_millis(5);
/// What the routers send each other, and the receiver: every path.
fn every_path() -> Arc<ExportPolicy> {
Arc::new(ExportPolicy {
learned_routes: true,
all_paths: true,
..ExportPolicy::default()
})
}
fn free_port() -> u16 {
std::net::TcpListener::bind((Ipv6Addr::LOCALHOST, 0))
.expect("a port")
.local_addr()
.expect("an address")
.port()
}
/// A router of the mesh, listening for feeders on IPv4 loopback and for
/// the other router on IPv6 loopback at `port`, which it dials at
/// `other_port`, knowing it as `other_name`.
async fn router(
other_name: &'static str,
id: u8,
port: u16,
other_port: u16,
with_receiver: bool,
collectors: u32,
) -> TestServer {
TestServer::start_with(ASN, |c| {
// Two routers of one AS must not share a BGP identifier.
c.router_id = Ipv4Addr::new(192, 0, 2, id);
c.cluster_id = c.router_id;
c.listen_endpoints = vec![
SocketAddr::new(Ipv4Addr::LOCALHOST.into(), 0),
SocketAddr::new(Ipv6Addr::LOCALHOST.into(), port),
];
c.peers = vec![PeerConfig {
add_path: Some(SendReceive::Both),
import: ImportPolicy::permissive(),
export: every_path(),
..peer_config(
other_name,
SocketAddr::new(Ipv6Addr::LOCALHOST.into(), other_port),
ASN,
)
}];
if with_receiver {
c.peers.push(PeerConfig {
add_path: Some(SendReceive::Send),
export: every_path(),
..peer_config("receiver", SocketAddr::new(RECEIVER.into(), 179), ASN)
});
}
for n in 1..=collectors {
c.peers.push(PeerConfig {
role: Some(PeerRole::Peer),
export: Arc::new(ExportPolicy {
learned_routes: true,
all_paths: false,
..ExportPolicy::default()
}),
..peer_config(
"collector",
SocketAddr::new(collector_address(n).into(), 179),
COLLECTOR_ASN + n,
)
});
}
c.connect_retry = Duration::from_millis(200);
})
.await
.expect("a router")
}
/// Connects `count` feeders to `router`, numbered from `first`, and
/// returns their sessions with the bytes each is to send.
async fn feeders(
router: &TestServer,
prefixes: &Prefixes,
shape: Shape,
first: u32,
count: u32,
marked: bool,
) -> Vec<(TestPeer, Vec<u8>)> {
let mut sessions = Vec::new();
for peer in first..first + count {
let stream = if marked {
// As having come across from a peer (RFC 9234): not for the
// collectors, which are peers too.
let mut updates = announcements(prefixes, shape, peer);
for update in &mut updates {
update
.path_attributes
.push(PathAttribute::OnlyToCustomer(64512 + peer));
}
encoded(&updates)
.iter()
.flat_map(|m| m.iter().copied())
.collect()
} else {
table_stream(prefixes, shape, peer)
};
let mut session = TestPeer::connect(router.addr).await.expect("a connection");
// No hold timer: a feeder has nothing to say after its table.
session
.handshake(64512 + peer, Ipv4Addr::new(10, 0, 0, peer as u8), 0)
.await
.expect("a session");
sessions.push((session, stream));
}
sessions
}
/// Every session of `registry`, one line each, for the panic when a
/// deadline passes: where the routes have got to, and what each session
/// was to be sent.
fn sessions_report(registry: &SessionRegistry) -> Vec<String> {
registry
.list()
.iter()
.map(|s| {
format!(
"{} {:?} export {:?} add_path_send {:?} routes {} rejected {} advertised {}",
s.peer,
s.name,
s.export,
s.add_path_send,
s.counters.routes(),
s.counters.rejected(),
s.counters.advertised()
)
})
.collect()
}
/// How big a run is, from the environment.
struct Settings {
peers_a: u32,
peers_b: u32,
shape: Shape,
collectors: u32,
}
impl Settings {
fn from_env() -> Self {
Self {
peers_a: env("BENCH_PEERS", 4) as u32,
peers_b: env("BENCH_PEERS_B", 2) as u32,
shape: Shape::INTERNET.scaled_down(env("BENCH_SCALE_DOWN", 1)),
collectors: env("BENCH_COLLECTORS", 0) as u32,
}
}
/// How many routes A's own feeders bring, and B's.
fn own_routes(&self) -> (usize, usize) {
let per_peer = self.shape.prefixes();
(
per_peer * self.peers_a as usize,
per_peer * self.peers_b as usize,
)
}
/// How many routes the receiver is to be sent: A's IPv4 routes from
/// its own peers.
fn receiver_routes(&self) -> usize {
self.shape.ipv4_prefixes * self.peers_a as usize
}
}
/// The two routers, up and peered, with the collectors' sessions once
/// they are connected.
struct Mesh {
settings: Settings,
prefixes: Prefixes,
a: TestServer,
b: TestServer,
/// The collectors' connections, drained, held until the end.
collectors: Vec<(OwnedWriteHalf, JoinHandle<()>)>,
}
/// The feeders sending, from when they began.
struct Feeding {
started: Instant,
/// Each hands its session back once its table is written.
sending: Vec<JoinHandle<TestPeer>>,
}
/// When each router had what: its own peers' routes, and the other's too.
struct Tables {
a_own: Duration,
b_own: Duration,
a_all: Duration,
b_all: Duration,
}
impl Mesh {
/// Starts the routers, with the collectors and receiver configured at
/// A, and waits for them to peer.
async fn start(settings: Settings, prefixes: Prefixes) -> Self {
let (port_a, port_b) = (free_port(), free_port());
let a = router("b", 1, port_a, port_b, true, settings.collectors).await;
let b = router("a", 2, port_b, port_a, false, 0).await;
a.wait_for_sessions(1).await.expect("the mesh session at A");
b.wait_for_sessions(1).await.expect("the mesh session at B");
Self {
settings,
prefixes,
a,
b,
collectors: Vec::new(),
}
}
/// Connects the collectors to A. They are there before the feeders, as
/// a router's peers are before its tables arrive after a restart. An
/// address that is not this machine's ends the connecting, with a
/// word.
async fn connect_collectors(&mut self) {
for n in 1..=self.settings.collectors {
let socket = tokio::net::TcpSocket::new_v4().expect("a socket");
if let Err(e) = socket.bind(SocketAddr::new(collector_address(n).into(), 0)) {
println!(
"collectors: skipped, {} is not ours ({e})",
collector_address(n)
);
break;
}
let stream = socket.connect(self.a.addr).await.expect("A listens");
let mut session = TestPeer::from_stream(stream);
session
.handshake(COLLECTOR_ASN + n, Ipv4Addr::new(192, 0, 3, n as u8), 0)
.await
.expect("A takes a collector's session");
self.collectors.push(session.drain());
}
if !self.collectors.is_empty() {
self.a
.wait_for_sessions(1 + self.collectors.len())
.await
.expect("the collectors' sessions at A");
}
}
/// Connects the feeders to both routers and starts them sending.
async fn feed(&self) -> Feeding {
let Settings {
peers_a,
peers_b,
shape,
..
} = self.settings;
let marked = !self.collectors.is_empty();
let feed_a = feeders(&self.a, &self.prefixes, shape, 1, peers_a, marked).await;
let feed_b = feeders(&self.b, &self.prefixes, shape, 1 + peers_a, peers_b, false).await;
let started = Instant::now();
let sending = feed_a
.into_iter()
.chain(feed_b)
.map(|(mut session, stream)| {
tokio::spawn(async move {
session.send_raw(&stream).await.expect("the router reads");
session
})
})
.collect();
Feeding { started, sending }
}
/// The receiver taking A's table while it fills, if `BENCH_RECEIVER_EARLY`
/// asks for that: as the feeders start (1), or once A has its own
/// peers' routes and B's are still coming (2), as a router's sessions
/// come up after a restart while its peers' tables arrive.
fn early_receiver(&self, deadline: Duration) -> Option<JoinHandle<()>> {
let when = env("BENCH_RECEIVER_EARLY", 0);
if when == 0 {
return None;
}
let (own_a, _) = self.settings.own_routes();
let routes = self.settings.receiver_routes();
let (addr, registry, rib) = (
self.a.addr,
self.a.registry.clone(),
Arc::clone(&self.a.rib),
);
Some(tokio::spawn(async move {
while when == 2 && rib.len() < own_a {
tokio::time::sleep(POLL).await;
}
resync(addr, registry, routes, deadline).await
}))
}
/// Waits until both routers have every route, noting when each had
/// what; or panics once `deadline` has passed since `started`.
async fn wait_for_tables(&self, started: Instant, deadline: Duration) -> Tables {
let (own_a, own_b) = self.settings.own_routes();
let mut a_own = None;
let mut b_own = None;
let mut a_all = None;
let mut b_all = None;
let (a_all, b_all) = loop {
tokio::time::sleep(POLL).await;
let (at_a, at_b) = (self.a.rib.len(), self.b.rib.len());
let now = started.elapsed();
if a_own.is_none() && at_a >= own_a {
a_own = Some(now);
}
if b_own.is_none() && at_b >= own_b {
b_own = Some(now);
}
if a_all.is_none() && at_a >= own_a + own_b {
a_all = Some(now);
}
if b_all.is_none() && at_b >= own_a + own_b {
b_all = Some(now);
}
if let (Some(a_all), Some(b_all)) = (a_all, b_all) {
break (a_all, b_all);
}
if now > deadline {
panic!(
"not all routes arrived: A has {at_a} {:?}, B has {at_b} {:?}",
sessions_report(&self.a.registry),
sessions_report(&self.b.registry)
);
}
};
Tables {
a_own: a_own.unwrap_or_default(),
b_own: b_own.unwrap_or_default(),
a_all,
b_all,
}
}
/// One row for each of the four moments of `tables`, with the rate the
/// routes that made the moment came at.
fn report(&self, tables: &Tables) {
let (own_a, own_b) = self.settings.own_routes();
let rate = |when: Duration, routes: usize| {
format!("{:.2} M routes/s", routes as f64 / when.as_secs_f64() / 1e6)
};
row(
"A has its own peers' routes",
tables.a_own,
rate(tables.a_own, own_a),
);
row(
"B has its own peers' routes",
tables.b_own,
rate(tables.b_own, own_b),
);
row(
"A has B's routes too (B's exporter, A's session)",
tables.a_all,
rate(tables.a_all, own_b),
);
row(
"B has A's routes too (A's exporter, B's session)",
tables.b_all,
rate(tables.b_all, own_a),
);
}
}
impl Feeding {
/// Waits for every feeder to have written its table, and returns the
/// sessions, which are to stay up until the end: a feeder going would
/// have its routes withdrawn from both tables meanwhile.
async fn finished(self) -> Vec<TestPeer> {
let mut feeders = Vec::new();
for session in self.sending {
feeders.push(session.await.expect("a feeder is done"));
}
feeders
}
}
fn main() {
// The routers' logs, with `BENCH_LOG` as a `RUST_LOG` filter.
if let Ok(filter) = std::env::var("BENCH_LOG") {
tracing_subscriber::fmt()
.with_env_filter(filter)
.with_writer(std::io::stderr)
.init();
}
let settings = Settings::from_env();
let deadline = Duration::from_secs(env("BENCH_DEADLINE", 600) as u64);
println!(
"== Two routers of AS{ASN}, every path both ways: A fed by {} peers, B by {}, {} prefixes each",
settings.peers_a,
settings.peers_b,
settings.shape.prefixes()
);
let prefixes = Prefixes::generate(settings.shape);
let runtime = tokio::runtime::Builder::new_multi_thread()
.enable_all()
.build()
.expect("a runtime");
runtime.block_on(async {
let mut mesh = Mesh::start(settings, prefixes).await;
mesh.connect_collectors().await;
let early_receiver = mesh.early_receiver(deadline);
let feeding = mesh.feed().await;
let tables = mesh.wait_for_tables(feeding.started, deadline).await;
mesh.report(&tables);
let feeders = feeding.finished().await;
match early_receiver {
Some(receiver) => receiver.await.expect("the receiver is done"),
None => {
resync(
mesh.a.addr,
mesh.a.registry.clone(),
mesh.settings.receiver_routes(),
deadline,
)
.await
}
}
drop(feeders);
drop(mesh);
});
}
/// The new peer taking a router's table: counting the UPDATEs it is sent
/// while keeping the session alive.
struct Receiver {
started: Instant,
/// When the first UPDATE arrived, in microseconds after `started`; 0
/// until it has.
first: Arc<AtomicU64>,
updates: Arc<AtomicU64>,
counting: JoinHandle<()>,
keepalives: JoinHandle<()>,
}
impl Receiver {
/// A session with the router at `addr` from [`RECEIVER`], as a
/// collector of every path; or `None`, with a word, where that address
/// is not this machine's to connect from.
async fn connect(addr: SocketAddr) -> Option<TestPeer> {
let socket = tokio::net::TcpSocket::new_v4().expect("a socket");
if let Err(e) = socket.bind(SocketAddr::new(RECEIVER.into(), 0)) {
println!("A new peer taking A's routes: skipped, {RECEIVER} is not ours ({e})");
return None;
}
let stream = socket.connect(addr).await.expect("A listens");
let mut session = TestPeer::from_stream(stream);
session
.handshake_with(
ASN,
Ipv4Addr::new(192, 0, 2, 9),
90,
collector_capabilities(ASN, &SUPPORTED, false),
)
.await
.expect("A takes the receiver's session");
Some(session)
}
/// Starts counting what `session` is sent, from now. Every millionth
/// UPDATE is noted, as is a NOTIFICATION or the connection closing:
/// what went wrong, if the table never all arrives.
fn count_updates(session: TestPeer) -> Self {
let started = Instant::now();
let (reader, mut writer) = session.into_stream().into_split();
// KEEPALIVEs, so that the session outlives the hold time.
let keepalives = tokio::spawn(async move {
let keepalive = TestPeer::keepalive_bytes();
loop {
tokio::time::sleep(Duration::from_secs(20)).await;
if writer.write_all(&keepalive).await.is_err() {
break;
}
}
});
let first = Arc::new(AtomicU64::new(0));
let updates = Arc::new(AtomicU64::new(0));
let counting = tokio::spawn({
let (first, updates) = (Arc::clone(&first), Arc::clone(&updates));
async move {
let mut reader = BufReader::with_capacity(1 << 20, reader);
let mut header = [0u8; 19];
let mut body = vec![0u8; 4096];
while reader.read_exact(&mut header).await.is_ok() {
let length = usize::from(u16::from_be_bytes([header[16], header[17]]));
if reader.read_exact(&mut body[..length - 19]).await.is_err() {
break;
}
match header[18] {
msg_type::UPDATE => {
let _ = first.compare_exchange(
0,
started.elapsed().as_micros() as u64,
Ordering::Relaxed,
Ordering::Relaxed,
);
let n = updates.fetch_add(1, Ordering::Relaxed) + 1;
if n % 1_000_000 == 0 {
println!(
" the receiver has read {n} UPDATEs after {:.2} s",
started.elapsed().as_secs_f64()
);
}
}
msg_type::NOTIFICATION => println!(
"the receiver was sent a NOTIFICATION, error {} subcode {}, after {} UPDATEs",
body[0],
body[1],
updates.load(Ordering::Relaxed)
),
_ => {}
}
}
println!(
"the receiver's connection closed after {} UPDATEs",
updates.load(Ordering::Relaxed)
);
}
});
Self {
started,
first,
updates,
counting,
keepalives,
}
}
/// Waits until the router whose sessions are in `registry` has
/// advertised `routes` routes to the receiver, and returns how many it
/// has; or panics once `deadline` has passed.
async fn wait_for_advertised(
&self,
registry: &SessionRegistry,
routes: usize,
deadline: Duration,
) -> u64 {
loop {
tokio::time::sleep(POLL).await;
let advertised = registry
.list()
.iter()
.find(|s| s.peer.addr.ip() == RECEIVER)
.map_or(0, |s| s.counters.advertised());
if advertised as usize >= routes {
return advertised;
}
if self.started.elapsed() > deadline {
panic!(
"the receiver has {advertised} of {routes} routes and read {} UPDATEs; sessions at A: {:#?}",
self.updates.load(Ordering::Relaxed),
sessions_report(registry)
);
}
}
}
/// Prints the row for the table being all there, `advertised` routes
/// of it, and stops counting.
fn report(self, advertised: u64) {
let took = self.started.elapsed();
self.counting.abort();
self.keepalives.abort();
let first = Duration::from_micros(self.first.load(Ordering::Relaxed));
row(
"A new peer has A's IPv4 routes (A's exporter)",
took,
format!(
"{:.2} M routes/s first UPDATE after {:.0} ms, {} UPDATEs",
advertised as f64 / took.as_secs_f64() / 1e6,
first.as_secs_f64() * 1e3,
self.updates.load(Ordering::Relaxed),
),
);
}
}
/// A new peer takes the table of the router at `addr`, whose sessions
/// are in `registry`: `routes` routes with every path. How long until
/// the first UPDATE, and until all of it is sent.
async fn resync(addr: SocketAddr, registry: SessionRegistry, routes: usize, deadline: Duration) {
let Some(session) = Receiver::connect(addr).await else {
return;
};
let receiver = Receiver::count_updates(session);
let advertised = receiver
.wait_for_advertised(&registry, routes, deadline)
.await;
receiver.report(advertised);
}
crates/bench/benches/real_table.rs
+16 -8
View File
@@ -11,20 +11,30 @@ use std::net::Ipv4Addr;
use std::path::PathBuf;
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering};
use std::time::Instant;
use std::time::{Duration, Instant};
use bgp_bench::memory::Region;
use bgp_bench::mrt::{Dump, parse_attributes};
use bgp_bench::report::{mib, per_second, row};
use bgp_packet::message::{NlriNextHop, PathAttribute};
use bgp_primitives::ip_prefix::IpPrefix;
use bgp_rib::table::{Event, PeerIndex, PeerInfo, RibTable};
/// Bytes allocated and not freed since `region` began; zero when memory
/// is not being counted (see `bgp_bench::memory`), and zero too in the
/// odd case of more freed than allocated, which is nothing held.
fn retained(region: &Region) -> usize {
usize::try_from(region.held().unwrap_or(0)).unwrap_or(0)
/// is not being counted (see `bgp_bench::memory`).
fn retained(region: &Region) -> i64 {
region.held().unwrap_or(0)
}
fn mib(bytes: i64) -> f64 {
bytes as f64 / (1024.0 * 1024.0)
}
fn per_second(count: usize, elapsed: Duration) -> String {
format!("{:.2} M/s", count as f64 / elapsed.as_secs_f64() / 1e6)
}
fn row(what: &str, elapsed: Duration, detail: String) {
println!("{what:<44} {:>9.3} s {detail}", elapsed.as_secs_f64());
}
fn dump_path() -> Option<PathBuf> {
@@ -271,9 +281,7 @@ fn add_peers(table: &RibTable, dump: &Dump, peers: &[usize]) -> Vec<PeerIndex> {
asn: dump.peers[peer].asn,
router_id: Ipv4Addr::from(peer as u32 + 1),
address: dump.peers[peer].address,
interface: 0,
ibgp: false,
reflector_client: false,
})
.expect("room for the peer")
})
crates/bench/benches/transit.rs
-146
View File
@@ -1,146 +0,0 @@
//! What passing routes on costs: one peer sends the table over a session,
//! and another is told of every route, over its own. The time is from the
//! first byte sent to the last route heard; alongside, the time for the
//! same table to be in our own table and no further.
use std::net::Ipv4Addr;
use std::sync::Arc;
use std::time::{Duration, Instant};
use bgp_bench::{Prefixes, Shape, announcements, encoded, env};
use bgp_integration_tests::test_peer::TestPeer;
use bgp_integration_tests::test_server::TestServer;
use bgp_packet::message::PathAttribute;
use bgp_server::marks::{Control, Marks};
use bgp_server::policy::ExportPolicy;
use bytes::BytesMut;
/// How a server passes routes on in one run.
#[derive(Clone, Copy)]
enum Passing {
Not,
Plain,
/// With the communities that say who is told, and a mark put on every
/// route.
Dressed,
}
/// Sends the table from one peer and, where it is passed on, hears all of
/// it at another. Returns when the table was in, and when it was all
/// heard.
fn run(
wire: &[BytesMut],
routes: usize,
heard_routes: usize,
passing: Passing,
) -> (Duration, Option<Duration>) {
let runtime = tokio::runtime::Builder::new_multi_thread()
.enable_all()
.build()
.expect("a runtime");
runtime.block_on(async {
let server = TestServer::start_with(4_200_000_000, |config| {
config.export = Arc::new(ExportPolicy {
learned_routes: !matches!(passing, Passing::Not),
all_paths: false,
marks: match passing {
Passing::Dressed => Marks::parse(&[], &["64601:7".to_string()], &[]).unwrap(),
_ => Marks::default(),
},
prepend: 0,
});
config.control = matches!(passing, Passing::Dressed).then_some(Control {
local_asn: 4_200_000_000,
});
})
.await
.expect("a server");
let mut hearer = TestPeer::connect(server.addr).await.expect("a connection");
hearer
// An AS number that is on none of the generated paths: a route
// is not passed to a peer whose AS it has been through.
.handshake(4_100_000_000, Ipv4Addr::new(10, 0, 0, 100), 90)
.await
.expect("a session");
let mut sender = TestPeer::connect(server.addr).await.expect("a connection");
sender
.handshake(64513, Ipv4Addr::new(10, 0, 0, 1), 90)
.await
.expect("a session");
let stream: Vec<u8> = wire.iter().flat_map(|m| m.iter().copied()).collect();
let started = Instant::now();
let sending = tokio::spawn(async move {
sender.send_raw(&stream).await.expect("the server reads");
sender
});
let hearing = tokio::spawn(async move {
if matches!(passing, Passing::Not) {
return (hearer, None, 0);
}
let mut heard = 0usize;
let mut messages = 0usize;
while heard < heard_routes {
let update = hearer.recv_update().await.expect("an UPDATE");
messages += 1;
heard += update.announced_routes.len();
heard += update
.path_attributes
.iter()
.map(|attribute| match attribute {
PathAttribute::MpReachNlri(mp) => mp.prefixes.len(),
_ => 0,
})
.sum::<usize>();
}
(hearer, Some(started.elapsed()), messages)
});
while server.rib.len() < routes {
tokio::time::sleep(Duration::from_millis(2)).await;
}
let in_table = started.elapsed();
let (hearer, heard, messages) = hearing.await.expect("the hearer");
if messages > 0 {
println!(
" ({messages} UPDATEs, {:.1} routes each)",
heard_routes as f64 / messages as f64
);
}
drop(sending.await);
drop(hearer);
(in_table, heard)
})
}
fn main() {
let shape = Shape::INTERNET.scaled_down(env("BENCH_SCALE_DOWN", 1));
let prefixes = Prefixes::generate(shape);
let wire = encoded(&announcements(&prefixes, shape, 1));
let routes = shape.prefixes();
// Over a session on IPv4 only the IPv4 routes go.
let heard_routes = shape.ipv4_prefixes;
println!(
"== One peer sends {routes} routes in {} UPDATEs over TCP; another, over IPv4, is told of the {heard_routes} IPv4 ones",
wire.len()
);
for (what, passing) in [
("not passed on", Passing::Not),
("passed on", Passing::Plain),
(
"passed on, with control communities and a mark",
Passing::Dressed,
),
] {
let (in_table, heard) = run(&wire, routes, heard_routes, passing);
let heard = heard.map_or(String::new(), |heard| {
format!(
" all heard by the other peer after {:>6.3} s ({:.2} M routes/s)",
heard.as_secs_f64(),
heard_routes as f64 / heard.as_secs_f64() / 1e6
)
});
println!(
"{what:<48} in the table after {:>6.3} s{heard}",
in_table.as_secs_f64()
);
}
}
crates/bench/examples/feed.rs
-181
View File
@@ -1,181 +0,0 @@
//! Feeds the synthetic Internet routing table (see the crate documentation)
//! to a running daemon over TCP, as one or more peers would, and holds the
//! sessions open afterwards. This is the load for profiling the daemon as
//! it takes in a full table; `scripts/profile/full-table.sh` runs it under
//! `perf`.
//!
//! Settings, from the environment:
//!
//! - `FEED_TO`: where the daemon listens (default `127.0.0.1:1790`).
//! - `FEED_PEERS`: how many peers send the table (default 1).
//! - `FEED_SCALE_DOWN`: as `BENCH_SCALE_DOWN`; 10 makes the table ten
//! times smaller (default 1).
//! - `FEED_FROM`: the first three octets of an IPv4 /24, "192.0.2". Peer N
//! connects from .N, which is also the next hop of its IPv4 routes.
//! Without it, peers connect from wherever the system puts them.
//! - `FEED_ASN`: the first peer's AS number (default 64513); the others
//! follow on from it.
//! - `FEED_HOLD_SECS`: how long the sessions stay open after the table is
//! sent (default 30).
//! - `FEED_COLLECTOR=1`: before the peers, one more session, as a route
//! collector would have: over IPv6, to `FEED_COLLECTOR_TO` (default
//! `[::1]:1790`) from `FEED_COLLECTOR_FROM` if set, offering to receive
//! several routes for a prefix (ADD-PATH) for both families and IPv4
//! routes with IPv6 next hops, and reading and dropping whatever it is
//! sent. Its AS number is 212232.
//!
//! What is printed: when the sessions are up and how long the writes took.
//! The writes end when the kernel's socket buffers have taken the last
//! bytes, which is before the daemon has read them: when the table is in
//! is for the daemon's API to say.
use std::net::{Ipv4Addr, SocketAddr};
use std::time::{Duration, Instant};
use bgp_bench::report::mib;
use bgp_bench::{Prefixes, Shape, env, table_stream};
use bgp_integration_tests::test_peer::{TestPeer, collector_capabilities};
use bgp_primitives::constants::AfiSafi;
use tokio::io::AsyncWriteExt;
use tokio::net::TcpSocket;
use tokio::net::tcp::OwnedWriteHalf;
/// How often KEEPALIVEs go out while the sessions are held open: well
/// inside the 90 s hold time.
const KEEPALIVE_EVERY: Duration = Duration::from_secs(20);
#[tokio::main]
async fn main() {
let to: SocketAddr = std::env::var("FEED_TO")
.unwrap_or_else(|_| "127.0.0.1:1790".into())
.parse()
.expect("FEED_TO is an address and port");
let peers = env("FEED_PEERS", 1) as u32;
let shape = Shape::INTERNET.scaled_down(env("FEED_SCALE_DOWN", 1));
let from = std::env::var("FEED_FROM").ok();
let first_asn = env("FEED_ASN", 64513) as u32;
let hold = Duration::from_secs(env("FEED_HOLD_SECS", 30) as u64);
let started = Instant::now();
let prefixes = Prefixes::generate(shape);
let tables: Vec<Vec<u8>> = (1..=peers)
.map(|peer| table_stream(&prefixes, shape, peer))
.collect();
let routes = shape.prefixes() * peers as usize;
let bytes: usize = tables.iter().map(Vec::len).sum();
println!(
"table: {} IPv4 + {} IPv6 prefixes from each of {peers} peer(s): {routes} routes, {:.1} MiB on the wire, made in {:.2} s",
shape.ipv4_prefixes,
shape.ipv6_prefixes,
mib(bytes),
started.elapsed().as_secs_f64()
);
// The collector first, so that it is there as the table arrives, as a
// router's collector session is after a restart.
let collector = env("FEED_COLLECTOR", 0) == 1;
let mut draining = Vec::new();
let mut collector_writer = None;
if collector {
let to: SocketAddr = std::env::var("FEED_COLLECTOR_TO")
.unwrap_or_else(|_| "[::1]:1790".into())
.parse()
.expect("FEED_COLLECTOR_TO is an IPv6 address and port");
let socket = TcpSocket::new_v6().expect("a socket");
if let Ok(from) = std::env::var("FEED_COLLECTOR_FROM") {
let address: std::net::Ipv6Addr = from.parse().expect("an IPv6 address");
socket
.bind(SocketAddr::new(address.into(), 0))
.expect("the collector's address is ours to connect from");
}
let stream = socket.connect(to).await.expect("the daemon listens");
let mut session = TestPeer::from_stream(stream);
session
.handshake_with(
212_232,
Ipv4Addr::new(10, 0, 0, 200),
90,
collector_capabilities(
212_232,
&[AfiSafi::IPV4_UNICAST, AfiSafi::IPV6_UNICAST],
true,
),
)
.await
.expect("the daemon takes the collector's session");
let (writer, sink) = session.drain();
draining.push(sink);
println!("collector session established");
collector_writer = Some(writer);
}
let mut sessions = Vec::new();
for peer in 1..=peers {
let socket = TcpSocket::new_v4().expect("a socket");
if let Some(from) = &from {
let address: Ipv4Addr = format!("{from}.{peer}")
.parse()
.expect("FEED_FROM is the first three octets of an IPv4 address");
socket
.bind(SocketAddr::new(address.into(), 0))
.expect("the peer's address is ours to connect from");
}
let stream = socket.connect(to).await.expect("the daemon listens");
let mut session = TestPeer::from_stream(stream);
session
.handshake(
first_asn + peer - 1,
Ipv4Addr::new(10, 0, 0, peer as u8),
90,
)
.await
.expect("the daemon takes the session");
// Whatever the daemon sends back, KEEPALIVEs as a rule, is read
// and dropped, so that the writer is never held up.
let (writer, sink) = session.drain();
draining.push(sink);
sessions.push(writer);
}
println!("{peers} session(s) established; sending");
let started = Instant::now();
let senders: Vec<_> = sessions
.into_iter()
.zip(tables)
.map(|(mut writer, table)| {
tokio::spawn(async move {
let started = Instant::now();
writer.write_all(&table).await.expect("the daemon reads");
(writer, started.elapsed())
})
})
.collect();
let mut writers: Vec<OwnedWriteHalf> = collector_writer.into_iter().collect();
for (index, sender) in senders.into_iter().enumerate() {
let (writer, took) = sender.await.expect("the sender runs");
println!(
"peer {}: table written in {:.3} s",
index + 1,
took.as_secs_f64()
);
writers.push(writer);
}
let elapsed = started.elapsed();
println!(
"all {routes} routes written in {:.3} s: {:.2} M routes/s into the sockets",
elapsed.as_secs_f64(),
routes as f64 / elapsed.as_secs_f64() / 1e6
);
let keepalive = TestPeer::keepalive_bytes();
let deadline = Instant::now() + hold;
while Instant::now() < deadline {
tokio::time::sleep(KEEPALIVE_EVERY.min(deadline - Instant::now())).await;
for writer in &mut writers {
// A session the daemon closed is left alone.
let _ = writer.write_all(&keepalive).await;
}
}
println!("sessions closed after {} s", hold.as_secs());
drop(draining);
}
crates/bench/examples/packing.rs
-71
View File
@@ -1,71 +0,0 @@
//! How many UPDATEs, and how many octets, a table takes when the routes
//! that share attributes are grouped only within slices of the table
//! taken in shard order, for a few slice sizes: what streaming a table
//! out in slices costs against grouping it whole.
use std::collections::HashMap;
use std::hash::{BuildHasher, RandomState};
use std::path::PathBuf;
use bgp_bench::mrt::Dump;
fn main() {
let path = std::env::var("BENCH_MRT")
.map(PathBuf::from)
.unwrap_or_else(|_| "crates/bench/data/rrc03-bview.20260915.0000.gz".into());
let dump = Dump::read(&path).expect("a readable dump");
let peers = dump.largest_peers(usize::MAX);
let full: Vec<usize> = peers
.iter()
.copied()
.filter(|&p| dump.routes[p].len() > 800_000)
.collect();
println!("{} peers with a full table", full.len());
let hasher = RandomState::new();
for (what, chosen) in [
("one full table", vec![full[0]]),
("every full table, every path", full.clone()),
] {
// (hash of prefix, attribute set, prefix octets) in shard order.
let mut routes: Vec<(u64, u32, usize)> = chosen
.iter()
.flat_map(|&p| dump.routes[p].iter())
.map(|(prefix, set)| {
(
hasher.hash_one(prefix),
*set,
1 + usize::from(prefix.length()).div_ceil(8) + 4,
)
})
.collect();
routes.sort_unstable_by_key(|r| r.0);
println!("== {what}: {} routes", routes.len());
for slice in [4_096usize, 16_384, 65_536, 262_144, usize::MAX] {
let mut updates = 0usize;
let mut octets = 0usize;
for chunk in routes.chunks(slice.min(routes.len())) {
let mut groups: HashMap<u32, usize> = HashMap::new();
for &(_, set, prefix_octets) in chunk {
*groups.entry(set).or_insert(0) += prefix_octets;
}
for (set, prefix_octets) in groups {
// One UPDATE per 4096 octets of prefixes, roughly.
let attrs = dump.attribute_sets[set as usize].len();
let messages = prefix_octets.div_ceil(4000 - attrs.min(3000)).max(1);
updates += messages;
octets += messages * (23 + attrs) + prefix_octets;
}
}
println!(
" slice {:>9}: {:>9} UPDATEs, {:>5.1} MB on the wire",
if slice == usize::MAX {
"whole".to_string()
} else {
slice.to_string()
},
updates,
octets as f64 / 1e6
);
}
}
}
crates/bench/src/lib.rs
+5 -98
View File
@@ -1,10 +1,9 @@
//! A synthetic copy of the Internet routing table, as the UPDATE messages a
//! peer would send to announce it.
//!
//! It is made up, not captured, so that the benchmarks of a synthetic table
//! need no data files and give the same input every run; the one of a real
//! table reads an MRT dump instead, see [`mrt`]. It is shaped like the real
//! thing where that matters for cost:
//! It is made up, not captured, so that the benchmarks need no data files
//! and give the same input every run. It is shaped like the real thing where
//! that matters for cost:
//!
//! - about a million IPv4 prefixes, most of them /24s, and about a quarter
//! of a million IPv6 prefixes, most of them /48s;
@@ -16,14 +15,6 @@
pub mod memory;
pub mod mrt;
pub mod report;
// The benchmarks allocate as the daemon does (bin/bgp/src/main.rs), with
// jemalloc; the C library's allocator would put its own locks in the
// profile. Not when memory is counted: that needs the counting one.
#[cfg(all(not(feature = "count-memory"), not(target_env = "msvc")))]
#[global_allocator]
static ALLOCATOR: tikv_jemallocator::Jemalloc = tikv_jemallocator::Jemalloc;
use std::collections::HashSet;
use std::net::{Ipv4Addr, Ipv6Addr};
@@ -172,7 +163,7 @@ pub fn announcements(prefixes: &Prefixes, shape: Shape, peer: u32) -> Vec<Update
let hops = 2 + rng.below(4) as usize;
let mut path = vec![peer_asn];
path.extend((0..hops).map(|_| 1 + rng.below(400_000) as u32));
let attributes = vec![
let mut attributes = vec![
PathAttribute::Origin(OriginPathAttribute::IGP),
PathAttribute::AsPath(AsPathAttribute {
segments: vec![AsPathSegment {
@@ -208,7 +199,7 @@ pub fn announcements(prefixes: &Prefixes, shape: Shape, peer: u32) -> Vec<Update
} else {
None
};
(attributes, later, large)
(std::mem::take(&mut attributes), later, large)
};
let mut updates = Vec::new();
@@ -267,16 +258,6 @@ pub fn announcements(prefixes: &Prefixes, shape: Shape, peer: u32) -> Vec<Update
updates
}
/// The whole table as the peer numbered `peer` sends it, one stream of
/// bytes: [`announcements`], [`encoded`] and run together, for a session
/// that writes the table in one go.
pub fn table_stream(prefixes: &Prefixes, shape: Shape, peer: u32) -> Vec<u8> {
encoded(&announcements(prefixes, shape, peer))
.iter()
.flat_map(|message| message.iter().copied())
.collect()
}
/// `updates` as they are on the wire, on a session with four-octet AS
/// numbers.
pub fn encoded(updates: &[UpdateMessage]) -> Vec<BytesMut> {
@@ -302,77 +283,3 @@ pub fn wire_context() -> ParserContext {
add_path: Vec::new(),
}
}
/// The value of the environment variable `name` as a number, or `default`
/// when it is not set or is not one: how the benchmarks take their size.
pub fn env(name: &str, default: usize) -> usize {
std::env::var(name)
.ok()
.and_then(|v| v.parse().ok())
.unwrap_or(default)
}
#[cfg(test)]
mod tests {
use super::*;
/// A small table, enough for every prefix length and attribute to
/// come up.
fn shape() -> Shape {
Shape::INTERNET.scaled_down(1000)
}
/// The benchmarks are compared from one run to the next, so the table
/// they are fed must not change between runs: the generator is
/// seeded, and nothing in it depends on where things are in memory.
#[test]
fn the_table_is_the_same_every_run() {
let shape = shape();
let once = Prefixes::generate(shape);
let again = Prefixes::generate(shape);
assert_eq!(once.ipv4, again.ipv4);
assert_eq!(once.ipv6, again.ipv6);
assert_eq!(once.ipv4.len(), shape.ipv4_prefixes);
assert_eq!(once.ipv6.len(), shape.ipv6_prefixes);
for peer in [1, 2] {
assert_eq!(
table_stream(&once, shape, peer),
table_stream(&again, shape, peer)
);
}
assert_ne!(
table_stream(&once, shape, 1),
table_stream(&once, shape, 2),
"each peer has attributes of its own"
);
}
/// What a peer of the benchmarks sends is well formed, and says what
/// it was meant to: every UPDATE reads back as it was made.
#[test]
fn every_announcement_decodes_to_itself() {
let shape = shape();
let prefixes = Prefixes::generate(shape);
let updates = announcements(&prefixes, shape, 1);
let ctx = wire_context();
let announced: usize = updates
.iter()
.map(|update| {
update.announced_routes.len()
+ update
.path_attributes
.iter()
.map(|attribute| match attribute {
PathAttribute::MpReachNlri(reach) => reach.prefixes.len(),
_ => 0,
})
.sum::<usize>()
})
.sum();
assert_eq!(announced, shape.prefixes());
for (update, wire) in updates.iter().zip(encoded(&updates)) {
let decoded = Message::decode(&ctx, &wire).expect("well formed");
assert_eq!(decoded, Message::Update(update.clone()));
}
}
}
crates/bench/src/mrt.rs
+26 -16
View File
@@ -10,7 +10,7 @@ use std::io::{self, BufReader, Read};
use std::net::{IpAddr, Ipv4Addr, Ipv6Addr};
use std::path::Path;
use bgp_packet::message::{NlriNextHop, PathAttribute, attr_type, split_attribute};
use bgp_packet::message::{NlriNextHop, PathAttribute};
use bgp_primitives::constants::IpVersion;
use bgp_primitives::ip_prefix::IpPrefix;
use flate2::read::MultiGzDecoder;
@@ -21,6 +21,7 @@ const TABLE_DUMP_V2: u16 = 13;
const PEER_INDEX_TABLE: u16 = 1;
const RIB_IPV4_UNICAST: u16 = 2;
const RIB_IPV6_UNICAST: u16 = 4;
const MP_REACH_NLRI: u8 = 14;
/// A peer of the collector.
#[derive(Debug, Clone)]
@@ -204,23 +205,32 @@ pub fn parse_attributes(mut wire: &[u8]) -> Option<(Vec<PathAttribute>, Option<N
let mut attributes = Vec::new();
let mut next_hop = None;
while !wire.is_empty() {
let (raw, rest) = split_attribute(wire).ok()?;
if raw.type_code == attr_type::MP_REACH_NLRI {
// The value is the next hop's length and the next hop, and
// nothing of the family, reserved octet and prefixes that
// follow in an UPDATE, so the attribute's own parser would
// refuse it.
let (length, address) = raw.value.split_first()?;
if usize::from(*length) != address.len() {
return None;
}
next_hop = Some(NlriNextHop::from_bytes(address)?);
} else {
let ([], attribute) = PathAttribute::from_wire(&ctx, raw.wire).ok()? else {
return None;
if wire.get(1) == Some(&MP_REACH_NLRI) {
let extended = wire[0] & 0x10 != 0;
let (length, header) = if extended {
(
usize::from(u16::from_be_bytes([*wire.get(2)?, *wire.get(3)?])),
4,
)
} else {
(usize::from(*wire.get(2)?), 3)
};
attributes.push(attribute);
let value = wire.get(header..header + length)?;
let address = |bytes: &[u8]| Some(Ipv6Addr::from(<[u8; 16]>::try_from(bytes).ok()?));
next_hop = match value {
[4, a, b, c, d] => Some(NlriNextHop::Ipv4(Ipv4Addr::new(*a, *b, *c, *d))),
[16, rest @ ..] => Some(NlriNextHop::Ipv6(address(rest)?)),
[32, rest @ ..] if rest.len() == 32 => Some(NlriNextHop::Ipv6WithLl {
global: address(&rest[..16])?,
link_local: address(&rest[16..])?,
}),
_ => return None,
};
wire = &wire[header + length..];
continue;
}
let (rest, attribute) = PathAttribute::from_wire(&ctx, wire).ok()?;
attributes.push(attribute);
wire = rest;
}
Some((attributes, next_hop))
crates/bench/src/report.rs
-31
View File
@@ -1,31 +0,0 @@
//! How the benchmarks print what they measured: one row a measurement,
//! what was measured in a column wide enough for the longest of them, the
//! time, and whatever else there is to say about it. The rows are what a
//! run is compared against, so their shape is settled here once.
use std::time::Duration;
/// How wide the column that says what was measured is. The widest row is
/// the mesh benchmark's, so that is what every benchmark lines up to.
pub const WIDTH: usize = 52;
/// `bytes` in mebibytes.
pub fn mib(bytes: usize) -> f64 {
bytes as f64 / (1024.0 * 1024.0)
}
/// `count` things in `elapsed`, as a rate: millions a second where there
/// are that many, thousands otherwise.
pub fn per_second(count: usize, elapsed: Duration) -> String {
let rate = count as f64 / elapsed.as_secs_f64();
if rate >= 1e6 {
format!("{:.2} M/s", rate / 1e6)
} else {
format!("{:.0} k/s", rate / 1e3)
}
}
/// Prints one row: `what` took `elapsed`, and `detail` says more.
pub fn row(what: &str, elapsed: Duration, detail: impl std::fmt::Display) {
println!("{what:<WIDTH$} {:>8.3} s {detail}", elapsed.as_secs_f64());
}
crates/fib/Cargo.toml
-32
View File
@@ -1,32 +0,0 @@
[package]
name = "bgp-fib"
authors.workspace = true
edition.workspace = true
homepage.workspace = true
license.workspace = true
repository.workspace = true
version.workspace = true
[dependencies]
bgp-packet.workspace = true
bgp-primitives.workspace = true
bgp-rib.workspace = true
bytes.workspace = true
foldhash.workspace = true
prometheus.workspace = true
thiserror.workspace = true
tokio = { workspace = true, features = ["full"] }
tracing.workspace = true
# The socket to the kernel. Only Linux has it; everything else in this
# crate builds, and is tested, anywhere.
[target.'cfg(target_os = "linux")'.dependencies]
netlink-sys = { version = "0.8", features = ["tokio_socket"] }
# For a socket option netlink-sys has no name for.
nix = { workspace = true, features = ["socket"] }
[dev-dependencies]
proptest.workspace = true
[lints]
workspace = true
crates/fib/src/interfaces.rs
-286
View File
@@ -1,286 +0,0 @@
//! The machine's interfaces and their addresses, as the kernel reports
//! them, and what they say about a next hop: whether it can be reached
//! directly, and from which interface.
use std::collections::{BTreeMap, HashSet};
use std::net::IpAddr;
use bgp_primitives::constants::IpVersion;
use bgp_primitives::ip_prefix::IpPrefix;
use crate::InterfaceStatus;
use crate::netlink::{Address, Link, Message};
/// The interface a neighbor is reached on.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Connected {
/// The interface's index.
pub interface: u32,
/// Our own address in the network the neighbor is in.
pub local: IpAddr,
}
/// The interfaces there are and the addresses they have.
#[derive(Debug, Clone, Default)]
pub struct Interfaces {
/// By index, which is the order they are given out in too.
links: BTreeMap<u32, Link>,
addresses: HashSet<Address>,
}
/// `address` as the prefix that is only it.
fn host(address: IpAddr) -> IpPrefix {
match address {
IpAddr::V4(v4) => IpPrefix::new(IpVersion::Ipv4, v4.octets(), 32),
IpAddr::V6(v6) => IpPrefix::new(IpVersion::Ipv6, v6.octets(), 128),
}
.expect("an address is a prefix of its own length")
}
impl Interfaces {
/// Takes in what a message from the kernel says about links and
/// addresses. Returns whether anything is different for it.
pub fn apply(&mut self, message: &Message) -> bool {
match message {
Message::Link {
present: true,
link,
} => self.links.insert(link.index, link.clone()).as_ref() != Some(link),
Message::Link {
present: false,
link,
} => {
self.addresses
.retain(|address| address.interface != link.index);
self.links.remove(&link.index).is_some()
}
Message::Address {
present: true,
address,
} => self.addresses.insert(*address),
Message::Address {
present: false,
address,
} => self.addresses.remove(address),
_ => false,
}
}
/// The interfaces, in order of index.
pub fn links(&self) -> impl Iterator<Item = &Link> {
self.links.values()
}
/// The interfaces and their addresses, for showing: in order of index,
/// each with its addresses in order.
pub fn status(&self) -> Vec<InterfaceStatus> {
self.links()
.map(|link| {
let mut addresses: Vec<(IpAddr, u8)> = self
.addresses_of(link.index)
.map(|address| (address.local, address.network.length()))
.collect();
addresses.sort();
InterfaceStatus {
index: link.index,
name: link.name.clone(),
up: link.up,
addresses,
}
})
.collect()
}
pub fn link(&self, index: u32) -> Option<&Link> {
self.links.get(&index)
}
/// The addresses of the interface with this index.
pub fn addresses_of(&self, interface: u32) -> impl Iterator<Item = &Address> {
self.addresses
.iter()
.filter(move |address| address.interface == interface)
}
/// Whether `address` is one of this machine's own.
pub fn is_local(&self, address: IpAddr) -> bool {
self.addresses.iter().any(|ours| ours.local == address)
}
/// Whether `prefix` is the network of one of this machine's addresses,
/// or inside one. Whether the interface is up does not matter: the
/// network is ours either way.
pub fn has_network_around(&self, prefix: &IpPrefix) -> bool {
self.addresses
.iter()
.any(|address| address.network.contains(prefix))
}
/// Where `neighbor` can be reached without a gateway: the interface
/// that is up and has an address in a network `neighbor` is in, the
/// most specific such network if there are several. `None` if there is
/// none, or if `neighbor` is this machine itself.
pub fn connected(&self, neighbor: IpAddr) -> Option<Connected> {
if self.is_local(neighbor) {
return None;
}
let neighbor = host(neighbor);
self.addresses
.iter()
.filter(|address| address.network.contains(&neighbor))
.filter(|address| {
self.links
.get(&address.interface)
.is_some_and(|link| link.up)
})
.max_by_key(|address| address.network.length())
.map(|address| Connected {
interface: address.interface,
local: address.local,
})
}
}
#[cfg(test)]
mod tests {
use super::*;
fn link(index: u32, name: &str, up: bool) -> Message {
Message::Link {
present: true,
link: Link {
index,
name: name.into(),
up,
},
}
}
fn address(interface: u32, local: &str, network: &str) -> Address {
Address {
interface,
local: local.parse().unwrap(),
network: IpPrefix::try_from(network).unwrap(),
}
}
fn present(address: Address) -> Message {
Message::Address {
present: true,
address,
}
}
fn some() -> Interfaces {
let mut interfaces = Interfaces::default();
for message in [
link(1, "lo", true),
link(2, "eth0", true),
link(3, "eth1", true),
link(4, "eth2", false),
present(address(1, "127.0.0.1", "127.0.0.0/8")),
present(address(2, "192.0.2.1", "192.0.2.0/24")),
present(address(2, "2001:db8::1", "2001:db8::/64")),
// A more specific network inside eth0's, on eth1.
present(address(3, "192.0.2.129", "192.0.2.128/25")),
present(address(4, "198.51.100.1", "198.51.100.0/24")),
] {
assert!(interfaces.apply(&message));
}
interfaces
}
fn connected(interfaces: &Interfaces, neighbor: &str) -> Option<(u32, String)> {
interfaces
.connected(neighbor.parse().unwrap())
.map(|c| (c.interface, c.local.to_string()))
}
#[test]
fn a_neighbor_is_reached_from_the_interface_whose_network_it_is_in() {
let interfaces = some();
assert_eq!(
connected(&interfaces, "192.0.2.7"),
Some((2, "192.0.2.1".into()))
);
assert_eq!(
connected(&interfaces, "2001:db8::7"),
Some((2, "2001:db8::1".into()))
);
// The most specific network wins.
assert_eq!(
connected(&interfaces, "192.0.2.200"),
Some((3, "192.0.2.129".into()))
);
// Not in any network of ours.
assert_eq!(connected(&interfaces, "203.0.113.1"), None);
assert_eq!(connected(&interfaces, "2001:db8:1::1"), None);
// In the network of an interface that is down.
assert_eq!(connected(&interfaces, "198.51.100.7"), None);
// Ourselves.
assert_eq!(connected(&interfaces, "192.0.2.1"), None);
assert!(interfaces.is_local("192.0.2.1".parse().unwrap()));
assert!(!interfaces.is_local("192.0.2.7".parse().unwrap()));
}
#[test]
fn what_the_kernel_reports_changes_what_is_reachable() {
let mut interfaces = some();
// Told again what is known: nothing changes.
assert!(!interfaces.apply(&link(2, "eth0", true)));
assert!(!interfaces.apply(&present(address(2, "192.0.2.1", "192.0.2.0/24"))));
// An interface comes up.
assert!(interfaces.apply(&link(4, "eth2", true)));
assert_eq!(
connected(&interfaces, "198.51.100.7"),
Some((4, "198.51.100.1".into()))
);
// An address goes.
assert!(interfaces.apply(&Message::Address {
present: false,
address: address(3, "192.0.2.129", "192.0.2.128/25"),
}));
assert_eq!(
connected(&interfaces, "192.0.2.200"),
Some((2, "192.0.2.1".into()))
);
// An interface goes, and its addresses with it.
assert!(interfaces.apply(&Message::Link {
present: false,
link: Link {
index: 2,
name: "eth0".into(),
up: true
},
}));
assert_eq!(connected(&interfaces, "192.0.2.7"), None);
assert_eq!(interfaces.addresses_of(2).count(), 0);
// What is not about interfaces changes nothing.
assert!(!interfaces.apply(&Message::Done));
}
#[test]
fn the_interfaces_are_shown_in_order() {
let mut interfaces = some();
// Learned last, listed in its place.
assert!(interfaces.apply(&link(0, "dummy0", false)));
let shown = interfaces.status();
let indexes: Vec<u32> = shown.iter().map(|interface| interface.index).collect();
assert_eq!(indexes, [0, 1, 2, 3, 4]);
assert_eq!(
shown[2],
InterfaceStatus {
index: 2,
name: "eth0".into(),
up: true,
addresses: vec![
("192.0.2.1".parse().unwrap(), 24),
("2001:db8::1".parse().unwrap(), 64),
],
}
);
assert_eq!((shown[4].up, shown[4].addresses.len()), (false, 1));
assert!(shown[0].addresses.is_empty());
}
}
crates/fib/src/kernel.rs
-1349
View File
File diff suppressed because it is too large. Load diff
crates/fib/src/lib.rs
-331
View File
@@ -1,331 +0,0 @@
//! Putting the best routes into the kernel's routing table, so that the
//! machine forwards packets by them (Linux, over rtnetlink).
//!
//! The routing table chooses a best route for each prefix. This crate
//! follows those choices and keeps the kernel's table the same: one route
//! for each prefix, to the best route's next hop. It also learns from the
//! kernel what interfaces and addresses the machine has, which is what
//! says whether a next hop can be reached at all.
//!
//! - [`netlink`] is the kernel's messages, as bytes and back.
//! - [`interfaces`] is the machine's interfaces and what is reachable on
//! them; [`routes`] the kernel's own routes, which say the way to what
//! is not; [`topology`] the two together with the kernel's nexthop
//! objects, as one view.
//! - [`plan`] works out what to ask the kernel for.
//! - The rest, which only Linux has, asks it.
//!
//! # What is and is not touched
//!
//! Our routes are marked as BGP's ([`netlink::PROTOCOL_BGP`]) and carry
//! the configured metric, in the configured table. Only routes with that
//! mark, metric and table are ever replaced or removed; and all such
//! routes are taken to be ours, so two daemons must not share them.
//! Routes left behind by a run that ended badly are found at the next
//! start, and removed unless they are still wanted. A run that ends in
//! good order removes its routes.
//!
//! # What is installed
//!
//! A best route whose next hop can be reached: a neighbor, which is an
//! address in the network of one of the machine's interfaces, or an
//! address the kernel's own routes know the way to. In the second case
//! the route goes into the kernel with the gateway and interface of the
//! route that knows the way, since the kernel does not look a gateway up
//! in turn: a route whose next hop is a router of our own AS, reached over
//! a tunnel by a static route, is installed as a route into the tunnel.
//! Which of the kernel's routes count for this, and which never do, is the
//! business of [`routes`].
pub mod interfaces;
#[cfg(target_os = "linux")]
mod kernel;
pub mod netlink;
pub mod plan;
pub mod routes;
pub mod sockdiag;
pub mod topology;
use std::net::IpAddr;
use std::sync::{Arc, Mutex, PoisonError};
use std::time::Duration;
use bgp_rib::table::RibTable;
use tokio::sync::watch;
use tokio::task::JoinHandle;
/// The metric our routes get unless configured otherwise: the one other
/// routing daemons give routes from external BGP peers, so that ours rank
/// against static and connected routes as an operator expects.
pub const DEFAULT_METRIC: u32 = 20;
/// How long routes left by an earlier run are kept unless configured
/// otherwise: time for sessions to come up and tables to be sent.
pub const DEFAULT_STALE_TIME: Duration = Duration::from_secs(120);
/// Where our routes go in the kernel.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct FibConfig {
/// The routing table: [`netlink::TABLE_MAIN`] to forward by them.
pub table: u32,
/// The metric, which with the table says which routes are ours.
pub metric: u32,
/// Have routes name nexthop objects where the kernel has them (Linux
/// 5.3 and later), which makes installing many routes a good deal
/// faster. Without, every route carries its own gateway.
pub nexthop_objects: bool,
/// How long routes of ours that an earlier run left in the kernel are
/// kept after a start without the routing table having them. The
/// table is empty until the peers have sent theirs again; removing
/// everything at once would stop the machine forwarding meanwhile.
/// After this long, what has not been announced again goes.
pub stale_time: Duration,
/// The kernel's table whose routes say the way to a next hop that is
/// not a neighbor: normally `main`, with the connected and static
/// routes. BGP's own routes in it never count; see [`routes`].
pub resolution_table: u32,
/// Whether the default route counts as the way to a next hop.
pub resolve_via_default: bool,
}
impl Default for FibConfig {
fn default() -> Self {
Self {
table: netlink::TABLE_MAIN,
metric: DEFAULT_METRIC,
nexthop_objects: true,
stale_time: DEFAULT_STALE_TIME,
resolution_table: netlink::TABLE_MAIN,
resolve_via_default: false,
}
}
}
/// Why routes cannot be kept in the kernel.
#[derive(Debug, thiserror::Error)]
pub enum FibError {
#[error("installing routes in the kernel is only supported on Linux")]
Unsupported,
#[error("the kernel's routing socket failed: {0}")]
Io(#[from] std::io::Error),
#[error("the kernel sent something that is not netlink: {0}")]
Malformed(#[from] netlink::NetlinkError),
#[error("the kernel refused to list {what}: error {errno}")]
Dump { what: &'static str, errno: i32 },
#[error(
"not permitted to change the kernel's routes: the daemon needs the CAP_NET_ADMIN \
capability (or root)"
)]
NotPermitted,
}
/// One of the machine's interfaces, for showing.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct InterfaceStatus {
pub index: u32,
pub name: String,
pub up: bool,
/// Its addresses, each with the length of the network it is in.
pub addresses: Vec<(IpAddr, u8)>,
}
/// How things stand with the kernel, for showing.
#[derive(Debug, Clone, Default)]
pub struct Status {
/// Whether the kernel has been brought in step with the routing table
/// at least once since the start.
pub synced: bool,
/// How many routes of ours the kernel has.
pub installed: usize,
/// How many of them are kept from an earlier run though the routing
/// table does not have them (yet); see [`FibConfig::stale_time`].
pub kept: usize,
/// How many of them are blackhole routes, which drop the packets for
/// their prefix.
pub blackholes: usize,
/// Prefixes whose best route is not in the kernel because its next
/// hop is not usable, by why.
pub unresolved: Vec<(&'static str, usize)>,
/// Prefixes whose best route the kernel refused.
pub refused: usize,
/// How many requests the kernel has refused in all.
pub errors: u64,
/// The last such refusal.
pub last_error: Option<String>,
pub interfaces: Vec<InterfaceStatus>,
}
/// The machine's interfaces, routes and nexthop objects as the kernel
/// last described them, as far as next hops are concerned; `None` until
/// it has. Whoever holds one is told when they change.
pub type TopologyWatch = watch::Receiver<Option<Arc<topology::Topology>>>;
/// How the task that keeps the kernel's routes is to end.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum Stop {
/// Taking our routes out of the kernel.
Withdraw,
/// Leaving them there, for the next run to find.
Keep,
}
struct Shared {
config: FibConfig,
status: Mutex<Status>,
topology: watch::Sender<Option<Arc<topology::Topology>>>,
stop: watch::Sender<Option<Stop>>,
/// How many routes of an earlier run were found in the kernel at the
/// start, once that has been looked at.
adopted: watch::Sender<Option<usize>>,
/// Set to have what is kept of an earlier run go now.
release: watch::Sender<bool>,
/// Whether the task has ended.
ended: watch::Sender<bool>,
}
/// Handle to the task that keeps the kernel's routes. Cheap to clone.
#[derive(Clone)]
pub struct Fib {
shared: Arc<Shared>,
}
impl Fib {
/// Starts keeping the kernel's routes in step with the best routes of
/// `rib`, and returns a handle with the task that does it. The task
/// ends only by failing, or when [`Fib::stop`] or [`Fib::leave`] is
/// called.
///
/// Must be called from within a Tokio runtime. Fails at once where
/// there is no kernel to talk to in this way.
pub fn spawn(
rib: Arc<RibTable>,
config: FibConfig,
) -> Result<(Self, JoinHandle<Result<(), FibError>>), FibError> {
let (stop, stopping) = watch::channel(None);
let fib = Self {
shared: Arc::new(Shared {
config,
status: Mutex::default(),
topology: watch::channel(None).0,
stop,
adopted: watch::channel(None).0,
release: watch::channel(false).0,
ended: watch::channel(false).0,
}),
};
let task = start(rib, fib.clone(), stopping)?;
Ok((fib, task))
}
pub fn config(&self) -> FibConfig {
self.shared.config
}
/// The kernel's view of the machine, for those who want to know
/// whether a next hop can be reached before the kernel is asked to use
/// it.
pub fn topology(&self) -> TopologyWatch {
self.shared.topology.subscribe()
}
#[cfg_attr(not(target_os = "linux"), allow(dead_code))]
fn publish_topology(&self, topology: &topology::Topology) {
self.shared
.topology
.send_replace(Some(Arc::new(topology.clone())));
}
pub fn status(&self) -> Status {
self.shared
.status
.lock()
.unwrap_or_else(PoisonError::into_inner)
.clone()
}
/// Has the task take our routes out of the kernel and end, and waits
/// until it has. Returns at once if it has ended already, for whatever
/// reason; the task's own result says how.
pub async fn stop(&self) {
self.end(Stop::Withdraw).await;
}
/// Has the task stop following the routing table and end, leaving our
/// routes in the kernel for the next run to find and keep for a while
/// (see [`FibConfig::stale_time`]), and waits until it has. For a
/// restart that is to go unnoticed by the packets being forwarded.
pub async fn leave(&self) {
self.end(Stop::Keep).await;
}
async fn end(&self, how: Stop) {
let mut ended = self.shared.ended.subscribe();
self.shared.stop.send_if_modified(|stop| {
let first = stop.is_none();
if first {
*stop = Some(how);
}
first
});
// Fails only if the handle that holds the sender is gone, and this
// is one.
let _ = ended.wait_for(|ended| *ended).await;
}
/// How many routes of an earlier run the kernel had at the start, once
/// the task has looked; zero if it ended before it did.
pub async fn adopted(&self) -> usize {
let mut adopted = self.shared.adopted.subscribe();
let mut ended = self.shared.ended.subscribe();
tokio::select! {
found = adopted.wait_for(Option::is_some) => {
found.ok().and_then(|found| *found).unwrap_or(0)
}
_ = ended.wait_for(|ended| *ended) => 0,
}
}
/// Has the routes of an earlier run that the routing table does not
/// have go now, rather than when [`FibConfig::stale_time`] is up: the
/// peers have all sent their routes, and what is not among them is no
/// longer anybody's.
pub fn release_kept(&self) {
self.shared.release.send_replace(true);
}
/// Changes the status, and gives back what `change` does.
#[cfg_attr(not(target_os = "linux"), allow(dead_code))]
fn update<R>(&self, change: impl FnOnce(&mut Status) -> R) -> R {
change(
&mut self
.shared
.status
.lock()
.unwrap_or_else(PoisonError::into_inner),
)
}
}
#[cfg(target_os = "linux")]
fn start(
rib: Arc<RibTable>,
fib: Fib,
stopping: watch::Receiver<Option<Stop>>,
) -> Result<JoinHandle<Result<(), FibError>>, FibError> {
// Opened here, so that a machine that cannot is known of at once.
let sockets = kernel::Sockets::open()?;
Ok(tokio::spawn(async move {
let result = kernel::run(rib, fib.clone(), sockets, stopping).await;
fib.shared.ended.send_replace(true);
result
}))
}
#[cfg(not(target_os = "linux"))]
fn start(
_rib: Arc<RibTable>,
_fib: Fib,
_stopping: watch::Receiver<Option<Stop>>,
) -> Result<JoinHandle<Result<(), FibError>>, FibError> {
Err(FibError::Unsupported)
}
crates/fib/src/netlink.rs
-1189
View File
File diff suppressed because it is too large. Load diff
crates/fib/src/plan.rs
-1327
View File
File diff suppressed because it is too large. Load diff
crates/fib/src/routes.rs
-266
View File
@@ -1,266 +0,0 @@
//! The kernel's own routes, as far as next hops are concerned: the way to
//! a next hop that is not a neighbor on any of our interfaces.
//!
//! Only the routes of one table count, the resolution table, which is
//! normally `main`: the connected and static routes, and an IGP's if there
//! is one. Routes put there by BGP, ours or another daemon's, never count:
//! a next hop that is reached only by way of a BGP route is reached by way
//! of something that may itself depend on a next hop, and a table that
//! leans on itself like that is one that falls over in a way nobody can
//! predict. The default route counts only if asked for: on a router, the
//! default points at a transit provider, and a next hop that is "reached"
//! only by sending the packets to the Internet at large is not reached.
use std::net::IpAddr;
use bgp_primitives::constants::IpVersion;
use bgp_primitives::ip_prefix::IpPrefix;
use crate::netlink::{Message, PROTOCOL_BGP, Route};
use crate::plan::HashMap;
/// The routes of the resolution table that may say the way to a next hop.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct KernelRoutes {
table: u32,
/// By prefix, the routes there are for it: usually one, but the kernel
/// keeps one for each metric.
routes: HashMap<IpPrefix, Vec<Route>>,
/// How many prefixes there are of each length, by IP version: the
/// lengths a lookup has to try.
lengths: [[u32; 129]; 2],
}
/// `address` as the prefix that is only it.
pub(crate) fn host(address: IpAddr) -> IpPrefix {
match address {
IpAddr::V4(v4) => IpPrefix::new(IpVersion::Ipv4, v4.octets(), 32),
IpAddr::V6(v6) => IpPrefix::new(IpVersion::Ipv6, v6.octets(), 128),
}
.expect("an address is a prefix of its own length")
}
fn version_index(version: IpVersion) -> usize {
match version {
IpVersion::Ipv4 => 0,
IpVersion::Ipv6 => 1,
}
}
impl KernelRoutes {
pub fn new(table: u32) -> Self {
Self {
table,
routes: HashMap::default(),
lengths: [[0; 129]; 2],
}
}
/// The table the routes are of.
pub fn table(&self) -> u32 {
self.table
}
/// Whether `route` is one that may say the way to a next hop.
fn counts(&self, route: &Route) -> bool {
route.table == self.table && route.protocol != PROTOCOL_BGP
}
/// Takes in what a message from the kernel says about routes. Returns
/// whether anything is different for it.
pub fn apply(&mut self, message: &Message) -> bool {
let Message::Route { present, route } = message else {
return false;
};
if !self.counts(route) {
return false;
}
let version = version_index(route.prefix.address_family());
let length = usize::from(route.prefix.length());
if *present {
let routes = self.routes.entry(route.prefix).or_default();
// A route for a prefix with a metric there is one for already
// takes that one's place: the kernel keeps one per metric.
match routes.iter_mut().find(|known| known.metric == route.metric) {
Some(known) if known == route => false,
Some(known) => {
*known = *route;
true
}
None => {
if routes.is_empty() {
self.lengths[version][length] += 1;
}
// In order of metric, so that two tables with the
// same routes compare equal however they were told.
let at = routes.partition_point(|known| known.metric < route.metric);
routes.insert(at, *route);
true
}
}
} else {
let Some(routes) = self.routes.get_mut(&route.prefix) else {
return false;
};
let before = routes.len();
routes.retain(|known| known.metric != route.metric);
let (after, none_left) = (routes.len(), routes.is_empty());
if none_left {
self.routes.remove(&route.prefix);
self.lengths[version][length] -= 1;
}
before != after
}
}
/// How many prefixes have a route.
pub fn len(&self) -> usize {
self.routes.len()
}
pub fn is_empty(&self) -> bool {
self.routes.is_empty()
}
/// The route the kernel would send packets for `address` by, from
/// this table: the one for the longest prefix that covers it, and of
/// several for that prefix the one with the lowest metric. The
/// default route too, if there is one and nothing longer covers the
/// address.
pub fn lookup(&self, address: IpAddr) -> Option<&Route> {
let whole = host(address);
let lengths = &self.lengths[version_index(whole.address_family())];
(0..=whole.length())
.rev()
.filter(|length| lengths[usize::from(*length)] > 0)
.find_map(|length| self.routes.get(&whole.shortened(length)))
.and_then(|routes| routes.iter().min_by_key(|route| route.metric))
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::netlink::{KIND_UNICAST, TABLE_MAIN};
fn prefix(s: &str) -> IpPrefix {
IpPrefix::try_from(s).unwrap()
}
fn route(prefix_text: &str, gateway: Option<&str>, interface: u32, metric: u32) -> Route {
Route {
prefix: prefix(prefix_text),
table: TABLE_MAIN,
protocol: 4, // static
metric,
gateway: gateway.map(|g| g.parse().unwrap()),
interface: Some(interface),
nexthop: None,
kind: KIND_UNICAST,
}
}
fn present(route: Route) -> Message {
Message::Route {
present: true,
route,
}
}
fn gone(route: Route) -> Message {
Message::Route {
present: false,
route,
}
}
fn lookup(routes: &KernelRoutes, address: &str) -> Option<(String, Option<String>, u32)> {
routes.lookup(address.parse().unwrap()).map(|route| {
(
route.prefix.to_string(),
route.gateway.map(|g| g.to_string()),
route.metric,
)
})
}
#[test]
fn the_longest_prefix_with_the_lowest_metric_is_the_way() {
let mut routes = KernelRoutes::new(TABLE_MAIN);
for r in [
route("0.0.0.0/0", Some("192.0.2.1"), 2, 100),
route("10.0.0.0/8", Some("192.0.2.9"), 2, 0),
route("10.1.0.0/16", None, 7, 0),
route("10.1.0.0/16", Some("198.51.100.1"), 3, 50),
route("2001:db8::/32", Some("2001:db8::1"), 2, 1024),
] {
assert!(routes.apply(&present(r)));
}
assert_eq!(routes.len(), 4);
assert_eq!(
lookup(&routes, "10.2.3.4"),
Some(("10.0.0.0/8".into(), Some("192.0.2.9".into()), 0))
);
// Two routes for the prefix: the lower metric wins.
assert_eq!(
lookup(&routes, "10.1.2.3"),
Some(("10.1.0.0/16".into(), None, 0))
);
assert_eq!(
lookup(&routes, "203.0.113.1"),
Some(("0.0.0.0/0".into(), Some("192.0.2.1".into()), 100))
);
assert_eq!(
lookup(&routes, "2001:db8:1::1"),
Some(("2001:db8::/32".into(), Some("2001:db8::1".into()), 1024))
);
assert_eq!(lookup(&routes, "2001:db9::1"), None);
// Told again what is known: nothing changes.
assert!(!routes.apply(&present(route("10.0.0.0/8", Some("192.0.2.9"), 2, 0))));
// The same prefix and metric with another gateway: replaced.
assert!(routes.apply(&present(route("10.0.0.0/8", Some("192.0.2.8"), 2, 0))));
assert_eq!(
lookup(&routes, "10.2.3.4"),
Some(("10.0.0.0/8".into(), Some("192.0.2.8".into()), 0))
);
// The better of the two for 10.1/16 goes: the other is the way.
assert!(routes.apply(&gone(route("10.1.0.0/16", None, 7, 0))));
assert_eq!(
lookup(&routes, "10.1.2.3"),
Some(("10.1.0.0/16".into(), Some("198.51.100.1".into()), 50))
);
assert!(routes.apply(&gone(route("10.1.0.0/16", Some("198.51.100.1"), 3, 50))));
assert_eq!(
lookup(&routes, "10.1.2.3"),
Some(("10.0.0.0/8".into(), Some("192.0.2.8".into()), 0))
);
// Gone already: nothing changes.
assert!(!routes.apply(&gone(route("10.1.0.0/16", None, 7, 0))));
assert_eq!(routes.len(), 3);
}
#[test]
fn routes_of_other_tables_and_of_bgp_do_not_count() {
let mut routes = KernelRoutes::new(TABLE_MAIN);
let other_table = Route {
table: 200,
..route("10.0.0.0/8", Some("192.0.2.9"), 2, 0)
};
let bgp = Route {
protocol: PROTOCOL_BGP,
..route("10.0.0.0/8", Some("192.0.2.9"), 2, 20)
};
assert!(!routes.apply(&present(other_table)));
assert!(!routes.apply(&present(bgp)));
assert!(routes.is_empty());
assert_eq!(lookup(&routes, "10.2.3.4"), None);
// What is not about routes changes nothing either.
assert!(!routes.apply(&Message::Done));
let mut table_200 = KernelRoutes::new(200);
assert!(table_200.apply(&present(other_table)));
assert_eq!(table_200.table(), 200);
assert_eq!(table_200.len(), 1);
}
}
crates/fib/src/sockdiag.rs
-420
View File
@@ -1,420 +0,0 @@
//! What the kernel knows about one of our TCP connections: the round-trip
//! time, the windows, the queues and what has held the connection up,
//! asked over the socket diagnostics netlink family (`NETLINK_SOCK_DIAG`,
//! as `ss -ti` asks). For the status page, when someone wants to know
//! why a session is slow.
use std::io;
use std::net::{IpAddr, SocketAddr};
use crate::netlink::{
AF_INET, AF_INET6, HEADER_LEN, attributes, flag, message_type, ne_u16, ne_u32, ne_u64,
};
/// A TCP connection as the kernel sees it. Times are in microseconds,
/// sizes in octets, rates in octets per second, unless said otherwise.
/// Fields the running kernel does not report are zero.
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct TcpConnection {
/// The TCP state, by its number (1 is ESTABLISHED); see
/// [`TcpConnection::state_name`].
pub state: u8,
/// The congestion control algorithm, if the kernel named it.
pub congestion: Option<String>,
/// Octets received and not yet read by us.
pub recv_queue: u32,
/// Octets written by us and not yet acknowledged by the peer.
pub send_queue: u32,
pub rtt: u32,
pub rtt_variance: u32,
pub min_rtt: u32,
pub rto: u32,
pub mss: u32,
pub pmtu: u32,
/// The congestion window, in segments.
pub cwnd: u32,
/// The slow start threshold, in segments; `None` if there is none.
pub ssthresh: Option<u32>,
/// What the peer lets us send: its receive window.
pub send_window: u32,
/// What we let the peer send.
pub receive_window: u32,
pub receive_space: u32,
pub receive_ssthresh: u32,
pub send_window_scale: u8,
pub receive_window_scale: u8,
/// Segments sent and not acknowledged.
pub unacked: u32,
/// Segments retransmitted and still out.
pub retrans: u32,
pub lost: u32,
pub total_retrans: u32,
pub bytes_retrans: u64,
/// Octets in the socket's buffer that are not sent yet.
pub not_sent: u32,
pub pacing_rate: u64,
pub delivery_rate: u64,
pub bytes_sent: u64,
pub bytes_acked: u64,
pub bytes_received: u64,
pub segments_out: u32,
pub segments_in: u32,
/// How long the connection had something to send.
pub busy: u64,
/// Of that, how long it was held up by the peer's receive window.
pub limited_by_receive_window: u64,
/// And how long by our own send buffer being full.
pub limited_by_send_buffer: u64,
pub delivered: u32,
pub reordering: u32,
/// Milliseconds since data was last sent, received, and since the
/// last acknowledgement came.
pub last_data_sent_ms: u32,
pub last_data_received_ms: u32,
pub last_ack_received_ms: u32,
}
impl TcpConnection {
pub fn state_name(&self) -> &'static str {
match self.state {
1 => "established",
2 => "syn-sent",
3 => "syn-received",
4 => "fin-wait-1",
5 => "fin-wait-2",
6 => "time-wait",
7 => "close",
8 => "close-wait",
9 => "last-ack",
10 => "listen",
11 => "closing",
_ => "unknown",
}
}
}
const SOCK_DIAG_BY_FAMILY: u16 = 20;
const IPPROTO_TCP: u8 = 6;
const INET_DIAG_INFO: u16 = 2;
const INET_DIAG_CONG: u16 = 4;
/// `idiag_ext` bits asking for those two.
const EXTENSIONS: u8 = (1 << (INET_DIAG_INFO - 1)) | (1 << (INET_DIAG_CONG - 1));
const NO_COOKIE: u32 = u32::MAX;
const REQUEST_LEN: usize = HEADER_LEN + 56;
const DIAG_MSG_LEN: usize = 72;
/// The request for the connection from `local` to `peer`, with `sequence`.
pub fn request(local: SocketAddr, peer: SocketAddr, sequence: u32) -> Vec<u8> {
let family = match local {
SocketAddr::V4(_) => AF_INET,
SocketAddr::V6(_) => AF_INET6,
};
let mut buf = Vec::with_capacity(REQUEST_LEN);
buf.extend_from_slice(&(REQUEST_LEN as u32).to_ne_bytes());
buf.extend_from_slice(&SOCK_DIAG_BY_FAMILY.to_ne_bytes());
buf.extend_from_slice(&flag::REQUEST.to_ne_bytes());
buf.extend_from_slice(&sequence.to_ne_bytes());
buf.extend_from_slice(&0u32.to_ne_bytes());
// inet_diag_req_v2
buf.extend_from_slice(&[family, IPPROTO_TCP, EXTENSIONS, 0]);
buf.extend_from_slice(&u32::MAX.to_ne_bytes()); // any state
// inet_diag_sockid: ports in network order, addresses as 16 octets.
buf.extend_from_slice(&local.port().to_be_bytes());
buf.extend_from_slice(&peer.port().to_be_bytes());
buf.extend_from_slice(&address_octets(local.ip()));
buf.extend_from_slice(&address_octets(peer.ip()));
// The interface, which a socket on a link-local address is bound to
// and is found by.
let interface = match local {
SocketAddr::V6(v6) => v6.scope_id(),
SocketAddr::V4(_) => 0,
};
buf.extend_from_slice(&interface.to_ne_bytes());
buf.extend_from_slice(&NO_COOKIE.to_ne_bytes());
buf.extend_from_slice(&NO_COOKIE.to_ne_bytes());
debug_assert_eq!(buf.len(), REQUEST_LEN);
buf
}
fn address_octets(address: IpAddr) -> [u8; 16] {
let mut octets = [0u8; 16];
match address {
IpAddr::V4(v4) => octets[..4].copy_from_slice(&v4.octets()),
IpAddr::V6(v6) => octets = v6.octets(),
}
octets
}
/// What the kernel answered.
#[derive(Debug, PartialEq, Eq)]
pub enum Answer {
Connection(Box<TcpConnection>),
/// No such connection.
None,
/// The kernel refused, with this error number.
Refused(i32),
}
/// Reads the kernel's answer to a [`request`].
pub fn parse(buf: &[u8]) -> io::Result<Answer> {
let header = buf
.get(..HEADER_LEN)
.ok_or_else(|| io::Error::other("short netlink answer"))?;
let length = ne_u32(&header[0..4]) as usize;
let kind = ne_u16(&header[4..6]);
let message = buf
.get(HEADER_LEN..length.min(buf.len()))
.ok_or_else(|| io::Error::other("short netlink answer"))?;
if kind == message_type::ERROR {
let errno = -(message
.first_chunk::<4>()
.map_or(0, |e| i32::from_ne_bytes(*e)));
// ENOENT: no such socket.
return Ok(if errno == 2 {
Answer::None
} else {
Answer::Refused(errno)
});
}
if kind != SOCK_DIAG_BY_FAMILY || message.len() < DIAG_MSG_LEN {
return Err(io::Error::other("not a socket diagnostic"));
}
let mut connection = TcpConnection {
state: message[1],
recv_queue: ne_u32(&message[56..60]),
send_queue: ne_u32(&message[60..64]),
..TcpConnection::default()
};
for (kind, value) in attributes(&message[DIAG_MSG_LEN..]) {
match kind {
INET_DIAG_INFO => fill_from_tcp_info(&mut connection, value),
INET_DIAG_CONG => {
let name = value.split(|b| *b == 0).next().unwrap_or(&[]);
connection.congestion = Some(String::from_utf8_lossy(name).into_owned());
}
_ => {}
}
}
Ok(Answer::Connection(Box::new(connection)))
}
/// Fills in what `tcp_info` (linux/tcp.h) has, as far as the kernel sent
/// it: the structure has grown over the years, and a field that is not
/// there is left at zero.
fn fill_from_tcp_info(c: &mut TcpConnection, info: &[u8]) {
let u32_at = |at: usize| info.get(at..at + 4).map_or(0, ne_u32);
let u64_at = |at: usize| info.get(at..at + 8).map_or(0, ne_u64);
c.send_window_scale = info.get(6).map_or(0, |b| b & 0x0f);
c.receive_window_scale = info.get(6).map_or(0, |b| b >> 4);
c.rto = u32_at(8);
c.mss = u32_at(16);
c.unacked = u32_at(24);
c.lost = u32_at(32);
c.retrans = u32_at(36);
c.last_data_sent_ms = u32_at(44);
c.last_data_received_ms = u32_at(52);
c.last_ack_received_ms = u32_at(56);
c.pmtu = u32_at(60);
c.receive_ssthresh = u32_at(64);
c.rtt = u32_at(68);
c.rtt_variance = u32_at(72);
c.ssthresh = Some(u32_at(76)).filter(|t| *t < 0x7fff_ffff);
c.cwnd = u32_at(80);
c.reordering = u32_at(88);
c.receive_space = u32_at(96);
c.total_retrans = u32_at(100);
c.pacing_rate = u64_at(104);
c.bytes_acked = u64_at(120);
c.bytes_received = u64_at(128);
c.segments_out = u32_at(136);
c.segments_in = u32_at(140);
c.not_sent = u32_at(144);
c.min_rtt = u32_at(148);
c.delivery_rate = u64_at(160);
c.busy = u64_at(168);
c.limited_by_receive_window = u64_at(176);
c.limited_by_send_buffer = u64_at(184);
c.delivered = u32_at(192);
c.bytes_sent = u64_at(200);
c.bytes_retrans = u64_at(208);
c.send_window = u32_at(228);
c.receive_window = u32_at(232);
}
/// Asks the kernel about the TCP connection from `local` to `peer`.
/// `Ok(None)` if there is no such connection, or on an operating system
/// that cannot be asked.
#[cfg(target_os = "linux")]
pub async fn tcp_connection(
local: SocketAddr,
peer: SocketAddr,
) -> io::Result<Option<TcpConnection>> {
use netlink_sys::protocols::NETLINK_SOCK_DIAG;
use netlink_sys::{AsyncSocket, AsyncSocketExt, TokioSocket};
let mut socket = TokioSocket::new(NETLINK_SOCK_DIAG)?;
socket
.socket_mut()
.bind(&netlink_sys::SocketAddr::new(0, 0))?;
let sequence = 1;
socket.send(&request(local, peer, sequence)).await?;
let mut buf = bytes::BytesMut::with_capacity(4096);
tokio::time::timeout(std::time::Duration::from_secs(2), socket.recv(&mut buf))
.await
.map_err(|_| io::Error::new(io::ErrorKind::TimedOut, "the kernel did not answer"))??;
match parse(&buf)? {
Answer::Connection(connection) => Ok(Some(*connection)),
Answer::None => Ok(None),
Answer::Refused(errno) => Err(io::Error::from_raw_os_error(errno)),
}
}
#[cfg(not(target_os = "linux"))]
pub async fn tcp_connection(
_local: SocketAddr,
_peer: SocketAddr,
) -> io::Result<Option<TcpConnection>> {
Ok(None)
}
#[cfg(test)]
mod tests {
use super::*;
/// A reply as the kernel writes it, for an established connection,
/// with the first `info_len` octets of a `tcp_info` of the full
/// modern length: an older kernel sends a shorter one.
fn reply(info_len: usize) -> Vec<u8> {
let mut info = vec![0u8; 248];
info[0] = 1; // established
info[6] = 0x7a; // snd_wscale 10, rcv_wscale 7
let put32 = |info: &mut Vec<u8>, at: usize, v: u32| {
info[at..at + 4].copy_from_slice(&v.to_ne_bytes())
};
let put64 = |info: &mut Vec<u8>, at: usize, v: u64| {
info[at..at + 8].copy_from_slice(&v.to_ne_bytes())
};
put32(&mut info, 16, 1448); // mss
put32(&mut info, 24, 12); // unacked
put32(&mut info, 68, 2300); // rtt
put32(&mut info, 76, 0x7fff_ffff); // no ssthresh
put32(&mut info, 80, 42); // cwnd
put32(&mut info, 144, 65536); // notsent
put64(&mut info, 176, 900_000); // rwnd limited
put64(&mut info, 168, 1_000_000); // busy
put32(&mut info, 228, 131072); // snd_wnd
put32(&mut info, 232, 262144); // rcv_wnd
info.truncate(info_len);
let cong = b"cubic\0";
let mut message = vec![0u8; DIAG_MSG_LEN];
message[1] = 1;
message[60..64].copy_from_slice(&80_000u32.to_ne_bytes()); // wqueue
// attributes: INFO, then CONG
let mut attrs = Vec::new();
attrs.extend_from_slice(&((4 + info.len()) as u16).to_ne_bytes());
attrs.extend_from_slice(&INET_DIAG_INFO.to_ne_bytes());
attrs.extend_from_slice(&info);
attrs.extend_from_slice(&((4 + cong.len()) as u16).to_ne_bytes());
attrs.extend_from_slice(&INET_DIAG_CONG.to_ne_bytes());
attrs.extend_from_slice(cong);
while attrs.len() % 4 != 0 {
attrs.push(0);
}
let length = HEADER_LEN + message.len() + attrs.len();
let mut buf = Vec::new();
buf.extend_from_slice(&(length as u32).to_ne_bytes());
buf.extend_from_slice(&SOCK_DIAG_BY_FAMILY.to_ne_bytes());
buf.extend_from_slice(&0u16.to_ne_bytes()); // flags
buf.extend_from_slice(&1u32.to_ne_bytes()); // sequence
buf.extend_from_slice(&0u32.to_ne_bytes()); // port
buf.extend_from_slice(&message);
buf.extend_from_slice(&attrs);
buf
}
#[test]
fn a_reply_is_read_field_by_field() {
let Answer::Connection(c) = parse(&reply(248)).unwrap() else {
panic!("a connection");
};
assert_eq!(c.state_name(), "established");
assert_eq!(c.congestion.as_deref(), Some("cubic"));
assert_eq!(c.send_queue, 80_000);
assert_eq!((c.send_window_scale, c.receive_window_scale), (10, 7));
assert_eq!((c.mss, c.unacked, c.rtt, c.cwnd), (1448, 12, 2300, 42));
assert_eq!(c.ssthresh, None);
assert_eq!(c.not_sent, 65536);
assert_eq!((c.busy, c.limited_by_receive_window), (1_000_000, 900_000));
assert_eq!((c.send_window, c.receive_window), (131072, 262144));
}
/// Linux 4.x sends 104 octets of `tcp_info`, up to `tcpi_total_retrans`:
/// what is there is read, and what came later is left at zero rather
/// than read from the attribute after it.
#[test]
fn a_short_tcp_info_from_an_old_kernel_leaves_later_fields_zero() {
let Answer::Connection(c) = parse(&reply(104)).unwrap() else {
panic!("a connection");
};
assert_eq!(c.congestion.as_deref(), Some("cubic"));
assert_eq!((c.mss, c.rtt, c.cwnd), (1448, 2300, 42));
assert_eq!((c.busy, c.send_window), (0, 0));
}
#[test]
fn no_such_connection_and_a_refusal_are_told_apart() {
let error = |errno: i32| {
let mut buf = vec![0u8; 20];
buf[0..4].copy_from_slice(&20u32.to_ne_bytes());
buf[4..6].copy_from_slice(&message_type::ERROR.to_ne_bytes());
buf[16..20].copy_from_slice(&(-errno).to_ne_bytes());
buf
};
assert_eq!(parse(&error(2)).unwrap(), Answer::None);
assert_eq!(parse(&error(13)).unwrap(), Answer::Refused(13));
}
#[test]
fn a_request_names_the_connection() {
let local: SocketAddr = "[2001:db8::1]:179".parse().unwrap();
let peer: SocketAddr = "[2001:db8::2]:40000".parse().unwrap();
let request = request(local, peer, 7);
assert_eq!(request.len(), REQUEST_LEN);
assert_eq!(request[16], AF_INET6);
assert_eq!(&request[24..26], &179u16.to_be_bytes());
assert_eq!(&request[26..28], &40000u16.to_be_bytes());
assert_eq!(
&request[28..44],
&"2001:db8::1"
.parse::<std::net::Ipv6Addr>()
.unwrap()
.octets()
);
}
#[cfg(target_os = "linux")]
#[tokio::test]
async fn the_kernel_tells_of_a_connection_of_ours() {
use tokio::io::AsyncWriteExt;
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let mut client = tokio::net::TcpStream::connect(listener.local_addr().unwrap())
.await
.unwrap();
let (_server, _) = listener.accept().await.unwrap();
client.write_all(&[0u8; 1000]).await.unwrap();
let connection = tcp_connection(client.local_addr().unwrap(), client.peer_addr().unwrap())
.await
.unwrap()
.expect("the kernel knows it");
assert_eq!(connection.state_name(), "established");
assert!(connection.mss > 0 && connection.cwnd > 0);
assert!(connection.bytes_sent >= 1000 || connection.bytes_acked >= 1000);
let none = tcp_connection(
"127.0.0.1:1".parse().unwrap(),
"127.0.0.1:2".parse().unwrap(),
)
.await
.unwrap();
assert!(none.is_none());
}
}
crates/fib/src/topology.rs
-123
View File
@@ -1,123 +0,0 @@
//! What the kernel knows that says whether, and how, a next hop can be
//! reached: the machine's interfaces and addresses, the routes of the
//! resolution table, and the kernel's nexthop objects that those routes
//! may name. One value, replaced as a whole when any of it changes, so
//! that everyone judging next hops judges them by one moment's view.
use crate::interfaces::Interfaces;
use crate::netlink::{Message, Nexthop};
use crate::plan::HashMap;
use crate::routes::KernelRoutes;
/// The kernel's view of the machine, as far as next hops are concerned.
#[derive(Debug, Clone)]
pub struct Topology {
pub interfaces: Interfaces,
/// The routes a next hop that is not a neighbor is reached by.
pub routes: KernelRoutes,
/// Every nexthop object the kernel has, by number, whoever made it:
/// a route of the resolution table may name one instead of carrying
/// its own gateway.
pub nexthops: HashMap<u32, Nexthop>,
/// Whether the default route counts as the way to a next hop.
pub via_default: bool,
}
impl Topology {
/// Nothing known yet, with next hops to be resolved by the routes of
/// `resolution_table`.
pub fn new(resolution_table: u32, via_default: bool) -> Self {
Self {
interfaces: Interfaces::default(),
routes: KernelRoutes::new(resolution_table),
nexthops: HashMap::default(),
via_default,
}
}
/// Takes in what a message from the kernel says. Returns whether
/// anything is different for it: then what any next hop resolves to
/// may be different too.
pub fn apply(&mut self, message: &Message) -> bool {
match message {
Message::Nexthop {
present: true,
nexthop,
} => self.nexthops.insert(nexthop.id, *nexthop) != Some(*nexthop),
Message::Nexthop {
present: false,
nexthop,
} => self.nexthops.remove(&nexthop.id).is_some(),
Message::Route { .. } => self.routes.apply(message),
_ => self.interfaces.apply(message),
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::netlink::{KIND_UNICAST, Link, PROTOCOL_BGP, Route, TABLE_MAIN};
use bgp_primitives::ip_prefix::IpPrefix;
#[test]
fn each_kind_of_news_goes_where_it_belongs() {
let mut topology = Topology::new(TABLE_MAIN, false);
let nexthop = Nexthop {
id: 7,
protocol: 4,
gateway: Some("192.0.2.1".parse().unwrap()),
interface: Some(2),
};
let present = |nexthop| Message::Nexthop {
present: true,
nexthop,
};
assert!(topology.apply(&present(nexthop)));
assert!(!topology.apply(&present(nexthop)));
assert!(topology.apply(&present(Nexthop {
interface: Some(3),
..nexthop
})));
assert_eq!(topology.nexthops[&7].interface, Some(3));
assert!(topology.apply(&Message::Nexthop {
present: false,
nexthop,
}));
assert!(topology.nexthops.is_empty());
let route = Route {
prefix: IpPrefix::try_from("10.0.0.0/8").unwrap(),
table: TABLE_MAIN,
protocol: 4,
metric: 0,
gateway: Some("192.0.2.1".parse().unwrap()),
interface: Some(2),
nexthop: None,
kind: KIND_UNICAST,
};
assert!(topology.apply(&Message::Route {
present: true,
route,
}));
assert_eq!(topology.routes.len(), 1);
// BGP's routes are not the way to anything.
assert!(!topology.apply(&Message::Route {
present: true,
route: Route {
protocol: PROTOCOL_BGP,
..route
},
}));
assert!(topology.apply(&Message::Link {
present: true,
link: Link {
index: 2,
name: "eth0".into(),
up: true,
},
}));
assert!(topology.interfaces.link(2).is_some());
assert!(!topology.apply(&Message::Done));
}
}
crates/fib/tests/kernel.rs
-910
View File
@@ -1,910 +0,0 @@
//! The kernel's routes against a real kernel: what `ip route` shows after
//! the routing table changes.
//!
//! These change the machine's interfaces and routes, so they are not run
//! by `cargo test`. `scripts/kernel-tests.sh` runs them in a network
//! namespace of their own, with the capability they need
//! (CAP_NET_ADMIN) and nothing more.
#![cfg(target_os = "linux")]
use std::collections::BTreeSet;
use std::net::{IpAddr, Ipv4Addr};
use std::process::Command;
use std::sync::Arc;
use std::time::{Duration, Instant};
use bgp_fib::{Fib, FibConfig, FibError};
use bgp_packet::message::{
AsPathAttribute, CommunitiesPathAttribute, MultiExitDiscPathAttribute, NextHopPathAttribute,
NlriNextHop, OriginPathAttribute, PathAttribute,
};
use bgp_primitives::ip_prefix::IpPrefix;
use bgp_rib::table::{AttrSet, PeerIndex, PeerInfo, RibTable};
use tokio::task::JoinHandle;
const TIMEOUT: Duration = Duration::from_secs(10);
fn prefix(s: &str) -> IpPrefix {
IpPrefix::try_from(s).unwrap()
}
/// Runs `ip` and returns what it printed. Panics if it fails.
#[track_caller]
fn ip(arguments: &str) -> String {
let output = Command::new("ip")
.args(arguments.split_whitespace())
.output()
.expect("the ip command is there");
assert!(
output.status.success(),
"ip {arguments}: {}",
String::from_utf8_lossy(&output.stderr)
);
String::from_utf8_lossy(&output.stdout).into_owned()
}
/// What a test has of the machine: an interface of its own with
/// 10.`n`.0.1/24 and 2001:db8:`n`::1/64 on it, and a routing table of its
/// own. Tests run side by side in one network namespace, and `n` keeps
/// them apart.
struct Machine {
n: u8,
rib: Arc<RibTable>,
/// Whether routes name nexthop objects, which is how it is unless a
/// test says otherwise.
nexthop_objects: bool,
/// How long routes of an earlier run are kept: not at all, unless a
/// test says otherwise.
stale_time: Duration,
}
impl Machine {
fn new(n: u8) -> Self {
let machine = Self {
n,
rib: Arc::new(RibTable::new(4)),
nexthop_objects: true,
stale_time: Duration::ZERO,
};
ip(&format!("link add {} type dummy", machine.interface()));
machine.add_addresses();
ip(&format!("link set {} up", machine.interface()));
machine
}
fn interface(&self) -> String {
format!("ferrix{}", self.n)
}
fn add_addresses(&self) {
let (n, interface) = (self.n, self.interface());
ip(&format!("addr add 10.{n}.0.1/24 dev {interface}"));
ip(&format!(
"-6 addr add 2001:db8:{n}::1/64 dev {interface} nodad"
));
}
fn add_addresses_v4(&self) {
ip(&format!(
"addr add 10.{}.0.1/24 dev {}",
self.n,
self.interface()
));
}
fn table(&self) -> u32 {
1000 + u32::from(self.n)
}
/// A neighbor on the test's interface.
fn neighbor(&self, host: u8) -> Ipv4Addr {
Ipv4Addr::new(10, self.n, 0, host)
}
fn start(&self) -> (Fib, JoinHandle<Result<(), FibError>>) {
Fib::spawn(
Arc::clone(&self.rib),
FibConfig {
table: self.table(),
metric: 20,
nexthop_objects: self.nexthop_objects,
stale_time: self.stale_time,
..FibConfig::default()
},
)
.expect("the kernel can be talked to")
}
fn peer(&self, host: u8) -> PeerIndex {
self.rib
.add_peer(PeerInfo {
asn: 65000 + u32::from(host),
router_id: self.neighbor(host),
address: self.neighbor(host).into(),
interface: 0,
ibgp: false,
reflector_client: false,
})
.unwrap()
}
/// Attributes with the next hop `next_hop` and, to say which of two
/// routes is the better, `med`.
fn via(&self, next_hop: IpAddr, med: u32) -> Arc<AttrSet> {
let mut attributes = vec![
PathAttribute::Origin(OriginPathAttribute::IGP),
AsPathAttribute::from_asns(vec![65001]),
PathAttribute::MultiExitDisc(MultiExitDiscPathAttribute(med)),
];
let mp_next_hop = match next_hop {
IpAddr::V4(v4) => {
attributes.push(PathAttribute::NextHop(NextHopPathAttribute(v4)));
None
}
IpAddr::V6(v6) => Some(NlriNextHop::Ipv6(v6)),
};
self.rib
.attrs()
.intern(&attributes, mp_next_hop.as_ref())
.unwrap()
}
/// The routes in the test's table, as `ip route` prints them.
fn kernel_routes(&self) -> BTreeSet<String> {
let table = self.table();
let mut routes = BTreeSet::new();
for family in ["-4", "-6"] {
// A table nothing was ever put in does not exist yet.
let output = Command::new("ip")
.args([family, "route", "show", "table", &table.to_string()])
.output()
.unwrap();
routes.extend(
String::from_utf8_lossy(&output.stdout)
.lines()
.map(|line| line.split_whitespace().collect::<Vec<_>>().join(" "))
// IPv6 routes are shown with "pref medium".
.map(|line| line.replace(" pref medium", ""))
// A route that names a nexthop object is shown with
// its number, which is the kernel's to choose.
.map(|line| match line.split_once(" nhid ") {
Some((prefix, rest)) => {
let rest = rest.split_once(' ').map_or("", |(_, rest)| rest);
format!("{prefix} {rest}")
}
None => line,
})
.filter(|line| !line.is_empty()),
);
}
routes
}
/// Waits until the kernel's table holds exactly `expected`.
#[track_caller]
fn kernel_becomes(&self, expected: &[String]) {
let expected: BTreeSet<String> = expected.iter().cloned().collect();
let start = Instant::now();
while self.kernel_routes() != expected {
assert!(
start.elapsed() < TIMEOUT,
"the kernel has {:#?}; expected {expected:#?}",
self.kernel_routes()
);
std::thread::sleep(Duration::from_millis(20));
}
}
/// How `ip route` shows a route of ours.
fn shown(&self, prefix: &str, gateway: impl std::fmt::Display) -> String {
format!(
"{prefix} via {gateway} dev {} proto bgp metric 20",
self.interface()
)
}
}
impl Drop for Machine {
fn drop(&mut self) {
let _ = Command::new("ip")
.args(["link", "del", &self.interface()])
.output();
for family in ["-4", "-6"] {
let _ = Command::new("ip")
.args([family, "route", "flush", "table", &self.table().to_string()])
.output();
}
}
}
/// Waits for `condition` to hold of the status.
#[track_caller]
fn status_becomes(fib: &Fib, what: &str, condition: impl Fn(&bgp_fib::Status) -> bool) {
let start = Instant::now();
while !condition(&fib.status()) {
assert!(
start.elapsed() < TIMEOUT,
"{what}: the status is {:#?}",
fib.status()
);
std::thread::sleep(Duration::from_millis(20));
}
}
/// The best route of each prefix is in the kernel, and follows the table:
/// when the best route changes, when it goes, and when there is none whose
/// next hop can be reached.
#[tokio::test(flavor = "multi_thread")]
#[ignore = "changes the machine's routes; run scripts/kernel-tests.sh"]
async fn the_kernel_has_the_best_route_of_each_prefix() {
let machine = Machine::new(1);
let (near, far) = (machine.peer(2), machine.peer(3));
let via_near = machine.via(machine.neighbor(2).into(), 50);
// What was there before the daemon started is installed too.
machine
.rib
.announce(near, &via_near, &[prefix("203.0.113.0/24")]);
let (fib, task) = machine.start();
machine.kernel_becomes(&[machine.shown("203.0.113.0/24", machine.neighbor(2))]);
status_becomes(&fib, "in step", |status| {
status.synced && status.installed == 1
});
// More routes, IPv6 among them.
let v6_neighbor: IpAddr = "2001:db8:1::2".parse().unwrap();
machine.rib.announce(
near,
&machine.via(v6_neighbor, 50),
&[prefix("2001:db8:100::/48")],
);
machine
.rib
.announce(near, &via_near, &[prefix("198.18.0.0/15")]);
machine.kernel_becomes(&[
machine.shown("203.0.113.0/24", machine.neighbor(2)),
machine.shown("198.18.0.0/15", machine.neighbor(2)),
machine.shown("2001:db8:100::/48", v6_neighbor),
]);
// A better route from another peer takes the first one's place.
let via_far = machine.via(machine.neighbor(3).into(), 10);
machine
.rib
.announce(far, &via_far, &[prefix("203.0.113.0/24")]);
machine.kernel_becomes(&[
machine.shown("203.0.113.0/24", machine.neighbor(3)),
machine.shown("198.18.0.0/15", machine.neighbor(2)),
machine.shown("2001:db8:100::/48", v6_neighbor),
]);
// It goes: the first is the best again.
machine.rib.withdraw(far, &[prefix("203.0.113.0/24")]);
machine.kernel_becomes(&[
machine.shown("203.0.113.0/24", machine.neighbor(2)),
machine.shown("198.18.0.0/15", machine.neighbor(2)),
machine.shown("2001:db8:100::/48", v6_neighbor),
]);
// A best route whose next hop is not a neighbor is not installed, and
// the route it replaced does not stay.
let unreachable = machine.via("172.31.9.9".parse().unwrap(), 5);
machine
.rib
.announce(far, &unreachable, &[prefix("198.18.0.0/15")]);
machine.kernel_becomes(&[
machine.shown("203.0.113.0/24", machine.neighbor(2)),
machine.shown("2001:db8:100::/48", v6_neighbor),
]);
status_becomes(&fib, "one unresolved", |status| {
status.installed == 2 && status.unresolved == [("no route to the next hop", 1)]
});
// A session ends: its routes go.
machine.rib.remove_peer(near);
machine.kernel_becomes(&[]);
status_becomes(&fib, "nothing installed", |status| status.installed == 0);
fib.stop().await;
task.await.unwrap().unwrap();
}
/// Routes of ours left by a run that ended badly are removed at the next
/// start unless they are still wanted. Routes that are not ours are left
/// alone, whatever they are for.
#[tokio::test(flavor = "multi_thread")]
#[ignore = "changes the machine's routes; run scripts/kernel-tests.sh"]
async fn routes_left_behind_are_cleared_and_others_are_not_touched() {
let machine = Machine::new(2);
let (table, interface) = (machine.table(), machine.interface());
let add = |route: &str| ip(&format!("route add {route} dev {interface} table {table}"));
// Ours, from before: one that is still right, one that has another
// next hop now, one that nobody announces any more.
add("203.0.113.0/24 via 10.2.0.2 proto bgp metric 20");
add("198.18.0.0/15 via 10.2.0.9 proto bgp metric 20");
add("192.88.99.0/24 via 10.2.0.2 proto bgp metric 20");
// Not ours: someone's static route, and a BGP route with another
// metric, which is how another daemon's would differ.
add("192.0.2.0/24 via 10.2.0.7 proto static metric 20");
add("192.88.99.0/24 via 10.2.0.7 proto bgp metric 30");
let others = [
format!("192.0.2.0/24 via 10.2.0.7 dev {interface} proto static metric 20"),
format!("192.88.99.0/24 via 10.2.0.7 dev {interface} proto bgp metric 30"),
];
let peer = machine.peer(2);
let via = machine.via(machine.neighbor(2).into(), 0);
machine.rib.announce(
peer,
&via,
&[prefix("203.0.113.0/24"), prefix("198.18.0.0/15")],
);
let (fib, task) = machine.start();
machine.kernel_becomes(&[
machine.shown("203.0.113.0/24", machine.neighbor(2)),
machine.shown("198.18.0.0/15", machine.neighbor(2)),
others[0].clone(),
others[1].clone(),
]);
// A run that ends in good order takes its routes with it, and only
// its own.
fib.stop().await;
task.await.unwrap().unwrap();
machine.kernel_becomes(&others);
}
/// What a next hop resolves to depends on the machine's interfaces. When
/// they change, the routes are worked out again.
#[tokio::test(flavor = "multi_thread")]
#[ignore = "changes the machine's routes; run scripts/kernel-tests.sh"]
async fn routes_follow_the_machines_interfaces() {
let machine = Machine::new(3);
let interface = machine.interface();
// The interface has no addresses yet: nothing is a neighbor.
ip(&format!("addr flush dev {interface}"));
let peer = machine.peer(2);
let via = machine.via(machine.neighbor(2).into(), 0);
machine
.rib
.announce(peer, &via, &[prefix("203.0.113.0/24")]);
let (fib, task) = machine.start();
status_becomes(&fib, "unresolved", |status| {
status.synced && status.unresolved == [("no route to the next hop", 1)]
});
assert!(machine.kernel_routes().is_empty());
// It gets its address: the next hop is a neighbor now.
machine.add_addresses();
machine.kernel_becomes(&[machine.shown("203.0.113.0/24", machine.neighbor(2))]);
status_becomes(&fib, "the interface is shown", |status| {
status.unresolved.is_empty()
&& status.interfaces.iter().any(|shown| {
shown.name == interface
&& shown.up
&& shown
.addresses
.contains(&(IpAddr::from(machine.neighbor(1)), 24))
})
});
// It loses it. The kernel drops the route by itself; what matters is
// that we know it has.
ip(&format!("addr del 10.3.0.1/24 dev {interface}"));
machine.kernel_becomes(&[]);
status_becomes(&fib, "unresolved again", |status| {
status.installed == 0 && status.unresolved.len() == 1
});
// The interface goes down and comes straight back, faster than we
// work the routes out again. The kernel dropped the route when it went
// down, though nothing about the route has changed for us.
machine.add_addresses_v4();
machine.kernel_becomes(&[machine.shown("203.0.113.0/24", machine.neighbor(2))]);
ip(&format!("link set {interface} down"));
ip(&format!("link set {interface} up"));
assert!(machine.kernel_routes().is_empty(), "the kernel dropped it");
machine.kernel_becomes(&[machine.shown("203.0.113.0/24", machine.neighbor(2))]);
ip(&format!("addr del 10.3.0.1/24 dev {interface}"));
machine.kernel_becomes(&[]);
// And gets it back, with the interface having been down meanwhile.
ip(&format!("link set {interface} down"));
ip(&format!("addr add 10.3.0.1/24 dev {interface}"));
ip(&format!("link set {interface} up"));
machine.kernel_becomes(&[machine.shown("203.0.113.0/24", machine.neighbor(2))]);
fib.stop().await;
task.await.unwrap().unwrap();
machine.kernel_becomes(&[]);
}
/// A next hop that is no neighbor goes the way the kernel's own routes
/// say: a static route's gateway, a tunnel's interface, a nexthop object.
/// When those routes change, so do ours; a route that discards packets is
/// the way to nowhere.
#[tokio::test(flavor = "multi_thread")]
#[ignore = "changes the machine's routes; run scripts/kernel-tests.sh"]
async fn a_far_next_hop_goes_the_way_the_kernels_routes_say() {
let machine = Machine::new(10);
let interface = machine.interface();
let peer = machine.peer(2);
// Next hops that are not in any network of ours.
let far = machine.via("172.31.10.9".parse().unwrap(), 0);
let farther = machine.via("172.31.11.9".parse().unwrap(), 0);
machine
.rib
.announce(peer, &far, &[prefix("203.0.113.0/24")]);
machine
.rib
.announce(peer, &farther, &[prefix("198.18.0.0/15")]);
let (fib, task) = machine.start();
status_becomes(&fib, "no way to either", |status| {
status.synced && status.unresolved == [("no route to the next hop", 2)]
});
assert!(machine.kernel_routes().is_empty());
// A static route with a gateway says the way to one: the route goes
// in with that gateway.
ip(&format!(
"route add 172.31.10.0/24 via 10.10.0.9 dev {interface}"
));
machine.kernel_becomes(&[machine.shown("203.0.113.0/24", "10.10.0.9")]);
// A route onto the interface, as a tunnel's is, says the way to the
// other: the next hop itself is the neighbor there.
ip(&format!("route add 172.31.11.0/24 dev {interface}"));
machine.kernel_becomes(&[
machine.shown("203.0.113.0/24", "10.10.0.9"),
machine.shown("198.18.0.0/15", "172.31.11.9"),
]);
status_becomes(&fib, "both resolved", |status| {
status.unresolved.is_empty() && status.installed == 2
});
// The way changes: so does the route. Then it goes, and so does the
// route.
ip(&format!(
"route replace 172.31.10.0/24 via 10.10.0.8 dev {interface}"
));
machine.kernel_becomes(&[
machine.shown("203.0.113.0/24", "10.10.0.8"),
machine.shown("198.18.0.0/15", "172.31.11.9"),
]);
ip("route del 172.31.10.0/24");
machine.kernel_becomes(&[machine.shown("198.18.0.0/15", "172.31.11.9")]);
status_becomes(&fib, "one unresolved again", |status| {
status.unresolved == [("no route to the next hop", 1)]
});
// A route that names a nexthop object goes where the object says.
ip(&format!(
"nexthop add id 1010 via 10.10.0.7 dev {interface}"
));
ip("route add 172.31.10.0/24 nhid 1010");
machine.kernel_becomes(&[
machine.shown("203.0.113.0/24", "10.10.0.7"),
machine.shown("198.18.0.0/15", "172.31.11.9"),
]);
// The tunnel's route becomes a blackhole: the way to nowhere.
ip("route replace blackhole 172.31.11.0/24");
machine.kernel_becomes(&[machine.shown("203.0.113.0/24", "10.10.0.7")]);
status_becomes(&fib, "discarded", |status| {
status.unresolved == [("the route to the next hop discards packets", 1)]
});
ip("route del 172.31.10.0/24");
ip("route del blackhole 172.31.11.0/24");
ip("nexthop del id 1010");
fib.stop().await;
task.await.unwrap().unwrap();
machine.kernel_becomes(&[]);
}
/// A route that asks for its packets to be dropped goes into the kernel
/// as a blackhole route, whatever its next hop, and comes out again with
/// the request.
#[tokio::test(flavor = "multi_thread")]
#[ignore = "changes the machine's routes; run scripts/kernel-tests.sh"]
async fn a_blackhole_request_becomes_a_blackhole_route() {
let machine = Machine::new(13);
let peer = machine.peer(2);
// Asked for from a next hop nothing is a neighbor of, and with one
// ordinary route beside it to show the two apart.
let mut attributes = vec![
PathAttribute::Origin(OriginPathAttribute::IGP),
AsPathAttribute::from_asns(vec![65002]),
PathAttribute::NextHop(NextHopPathAttribute(Ipv4Addr::new(172, 31, 11, 9))),
PathAttribute::Communities(CommunitiesPathAttribute(vec![(65002, 1), (0xFFFF, 0x029A)])),
];
let blackhole = machine.rib.attrs().intern(&attributes, None).unwrap();
attributes.pop();
attributes.push(PathAttribute::Communities(CommunitiesPathAttribute(vec![
(65002, 1),
])));
let plain = machine.rib.attrs().intern(&attributes, None).unwrap();
machine.rib.announce(
peer,
&blackhole,
&[prefix("203.0.113.9/32"), prefix("2001:db8:100::9/128")],
);
machine
.rib
.announce(peer, &plain, &[prefix("198.18.0.0/15")]);
let (fib, task) = machine.start();
// The kernel puts an IPv6 blackhole route on lo, and says so.
machine.kernel_becomes(&[
"blackhole 203.0.113.9 proto bgp metric 20".into(),
"blackhole 2001:db8:100::9 dev lo proto bgp metric 20".into(),
]);
status_becomes(&fib, "two blackholes, one unresolved", |status| {
status.synced
&& status.installed == 2
&& status.blackholes == 2
&& status.unresolved == [("no route to the next hop", 1)]
});
// The request is withdrawn: the blackhole goes.
machine.rib.withdraw(peer, &[prefix("203.0.113.9/32")]);
machine.kernel_becomes(&["blackhole 2001:db8:100::9 dev lo proto bgp metric 20".into()]);
// A blackhole left by an earlier run is found and kept like any other.
fib.leave().await;
task.await.unwrap().unwrap();
let (fib, task) = machine.start();
status_becomes(&fib, "the blackhole is adopted", |status| {
status.synced && status.installed == 1 && status.blackholes == 1
});
fib.stop().await;
task.await.unwrap().unwrap();
machine.kernel_becomes(&[]);
}
/// A table's worth of routes goes in, and comes out, in reasonable time.
#[tokio::test(flavor = "multi_thread")]
#[ignore = "changes the machine's routes; run scripts/kernel-tests.sh"]
async fn many_routes_are_installed_and_removed() {
// FERRIX_ROUTES=1000000 for the size of the Internet's IPv4 table.
let routes: usize = std::env::var("FERRIX_ROUTES")
.ok()
.and_then(|routes| routes.parse().ok())
.unwrap_or(200_000);
let machine = Machine::new(4);
let peer = machine.peer(2);
let via = machine.via(machine.neighbor(2).into(), 0);
let prefixes: Vec<IpPrefix> = (0..routes as u32)
.map(|n| {
let [_, a, b, c] = n.to_be_bytes();
IpPrefix::new(
bgp_primitives::constants::IpVersion::Ipv4,
[100 + a, b, c, 0],
24,
)
.unwrap()
})
.collect();
machine.rib.announce(peer, &via, &prefixes);
let started = Instant::now();
let (fib, task) = machine.start();
status_becomes(&fib, "all installed", |status| {
status.synced && status.installed == routes
});
let installed_in = started.elapsed();
assert_eq!(machine.kernel_routes().len(), routes);
assert_eq!(fib.status().errors, 0, "{:?}", fib.status().last_error);
let started = Instant::now();
fib.stop().await;
task.await.unwrap().unwrap();
let removed_in = started.elapsed();
assert!(machine.kernel_routes().is_empty());
println!(
"{routes} routes: installed in {installed_in:?} ({:.0}/s), removed in {removed_in:?}",
routes as f64 / installed_in.as_secs_f64()
);
}
/// Someone empties the table behind our back (`ip route flush`), and then
/// the peers withdraw everything: the kernel refuses every removal, a
/// table's worth of rejections at once. That is more than the socket holds
/// answers for, and must be no more than a reason to look again.
#[tokio::test(flavor = "multi_thread")]
#[ignore = "changes the machine's routes; run scripts/kernel-tests.sh"]
async fn a_flood_of_refusals_does_not_stop_the_daemon() {
const ROUTES: u32 = 20_000;
let mut machine = Machine::new(11);
machine.nexthop_objects = false;
let peer = machine.peer(2);
let via = machine.via(machine.neighbor(2).into(), 0);
let prefixes: Vec<IpPrefix> = (0..ROUTES)
.map(|n| {
let [_, a, b, c] = n.to_be_bytes();
IpPrefix::new(
bgp_primitives::constants::IpVersion::Ipv4,
[100 + a, b, c, 0],
24,
)
.unwrap()
})
.collect();
machine.rib.announce(peer, &via, &prefixes);
let (fib, task) = machine.start();
status_becomes(&fib, "all installed", |status| {
status.synced && status.installed == ROUTES as usize
});
ip(&format!("route flush table {}", machine.table()));
assert!(machine.kernel_routes().is_empty());
let routes: Vec<bgp_primitives::ip_prefix::Nlri> =
prefixes.iter().map(|prefix| (*prefix).into()).collect();
machine.rib.withdraw(peer, &routes);
status_becomes(&fib, "nothing installed", |status| {
status.synced && status.installed == 0
});
assert!(
!task.is_finished(),
"the task that talks to the kernel ended"
);
// And it goes on working.
machine.rib.announce(peer, &via, &prefixes[..3]);
status_becomes(&fib, "three installed", |status| {
status.synced && status.installed == 3
});
assert_eq!(machine.kernel_routes().len(), 3);
assert!(!task.is_finished());
fib.stop().await;
task.await.unwrap().unwrap();
}
/// Routes name a nexthop object for their gateway where the kernel has
/// such objects. The object is ours: made when first needed, put back if
/// someone removes it, and removed when we stop. Other objects are not
/// touched.
#[tokio::test(flavor = "multi_thread")]
#[ignore = "changes the machine's routes; run scripts/kernel-tests.sh"]
async fn routes_name_a_nexthop_object_that_is_ours_to_keep() {
let machine = Machine::new(5);
let interface = machine.interface();
// Someone else's object, to the same gateway even.
ip(&format!(
"nexthop add id 50005 via 10.5.0.2 dev {interface} protocol bgp"
));
let nexthops = || -> Vec<String> {
ip("nexthop show")
.lines()
.filter(|line| line.contains(&format!("dev {interface} ")))
.map(|line| line.trim().to_string())
.collect()
};
let peer = machine.peer(2);
let via = machine.via(machine.neighbor(2).into(), 0);
machine.rib.announce(
peer,
&via,
&[prefix("203.0.113.0/24"), prefix("198.18.0.0/15")],
);
let (fib, task) = machine.start();
let both = [
machine.shown("203.0.113.0/24", machine.neighbor(2)),
machine.shown("198.18.0.0/15", machine.neighbor(2)),
];
machine.kernel_becomes(&both);
// One object of ours beside the other, and both routes name it.
let ours: Vec<String> = nexthops()
.into_iter()
.filter(|line| !line.starts_with("id 50005 "))
.collect();
assert_eq!(ours.len(), 1, "{:?}", nexthops());
assert!(ours[0].contains("via 10.5.0.2") && ours[0].contains("proto bgp"));
let id = ours[0].split_whitespace().nth(1).unwrap().to_string();
let raw = ip(&format!("route show table {}", machine.table()));
assert_eq!(raw.matches(&format!("nhid {id} ")).count(), 2, "{raw}");
// Someone removes our object, and the kernel our routes with it. We
// are told, and put both back.
ip(&format!("nexthop del id {id}"));
machine.kernel_becomes(&both);
// When we stop, our object goes with our routes; the other stays.
fib.stop().await;
task.await.unwrap().unwrap();
machine.kernel_becomes(&[]);
assert_eq!(nexthops().len(), 1, "{:?}", nexthops());
assert!(nexthops()[0].starts_with("id 50005 "));
ip("nexthop del id 50005");
}
/// On a kernel without nexthop objects, or where the configuration says
/// so, every route carries its own gateway.
#[tokio::test(flavor = "multi_thread")]
#[ignore = "changes the machine's routes; run scripts/kernel-tests.sh"]
async fn routes_can_carry_their_own_gateway() {
let mut machine = Machine::new(6);
machine.nexthop_objects = false;
let peer = machine.peer(2);
let via = machine.via(machine.neighbor(2).into(), 0);
machine
.rib
.announce(peer, &via, &[prefix("203.0.113.0/24")]);
let (fib, task) = machine.start();
machine.kernel_becomes(&[machine.shown("203.0.113.0/24", machine.neighbor(2))]);
let raw = ip(&format!("route show table {}", machine.table()));
assert!(!raw.contains("nhid"), "{raw}");
assert!(
!ip("nexthop show").contains(&format!("dev {} ", machine.interface())),
"no nexthop object was made"
);
// The next hop changes, and the route goes.
let other = machine.via(machine.neighbor(3).into(), 0);
machine
.rib
.announce(peer, &other, &[prefix("203.0.113.0/24")]);
machine.kernel_becomes(&[machine.shown("203.0.113.0/24", machine.neighbor(3))]);
machine.rib.withdraw(peer, &[prefix("203.0.113.0/24")]);
machine.kernel_becomes(&[]);
fib.stop().await;
task.await.unwrap().unwrap();
}
/// When the daemon starts, its table is empty until its peers have sent
/// theirs. The routes an earlier run left are kept for a while, so that
/// the machine goes on forwarding; then those nobody announced again go.
#[tokio::test(flavor = "multi_thread")]
#[ignore = "changes the machine's routes; run scripts/kernel-tests.sh"]
async fn routes_of_an_earlier_run_are_kept_while_peers_come_back() {
let mut machine = Machine::new(7);
machine.stale_time = Duration::from_secs(3);
let (table, interface) = (machine.table(), machine.interface());
let add = |route: &str| ip(&format!("route add {route} dev {interface} table {table}"));
add("203.0.113.0/24 via 10.7.0.2 proto bgp metric 20");
add("198.18.0.0/15 via 10.7.0.2 proto bgp metric 20");
let left = [
machine.shown("203.0.113.0/24", machine.neighbor(2)),
machine.shown("198.18.0.0/15", machine.neighbor(2)),
];
// The daemon starts with nothing in its table: both stay.
let (fib, task) = machine.start();
status_becomes(&fib, "in step, keeping two", |status| {
status.synced && status.installed == 2 && status.kept == 2
});
assert_eq!(machine.kernel_routes(), left.iter().cloned().collect());
// A peer comes back with one of them.
let peer = machine.peer(2);
let via = machine.via(machine.neighbor(2).into(), 0);
machine
.rib
.announce(peer, &via, &[prefix("203.0.113.0/24")]);
status_becomes(&fib, "one confirmed", |status| status.kept == 1);
assert_eq!(machine.kernel_routes(), left.iter().cloned().collect());
// Time is up: the one nobody announced goes.
machine.kernel_becomes(&[machine.shown("203.0.113.0/24", machine.neighbor(2))]);
status_becomes(&fib, "none kept", |status| {
status.installed == 1 && status.kept == 0
});
fib.stop().await;
task.await.unwrap().unwrap();
}
/// A restart that is to go unnoticed (graceful restart): the routes are
/// left in the kernel, by way of their nexthop objects; the next run
/// finds them, says how many, and lets those nobody announced again go
/// as soon as it is told the peers have all sent theirs.
#[tokio::test(flavor = "multi_thread")]
#[ignore = "changes the machine's routes; run scripts/kernel-tests.sh"]
async fn routes_stay_in_the_kernel_over_a_restart() {
let mut machine = Machine::new(12);
machine.stale_time = Duration::from_secs(60);
let (fib, task) = machine.start();
assert_eq!(fib.adopted().await, 0);
let peer = machine.peer(2);
let via = machine.via(machine.neighbor(2).into(), 0);
machine.rib.announce(
peer,
&via,
&[prefix("203.0.113.0/24"), prefix("198.18.0.0/15")],
);
let both = [
machine.shown("198.18.0.0/15", machine.neighbor(2)),
machine.shown("203.0.113.0/24", machine.neighbor(2)),
];
machine.kernel_becomes(&both);
fib.leave().await;
task.await.unwrap().unwrap();
assert_eq!(machine.kernel_routes(), both.iter().cloned().collect());
// The next run, with a table that is empty until the peers are back.
machine.rib = Arc::new(RibTable::new(4));
let (fib, task) = machine.start();
assert_eq!(fib.adopted().await, 2);
status_becomes(&fib, "keeping both", |status| status.kept == 2);
let peer = machine.peer(2);
let via = machine.via(machine.neighbor(2).into(), 0);
machine
.rib
.announce(peer, &via, &[prefix("203.0.113.0/24")]);
status_becomes(&fib, "one confirmed", |status| status.kept == 1);
// Every peer has sent its routes: what was not among them goes now,
// and not a minute later.
fib.release_kept();
machine.kernel_becomes(&[machine.shown("203.0.113.0/24", machine.neighbor(2))]);
fib.stop().await;
task.await.unwrap().unwrap();
}
/// An IPv4 route whose next hop is an IPv6 neighbor: routes over a link
/// that has only IPv6 addresses (RFC 8950). The kernel takes such a route,
/// by way of a nexthop object or with its gateway spelled out.
#[tokio::test(flavor = "multi_thread")]
#[ignore = "changes the machine's routes; run scripts/kernel-tests.sh"]
async fn an_ipv4_route_goes_by_way_of_an_ipv6_neighbor() {
for (n, nexthop_objects) in [(8, true), (9, false)] {
let mut machine = Machine::new(n);
machine.nexthop_objects = nexthop_objects;
// Only IPv6 on the link.
ip(&format!(
"addr del 10.{n}.0.1/24 dev {}",
machine.interface()
));
let neighbor: IpAddr = format!("2001:db8:{n}::2").parse().unwrap();
let peer = machine
.rib
.add_peer(PeerInfo {
asn: 65002,
router_id: machine.neighbor(2),
address: neighbor,
interface: 0,
ibgp: false,
reflector_client: false,
})
.unwrap();
// Both families by way of the same neighbor.
let via = machine.via(neighbor, 0);
machine.rib.announce(
peer,
&via,
&[prefix("203.0.113.0/24"), prefix("2001:db8:100::/48")],
);
// And an IPv4 route whose next hop has a link-local address too,
// with a global one that is not on any link of ours.
let link_local = machine
.rib
.attrs()
.intern(
&[
PathAttribute::Origin(OriginPathAttribute::IGP),
AsPathAttribute::from_asns(vec![65002]),
],
Some(&NlriNextHop::Ipv6WithLl {
global: "2001:db8:ffff::2".parse().unwrap(),
link_local: "fe80::2".parse().unwrap(),
}),
)
.unwrap();
machine
.rib
.announce(peer, &link_local, &[prefix("198.18.0.0/15")]);
let (fib, task) = machine.start();
let interface = machine.interface();
machine.kernel_becomes(&[
format!("203.0.113.0/24 via inet6 {neighbor} dev {interface} proto bgp metric 20"),
format!("198.18.0.0/15 via inet6 fe80::2 dev {interface} proto bgp metric 20"),
machine.shown("2001:db8:100::/48", neighbor),
]);
assert_eq!(fib.status().errors, 0, "{:?}", fib.status().last_error);
fib.stop().await;
task.await.unwrap().unwrap();
machine.kernel_becomes(&[]);
}
}
crates/integration-tests/Cargo.toml
+1 -6
View File
@@ -9,22 +9,17 @@ version.workspace = true
publish = false
[dependencies]
bgp-fib.workspace = true
bgp-packet.workspace = true
bgp-primitives.workspace = true
bgp-rib.workspace = true
bgp-rpki.workspace = true
bgp-server.workspace = true
bgp-web.workspace = true
bytes.workspace = true
clap.workspace = true
eyre.workspace = true
reqwest.workspace = true
serde_json.workspace = true
tokio.workspace = true
# Used by the examples only.
[dev-dependencies]
clap.workspace = true
[lints]
workspace = true
crates/integration-tests/examples/fake_peer.rs
+35 -14
View File
@@ -16,9 +16,11 @@
use std::net::Ipv4Addr;
use std::time::Duration;
use bgp_integration_tests::announcement::{Announcement, withdrawal};
use bgp_integration_tests::test_peer::TestPeer;
use bgp_packet::message::{Message, PathAttribute, UpdateMessage};
use bgp_packet::message::{
AsPathAttribute, CommunitiesPathAttribute, Message, NextHopPathAttribute, OriginPathAttribute,
PathAttribute, UpdateMessage,
};
use bgp_primitives::ip_prefix::IpPrefix;
use clap::Parser;
use eyre::Result;
@@ -86,8 +88,8 @@ async fn run(args: &Args, mut peer: TestPeer, addr: std::net::SocketAddr) -> Res
for p in &args.announce {
let prefix = IpPrefix::try_from(p.as_str())?;
peer.announce(announce(prefix, vec![args.asn], vec![(args.asn as u16, 1)]))
.await?;
let update = announce(prefix, vec![args.asn], vec![(args.asn as u16, 1)]);
peer.send(&Message::Update(update)).await?;
println!("announced {prefix}");
}
@@ -96,8 +98,7 @@ async fn run(args: &Args, mut peer: TestPeer, addr: std::net::SocketAddr) -> Res
loop {
tokio::select! {
_ = ticker.tick() => peer.send(&Message::KeepAlive).await?,
// However long the server is quiet: this peer stays on the line.
msg = peer.recv_untimed() => match msg? {
msg = peer.recv() => match msg? {
Message::KeepAlive => {}
Message::Update(u) => {
for w in &u.withdrawn_routes {
@@ -120,10 +121,22 @@ async fn run(args: &Args, mut peer: TestPeer, addr: std::net::SocketAddr) -> Res
}
}
fn announce(prefix: IpPrefix, as_path: Vec<u32>, communities: Vec<(u16, u16)>) -> Announcement {
Announcement::of([prefix.into()], &as_path)
.with_next_hop(Ipv4Addr::new(10, 0, 0, 2))
.with_communities(&communities)
fn announce(prefix: IpPrefix, as_path: Vec<u32>, communities: Vec<(u16, u16)>) -> UpdateMessage {
let mut path_attributes = vec![
PathAttribute::Origin(OriginPathAttribute::IGP),
AsPathAttribute::from_asns(as_path),
PathAttribute::NextHop(NextHopPathAttribute(Ipv4Addr::new(10, 0, 0, 2))),
];
if !communities.is_empty() {
path_attributes.push(PathAttribute::Communities(CommunitiesPathAttribute(
communities,
)));
}
UpdateMessage {
withdrawn_routes: vec![],
path_attributes,
announced_routes: vec![prefix.into()],
}
}
async fn echo_back(peer: &mut TestPeer, u: &UpdateMessage, asn: u32) -> Result<()> {
@@ -144,13 +157,21 @@ async fn echo_back(peer: &mut TestPeer, u: &UpdateMessage, asn: u32) -> Result<(
}
for nlri in &u.announced_routes {
let prefix = nlri.prefix;
peer.announce(announce(prefix, as_path.clone(), communities.clone()))
.await?;
peer.send(&Message::Update(announce(
prefix,
as_path.clone(),
communities.clone(),
)))
.await?;
println!(" echoed {prefix} back with path {as_path:?}");
}
if !u.withdrawn_routes.is_empty() {
peer.announce(withdrawal(u.withdrawn_routes.iter().cloned()))
.await?;
peer.send(&Message::Update(UpdateMessage {
withdrawn_routes: u.withdrawn_routes.clone(),
path_attributes: vec![],
announced_routes: vec![],
}))
.await?;
println!(
" echoed withdraw of {:?}",
u.withdrawn_routes
crates/integration-tests/src/announcement.rs
-110
View File
@@ -1,110 +0,0 @@
//! The UPDATEs a test peer sends, built from what a test has to say about
//! them and nothing more.
use std::net::Ipv4Addr;
use bgp_packet::message::{
AsPathAttribute, CommunitiesPathAttribute, Message, NextHopPathAttribute, OriginPathAttribute,
PathAttribute, UpdateMessage,
};
use bgp_primitives::ip_prefix::Nlri;
use crate::prefix;
/// An UPDATE announcing IPv4 routes as the peers of these tests do: with
/// ORIGIN IGP, the AS path given, and 192.0.2.1 as the next hop unless
/// another is set. Attributes are written in order of type code.
#[derive(Debug, Clone)]
pub struct Announcement {
routes: Vec<Nlri>,
path: Vec<u32>,
next_hop: Ipv4Addr,
only_to_customer: Option<u32>,
communities: Vec<(u16, u16)>,
more: Vec<PathAttribute>,
}
/// Announces `prefixes`, written as text, with the AS path `path`.
pub fn announcement(prefixes: &[&str], path: &[u32]) -> Announcement {
Announcement::of(prefixes.iter().map(|p| prefix(p).into()), path)
}
/// Withdraws `routes`.
pub fn withdrawal(routes: impl IntoIterator<Item = Nlri>) -> UpdateMessage {
UpdateMessage {
withdrawn_routes: routes.into_iter().collect(),
path_attributes: vec![],
announced_routes: vec![],
}
}
impl Announcement {
/// Announces `routes`, prefixes with or without path identifiers, with
/// the AS path `path`.
pub fn of(routes: impl IntoIterator<Item = Nlri>, path: &[u32]) -> Self {
Self {
routes: routes.into_iter().collect(),
path: path.to_vec(),
next_hop: Ipv4Addr::new(192, 0, 2, 1),
only_to_customer: None,
communities: Vec::new(),
more: Vec::new(),
}
}
pub fn with_next_hop(mut self, next_hop: Ipv4Addr) -> Self {
self.next_hop = next_hop;
self
}
/// With a COMMUNITIES attribute carrying `communities`, if there are
/// any.
pub fn with_communities(mut self, communities: &[(u16, u16)]) -> Self {
self.communities = communities.to_vec();
self
}
/// Marked as only to go to customers, by `asn` (RFC 9234).
pub fn with_only_to_customer(mut self, asn: u32) -> Self {
self.only_to_customer = Some(asn);
self
}
/// With `attributes` after the mandatory ones, as given.
pub fn with_attributes(mut self, attributes: impl IntoIterator<Item = PathAttribute>) -> Self {
self.more.extend(attributes);
self
}
}
impl From<Announcement> for UpdateMessage {
fn from(announcement: Announcement) -> Self {
let mut path_attributes = vec![
PathAttribute::Origin(OriginPathAttribute::IGP),
AsPathAttribute::from_asns(announcement.path),
PathAttribute::NextHop(NextHopPathAttribute(announcement.next_hop)),
];
if !announcement.communities.is_empty() {
path_attributes.push(PathAttribute::Communities(CommunitiesPathAttribute(
announcement.communities,
)));
}
path_attributes.extend(
announcement
.only_to_customer
.map(PathAttribute::OnlyToCustomer),
);
path_attributes.extend(announcement.more);
UpdateMessage {
withdrawn_routes: vec![],
path_attributes,
announced_routes: announcement.routes,
}
}
}
impl From<Announcement> for Message {
fn from(announcement: Announcement) -> Self {
Message::Update(announcement.into())
}
}
crates/integration-tests/src/lib.rs
-99
View File
@@ -1,101 +1,2 @@
pub mod announcement;
pub mod test_cache;
pub mod test_peer;
pub mod test_server;
use std::time::Duration;
use bgp_primitives::ip_prefix::IpPrefix;
use bgp_server::address::Endpoint;
use bgp_server::config::PeerConfig;
use bgp_server::family::FamilySet;
use eyre::{Result, bail};
use serde_json::Value;
/// How long a test waits for something the server does on its own: a
/// route reaching the table, a session coming up, a message arriving.
/// Everything happens on the loopback, so it is generous.
pub const TIMEOUT: Duration = Duration::from_secs(5);
/// The policy a peer has with nothing configured, but for one thing: the
/// peers of these tests are all at 127.0.0.1, which no route can have as
/// its next hop, so their routes name another.
pub fn default_policy() -> bgp_server::policy::ImportPolicy {
bgp_server::policy::ImportPolicy {
accept_third_party_next_hop: true,
accept_blackholes: false,
..Default::default()
}
}
/// A prefix written as text. The tests' prefixes are literals, so a bad
/// one is a mistake in the test.
pub fn prefix(s: &str) -> IpPrefix {
IpPrefix::try_from(s).unwrap()
}
/// A configured peer with nothing said about it but where it is and what
/// AS it is in: the default hold time, every family, the default policy of
/// these tests, and no role, limits, groups or ADD-PATH. A test says what
/// else with struct update syntax: `PeerConfig { role: Some(..),
/// ..peer_config(..) }`.
pub fn peer_config(name: &str, endpoint: impl Into<Endpoint>, remote_asn: u32) -> PeerConfig {
PeerConfig {
name: name.into(),
endpoint: endpoint.into(),
remote_asn,
hold_time_secs: 90,
local_address: None,
families: FamilySet::supported(),
add_path: None,
import: default_policy(),
role: None,
require_role: false,
export: Default::default(),
export_limits: None,
graceful_restart: Default::default(),
groups: Vec::new(),
reflector_client: false,
passive: false,
expected_routes: None,
}
}
/// Waits until `done` holds, looking every few milliseconds, or fails
/// with `what` once [`TIMEOUT`] has passed.
pub async fn wait_until(what: &str, mut done: impl FnMut() -> bool) -> Result<()> {
let waited = tokio::time::timeout(TIMEOUT, async {
while !done() {
tokio::time::sleep(Duration::from_millis(5)).await;
}
})
.await;
if waited.is_err() {
bail!("timed out waiting until {what}");
}
Ok(())
}
/// Waits until `get` has something, and returns it; fails with `what`
/// once [`TIMEOUT`] has passed.
pub async fn wait_for<T>(what: &str, mut get: impl FnMut() -> Option<T>) -> Result<T> {
let mut got = None;
wait_until(what, || {
got = get();
got.is_some()
})
.await?;
Ok(got.expect("the wait ended with something"))
}
/// `GET`s `url` and reads the JSON it answers with. An error status is an
/// error.
pub async fn get_json(client: &reqwest::Client, url: &str) -> Result<Value> {
Ok(client
.get(url)
.send()
.await?
.error_for_status()?
.json()
.await?)
}
crates/integration-tests/src/test_cache.rs
-156
View File
@@ -1,156 +0,0 @@
//! An RPKI cache for tests: an RTR server that hands out whatever VRPs and
//! ASPAs the test says there are.
use std::net::{Ipv4Addr, SocketAddr};
use std::sync::{Arc, Mutex};
use bgp_rpki::Vrp;
use bgp_rpki::rtr::{Pdu, Timers, VERSION_2};
use bytes::BytesMut;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::{TcpListener, TcpStream};
use tokio::sync::watch;
use tokio::task::JoinHandle;
const SESSION: u16 = 0x1234;
/// What the cache has to give.
#[derive(Clone, Default)]
struct Data {
vrps: Vec<Vrp>,
/// Customer ASes and their providers.
aspas: Vec<(u32, Vec<u32>)>,
serial: u32,
}
/// The cache. Stops, closing its connections, when dropped.
pub struct TestCache {
address: SocketAddr,
data: Arc<Mutex<Data>>,
/// The serial, for telling connected routers there is something new.
serial: watch::Sender<u32>,
task: JoinHandle<()>,
}
impl TestCache {
pub async fn start(vrps: Vec<Vrp>) -> Self {
Self::start_with(vrps, vec![]).await
}
/// A cache with ASPAs too: customer ASes and their providers.
pub async fn start_with(vrps: Vec<Vrp>, aspas: Vec<(u32, Vec<u32>)>) -> Self {
let listener = TcpListener::bind((Ipv4Addr::LOCALHOST, 0)).await.unwrap();
let address = listener.local_addr().unwrap();
let data = Arc::new(Mutex::new(Data {
vrps,
aspas,
serial: 1,
}));
let (serial, _) = watch::channel(1);
let task = tokio::spawn({
let (data, serial) = (Arc::clone(&data), serial.clone());
async move {
// Connections end with this task: they are part of it.
let mut connections = tokio::task::JoinSet::new();
loop {
let Ok((stream, _)) = listener.accept().await else {
return;
};
connections.spawn(serve(stream, Arc::clone(&data), serial.subscribe()));
}
}
});
Self {
address,
data,
serial,
task,
}
}
pub fn address(&self) -> SocketAddr {
self.address
}
/// Replaces the VRPs and tells connected routers.
pub fn set(&self, vrps: Vec<Vrp>) {
self.change(|data| data.vrps = vrps);
}
/// Replaces the ASPAs and tells connected routers.
pub fn set_aspas(&self, aspas: Vec<(u32, Vec<u32>)>) {
self.change(|data| data.aspas = aspas);
}
fn change(&self, change: impl FnOnce(&mut Data)) {
let serial = {
let mut data = self.data.lock().unwrap();
change(&mut data);
data.serial += 1;
data.serial
};
self.serial.send_replace(serial);
}
}
impl Drop for TestCache {
fn drop(&mut self) {
self.task.abort();
}
}
/// One router's connection. Asked for changes, this cache always says to
/// ask for everything: the router works out what changed.
async fn serve(mut stream: TcpStream, data: Arc<Mutex<Data>>, mut serial: watch::Receiver<u32>) {
let mut buf = BytesMut::new();
// The version the router speaks, once it has said something.
let mut spoken = VERSION_2;
loop {
let mut out = BytesMut::new();
tokio::select! {
read = stream.read_buf(&mut buf) => {
if !matches!(read, Ok(read) if read > 0) {
return;
}
while let Ok(Some((version, pdu, used))) = Pdu::decode(&buf) {
let _ = buf.split_to(used);
spoken = version;
match pdu {
Pdu::SerialQuery { .. } => Pdu::CacheReset.encode(version, &mut out),
Pdu::ResetQuery => {
let data = data.lock().unwrap().clone();
Pdu::CacheResponse { session_id: SESSION }.encode(version, &mut out);
for vrp in data.vrps {
Pdu::Prefix { announce: true, vrp }.encode(version, &mut out);
}
// Only version 2 has ASPAs.
for (customer, providers) in data.aspas {
if version >= VERSION_2 {
Pdu::Aspa { announce: true, customer, providers }
.encode(version, &mut out);
}
}
Pdu::EndOfData {
session_id: SESSION,
serial: data.serial,
timers: Some(Timers::default()),
}
.encode(version, &mut out);
}
_ => return,
}
}
}
changed = serial.changed() => {
if changed.is_err() {
return;
}
let serial = *serial.borrow_and_update();
Pdu::SerialNotify { session_id: SESSION, serial }.encode(spoken, &mut out);
}
}
if stream.write_all(&out).await.is_err() {
return;
}
}
}
crates/integration-tests/src/test_peer.rs
+18 -162
View File
@@ -1,55 +1,13 @@
use std::net::{Ipv4Addr, SocketAddr};
use std::time::Duration;
use bgp_packet::message::{BGP_HEADER_LEN, Message, NotificationMessage, UpdateMessage};
use bgp_packet::open::{
AddPathEntry, Capability, ExtendedNextHopEntry, OpenMessage, OpenOption, SendReceive,
};
use bgp_packet::message::{BGP_HEADER_LEN, Message, UpdateMessage};
use bgp_packet::open::{Capability, OpenMessage, OpenOption};
use bgp_primitives::constants::{Afi, AfiSafi, Safi};
use bgp_primitives::parser::ParserContext;
use bytes::BytesMut;
use eyre::{Result, bail, eyre};
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::tcp::OwnedWriteHalf;
use tokio::net::{TcpListener, TcpStream};
use tokio::task::JoinHandle;
use crate::TIMEOUT;
/// The capabilities a route collector offers in its OPEN: four-octet AS
/// numbers as `asn`, each of `families`, and to be sent several routes
/// for a prefix (ADD-PATH, RFC 7911) in each of them; with
/// `ipv4_over_ipv6`, IPv4 routes with IPv6 next hops too (RFC 8950), as a
/// collector on an IPv6-only session wants. For the tests and benchmarks
/// that play a collector taking a router's whole table.
pub fn collector_capabilities(
asn: u32,
families: &[AfiSafi],
ipv4_over_ipv6: bool,
) -> Vec<Capability> {
let mut capabilities = vec![Capability::FourByteAsn { asn }];
capabilities.extend(families.iter().copied().map(Capability::multiprotocol));
capabilities.push(Capability::AddPath {
entries: families
.iter()
.map(|family| AddPathEntry {
afi: family.afi,
safi: family.safi,
send_recv: SendReceive::Receive,
})
.collect(),
});
if ipv4_over_ipv6 {
capabilities.push(Capability::ExtendedNextHop {
entries: vec![ExtendedNextHopEntry {
afi: Afi::IPV4,
safi: Safi::UNICAST,
nh_afi: Afi::IPV6,
}],
});
}
capabilities
}
use tokio::net::TcpStream;
/// A minimal BGP peer for use in tests. Connects over TCP and can exchange
/// framed BGP messages using the `bgp-packet` wire format.
@@ -72,13 +30,6 @@ impl TestPeer {
Ok(Self::from_stream(stream))
}
/// The next connection to `listener`, as the server dials it. It is
/// waited for at most [`TIMEOUT`].
pub async fn accept(listener: &TcpListener) -> Result<Self> {
let (stream, _) = tokio::time::timeout(TIMEOUT, listener.accept()).await??;
Ok(Self::from_stream(stream))
}
/// Wraps an already-connected stream, e.g. one accepted from a server
/// that dialled us.
pub fn from_stream(stream: TcpStream) -> Self {
@@ -90,43 +41,6 @@ impl TestPeer {
}
}
/// The connection itself, for a test or a tool that goes on without
/// the framing, once the handshake is done.
pub fn into_stream(self) -> TcpStream {
self.stream
}
/// The connection with a task reading and dropping whatever the server
/// sends over it, and the half that is left for writing: for a peer
/// that has nothing to hear, as one feeding a table or taking one by
/// the megabyte has not, and whose writer must never be held up by
/// what it has not read. The task ends with the connection, or when
/// its handle is dropped.
pub fn drain(self) -> (OwnedWriteHalf, JoinHandle<()>) {
let (mut reader, writer) = self.stream.into_split();
let sink = tokio::spawn(async move {
let mut sink = vec![0u8; 1 << 16];
while matches!(reader.read(&mut sink).await, Ok(n) if n > 0) {}
});
(writer, sink)
}
/// A KEEPALIVE as it goes on the wire, for whoever writes to the
/// connection without the framing and has to keep the session alive
/// past its hold time.
pub fn keepalive_bytes() -> Vec<u8> {
let mut wire = BytesMut::new();
Message::KeepAlive
.to_wire(&ParserContext::default(), &mut wire)
.expect("a KEEPALIVE is nineteen octets");
wire.to_vec()
}
/// Our end of the connection: what the server sees the peer as.
pub fn local_addr(&self) -> Result<SocketAddr> {
Ok(self.stream.local_addr()?)
}
/// Send a BGP message.
pub async fn send(&mut self, msg: &Message) -> Result<()> {
let send_ctx = ParserContext {
@@ -139,32 +53,8 @@ impl TestPeer {
Ok(())
}
/// Send an UPDATE: an [`Announcement`](crate::announcement::Announcement),
/// a [`withdrawal`](crate::announcement::withdrawal), or one built by
/// hand.
pub async fn announce(&mut self, update: impl Into<UpdateMessage>) -> Result<()> {
self.send(&Message::Update(update.into())).await
}
/// Read exactly one BGP message, waiting at most [`TIMEOUT`] for it. A
/// test that expects nothing to arrive says for how long with
/// [`TestPeer::recv_within`].
/// Read exactly one BGP message.
pub async fn recv(&mut self) -> Result<Message> {
self.recv_within(TIMEOUT).await
}
/// Read exactly one BGP message, waiting at most `wait` for it.
pub async fn recv_within(&mut self, wait: Duration) -> Result<Message> {
match tokio::time::timeout(wait, self.recv_untimed()).await {
Ok(message) => message,
Err(_) => Err(eyre::eyre!("no message from the server within {wait:?}")),
}
}
/// Read exactly one BGP message, however long it takes to come. For a
/// peer that is to stay on the line, like the `fake_peer` example;
/// tests want [`TestPeer::recv`].
pub async fn recv_untimed(&mut self) -> Result<Message> {
let mut header = [0u8; BGP_HEADER_LEN as usize];
self.stream.read_exact(&mut header).await?;
@@ -189,19 +79,7 @@ impl TestPeer {
match self.recv().await? {
Message::Update(u) => return Ok(u),
Message::KeepAlive => {}
other => return Err(eyre::eyre!("Expected UPDATE, got {:?}", other)),
}
}
}
/// Read messages until a NOTIFICATION arrives, skipping KEEPALIVEs. Any
/// other message type is an error.
pub async fn expect_notification(&mut self) -> Result<NotificationMessage> {
loop {
match self.recv().await? {
Message::Notification(n) => return Ok(n),
Message::KeepAlive => {}
other => return Err(eyre::eyre!("expected NOTIFICATION, got {other:?}")),
other => bail!("Expected UPDATE, got {:?}", other),
}
}
}
@@ -210,9 +88,7 @@ impl TestPeer {
pub async fn recv_open(&mut self) -> Result<OpenMessage> {
match self.recv().await? {
Message::Open(o) => {
// AS numbers are four octets wide only if both sides
// offered it (RFC 6793 §4.1): the server's offer is read
// here, ours was noted when the OPEN went out.
// Enable 4-octet ASN parsing if the peer advertised it.
let four_byte = o.options.iter().any(|opt| {
matches!(opt, OpenOption::Capabilities(caps)
if caps.iter().any(|c| matches!(c, Capability::FourByteAsn { .. })))
@@ -220,7 +96,7 @@ impl TestPeer {
self.ctx.four_octet_asn = four_byte && self.offered_four_octet_asn;
Ok(o)
}
other => Err(eyre::eyre!("Expected OPEN, got {:?}", other)),
other => bail!("Expected OPEN, got {:?}", other),
}
}
@@ -310,48 +186,28 @@ impl TestPeer {
async fn finish_handshake(&mut self) -> Result<OpenMessage> {
let server_open = self.recv_open().await?;
// The server's KEEPALIVE says it accepted our OPEN; anything else
// is its refusal, or a server out of step.
// Expect server KEEPALIVE
match self.recv().await? {
Message::KeepAlive => {}
other => {
return Err(eyre::eyre!(
"Expected KEEPALIVE from server, got {:?}",
other
));
}
other => bail!("Expected KEEPALIVE from server, got {:?}", other),
}
// Ours says the same of its OPEN, and the session is established.
// Send our KEEPALIVE
self.send(&Message::KeepAlive).await?;
Ok(server_open)
}
/// Writes `bytes` as they are, with no framing of its own: for a test
/// that sends what no message would be, and for the benchmarks, which
/// send a table encoded once as one stream rather than message by
/// message.
/// Send raw bytes without framing — useful for testing invalid input.
pub async fn send_raw(&mut self, bytes: &[u8]) -> Result<()> {
self.stream.write_all(bytes).await?;
Ok(())
}
/// Whether the server closes the connection within `wait`. Whatever it
/// still sends first, KEEPALIVEs as a rule, is read past: the end of
/// the stream, or anything unreadable, is what counts.
pub async fn closed_within(&mut self, wait: Duration) -> bool {
tokio::time::timeout(wait, async { while self.recv_untimed().await.is_ok() {} })
.await
.is_ok()
}
/// Whether the server closes the connection within [`TIMEOUT`] without
/// having sent anything: what a peer that is not let in sees. A server
/// that sends its OPEN first, or that leaves the connection open, is
/// answering.
pub async fn closed_unanswered(&mut self) -> bool {
matches!(
tokio::time::timeout(TIMEOUT, self.recv_untimed()).await,
Ok(Err(_))
)
/// Returns true if the remote end has closed the connection.
pub async fn is_closed(&mut self) -> bool {
let mut buf = [0u8; 1];
match self.stream.read(&mut buf).await {
Ok(0) | Err(_) => true,
Ok(_) => false,
}
}
}
crates/integration-tests/src/test_server.rs
+8 -172
View File
@@ -1,51 +1,31 @@
use std::collections::BTreeSet;
use std::net::{IpAddr, Ipv4Addr, SocketAddr};
use std::sync::Arc;
use bgp_fib::Fib;
use bgp_packet::message::{NlriNextHop, PathAttribute};
use bgp_packet::open::Capability;
use bgp_primitives::ip_prefix::IpPrefix;
use bgp_rib::LocalRib;
use bgp_rib::table::{Event, PeerIndex, Reader, RibTable};
use bgp_rpki::Rpki;
use bgp_server::config::{Config, PeerConfig};
use bgp_server::config::Config;
use bgp_server::error::ServerError;
use bgp_server::rejections::RejectionLog;
use bgp_server::restart::Restarts;
use bgp_server::server::BgpServer;
use bgp_server::session::{SessionHandle, SessionRegistry};
use bgp_server::session::SessionRegistry;
use bgp_web::{AppState, RouterInfo};
use eyre::{Result, eyre};
use eyre::Result;
use tokio::net::TcpListener;
use tokio::task::JoinHandle;
use crate::test_peer::TestPeer;
use crate::{peer_config, wait_for, wait_until};
/// A BGP server started on an ephemeral port for testing.
/// Aborts the server task when dropped.
pub struct TestServer {
handle: JoinHandle<Result<(), ServerError>>,
/// The single IPv4 loopback address the server is bound to.
pub addr: SocketAddr,
/// Every address it is bound to, in the order configured: `addr`
/// first.
pub addrs: Vec<SocketAddr>,
/// The server's routing table.
pub rib: Arc<RibTable>,
/// Handle to the server's locally originated routes.
pub local_rib: LocalRib,
/// Live view of the server's established sessions.
pub registry: SessionRegistry,
/// What peers announced that was not taken.
pub rejections: RejectionLog,
/// The data for route origin validation, if the server validates.
pub rpki: Option<Rpki>,
/// What keeps the best routes in the kernel, if the server does.
pub kernel: Option<Fib>,
/// The routes kept of restarting peers (graceful restart).
pub restarts: Arc<Restarts>,
pub local_asn: u32,
}
@@ -65,177 +45,37 @@ impl TestServer {
pub async fn start_with(local_asn: u32, customize: impl FnOnce(&mut Config)) -> Result<Self> {
let mut config = Config {
router_id: Ipv4Addr::new(127, 0, 0, 1),
cluster_id: Ipv4Addr::new(127, 0, 0, 1),
local_asn,
hold_time_secs: 90,
listen_endpoints: vec![SocketAddr::new(IpAddr::V4(Ipv4Addr::LOCALHOST), 0)],
peers: vec![],
connect_retry: std::time::Duration::from_secs(5),
collision_detect_established_state: false,
// Most tests play a peer that connects from wherever the
// system puts it; the ones about who may connect turn this off.
accept_unconfigured_peers: true,
originate: vec![],
http_endpoint: None,
rpki: None,
kernel: None,
export: Default::default(),
export_limits: None,
community_names: Vec::new(),
control: None,
graceful_restart: Default::default(),
};
customize(&mut config);
let bound = BgpServer::bind(config).await?;
let addrs = bound.local_addrs();
let addr = *addrs.first().expect("at least one listener");
let addr = bound
.local_addrs()
.into_iter()
.next()
.expect("at least one listener");
let rib = Arc::clone(&bound.rib);
let local_rib = bound.local_rib.clone();
let registry = bound.registry.clone();
let rejections = bound.rejections.clone();
let rpki = bound.rpki.clone();
let kernel = bound.kernel.clone();
let restarts = Arc::clone(&bound.restarts);
let handle = tokio::spawn(bound.run());
Ok(Self {
handle,
addr,
addrs,
rib,
local_rib,
registry,
rejections,
rpki,
kernel,
restarts,
local_asn,
})
}
/// A server in `local_asn` that takes connections from one configured
/// peer only: "r1", in `remote_asn`, at the loopback address the test
/// connects from. `customize` edits the configuration and the peer's
/// before the server starts. The server dials the peer too; where that
/// goes is returned with it, for the test to answer or to leave alone.
pub async fn with_peer(
local_asn: u32,
remote_asn: u32,
customize: impl FnOnce(&mut Config, &mut PeerConfig),
) -> Result<ServerWithPeer> {
let listener = TcpListener::bind((Ipv4Addr::LOCALHOST, 0)).await?;
let endpoint = listener.local_addr()?;
let server = Self::start_with(local_asn, |config| {
config.accept_unconfigured_peers = false;
let mut peer = peer_config("r1", endpoint, remote_asn);
customize(config, &mut peer);
config.peers = vec![peer];
})
.await?;
Ok(ServerWithPeer { server, listener })
}
}
/// A server with one configured peer, and the listener at the peer's
/// address, which the server's dialling goes to.
pub struct ServerWithPeer {
pub server: TestServer,
pub listener: TcpListener,
}
impl TestServer {
/// A session with the server from a peer in `asn` with `router_id`,
/// offering the capabilities of a typical dual-stack speaker. Returns
/// once the server lists the session as established. Distinct peers
/// need distinct identifiers, or the second connection is (correctly)
/// treated as a collision with the first.
pub async fn established(&self, asn: u32, router_id: Ipv4Addr) -> Result<TestPeer> {
let mut peer = TestPeer::connect(self.addr).await?;
peer.handshake(asn, router_id, 90).await?;
self.listed(&peer).await?;
Ok(peer)
}
/// As [`TestServer::established`], with an OPEN carrying exactly
/// `capabilities`.
pub async fn established_with(
&self,
asn: u32,
router_id: Ipv4Addr,
capabilities: Vec<Capability>,
) -> Result<TestPeer> {
let mut peer = TestPeer::connect(self.addr).await?;
peer.handshake_with(asn, router_id, 90, capabilities)
.await?;
self.listed(&peer).await?;
Ok(peer)
}
/// Waits until the registry lists `peer`'s session.
async fn listed(&self, peer: &TestPeer) -> Result<()> {
let from = peer.local_addr()?;
wait_until("the session is established", || {
self.registry
.list()
.iter()
.any(|session| session.peer.addr == from)
})
.await
}
/// Waits until the registry lists `n` established sessions.
pub async fn wait_for_sessions(&self, n: usize) -> Result<()> {
wait_until(&format!("{n} sessions are established"), || {
self.registry.list().len() == n
})
.await
}
/// Waits until a session is established, and returns it.
pub async fn wait_for_session(&self) -> Result<SessionHandle> {
wait_for("a session is established", || {
self.registry.list().into_iter().next()
})
.await
}
/// Waits until the table holds `routes` routes.
pub async fn wait_for_routes(&self, routes: usize) -> Result<()> {
wait_until(&format!("the table holds {routes} routes"), || {
self.rib.len() == routes
})
.await
}
/// Waits until the table holds exactly the prefixes `expected` and the
/// peer has `held` routes held aside.
pub async fn table_becomes(&self, expected: &[&str], held: u64) -> Result<()> {
let expected: BTreeSet<String> = expected.iter().map(|p| p.to_string()).collect();
let table = || -> BTreeSet<String> {
self.received_routes()
.iter()
.map(|route| route.prefix.to_string())
.collect()
};
let held_now = || {
self.registry
.list()
.first()
.map_or(0, |session| session.counters.held())
};
wait_until("the table is as expected", || {
table() == expected && held_now() == held
})
.await
.map_err(|_| {
eyre!(
"the table has {:?} with {} held; expected {expected:?} with {held} held",
table(),
held_now()
)
})
}
}
/// A route in the server's table, as a test wants to look at it.
@@ -350,10 +190,6 @@ impl TestServer {
registry: self.registry.clone(),
local_rib: self.local_rib.clone(),
rib: Arc::clone(&self.rib),
rejections: self.rejections.clone(),
rpki: self.rpki.clone(),
kernel: self.kernel.clone(),
restarts: Arc::clone(&self.restarts),
};
let handle = tokio::spawn(bgp_web::serve_on(state, listener));
Ok((base_url, HttpGuard { handle }))
crates/integration-tests/tests/add_path.rs
+85 -366
View File
@@ -1,45 +1,51 @@
//! End-to-end tests of ADD-PATH (RFC 7911): a peer sending several routes
//! for one prefix, told apart by path identifiers.
use std::collections::BTreeMap;
use std::net::{Ipv4Addr, Ipv6Addr, SocketAddr};
use std::sync::Arc;
use std::net::{Ipv4Addr, SocketAddr};
use std::time::Duration;
use bgp_integration_tests::announcement::{Announcement, withdrawal};
use bgp_integration_tests::test_peer::{TestPeer, collector_capabilities};
use bgp_integration_tests::test_peer::TestPeer;
use bgp_integration_tests::test_server::TestServer;
use bgp_integration_tests::{peer_config, prefix, wait_until};
use bgp_packet::message::{
AsPathAttribute, Message, MpReachNlriPathAttribute, NlriNextHop, OriginPathAttribute,
PathAttribute, UpdateMessage,
AsPathAttribute, Message, NextHopPathAttribute, OriginPathAttribute, PathAttribute,
UpdateMessage,
};
use bgp_packet::open::{AddPathEntry, Capability, OpenMessage, SendReceive};
use bgp_packet::open::{AddPathEntry, AddPathSendRecv, Capability, OpenMessage};
use bgp_primitives::constants::AfiSafi;
use bgp_primitives::ip_prefix::Nlri;
use bgp_primitives::ip_prefix::{IpPrefix, Nlri};
use bgp_rib::{LocalRoute, LocalRouteAttrs};
use bgp_server::config::PeerConfig;
use bgp_server::family::SUPPORTED;
use bgp_server::policy::{ExportPolicy, ImportPolicy};
use bgp_server::family::{FamilySet, SUPPORTED};
use eyre::Result;
use serde_json::Value;
const LOCAL_ASN: u32 = 65000;
const PEER_ASN: u32 = 65001;
const TIMEOUT: Duration = Duration::from_secs(5);
fn prefix(s: &str) -> IpPrefix {
IpPrefix::try_from(s).unwrap()
}
/// A server on which connections from this machine are those of a
/// configured peer, with ADD-PATH offered to it as `add_path` says.
async fn server_offering(add_path: Option<SendReceive>, originate: &[&str]) -> Result<TestServer> {
async fn server_offering(
add_path: Option<AddPathSendRecv>,
originate: &[&str],
) -> Result<TestServer> {
let originate: Vec<LocalRoute> = originate
.iter()
.map(|p| LocalRoute::new(prefix(p), LocalRouteAttrs::default()))
.collect();
TestServer::start_with(LOCAL_ASN, |c| {
let at = SocketAddr::new(Ipv4Addr::LOCALHOST.into(), 1);
c.peers = vec![PeerConfig {
name: "route-server-client".into(),
endpoint: SocketAddr::new(Ipv4Addr::LOCALHOST.into(), 1),
remote_asn: PEER_ASN,
hold_time_secs: 90,
local_address: None,
families: FamilySet::supported(),
add_path,
import: ImportPolicy::permissive(),
..peer_config("route-server-client", at, PEER_ASN)
}];
c.originate = originate;
// Keep the dialer out of the way.
@@ -52,7 +58,7 @@ async fn server_offering(add_path: Option<SendReceive>, originate: &[&str]) -> R
/// IPv4 unicast. Returns the peer and the server's OPEN.
async fn peer_offering(
server: &TestServer,
offer: Option<SendReceive>,
offer: Option<AddPathSendRecv>,
) -> Result<(TestPeer, OpenMessage)> {
let mut capabilities = vec![Capability::FourByteAsn { asn: PEER_ASN }];
capabilities.extend(SUPPORTED.into_iter().map(Capability::multiprotocol));
@@ -69,11 +75,16 @@ async fn peer_offering(
let open = peer
.handshake_with(PEER_ASN, Ipv4Addr::new(10, 0, 0, 1), 90, capabilities)
.await?;
server.wait_for_session().await?;
tokio::time::timeout(TIMEOUT, async {
while server.registry.list().is_empty() {
tokio::time::sleep(Duration::from_millis(5)).await;
}
})
.await?;
Ok((peer, open))
}
fn add_path_offered(open: &OpenMessage) -> Vec<(AfiSafi, SendReceive)> {
fn add_path_offered(open: &OpenMessage) -> Vec<(AfiSafi, AddPathSendRecv)> {
open.capabilities()
.filter_map(|capability| match capability {
Capability::AddPath { entries } => Some(entries),
@@ -84,8 +95,26 @@ fn add_path_offered(open: &OpenMessage) -> Vec<(AfiSafi, SendReceive)> {
.collect()
}
fn announcement(nlri: Nlri, as_path: &[u32]) -> Announcement {
Announcement::of([nlri], as_path)
fn announcement(nlri: Nlri, as_path: &[u32]) -> Message {
Message::Update(UpdateMessage {
withdrawn_routes: vec![],
path_attributes: vec![
PathAttribute::Origin(OriginPathAttribute::IGP),
AsPathAttribute::from_asns(as_path.to_vec()),
PathAttribute::NextHop(NextHopPathAttribute(Ipv4Addr::new(192, 0, 2, 1))),
],
announced_routes: vec![nlri],
})
}
async fn wait_for_routes(server: &TestServer, routes: usize) -> Result<()> {
tokio::time::timeout(TIMEOUT, async {
while server.rib.len() != routes {
tokio::time::sleep(Duration::from_millis(5)).await;
}
})
.await?;
Ok(())
}
/// The routes the server holds, as (path identifier, AS path, best).
@@ -113,14 +142,14 @@ fn held(server: &TestServer) -> Vec<(u32, Vec<u32>, bool)> {
/// of them is chosen, and each is withdrawn on its own.
#[tokio::test]
async fn test_several_routes_for_a_prefix_from_one_peer() -> Result<()> {
let server = server_offering(Some(SendReceive::Receive), &[]).await?;
let server = server_offering(Some(AddPathSendRecv::Receive), &[]).await?;
let (base, _http) = server.serve_http().await?;
let (mut peer, open) = peer_offering(&server, Some(SendReceive::Send)).await?;
let (mut peer, open) = peer_offering(&server, Some(AddPathSendRecv::Send)).await?;
// The server offered to receive, for every family it offers.
assert_eq!(
add_path_offered(&open),
SUPPORTED.map(|family| (family, SendReceive::Receive))
SUPPORTED.map(|family| (family, AddPathSendRecv::Receive))
);
// In use for the one family the peer sends for, and in that direction.
let session = &server.registry.list()[0];
@@ -133,11 +162,11 @@ async fn test_several_routes_for_a_prefix_from_one_peer() -> Result<()> {
let p = prefix("203.0.113.0/24");
let path = |path_id| Nlri { prefix: p, path_id };
peer.announce(announcement(path(1), &[PEER_ASN, 64999, 64998]))
peer.send(&announcement(path(1), &[PEER_ASN, 64999, 64998]))
.await?;
peer.announce(announcement(path(2), &[PEER_ASN, 64998]))
peer.send(&announcement(path(2), &[PEER_ASN, 64998]))
.await?;
server.wait_for_routes(2).await?;
wait_for_routes(&server, 2).await?;
assert_eq!(
held(&server),
[
@@ -152,7 +181,7 @@ async fn test_several_routes_for_a_prefix_from_one_peer() -> Result<()> {
.await?
.json()
.await?;
let mut ids: Vec<u64> = routes["routes"]
let mut ids: Vec<u64> = routes
.as_array()
.unwrap()
.iter()
@@ -170,24 +199,34 @@ async fn test_several_routes_for_a_prefix_from_one_peer() -> Result<()> {
);
// Announcing a path again replaces that path only.
peer.announce(announcement(path(1), &[PEER_ASN])).await?;
wait_until("the path is replaced", || held(&server)[0].1.len() == 1).await?;
peer.send(&announcement(path(1), &[PEER_ASN])).await?;
tokio::time::timeout(TIMEOUT, async {
while held(&server)[0].1.len() != 1 {
tokio::time::sleep(Duration::from_millis(5)).await;
}
})
.await?;
assert_eq!(
held(&server),
[(1, vec![PEER_ASN], true), (2, vec![PEER_ASN, 64998], false),]
);
// And withdrawing one leaves the other.
peer.announce(withdrawal([path(1)])).await?;
server.wait_for_routes(1).await?;
peer.send(&Message::Update(UpdateMessage {
withdrawn_routes: vec![path(1)],
path_attributes: vec![],
announced_routes: vec![],
}))
.await?;
wait_for_routes(&server, 1).await?;
assert_eq!(held(&server), [(2, vec![PEER_ASN, 64998], true)]);
// When the session ends, all of its routes go.
peer.announce(announcement(path(3), &[PEER_ASN, 1, 2, 3]))
peer.send(&announcement(path(3), &[PEER_ASN, 1, 2, 3]))
.await?;
server.wait_for_routes(2).await?;
wait_for_routes(&server, 2).await?;
drop(peer);
server.wait_for_routes(0).await?;
wait_for_routes(&server, 0).await?;
Ok(())
}
@@ -197,24 +236,24 @@ async fn test_several_routes_for_a_prefix_from_one_peer() -> Result<()> {
#[tokio::test]
async fn test_add_path_is_not_used_unless_both_sides_offer_it() -> Result<()> {
let server = server_offering(None, &["10.100.0.0/16"]).await?;
let (mut peer, open) = peer_offering(&server, Some(SendReceive::Both)).await?;
let (mut peer, open) = peer_offering(&server, Some(AddPathSendRecv::Both)).await?;
assert!(add_path_offered(&open).is_empty());
let session = &server.registry.list()[0];
assert!(session.add_path_receive.is_empty() && session.add_path_send.is_empty());
// Plain prefixes both ways.
let update = peer.recv_update().await?;
let update = tokio::time::timeout(TIMEOUT, peer.recv_update()).await??;
assert_eq!(update.announced_routes, vec![prefix("10.100.0.0/16")]);
peer.announce(announcement(prefix("203.0.113.0/24").into(), &[PEER_ASN]))
peer.send(&announcement(prefix("203.0.113.0/24").into(), &[PEER_ASN]))
.await?;
server.wait_for_routes(1).await?;
wait_for_routes(&server, 1).await?;
assert_eq!(held(&server), [(0, vec![PEER_ASN], true)]);
// And the other way round: offered by the server, not by the peer.
let server = server_offering(Some(SendReceive::Both), &["10.100.0.0/16"]).await?;
let server = server_offering(Some(AddPathSendRecv::Both), &["10.100.0.0/16"]).await?;
let (mut peer, open) = peer_offering(&server, None).await?;
assert_eq!(add_path_offered(&open).len(), SUPPORTED.len());
let update = peer.recv_update().await?;
let update = tokio::time::timeout(TIMEOUT, peer.recv_update()).await??;
assert_eq!(update.announced_routes, vec![prefix("10.100.0.0/16")]);
Ok(())
}
@@ -224,8 +263,8 @@ async fn test_add_path_is_not_used_unless_both_sides_offer_it() -> Result<()> {
/// understands them.
#[tokio::test]
async fn test_prefixes_are_sent_with_path_identifiers_when_agreed() -> Result<()> {
let server = server_offering(Some(SendReceive::Both), &["10.100.0.0/16"]).await?;
let (mut peer, _) = peer_offering(&server, Some(SendReceive::Receive)).await?;
let server = server_offering(Some(AddPathSendRecv::Both), &["10.100.0.0/16"]).await?;
let (mut peer, _) = peer_offering(&server, Some(AddPathSendRecv::Receive)).await?;
let session = &server.registry.list()[0];
assert_eq!(
session.add_path_send,
@@ -235,335 +274,15 @@ async fn test_prefixes_are_sent_with_path_identifiers_when_agreed() -> Result<()
// Read as agreed: with identifiers.
peer.ctx.add_path = vec![AfiSafi::IPV4_UNICAST];
let update = peer.recv_update().await?;
let update = tokio::time::timeout(TIMEOUT, peer.recv_update()).await??;
assert_eq!(update.announced_routes.len(), 1);
assert_eq!(update.announced_routes[0].prefix, prefix("10.100.0.0/16"));
// The peer did not offer to send, so the server reads its prefixes
// plain.
peer.announce(announcement(prefix("203.0.113.0/24").into(), &[PEER_ASN]))
peer.send(&announcement(prefix("203.0.113.0/24").into(), &[PEER_ASN]))
.await?;
server.wait_for_routes(1).await?;
wait_for_routes(&server, 1).await?;
assert_eq!(held(&server), [(0, vec![PEER_ASN], true)]);
Ok(())
}
const COLLECTOR_ASN: u32 = 65100;
/// What a collector holds: for each prefix and path identifier, the
/// route's AS path.
type Held = BTreeMap<(String, u32), Vec<u32>>;
/// Reads what the collector is told, into `held`, until `done` says so.
async fn told(
collector: &mut TestPeer,
held: &mut Held,
done: impl Fn(&Held) -> bool,
) -> Result<()> {
while !done(held) {
let update = collector.recv_update().await?;
let path = update
.path_attributes
.iter()
.find_map(|attribute| match attribute {
PathAttribute::AsPath(path) => Some(path.segments[0].path.clone()),
_ => None,
})
.unwrap_or_default();
for nlri in &update.withdrawn_routes {
held.remove(&(nlri.prefix.to_string(), nlri.path_id));
}
for nlri in &update.announced_routes {
held.insert((nlri.prefix.to_string(), nlri.path_id), path.clone());
}
// And what came and went in the multiprotocol attributes.
for attribute in &update.path_attributes {
match attribute {
PathAttribute::MpReachNlri(reach) => {
for nlri in &reach.prefixes {
held.insert((nlri.prefix.to_string(), nlri.path_id), path.clone());
}
}
PathAttribute::MpUnreachNlri(unreach) => {
for nlri in &unreach.prefixes {
held.remove(&(nlri.prefix.to_string(), nlri.path_id));
}
}
_ => {}
}
}
}
Ok(())
}
/// The routes held for 203.0.113.0/24, by path identifier.
fn paths(held: &Held) -> Vec<(u32, Vec<u32>)> {
held.iter()
.map(|((prefix, id), path)| {
assert_eq!(prefix, "203.0.113.0/24");
(*id, path.clone())
})
.collect()
}
/// A collector that agreed to be sent several routes for a prefix, and
/// whose export policy says to send it every route, is told of every
/// peer's route for a prefix, each with a path identifier of its own that
/// stays with the route; a route that goes is withdrawn by its identifier.
#[tokio::test]
async fn test_every_route_goes_to_a_collector_told_apart_by_path_identifiers() -> Result<()> {
let server = TestServer::start_with(LOCAL_ASN, |c| {
// The collector is the configured peer at the IPv4 loopback
// address; the feeders come over IPv6 loopback as peers of no
// account.
c.listen_endpoints
.push(SocketAddr::new(Ipv6Addr::LOCALHOST.into(), 0));
c.peers = vec![PeerConfig {
add_path: Some(SendReceive::Send),
import: ImportPolicy::permissive(),
export: Arc::new(ExportPolicy {
learned_routes: true,
all_paths: true,
..ExportPolicy::default()
}),
..peer_config(
"collector",
SocketAddr::new(Ipv4Addr::LOCALHOST.into(), 1),
COLLECTOR_ASN,
)
}];
c.connect_retry = Duration::from_secs(3600);
})
.await?;
let route = |path: &[u32]| Announcement::of([prefix("203.0.113.0/24").into()], path);
let feeder = async |n: u32| -> Result<TestPeer> {
let mut peer = TestPeer::connect(server.addrs[1]).await?;
peer.handshake(PEER_ASN + n, Ipv4Addr::new(10, 0, 0, n as u8 + 1), 90)
.await?;
Ok(peer)
};
let mut a = feeder(0).await?;
let mut b = feeder(1).await?;
server.wait_for_sessions(2).await?;
a.announce(route(&[PEER_ASN, 64510])).await?;
b.announce(route(&[PEER_ASN + 1, 64510])).await?;
server.wait_for_routes(2).await?;
let mut collector = TestPeer::connect(server.addr).await?;
collector
.handshake_with(
COLLECTOR_ASN,
Ipv4Addr::new(10, 0, 0, 100),
90,
collector_capabilities(COLLECTOR_ASN, &SUPPORTED, false),
)
.await?;
collector.ctx.add_path = SUPPORTED.to_vec();
// Both routes, told apart.
let mut held = Held::new();
told(&mut collector, &mut held, |held| held.len() == 2).await?;
assert_eq!(
paths(&held),
[
(1, vec![LOCAL_ASN, PEER_ASN, 64510]),
(2, vec![LOCAL_ASN, PEER_ASN + 1, 64510]),
]
);
// The second peer's route goes: withdrawn by its identifier, the
// first's untouched.
b.announce(withdrawal([prefix("203.0.113.0/24").into()]))
.await?;
told(&mut collector, &mut held, |held| held.len() == 1).await?;
assert_eq!(paths(&held), [(1, vec![LOCAL_ASN, PEER_ASN, 64510])]);
// The first peer's route changes: it goes out again with the same
// identifier. A third peer's route gets the lowest free one.
a.announce(route(&[PEER_ASN, 64511, 64510])).await?;
let mut c = feeder(2).await?;
server.wait_for_sessions(4).await?;
c.announce(route(&[PEER_ASN + 2, 64510])).await?;
let changed = |held: &Held| {
held.len() == 2
&& held.get(&("203.0.113.0/24".to_string(), 1))
== Some(&vec![LOCAL_ASN, PEER_ASN, 64511, 64510])
};
told(&mut collector, &mut held, changed).await?;
assert_eq!(
paths(&held),
[
(1, vec![LOCAL_ASN, PEER_ASN, 64511, 64510]),
(2, vec![LOCAL_ASN, PEER_ASN + 2, 64510]),
]
);
Ok(())
}
/// A collector that is connected before the table arrives is told the
/// whole table, however large: the walk of the table that a new follower
/// starts with takes many batches with a table of some size, and goes on
/// while the feeders keep sending.
#[tokio::test]
async fn test_a_large_table_reaches_a_collector_that_was_there_first() -> Result<()> {
const PREFIXES: u32 = 40_000;
let server = TestServer::start_with(LOCAL_ASN, |c| {
c.listen_endpoints
.push(SocketAddr::new(Ipv6Addr::LOCALHOST.into(), 0));
c.peers = vec![PeerConfig {
add_path: Some(SendReceive::Send),
import: ImportPolicy::permissive(),
export: Arc::new(ExportPolicy {
learned_routes: true,
all_paths: true,
..ExportPolicy::default()
}),
..peer_config(
"collector",
SocketAddr::new(Ipv4Addr::LOCALHOST.into(), 1),
COLLECTOR_ASN,
)
}];
c.connect_retry = Duration::from_secs(3600);
})
.await?;
let mut collector = TestPeer::connect(server.addr).await?;
collector
.handshake_with(
COLLECTOR_ASN,
Ipv4Addr::new(10, 0, 0, 100),
90,
collector_capabilities(COLLECTOR_ASN, &SUPPORTED, false),
)
.await?;
collector.ctx.add_path = SUPPORTED.to_vec();
server.wait_for_sessions(1).await?;
// Two feeders, each sending the same prefixes a hundred to an UPDATE.
let nth = |n: u32| {
let [_, b, c, _] = (0x0a00_0000u32 + n * 256).to_be_bytes();
prefix(&format!("10.{b}.{c}.0/24"))
};
let mut feeders = Vec::new();
for n in 0..2u32 {
let mut peer = TestPeer::connect(server.addrs[1]).await?;
peer.handshake(PEER_ASN + n, Ipv4Addr::new(10, 0, 0, n as u8 + 1), 90)
.await?;
feeders.push(peer);
}
server.wait_for_sessions(3).await?;
for start in (0..PREFIXES).step_by(100) {
for (n, feeder) in feeders.iter_mut().enumerate() {
feeder
.announce(Announcement::of(
(start..start + 100).map(|i| nth(i).into()),
&[PEER_ASN + n as u32, 64510],
))
.await?;
}
}
let mut held = Held::new();
told(&mut collector, &mut held, |held| {
held.len() == 2 * PREFIXES as usize
})
.await?;
let ids: std::collections::BTreeSet<u32> = held.keys().map(|(_, id)| *id).collect();
assert_eq!(ids.into_iter().collect::<Vec<_>>(), [1, 2]);
Ok(())
}
/// As a router at an exchange feeds a collector: one IPv6 session that
/// carries both families, with ADD-PATH for both and IPv4 routes with
/// IPv6 next hops (RFC 8950), and a table of some size from two feeders.
#[tokio::test]
async fn test_both_families_reach_a_collector_over_one_ipv6_session() -> Result<()> {
const V4: u32 = 20_000;
const V6: u32 = 1_000;
let server = TestServer::start_with(LOCAL_ASN, |c| {
c.listen_endpoints
.push(SocketAddr::new(Ipv6Addr::LOCALHOST.into(), 0));
c.peers = vec![PeerConfig {
add_path: Some(SendReceive::Send),
import: ImportPolicy::permissive(),
export: Arc::new(ExportPolicy {
learned_routes: true,
all_paths: true,
..ExportPolicy::default()
}),
..peer_config(
"collector",
SocketAddr::new(Ipv6Addr::LOCALHOST.into(), 1),
COLLECTOR_ASN,
)
}];
c.connect_retry = Duration::from_secs(3600);
})
.await?;
let mut collector = TestPeer::connect(server.addrs[1]).await?;
collector
.handshake_with(
COLLECTOR_ASN,
Ipv4Addr::new(10, 0, 0, 100),
90,
collector_capabilities(COLLECTOR_ASN, &SUPPORTED, true),
)
.await?;
collector.ctx.add_path = SUPPORTED.to_vec();
server.wait_for_sessions(1).await?;
let session = &server.registry.list()[0];
assert!(session.extended_next_hop, "RFC 8950 agreed");
assert_eq!(session.add_path_send.iter().count(), 2);
let v4 = |n: u32| {
let [_, b, c, _] = (0x0a00_0000u32 + n * 256).to_be_bytes();
prefix(&format!("10.{b}.{c}.0/24"))
};
let v6 = |n: u32| prefix(&format!("2001:db8:{:x}::/48", 0x1000 + n));
let mut feeders = Vec::new();
for n in 0..2u32 {
let mut peer = TestPeer::connect(server.addr).await?;
peer.handshake(PEER_ASN + n, Ipv4Addr::new(10, 0, 0, n as u8 + 1), 90)
.await?;
feeders.push(peer);
}
server.wait_for_sessions(3).await?;
for start in (0..V4).step_by(100) {
for (n, feeder) in feeders.iter_mut().enumerate() {
feeder
.announce(Announcement::of(
(start..start + 100).map(|i| v4(i).into()),
&[PEER_ASN + n as u32, 64510],
))
.await?;
}
}
for start in (0..V6).step_by(100) {
for (n, feeder) in feeders.iter_mut().enumerate() {
let update = UpdateMessage {
withdrawn_routes: vec![],
path_attributes: vec![
PathAttribute::Origin(OriginPathAttribute::IGP),
AsPathAttribute::from_asns(vec![PEER_ASN + n as u32, 64510]),
PathAttribute::MpReachNlri(MpReachNlriPathAttribute {
family: AfiSafi::IPV6_UNICAST,
next_hop: NlriNextHop::Ipv6("2001:db8::9".parse().unwrap()),
prefixes: (start..start + 100).map(|i| v6(i).into()).collect(),
}),
],
announced_routes: vec![],
};
feeder.send(&Message::Update(update)).await?;
}
}
let mut held = Held::new();
told(&mut collector, &mut held, |held| {
held.len() == 2 * (V4 + V6) as usize
})
.await?;
let v6_routes = held
.keys()
.filter(|(prefix, _)| prefix.contains(':'))
.count();
assert_eq!(v6_routes, 2 * V6 as usize);
Ok(())
}
crates/integration-tests/tests/admission.rs
-89
View File
@@ -1,89 +0,0 @@
//! Who may connect: only the addresses of configured peers, and each of
//! them only so many times at once.
use std::net::{Ipv4Addr, SocketAddr};
use std::time::Duration;
use bgp_integration_tests::test_peer::TestPeer;
use bgp_integration_tests::test_server::{ServerWithPeer, TestServer};
use bgp_integration_tests::{TIMEOUT, peer_config};
use bgp_server::config::PeerConfig;
use bgp_server::policy::ImportPolicy;
use eyre::Result;
const LOCAL_ASN: u32 = 65000;
const PEER_ASN: u32 = 65001;
/// A host that is not a configured peer gets nothing: no OPEN to answer,
/// no session, no routes taken from it. Any host used to be able to open a
/// session as any AS it named.
#[tokio::test]
async fn test_a_host_that_is_not_a_configured_peer_is_refused() -> Result<()> {
// The only peer configured is somewhere else (documentation space).
let server = TestServer::start_with(LOCAL_ASN, |c| {
c.accept_unconfigured_peers = false;
let elsewhere: SocketAddr = "192.0.2.9:179".parse().unwrap();
c.peers = vec![PeerConfig {
import: ImportPolicy::permissive(),
..peer_config("r1", elsewhere, PEER_ASN)
}];
})
.await?;
let mut stranger = TestPeer::connect(server.addr).await?;
assert!(stranger.closed_unanswered().await);
assert!(server.registry.list().is_empty());
Ok(())
}
/// The address of a configured peer is let in, and held to what is
/// configured for that peer.
#[tokio::test]
async fn test_a_configured_peer_is_let_in() -> Result<()> {
// The server's own dialling goes to the listener; it is never answered.
let ServerWithPeer {
server,
listener: _listener,
} = TestServer::with_peer(LOCAL_ASN, PEER_ASN, |_, peer| {
peer.import = ImportPolicy::permissive();
})
.await?;
let _peer = server
.established(PEER_ASN, Ipv4Addr::new(10, 0, 0, 2))
.await?;
assert_eq!(server.registry.list()[0].name.as_deref(), Some("r1"));
Ok(())
}
/// One address may hold only a few connections at a time, so that one host
/// cannot use up the server's sockets with connections that never get as
/// far as a session.
#[tokio::test]
async fn test_one_address_may_open_only_a_few_connections() -> Result<()> {
let server = TestServer::start(LOCAL_ASN).await?;
// Four that say nothing: each is sent the server's OPEN and waited on.
let mut held = Vec::new();
for _ in 0..4 {
let mut peer = TestPeer::connect(server.addr).await?;
peer.recv_open().await?;
held.push(peer);
}
let mut one_too_many = TestPeer::connect(server.addr).await?;
assert!(one_too_many.closed_unanswered().await);
// When one of them goes, there is room again.
held.pop();
tokio::time::timeout(TIMEOUT, async {
loop {
let mut peer = TestPeer::connect(server.addr).await?;
if peer.recv_open().await.is_ok() {
return Ok::<_, eyre::Report>(());
}
tokio::time::sleep(Duration::from_millis(20)).await;
}
})
.await??;
Ok(())
}
crates/integration-tests/tests/announce.rs
+59 -23
View File
@@ -4,21 +4,25 @@
use std::net::Ipv4Addr;
use std::time::Duration;
use bgp_integration_tests::announcement::announcement;
use bgp_integration_tests::test_peer::TestPeer;
use bgp_integration_tests::test_server::{RouteChange, TestServer};
use bgp_integration_tests::{TIMEOUT, prefix};
use bgp_packet::message::{
AsPathAttribute, CommunitiesPathAttribute, LargeCommunitiesPathAttribute, LargeCommunity,
LocalPrefPathAttribute, NextHopPathAttribute, OriginPathAttribute, PathAttribute,
UpdateMessage,
};
use bgp_primitives::ip_prefix::IpPrefix;
use bgp_rib::{LocalRoute, LocalRouteAttrs};
use bgp_server::session::SessionEvent;
use eyre::Result;
const LOCAL_ASN: u32 = 65000;
const PEER_ASN: u32 = 65001;
const TIMEOUT: Duration = Duration::from_secs(5);
fn prefix(s: &str) -> IpPrefix {
IpPrefix::try_from(s).unwrap()
}
fn route(p: &str, attrs: LocalRouteAttrs) -> LocalRoute {
LocalRoute::new(prefix(p), attrs)
@@ -28,12 +32,27 @@ async fn established_peer(server: &TestServer) -> Result<TestPeer> {
established_peer_with_id(server, Ipv4Addr::new(10, 0, 0, 1)).await
}
/// Distinct routers need distinct BGP Identifiers, or the second connection
/// is (correctly) treated as a collision with the first.
async fn established_peer_with_id(server: &TestServer, router_id: Ipv4Addr) -> Result<TestPeer> {
server.established(PEER_ASN, router_id).await
let mut peer = TestPeer::connect(server.addr).await?;
peer.handshake(PEER_ASN, router_id, 90).await?;
Ok(peer)
}
async fn next_update(peer: &mut TestPeer) -> Result<UpdateMessage> {
peer.recv_update().await
tokio::time::timeout(TIMEOUT, peer.recv_update()).await?
}
/// Waits until the registry reports `n` established sessions.
async fn wait_for_sessions(server: &TestServer, n: usize) -> Result<()> {
tokio::time::timeout(TIMEOUT, async {
while server.registry.list().len() != n {
tokio::time::sleep(Duration::from_millis(10)).await;
}
})
.await?;
Ok(())
}
// ---------------------------------------------------------------------------
@@ -78,7 +97,7 @@ async fn test_registry_tracks_session_lifecycle() -> Result<()> {
async fn test_announce_after_session_up() -> Result<()> {
let server = TestServer::start(LOCAL_ASN).await?;
let mut peer = established_peer(&server).await?;
server.wait_for_sessions(1).await?;
wait_for_sessions(&server, 1).await?;
server
.local_rib
@@ -155,7 +174,7 @@ async fn test_announce_reaches_all_sessions() -> Result<()> {
let server = TestServer::start(LOCAL_ASN).await?;
let mut a = established_peer_with_id(&server, Ipv4Addr::new(10, 0, 0, 1)).await?;
let mut b = established_peer_with_id(&server, Ipv4Addr::new(10, 0, 0, 2)).await?;
server.wait_for_sessions(2).await?;
wait_for_sessions(&server, 2).await?;
server
.local_rib
@@ -173,7 +192,7 @@ async fn test_announce_reaches_all_sessions() -> Result<()> {
async fn test_withdraw_sends_withdrawn_route() -> Result<()> {
let server = TestServer::start(LOCAL_ASN).await?;
let mut peer = established_peer(&server).await?;
server.wait_for_sessions(1).await?;
wait_for_sessions(&server, 1).await?;
server
.local_rib
@@ -194,7 +213,7 @@ async fn test_withdraw_sends_withdrawn_route() -> Result<()> {
async fn test_reannounce_with_prepend_and_med() -> Result<()> {
let server = TestServer::start(LOCAL_ASN).await?;
let mut peer = established_peer(&server).await?;
server.wait_for_sessions(1).await?;
wait_for_sessions(&server, 1).await?;
server
.local_rib
@@ -239,7 +258,7 @@ async fn test_keepalives_continue_alongside_updates() -> Result<()> {
let mut peer = TestPeer::connect(server.addr).await?;
peer.handshake(PEER_ASN, Ipv4Addr::new(10, 0, 0, 1), 3)
.await?;
server.wait_for_sessions(1).await?;
wait_for_sessions(&server, 1).await?;
server
.local_rib
@@ -270,12 +289,22 @@ async fn test_received_routes_removed_when_session_closes() -> Result<()> {
let mut rib_events = server.route_changes();
let mut session_events = server.registry.subscribe();
let mut peer = established_peer(&server).await?;
server.wait_for_sessions(1).await?;
wait_for_sessions(&server, 1).await?;
peer.announce(
announcement(&["203.0.113.0/24", "198.51.100.0/24"], &[PEER_ASN])
.with_next_hop(Ipv4Addr::new(10, 0, 0, 1)),
)
peer.send(&bgp_packet::message::Message::Update(
bgp_packet::message::UpdateMessage {
withdrawn_routes: vec![],
path_attributes: vec![
PathAttribute::Origin(OriginPathAttribute::IGP),
AsPathAttribute::from_asns(vec![PEER_ASN]),
PathAttribute::NextHop(NextHopPathAttribute(Ipv4Addr::new(10, 0, 0, 1))),
],
announced_routes: vec![
prefix("203.0.113.0/24").into(),
prefix("198.51.100.0/24").into(),
],
},
))
.await?;
for _ in 0..2 {
let ev = tokio::time::timeout(TIMEOUT, rib_events.recv()).await??;
@@ -322,7 +351,7 @@ const ROUTE_REFRESH_IPV4: &[u8] = &[
async fn test_route_refresh_resends_local_routes() -> Result<()> {
let server = TestServer::start(LOCAL_ASN).await?;
let mut peer = established_peer(&server).await?;
server.wait_for_sessions(1).await?;
wait_for_sessions(&server, 1).await?;
server
.local_rib
@@ -357,26 +386,33 @@ async fn test_route_refresh_resends_local_routes() -> Result<()> {
#[tokio::test]
async fn test_shutdown_sends_cease_to_every_peer() -> Result<()> {
use bgp_packet::errors::{BgpError, CeaseSubcode};
use bgp_packet::message::Message;
let server = TestServer::start(LOCAL_ASN).await?;
let mut a = established_peer_with_id(&server, Ipv4Addr::new(10, 0, 0, 1)).await?;
let mut b = established_peer_with_id(&server, Ipv4Addr::new(10, 0, 0, 2)).await?;
server.wait_for_sessions(2).await?;
wait_for_sessions(&server, 2).await?;
let left = server
.registry
.leave(bgp_server::restart::Leave::Stop, Duration::from_secs(2))
.await;
let left = server.registry.shutdown(Duration::from_secs(2)).await;
assert_eq!(left, 0, "sessions still open after shutdown");
assert!(server.registry.is_shutting_down());
for peer in [&mut a, &mut b] {
let notif = peer.expect_notification().await?;
let notif = tokio::time::timeout(TIMEOUT, async {
loop {
match peer.recv().await? {
Message::Notification(n) => return Ok::<_, eyre::Error>(n),
Message::KeepAlive => {}
other => eyre::bail!("unexpected {other:?}"),
}
}
})
.await??;
assert!(
matches!(notif.error, BgpError::Cease(CeaseSubcode::AdminShutdown)),
"{notif:?}"
);
assert!(peer.closed_within(TIMEOUT).await);
assert!(peer.is_closed().await);
}
Ok(())
}
@@ -431,7 +467,7 @@ async fn test_a_peer_in_our_own_as_gets_an_empty_path_and_a_local_pref() -> Resu
// The peer reads as a router in the server's AS does: one outside it
// would drop the LOCAL_PREF.
peer.ctx.ibgp = true;
server.wait_for_sessions(1).await?;
wait_for_sessions(&server, 1).await?;
assert!(server.registry.list()[0].ibgp);
server
crates/integration-tests/tests/aspa.rs
-275
View File
@@ -1,275 +0,0 @@
//! AS path verification with ASPAs, end to end: a server that gets ASPAs
//! from a cache, a peer that announces routes, and the ASPAs changing
//! under both.
use std::collections::BTreeSet;
use std::net::Ipv4Addr;
use std::time::Duration;
use bgp_integration_tests::announcement::announcement;
use bgp_integration_tests::test_cache::TestCache;
use bgp_integration_tests::test_peer::TestPeer;
use bgp_integration_tests::test_server::{ServerWithPeer, TestServer};
use bgp_integration_tests::{default_policy, prefix, wait_until};
use bgp_packet::message::PathAttribute;
use bgp_rpki::{CacheState, RpkiConfig};
use bgp_server::policy::{ImportPolicy, PeerRole};
use eyre::Result;
use serde_json::Value;
const LOCAL_ASN: u32 = 1299;
const PEER_ASN: u32 = 3333;
const PEER_ID: Ipv4Addr = Ipv4Addr::new(10, 0, 0, 2);
/// A server that validates against `cache`, with one configured peer at
/// the address the test connects from, which is `role` to it. Returns once
/// the server has the cache's data.
async fn server_with(
cache: &TestCache,
role: Option<PeerRole>,
import: ImportPolicy,
) -> Result<ServerWithPeer> {
let cache = cache.address();
let setup = TestServer::with_peer(LOCAL_ASN, PEER_ASN, |c, peer| {
c.rpki = Some(RpkiConfig {
caches: vec![cache],
retry: Some(Duration::from_millis(100)),
..RpkiConfig::default()
});
peer.import = import;
peer.role = role;
})
.await?;
let rpki = setup.server.rpki.clone().expect("validation is configured");
wait_until("the server has the cache's data", || {
rpki.status().state == CacheState::Synchronised
})
.await?;
Ok(setup)
}
async fn established(server: &TestServer) -> Result<TestPeer> {
server.established(PEER_ASN, PEER_ID).await
}
/// Announces `prefixes` with the AS path `path`.
async fn announce(peer: &mut TestPeer, prefixes: &[&str], path: &[u32]) -> Result<()> {
peer.announce(announcement(prefixes, path)).await
}
/// AS 15169 buys from 3356 and says so; 3356 and the peer, 3333, have no
/// providers.
fn aspas() -> Vec<(u32, Vec<u32>)> {
vec![(15169, vec![3356]), (3356, vec![0]), (PEER_ASN, vec![0])]
}
/// A peer that hands us a route it got from one of its own peers is
/// leaking it. The route is refused; routes whose paths are attested, or
/// that nothing is known about, are taken.
#[tokio::test]
async fn test_a_leaked_route_is_refused_and_the_rest_are_taken() -> Result<()> {
let cache = TestCache::start_with(vec![], aspas()).await;
let ServerWithPeer { server, .. } =
server_with(&cache, Some(PeerRole::Peer), default_policy()).await?;
let (base, _http) = server.serve_http().await?;
let mut peer = established(&server).await?;
// The peer's own route; one of a customer of its that has no ASPA;
// and 15169's route by way of 3356, which is not for 3333 to pass on
// to a peer: 3356 is not its customer.
announce(&mut peer, &["193.0.0.0/21"], &[PEER_ASN]).await?;
announce(&mut peer, &["194.0.0.0/24"], &[PEER_ASN, 2121]).await?;
announce(&mut peer, &["8.8.8.0/24"], &[PEER_ASN, 3356, 15169]).await?;
server
.table_becomes(&["193.0.0.0/21", "194.0.0.0/24"], 1)
.await?;
let client = reqwest::Client::new();
let get = |path: &str| {
let request = client.get(format!("{base}{path}"));
async move { Ok::<Value, eyre::Report>(request.send().await?.error_for_status()?.json().await?) }
};
let routes = get("/api/routes/received").await?["routes"].take();
let validity: Vec<(&str, &str)> = routes
.as_array()
.unwrap()
.iter()
.map(|r| (r["prefix"].as_str().unwrap(), r["aspa"].as_str().unwrap()))
.collect();
assert_eq!(
validity,
[("193.0.0.0/21", "valid"), ("194.0.0.0/24", "unknown")]
);
let refused = get("/api/routes/rejected").await?;
assert_eq!(
refused[0]["by_reason"],
serde_json::json!({"ASPA invalid": 1})
);
let peers = get("/api/peers").await?;
assert_eq!(peers[0]["role"], "peer");
assert_eq!(peers[0]["held_routes"], 1);
let rpki = get("/api/rpki").await?;
assert_eq!(rpki["aspas"], 3);
assert_eq!(rpki["protocol_version"], 2);
Ok(())
}
/// What a path may look like depends on who it comes from. The same route
/// that a peer may not pass on is in order from a provider, which may hand
/// us anything it has. Where it is not said what the peer is, paths are
/// not verified.
#[tokio::test]
async fn test_a_route_that_has_crossed_a_peering_is_taken_from_a_provider_only() -> Result<()> {
for (role, taken) in [
(Some(PeerRole::Provider), true),
(Some(PeerRole::Customer), false),
(Some(PeerRole::Peer), false),
(Some(PeerRole::RouteServerClient), false),
(None, true),
] {
let cache = TestCache::start_with(vec![], aspas()).await;
let ServerWithPeer { server, .. } = server_with(&cache, role, default_policy()).await?;
let mut peer = established(&server).await?;
announce(&mut peer, &["8.8.8.0/24"], &[PEER_ASN, 3356, 15169]).await?;
let expected = if taken {
server.table_becomes(&["8.8.8.0/24"], 0).await
} else {
server.table_becomes(&[], 1).await
};
expected.map_err(|e| eyre::eyre!("from a peer that is {role:?}: {e}"))?;
}
Ok(())
}
/// An Internet exchange's route server does not put its AS on the path, so
/// the path need not begin with the route server's AS. A leak is refused
/// across it all the same.
#[tokio::test]
async fn test_paths_from_a_route_server_do_not_begin_with_its_as() -> Result<()> {
let cache = TestCache::start_with(vec![], aspas()).await;
let import = ImportPolicy {
accept_any_first_as: true,
..default_policy()
};
let ServerWithPeer { server, .. } =
server_with(&cache, Some(PeerRole::RouteServer), import).await?;
let mut peer = established(&server).await?;
// 3356's customer route, as 3356 gave it to the route server.
announce(&mut peer, &["8.8.8.0/24"], &[3356, 15169]).await?;
// The same route leaked to the route server by another member.
announce(&mut peer, &["8.8.4.0/24"], &[2121, 3356, 15169]).await?;
server.table_becomes(&["8.8.8.0/24"], 1).await
}
/// Validity changes with the ASPAs, and the routes follow: one that was
/// taken goes when its path becomes invalid, and one that was refused is
/// taken when it no longer is. The peer sends nothing more meanwhile.
#[tokio::test]
async fn test_routes_follow_the_aspas() -> Result<()> {
let cache = TestCache::start_with(vec![], aspas()).await;
let ServerWithPeer { server, .. } =
server_with(&cache, Some(PeerRole::Peer), default_policy()).await?;
let mut peer = established(&server).await?;
announce(&mut peer, &["194.0.0.0/24"], &[PEER_ASN, 2121]).await?;
announce(&mut peer, &["8.8.8.0/24"], &[PEER_ASN, 3356, 15169]).await?;
server.table_becomes(&["194.0.0.0/24"], 1).await?;
// 2121 says that its only provider is 1101: its route by way of 3333
// is now invalid, and goes.
let mut changed = aspas();
changed.push((2121, vec![1101]));
cache.set_aspas(changed);
server.table_becomes(&[], 2).await?;
let refused = &server.rejections.peers()[0].1;
assert_eq!(refused.total(), 2);
assert_eq!(refused.recent()[0].prefix, prefix("194.0.0.0/24"));
assert_eq!(&refused.recent()[0].as_path[..], [PEER_ASN, 2121]);
// 3356 becomes a customer of 3333, and 2121 too: both routes come
// back, with the paths they were announced with.
cache.set_aspas(vec![
(15169, vec![3356]),
(3356, vec![PEER_ASN]),
(PEER_ASN, vec![0]),
(2121, vec![1101, PEER_ASN]),
]);
server
.table_becomes(&["194.0.0.0/24", "8.8.8.0/24"], 0)
.await?;
let paths: BTreeSet<Vec<u32>> = server
.received_routes()
.iter()
.flat_map(|route| &route.attributes)
.filter_map(|attribute| match attribute {
PathAttribute::AsPath(path) => Some(path.segments[0].path.clone()),
_ => None,
})
.collect();
assert_eq!(
paths,
[vec![PEER_ASN, 2121], vec![PEER_ASN, 3356, 15169]].into()
);
// The validator loses its ASPAs altogether: nothing is known about
// any path, and nothing is refused.
cache.set_aspas(vec![
(15169, vec![3356]),
(3356, vec![0]),
(PEER_ASN, vec![0]),
]);
server.table_becomes(&["194.0.0.0/24"], 1).await?;
cache.set_aspas(vec![]);
server
.table_becomes(&["194.0.0.0/24", "8.8.8.0/24"], 0)
.await
}
/// A peer can be exempt: its routes with invalid paths are taken, and
/// shown for what they are.
#[tokio::test]
async fn test_a_peer_can_be_exempt_from_path_verification() -> Result<()> {
let cache = TestCache::start_with(vec![], aspas()).await;
let import = ImportPolicy {
accept_aspa_invalid: true,
..default_policy()
};
let ServerWithPeer { server, .. } = server_with(&cache, Some(PeerRole::Peer), import).await?;
let (base, _http) = server.serve_http().await?;
let mut peer = established(&server).await?;
announce(&mut peer, &["8.8.8.0/24"], &[PEER_ASN, 3356, 15169]).await?;
server.table_becomes(&["8.8.8.0/24"], 0).await?;
let routes: Value = reqwest::get(format!("{base}/api/routes/received"))
.await?
.json()
.await?;
assert_eq!(routes["routes"][0]["aspa"], "invalid");
assert!(server.rejections.peers().is_empty());
Ok(())
}
/// A route can fail both checks. It is taken only when it passes both.
#[tokio::test]
async fn test_a_route_is_held_until_origin_and_path_are_both_in_order() -> Result<()> {
let google = |asn| bgp_rpki::Vrp {
prefix: prefix("8.8.8.0/24"),
max_length: 24,
asn,
};
let cache = TestCache::start_with(vec![google(13335)], aspas()).await;
let ServerWithPeer { server, .. } =
server_with(&cache, Some(PeerRole::Peer), default_policy()).await?;
let mut peer = established(&server).await?;
announce(&mut peer, &["8.8.8.0/24"], &[PEER_ASN, 3356, 15169]).await?;
server.table_becomes(&[], 1).await?;
// The origin is put right; the path is still a leak.
cache.set(vec![google(15169)]);
tokio::time::sleep(Duration::from_millis(500)).await;
server.table_becomes(&[], 1).await?;
cache.set_aspas(vec![(15169, vec![3356]), (3356, vec![PEER_ASN])]);
server.table_becomes(&["8.8.8.0/24"], 0).await
}
crates/integration-tests/tests/basic_connection.rs
+38 -10
View File
@@ -4,7 +4,6 @@ use std::time::Duration;
use bgp_integration_tests::test_peer::TestPeer;
use bgp_integration_tests::test_server::RouteChange;
use bgp_integration_tests::test_server::TestServer;
use bgp_integration_tests::{TIMEOUT, wait_until};
use bgp_packet::errors::{BgpError, CeaseSubcode, FsmSubcode, MsgHeaderSubcode, UpdateMsgSubcode};
use bgp_packet::message::{
AsPathAttribute, Message, MpReachNlriPathAttribute, MpUnreachNlriPathAttribute,
@@ -15,6 +14,8 @@ use bgp_primitives::constants::{AfiSafi, IpVersion};
use bgp_primitives::ip_prefix::IpPrefix;
use eyre::Result;
const TIMEOUT: Duration = Duration::from_secs(5);
/// The attributes every announcement of IPv4 routes must carry. An UPDATE
/// without them has its routes withdrawn instead (RFC 7606).
fn mandatory_attributes() -> Vec<PathAttribute> {
@@ -97,6 +98,20 @@ async fn test_bad_marker_does_not_crash_server() -> Result<()> {
Ok(())
}
/// Reads until a NOTIFICATION arrives, skipping KEEPALIVEs.
async fn expect_notification(peer: &mut TestPeer) -> Result<NotificationMessage> {
tokio::time::timeout(TIMEOUT, async {
loop {
match peer.recv().await? {
Message::Notification(n) => return Ok(n),
Message::KeepAlive => {}
other => eyre::bail!("expected NOTIFICATION, got {other:?}"),
}
}
})
.await?
}
/// A header with the given length and type, followed by `body`.
fn framed(length: u16, msg_type: u8, body: &[u8]) -> Vec<u8> {
let mut buf = vec![0xFF; 16];
@@ -155,7 +170,7 @@ async fn test_malformed_messages_are_answered_with_the_matching_notification() -
peer.handshake(65001, Ipv4Addr::new(10, 0, 1, i as u8 + 1), 90)
.await?;
peer.send_raw(&bytes).await?;
let notification = peer.expect_notification().await?;
let notification = expect_notification(&mut peer).await?;
assert_eq!(
(notification.error, notification.data),
(error.clone(), data.clone()),
@@ -166,7 +181,7 @@ async fn test_malformed_messages_are_answered_with_the_matching_notification() -
let mut peer = TestPeer::connect(server.addr).await?;
let _ = peer.recv_open().await?;
peer.send_raw(&bytes).await?;
let notification = peer.expect_notification().await?;
let notification = expect_notification(&mut peer).await?;
assert_eq!(
(notification.error, notification.data),
(error, data),
@@ -179,7 +194,7 @@ async fn test_malformed_messages_are_answered_with_the_matching_notification() -
let mut peer = TestPeer::connect(server.addr).await?;
let _ = peer.recv_open().await?;
peer.send(&Message::KeepAlive).await?;
let notification = peer.expect_notification().await?;
let notification = expect_notification(&mut peer).await?;
assert_eq!(
notification.error,
BgpError::FsmError(FsmSubcode::UnexpectedOpenSent)
@@ -213,10 +228,13 @@ async fn test_message_split_across_a_keepalive_tick_is_still_one_message() -> Re
tokio::time::sleep(Duration::from_millis(1300)).await;
peer.send_raw(&update[10..]).await?;
wait_until("the split UPDATE reaches the RIB", || {
server.received_routes().iter().any(|e| e.prefix == prefix)
tokio::time::timeout(TIMEOUT, async {
while !server.received_routes().iter().any(|e| e.prefix == prefix) {
tokio::time::sleep(Duration::from_millis(20)).await;
}
})
.await?;
.await
.map_err(|_| eyre::eyre!("the split UPDATE never reached the RIB"))?;
assert_eq!(server.registry.list().len(), 1, "session was dropped");
Ok(())
}
@@ -248,7 +266,7 @@ async fn test_server_sends_keepalives() -> Result<()> {
peer.handshake(65001, Ipv4Addr::new(10, 0, 0, 1), 3).await?;
// We should receive a KEEPALIVE within ~1 second (hold/3).
let msg = peer.recv_within(Duration::from_secs(3)).await?;
let msg = tokio::time::timeout(Duration::from_secs(3), peer.recv()).await??;
assert!(
matches!(msg, Message::KeepAlive),
"Expected KEEPALIVE, got {:?}",
@@ -268,7 +286,16 @@ async fn test_hold_timer_expires() -> Result<()> {
// Don't send any keepalives. The server will send keepalives until its own
// hold timer fires and it sends a NOTIFICATION. Drain messages until we
// see it (or the outer timeout fires).
let notif = tokio::time::timeout(Duration::from_secs(6), peer.expect_notification()).await??;
let notif = tokio::time::timeout(Duration::from_secs(6), async {
loop {
match peer.recv().await? {
Message::Notification(n) => return Ok::<_, eyre::Error>(n),
Message::KeepAlive => {} // server keepalive — keep waiting
other => eyre::bail!("Unexpected message: {:?}", other),
}
}
})
.await??;
assert!(
matches!(notif.error, BgpError::HoldTimer),
"Expected HoldTimer NOTIFICATION, got {:?}",
@@ -292,7 +319,8 @@ async fn test_peer_sends_notification() -> Result<()> {
.await?;
// Server should close the connection cleanly.
assert!(peer.closed_within(TIMEOUT).await);
tokio::time::sleep(Duration::from_millis(100)).await;
assert!(peer.is_closed().await);
Ok(())
}
crates/integration-tests/tests/best_route.rs
+72 -43
View File
@@ -2,34 +2,66 @@
//! peers, which of them the server chooses, and how that is shown.
use std::net::Ipv4Addr;
use std::time::Duration;
use bgp_integration_tests::announcement::{Announcement, withdrawal as withdrawal_of};
use bgp_integration_tests::test_peer::TestPeer;
use bgp_integration_tests::test_server::{RouteChange, TestServer};
use bgp_integration_tests::{TIMEOUT, get_json, prefix};
use bgp_packet::message::{PathAttribute, UpdateMessage};
use bgp_packet::message::{
AsPathAttribute, Message, NextHopPathAttribute, OriginPathAttribute, PathAttribute,
UpdateMessage,
};
use bgp_primitives::ip_prefix::{IpPrefix, Nlri};
use eyre::Result;
use serde_json::Value;
const LOCAL_ASN: u32 = 65000;
const TIMEOUT: Duration = Duration::from_secs(5);
fn prefix(s: &str) -> IpPrefix {
IpPrefix::try_from(s).unwrap()
}
/// A session from the peer numbered `number`, in AS 65000 + `number`.
async fn peer(server: &TestServer, number: u8) -> Result<TestPeer> {
server
.established(
LOCAL_ASN + u32::from(number),
Ipv4Addr::new(10, 0, 0, number),
)
.await
let mut peer = TestPeer::connect(server.addr).await?;
peer.handshake(
LOCAL_ASN + u32::from(number),
Ipv4Addr::new(10, 0, 0, number),
90,
)
.await?;
Ok(peer)
}
fn announcement(prefixes: &[IpPrefix], as_path: &[u32]) -> Announcement {
Announcement::of(Nlri::plain(prefixes.iter().copied()), as_path)
fn announcement(prefixes: &[IpPrefix], as_path: &[u32]) -> Message {
Message::Update(UpdateMessage {
withdrawn_routes: vec![],
path_attributes: vec![
PathAttribute::Origin(OriginPathAttribute::IGP),
AsPathAttribute::from_asns(as_path.to_vec()),
PathAttribute::NextHop(NextHopPathAttribute(Ipv4Addr::new(192, 0, 2, 1))),
],
announced_routes: Nlri::plain(prefixes.iter().copied()),
})
}
fn withdrawal(prefixes: &[IpPrefix]) -> UpdateMessage {
withdrawal_of(Nlri::plain(prefixes.iter().copied()))
fn withdrawal(prefixes: &[IpPrefix]) -> Message {
Message::Update(UpdateMessage {
withdrawn_routes: Nlri::plain(prefixes.iter().copied()),
path_attributes: vec![],
announced_routes: vec![],
})
}
/// Waits until the table holds `routes` routes.
async fn wait_for_routes(server: &TestServer, routes: usize) -> Result<()> {
tokio::time::timeout(TIMEOUT, async {
while server.rib.len() != routes {
tokio::time::sleep(Duration::from_millis(5)).await;
}
})
.await?;
Ok(())
}
/// The AS path of the route the server holds as best for `prefix`.
@@ -46,6 +78,16 @@ fn best_path(server: &TestServer, prefix: &IpPrefix) -> Option<Vec<u32>> {
})
}
async fn get_json(client: &reqwest::Client, url: &str) -> Result<Value> {
Ok(client
.get(url)
.send()
.await?
.error_for_status()?
.json()
.await?)
}
/// Two peers offer the same prefix. The route with the shorter AS path is
/// chosen; when it goes, the other is.
#[tokio::test]
@@ -57,21 +99,21 @@ async fn test_the_best_route_is_chosen_and_replaced_when_it_goes() -> Result<()>
let p = prefix("203.0.113.0/24");
let mut far = peer(&server, 1).await?;
far.announce(announcement(&[p], &[65001, 64999, 64998]))
far.send(&announcement(&[p], &[65001, 64999, 64998]))
.await?;
server.wait_for_routes(1).await?;
wait_for_routes(&server, 1).await?;
assert_eq!(best_path(&server, &p), Some(vec![65001, 64999, 64998]));
let mut near = peer(&server, 2).await?;
near.announce(announcement(&[p], &[65002, 64998])).await?;
server.wait_for_routes(2).await?;
near.send(&announcement(&[p], &[65002, 64998])).await?;
wait_for_routes(&server, 2).await?;
assert_eq!(best_path(&server, &p), Some(vec![65002, 64998]));
// Both routes are kept and shown; one is marked.
let status = get_json(&client, &format!("{base}/api/status")).await?;
assert_eq!(status["received_routes"], 2);
assert_eq!(status["received_prefixes"], 1);
let routes = get_json(&client, &format!("{base}/api/routes/received")).await?["routes"].take();
let routes = get_json(&client, &format!("{base}/api/routes/received")).await?;
let best: Vec<&Value> = routes
.as_array()
.unwrap()
@@ -81,27 +123,14 @@ async fn test_the_best_route_is_chosen_and_replaced_when_it_goes() -> Result<()>
assert_eq!(routes.as_array().unwrap().len(), 2);
assert_eq!(best.len(), 1);
assert_eq!(best[0]["peer_asn"], 65002);
// Listed from the best down, each with its place in selection.
let ranks: Vec<(u64, u64)> = routes
.as_array()
.unwrap()
.iter()
.map(|route| {
(
route["rank"].as_u64().unwrap(),
route["peer_asn"].as_u64().unwrap(),
)
})
.collect();
assert_eq!(ranks, vec![(1, 65002), (2, 65001)]);
// The session with the best route ends: the other route takes over.
drop(near);
server.wait_for_routes(1).await?;
wait_for_routes(&server, 1).await?;
assert_eq!(best_path(&server, &p), Some(vec![65001, 64999, 64998]));
far.announce(withdrawal(&[p])).await?;
server.wait_for_routes(0).await?;
far.send(&withdrawal(&[p])).await?;
wait_for_routes(&server, 0).await?;
assert_eq!(best_path(&server, &p), None);
// A follower of the best routes is told of each change: the first
@@ -128,9 +157,9 @@ async fn test_changes_of_best_route_are_on_the_event_stream() -> Result<()> {
// A route that is there before the client connects is not an event.
let mut first = peer(&server, 1).await?;
first
.announce(announcement(&[prefix("198.51.100.0/24")], &[65001]))
.send(&announcement(&[prefix("198.51.100.0/24")], &[65001]))
.await?;
server.wait_for_routes(1).await?;
wait_for_routes(&server, 1).await?;
let mut stream = reqwest::Client::new()
.get(format!("{base}/api/events"))
@@ -138,10 +167,10 @@ async fn test_changes_of_best_route_are_on_the_event_stream() -> Result<()> {
.await?
.error_for_status()?;
first.announce(announcement(&[p], &[65001, 64998])).await?;
server.wait_for_routes(2).await?;
first.announce(withdrawal(&[p])).await?;
server.wait_for_routes(1).await?;
first.send(&announcement(&[p], &[65001, 64998])).await?;
wait_for_routes(&server, 2).await?;
first.send(&withdrawal(&[p])).await?;
wait_for_routes(&server, 1).await?;
let mut text = String::new();
let mut events: Vec<Value> = Vec::new();
@@ -194,10 +223,10 @@ async fn test_received_routes_are_returned_up_to_a_limit() -> Result<()> {
let prefixes: Vec<IpPrefix> = (0..40).map(|n| prefix(&format!("10.{n}.0.0/16"))).collect();
let mut peer = peer(&server, 1).await?;
peer.announce(announcement(&prefixes, &[65001])).await?;
server.wait_for_routes(40).await?;
peer.send(&announcement(&prefixes, &[65001])).await?;
wait_for_routes(&server, 40).await?;
let count = |page: Value| page["routes"].as_array().unwrap().len();
let count = |routes: Value| routes.as_array().unwrap().len();
let url = format!("{base}/api/routes/received");
assert_eq!(count(get_json(&client, &url).await?), 40);
assert_eq!(
crates/integration-tests/tests/blackhole.rs
-141
View File
@@ -1,141 +0,0 @@
//! Blackhole requests (RFC 7999): refused by default, and with
//! `accept_blackholes` taken for one address inside what the peer may
//! announce, whatever their next hop.
use std::collections::BTreeSet;
use std::net::Ipv4Addr;
use std::time::Duration;
use bgp_integration_tests::announcement::announcement;
use bgp_integration_tests::test_peer::TestPeer;
use bgp_integration_tests::test_server::{ServerWithPeer, TestServer};
use bgp_integration_tests::{default_policy, prefix, wait_until};
use bgp_server::policy::{ImportPolicy, PrefixSet};
use eyre::Result;
const LOCAL_ASN: u32 = 1299;
const PEER_ASN: u32 = 3333;
const PEER_ID: Ipv4Addr = Ipv4Addr::new(10, 0, 0, 2);
const BLACKHOLE: (u16, u16) = (0xFFFF, 0x029A);
async fn server_with(import: ImportPolicy) -> Result<ServerWithPeer> {
TestServer::with_peer(LOCAL_ASN, PEER_ASN, |c, peer| {
c.connect_retry = Duration::from_millis(50);
peer.import = import;
})
.await
}
/// Waits until the server has taken `taken` routes of the session's and
/// refused `refused`, and returns the prefixes in its table.
async fn table_after(server: &TestServer, taken: u64, refused: u64) -> Result<BTreeSet<String>> {
let counters = server.registry.list()[0].counters.clone();
wait_until("the routes are dealt with", || {
(counters.routes(), counters.rejected()) == (taken, refused)
})
.await
.map_err(|_| {
eyre::eyre!(
"the server took {} routes and refused {}; expected {taken} and {refused}",
counters.routes(),
counters.rejected()
)
})?;
Ok(server
.received_routes()
.iter()
.map(|route| route.prefix.to_string())
.collect())
}
/// The policy that lets the peer announce 193.0.0.0/21 and nothing
/// longer than a /24 inside it, as a customer's would.
fn customer_policy(accept_blackholes: bool) -> ImportPolicy {
let mut prefixes = PrefixSet::default();
prefixes.allow(prefix("193.0.0.0/21"), 24);
ImportPolicy {
prefixes: Some(prefixes),
accept_blackholes,
..default_policy()
}
}
#[tokio::test]
async fn test_blackhole_requests_are_refused_unless_accepted() -> Result<()> {
let with_peer = server_with(customer_policy(false)).await?;
let server = &with_peer.server;
let mut peer: TestPeer = server.established(PEER_ASN, PEER_ID).await?;
peer.announce(announcement(&["193.0.0.9/32"], &[PEER_ASN]).with_communities(&[BLACKHOLE]))
.await?;
peer.announce(announcement(&["193.0.0.0/24"], &[PEER_ASN]))
.await?;
assert_eq!(table_after(server, 1, 1).await?, set(&["193.0.0.0/24"]));
let refused = server.rejections.peers();
assert_eq!(refused.len(), 1);
assert_eq!(
refused[0]
.1
.by_reason()
.iter()
.map(|(reason, n)| (reason.as_str(), *n))
.collect::<Vec<_>>(),
[("blackhole request", 1)]
);
Ok(())
}
#[tokio::test]
async fn test_a_blackhole_request_is_taken_for_one_address_in_the_peers_space() -> Result<()> {
let with_peer = server_with(customer_policy(true)).await?;
let server = &with_peer.server;
let mut peer: TestPeer = server.established(PEER_ASN, PEER_ID).await?;
// One address inside the peer's prefix, from a next hop that is no
// neighbor and would not pass as an ordinary route's.
peer.announce(
announcement(&["193.0.0.9/32"], &[PEER_ASN])
.with_next_hop(Ipv4Addr::new(172, 31, 0, 9))
.with_communities(&[BLACKHOLE]),
)
.await?;
// More than one address: not a blackhole request that is taken.
peer.announce(announcement(&["193.0.0.0/25"], &[PEER_ASN]).with_communities(&[BLACKHOLE]))
.await?;
// One address outside the peer's space.
peer.announce(announcement(&["193.0.8.9/32"], &[PEER_ASN]).with_communities(&[BLACKHOLE]))
.await?;
// And an ordinary route, taken as ever.
peer.announce(announcement(&["193.0.0.0/24"], &[PEER_ASN]))
.await?;
assert_eq!(
table_after(server, 2, 2).await?,
set(&["193.0.0.0/24", "193.0.0.9/32"])
);
let refused = server.rejections.peers();
let mut reasons: Vec<(&str, u64)> = refused[0]
.1
.by_reason()
.iter()
.map(|(reason, n)| (reason.as_str(), *n))
.collect();
reasons.sort();
assert_eq!(
reasons,
[
("blackhole for more than one address", 1),
("prefix not allowed", 1)
]
);
// The route is in the table with its request on it, for the kernel
// and for whoever it goes on to.
let routes = server.received_routes();
let blackhole = routes
.iter()
.find(|route| route.prefix == prefix("193.0.0.9/32"))
.expect("the blackhole route is in the table");
assert!(ImportPolicy::is_blackhole_request(&blackhole.attributes));
Ok(())
}
fn set(prefixes: &[&str]) -> BTreeSet<String> {
prefixes.iter().map(|p| p.to_string()).collect()
}
crates/integration-tests/tests/collision.rs
+104 -46
View File
@@ -3,33 +3,46 @@
//! The test server's BGP Identifier is 127.0.0.1. A test peer picks an
//! identifier above or below that to steer which connection must survive.
use std::net::Ipv4Addr;
use std::net::{Ipv4Addr, SocketAddr};
use std::time::Duration;
use bgp_integration_tests::announcement::announcement;
use bgp_integration_tests::test_peer::TestPeer;
use bgp_integration_tests::test_server::{ServerWithPeer, TestServer};
use bgp_integration_tests::{TIMEOUT, prefix, wait_until};
use bgp_integration_tests::test_server::TestServer;
use bgp_packet::errors::{BgpError, CeaseSubcode, OpenMsgSubcode};
use bgp_packet::message::{Message, NotificationMessage};
use bgp_packet::message::{
AsPathAttribute, Message, NextHopPathAttribute, NotificationMessage, OriginPathAttribute,
PathAttribute, UpdateMessage,
};
use bgp_primitives::ip_prefix::IpPrefix;
use bgp_rib::{LocalRoute, LocalRouteAttrs};
use bgp_server::collision::Direction;
use bgp_server::policy::ImportPolicy;
use bgp_server::config::PeerConfig;
use bgp_server::family::FamilySet;
use eyre::Result;
use tokio::net::TcpListener;
const LOCAL_ASN: u32 = 65000;
const PEER_ASN: u32 = 65001;
const TIMEOUT: Duration = Duration::from_secs(5);
/// Above the server's 127.0.0.1.
const HIGH_ID: Ipv4Addr = Ipv4Addr::new(192, 0, 2, 1);
/// Below the server's 127.0.0.1.
const LOW_ID: Ipv4Addr = Ipv4Addr::new(10, 0, 0, 1);
/// A server that dials its peer "r1" every 200 ms until it has a session.
async fn server_dialing() -> Result<ServerWithPeer> {
TestServer::with_peer(LOCAL_ASN, PEER_ASN, |c, peer| {
async fn server_dialing(listener: &TcpListener) -> Result<TestServer> {
let endpoint: SocketAddr = listener.local_addr()?;
TestServer::start_with(LOCAL_ASN, |c| {
c.connect_retry = Duration::from_millis(200);
peer.import = ImportPolicy::permissive();
c.peers = vec![PeerConfig {
name: "r1".into(),
endpoint,
remote_asn: PEER_ASN,
hold_time_secs: 90,
local_address: None,
families: FamilySet::supported(),
add_path: None,
}];
})
.await
}
@@ -44,13 +57,27 @@ async fn open_only(peer: &mut TestPeer, router_id: Ipv4Addr) -> Result<()> {
/// Finishes the handshake begun by [`open_only`].
async fn confirm(peer: &mut TestPeer) -> Result<()> {
match peer.recv().await? {
match tokio::time::timeout(TIMEOUT, peer.recv()).await?? {
Message::KeepAlive => {}
other => eyre::bail!("expected KEEPALIVE, got {other:?}"),
}
peer.send(&Message::KeepAlive).await
}
/// Reads until a NOTIFICATION arrives, skipping KEEPALIVEs.
async fn expect_notification(peer: &mut TestPeer) -> Result<NotificationMessage> {
tokio::time::timeout(TIMEOUT, async {
loop {
match peer.recv().await? {
Message::Notification(n) => return Ok(n),
Message::KeepAlive => {}
other => eyre::bail!("expected NOTIFICATION, got {other:?}"),
}
}
})
.await?
}
fn is_collision_cease(n: &NotificationMessage) -> bool {
matches!(
n.error,
@@ -58,26 +85,38 @@ fn is_collision_cease(n: &NotificationMessage) -> bool {
)
}
async fn wait_for_sessions(server: &TestServer, n: usize) -> Result<()> {
tokio::time::timeout(TIMEOUT, async {
while server.registry.list().len() != n {
tokio::time::sleep(Duration::from_millis(10)).await;
}
})
.await?;
Ok(())
}
/// Peer has the higher identifier, so the connection the *peer* initiated
/// must survive: the server closes its own outbound connection.
#[tokio::test]
async fn test_simultaneous_open_peer_higher_keeps_inbound() -> Result<()> {
let ServerWithPeer { server, listener } = server_dialing().await?;
let listener = TcpListener::bind((Ipv4Addr::LOCALHOST, 0)).await?;
let server = server_dialing(&listener).await?;
// Server's outbound connection reaches OpenConfirm.
let mut dialled = TestPeer::accept(&listener).await?;
let (stream, _) = tokio::time::timeout(TIMEOUT, listener.accept()).await??;
let mut dialled = TestPeer::from_stream(stream);
open_only(&mut dialled, HIGH_ID).await?;
// The peer now also dials the server.
let mut inbound = TestPeer::connect(server.addr).await?;
open_only(&mut inbound, HIGH_ID).await?;
let notif = dialled.expect_notification().await?;
let notif = expect_notification(&mut dialled).await?;
assert!(is_collision_cease(&notif), "{notif:?}");
assert!(dialled.closed_within(TIMEOUT).await);
assert!(dialled.is_closed().await);
confirm(&mut inbound).await?;
server.wait_for_sessions(1).await?;
wait_for_sessions(&server, 1).await?;
let session = server.registry.list().remove(0);
assert_eq!(session.direction, Direction::Inbound);
assert_eq!(session.name.as_deref(), Some("r1"));
@@ -86,11 +125,11 @@ async fn test_simultaneous_open_peer_higher_keeps_inbound() -> Result<()> {
server
.local_rib
.announce(LocalRoute::new(
prefix("10.100.1.0/24"),
IpPrefix::try_from("10.100.1.0/24")?,
LocalRouteAttrs::default(),
))
.await?;
let update = inbound.recv_update().await?;
let update = tokio::time::timeout(TIMEOUT, inbound.recv_update()).await??;
assert_eq!(update.announced_routes[0].to_string(), "10.100.1.0/24");
// The dialer notices the surviving session and does not open another.
@@ -106,20 +145,22 @@ async fn test_simultaneous_open_peer_higher_keeps_inbound() -> Result<()> {
/// must survive: the peer's inbound connection is refused.
#[tokio::test]
async fn test_simultaneous_open_peer_lower_keeps_outbound() -> Result<()> {
let ServerWithPeer { server, listener } = server_dialing().await?;
let listener = TcpListener::bind((Ipv4Addr::LOCALHOST, 0)).await?;
let server = server_dialing(&listener).await?;
let mut dialled = TestPeer::accept(&listener).await?;
let (stream, _) = tokio::time::timeout(TIMEOUT, listener.accept()).await??;
let mut dialled = TestPeer::from_stream(stream);
open_only(&mut dialled, LOW_ID).await?;
let mut inbound = TestPeer::connect(server.addr).await?;
open_only(&mut inbound, LOW_ID).await?;
let notif = inbound.expect_notification().await?;
let notif = expect_notification(&mut inbound).await?;
assert!(is_collision_cease(&notif), "{notif:?}");
assert!(inbound.closed_within(TIMEOUT).await);
assert!(inbound.is_closed().await);
confirm(&mut dialled).await?;
server.wait_for_sessions(1).await?;
wait_for_sessions(&server, 1).await?;
assert_eq!(server.registry.list()[0].direction, Direction::Outbound);
Ok(())
}
@@ -128,20 +169,22 @@ async fn test_simultaneous_open_peer_lower_keeps_outbound() -> Result<()> {
/// outcome: here the inbound connection reaches OpenConfirm first.
#[tokio::test]
async fn test_outcome_is_independent_of_arrival_order() -> Result<()> {
let ServerWithPeer { server, listener } = server_dialing().await?;
let listener = TcpListener::bind((Ipv4Addr::LOCALHOST, 0)).await?;
let server = server_dialing(&listener).await?;
let mut dialled = TestPeer::accept(&listener).await?;
let (stream, _) = tokio::time::timeout(TIMEOUT, listener.accept()).await??;
let mut dialled = TestPeer::from_stream(stream);
// Inbound first this time; peer is lower so outbound must still win.
let mut inbound = TestPeer::connect(server.addr).await?;
open_only(&mut inbound, LOW_ID).await?;
open_only(&mut dialled, LOW_ID).await?;
let notif = inbound.expect_notification().await?;
let notif = expect_notification(&mut inbound).await?;
assert!(is_collision_cease(&notif), "{notif:?}");
confirm(&mut dialled).await?;
server.wait_for_sessions(1).await?;
wait_for_sessions(&server, 1).await?;
assert_eq!(server.registry.list()[0].direction, Direction::Outbound);
Ok(())
}
@@ -154,11 +197,11 @@ async fn test_established_session_wins_by_default() -> Result<()> {
let mut first = TestPeer::connect(server.addr).await?;
first.handshake(PEER_ASN, HIGH_ID, 90).await?;
server.wait_for_sessions(1).await?;
wait_for_sessions(&server, 1).await?;
let mut second = TestPeer::connect(server.addr).await?;
open_only(&mut second, HIGH_ID).await?;
let notif = second.expect_notification().await?;
let notif = expect_notification(&mut second).await?;
assert!(is_collision_cease(&notif), "{notif:?}");
// The original is untouched.
@@ -166,11 +209,11 @@ async fn test_established_session_wins_by_default() -> Result<()> {
server
.local_rib
.announce(LocalRoute::new(
prefix("10.100.1.0/24"),
IpPrefix::try_from("10.100.1.0/24")?,
LocalRouteAttrs::default(),
))
.await?;
let update = first.recv_update().await?;
let update = tokio::time::timeout(TIMEOUT, first.recv_update()).await??;
assert_eq!(update.announced_routes[0].to_string(), "10.100.1.0/24");
Ok(())
}
@@ -185,26 +228,41 @@ async fn test_restarted_peer_replaces_stale_session_when_enabled() -> Result<()>
let mut stale = TestPeer::connect(server.addr).await?;
stale.handshake(PEER_ASN, HIGH_ID, 90).await?;
server.wait_for_sessions(1).await?;
wait_for_sessions(&server, 1).await?;
stale
.announce(
announcement(&["203.0.113.0/24"], &[PEER_ASN])
.with_next_hop(Ipv4Addr::new(10, 0, 0, 9)),
)
.send(&Message::Update(UpdateMessage {
withdrawn_routes: vec![],
path_attributes: vec![
PathAttribute::Origin(OriginPathAttribute::IGP),
AsPathAttribute::from_asns(vec![PEER_ASN]),
PathAttribute::NextHop(NextHopPathAttribute(Ipv4Addr::new(10, 0, 0, 9))),
],
announced_routes: vec![IpPrefix::try_from("203.0.113.0/24")?.into()],
}))
.await?;
server.wait_for_routes(1).await?;
tokio::time::timeout(TIMEOUT, async {
while server.received_routes().is_empty() {
tokio::time::sleep(Duration::from_millis(10)).await;
}
})
.await?;
let stale_addr = server.registry.list()[0].peer.addr;
// "Restart": a fresh connection from the same router.
let mut fresh = TestPeer::connect(server.addr).await?;
fresh.handshake(PEER_ASN, HIGH_ID, 90).await?;
let notif = stale.expect_notification().await?;
let notif = expect_notification(&mut stale).await?;
assert!(is_collision_cease(&notif), "{notif:?}");
wait_until("the fresh session has replaced the stale one", || {
let sessions = server.registry.list();
sessions.len() == 1 && sessions[0].peer.addr != stale_addr
tokio::time::timeout(TIMEOUT, async {
loop {
let sessions = server.registry.list();
if sessions.len() == 1 && sessions[0].peer.addr != stale_addr {
break;
}
tokio::time::sleep(Duration::from_millis(10)).await;
}
})
.await?;
assert!(
@@ -222,7 +280,7 @@ async fn test_distinct_identifiers_do_not_collide() -> Result<()> {
a.handshake(PEER_ASN, HIGH_ID, 90).await?;
let mut b = TestPeer::connect(server.addr).await?;
b.handshake(PEER_ASN, LOW_ID, 90).await?;
server.wait_for_sessions(2).await?;
wait_for_sessions(&server, 2).await?;
Ok(())
}
@@ -232,13 +290,13 @@ async fn test_identifier_is_released_when_connection_ends() -> Result<()> {
let server = TestServer::start(LOCAL_ASN).await?;
let mut first = TestPeer::connect(server.addr).await?;
first.handshake(PEER_ASN, HIGH_ID, 90).await?;
server.wait_for_sessions(1).await?;
wait_for_sessions(&server, 1).await?;
drop(first);
server.wait_for_sessions(0).await?;
wait_for_sessions(&server, 0).await?;
let mut second = TestPeer::connect(server.addr).await?;
second.handshake(PEER_ASN, HIGH_ID, 90).await?;
server.wait_for_sessions(1).await?;
wait_for_sessions(&server, 1).await?;
Ok(())
}
@@ -248,7 +306,7 @@ async fn test_peer_with_our_identifier_is_refused() -> Result<()> {
let server = TestServer::start(LOCAL_ASN).await?;
let mut peer = TestPeer::connect(server.addr).await?;
open_only(&mut peer, Ipv4Addr::new(127, 0, 0, 1)).await?;
let notif = peer.expect_notification().await?;
let notif = expect_notification(&mut peer).await?;
assert!(
matches!(notif.error, BgpError::OpenMsg(OpenMsgSubcode::BadBgpId)),
"{notif:?}"
crates/integration-tests/tests/deadlock.rs
-92
View File
@@ -1,92 +0,0 @@
//! A peer that sends us a table while not reading what we send it must
//! not stop us reading it: two routers sending each other their tables at
//! once would otherwise each wait on the other for ever.
use std::net::Ipv4Addr;
use std::sync::Arc;
use std::time::Duration;
use bgp_integration_tests::announcement::Announcement;
use bgp_integration_tests::prefix;
use bgp_integration_tests::test_peer::TestPeer;
use bgp_integration_tests::test_server::TestServer;
use bgp_server::policy::ExportPolicy;
use eyre::Result;
const LOCAL_ASN: u32 = 64601;
const ROUTES: u32 = 100_000;
fn nth(n: u32) -> bgp_primitives::ip_prefix::IpPrefix {
let [a, b, c, _] = (0x0a00_0000u32 + n * 256).to_be_bytes();
prefix(&format!("{a}.{b}.{c}.0/24"))
}
#[tokio::test]
async fn a_peer_that_does_not_read_is_still_read() -> Result<()> {
let server = TestServer::start_with(LOCAL_ASN, |c| {
c.export = Arc::new(ExportPolicy {
learned_routes: true,
all_paths: false,
..ExportPolicy::default()
});
})
.await?;
// One peer's table, for the server to pass on.
let mut first = server
.established(64701, Ipv4Addr::new(10, 0, 0, 1))
.await?;
// A route to an UPDATE, each with attributes of its own, as a real
// table has them: passing it on is then megabytes, not a few hundred
// UPDATEs that the socket buffers would take whole.
for i in 0..ROUTES {
first
.announce(
Announcement::of([nth(i).into()], &[64701, 64510])
.with_communities(&[((i >> 16) as u16 + 1, i as u16)]),
)
.await?;
}
server.wait_for_routes(ROUTES as usize).await?;
// A second peer, which the server starts sending that table to as
// soon as it is up, sends a table of its own and reads nothing back:
// the server's writes to it fill its socket and stop. The server has
// to go on reading it all the same. The peer's receive buffer is
// small, as a real link's window would be, so that this happens with
// the first few hundred kilobytes and not after the loopback's
// megabytes.
let socket = tokio::net::TcpSocket::new_v4()?;
socket.set_recv_buffer_size(16 * 1024)?;
let stream = socket.connect(server.addr).await?;
let mut second = TestPeer::from_stream(stream);
second
.handshake(64702, Ipv4Addr::new(10, 0, 0, 2), 90)
.await?;
server.wait_for_sessions(2).await?;
// Sent over a couple of seconds, so that the server's export to the
// peer is well under way while the peer is still sending.
let sending = async {
for start in (0..ROUTES).step_by(100) {
second
.announce(Announcement::of(
(start..start + 100).map(|i| nth(i).into()),
&[64702, 64510],
))
.await?;
if start % 2000 == 0 {
tokio::time::sleep(Duration::from_millis(40)).await;
}
}
Ok::<_, eyre::Report>(())
};
tokio::time::timeout(Duration::from_secs(30), sending)
.await
.expect("the server reads the peer's table though the peer reads nothing")?;
tokio::time::timeout(
Duration::from_secs(30),
server.wait_for_routes(2 * ROUTES as usize),
)
.await
.expect("every route is in the table")?;
Ok(())
}
crates/integration-tests/tests/debugging.rs
-261
View File
@@ -1,261 +0,0 @@
//! Finding out what the server has and why: looking routes up, and seeing
//! what was refused.
use std::net::Ipv4Addr;
use bgp_integration_tests::announcement::announcement;
use bgp_integration_tests::test_server::{ServerWithPeer, TestServer};
use bgp_integration_tests::{default_policy, wait_until};
use bgp_server::policy::ImportPolicy;
use eyre::Result;
use serde_json::Value;
/// The routes a query returns, as "prefix from AS, best or not".
async fn routes(client: &reqwest::Client, base: &str, query: &str) -> Result<Vec<String>> {
let reply = client
.get(format!("{base}/api/routes/received?{query}"))
.send()
.await?
.error_for_status()?;
let page: Value = reply.json().await?;
Ok(page["routes"]
.as_array()
.unwrap()
.iter()
.map(|r| {
format!(
"{} AS{}{}",
r["prefix"].as_str().unwrap(),
r["peer_asn"],
if r["best"] == true { " best" } else { "" }
)
})
.collect())
}
/// The questions asked when something is wrong with a prefix: who sends it
/// and which route won, what covers an address, what is inside a block.
#[tokio::test]
async fn test_routes_can_be_looked_up() -> Result<()> {
let server = TestServer::start(65000).await?;
let (base, _http) = server.serve_http().await?;
let client = reqwest::Client::new();
let mut one = server
.established(65001, Ipv4Addr::new(10, 0, 0, 1))
.await?;
let mut two = server
.established(65002, Ipv4Addr::new(10, 0, 0, 2))
.await?;
one.announce(announcement(
&["10.0.0.0/8", "10.1.0.0/16", "10.1.2.0/24", "192.0.2.0/24"],
&[65001],
))
.await?;
// A longer path for one of the same prefixes: it loses.
two.announce(announcement(&["10.1.0.0/16"], &[65002, 65003]))
.await?;
server.wait_for_routes(5).await?;
// One prefix: every route for it, the best first.
assert_eq!(
routes(&client, &base, "prefix=10.1.0.0/16").await?,
["10.1.0.0/16 AS65001 best", "10.1.0.0/16 AS65002"]
);
assert!(
routes(&client, &base, "prefix=10.2.0.0/16")
.await?
.is_empty()
);
// An address: every prefix that covers it, the most specific first.
assert_eq!(
routes(&client, &base, "address=10.1.2.3").await?,
[
"10.1.2.0/24 AS65001 best",
"10.1.0.0/16 AS65001 best",
"10.1.0.0/16 AS65002",
"10.0.0.0/8 AS65001 best"
]
);
assert_eq!(
routes(&client, &base, "address=10.200.0.1").await?,
["10.0.0.0/8 AS65001 best"]
);
// A block: itself and everything inside it.
assert_eq!(
routes(&client, &base, "within=10.1.0.0/16").await?,
[
"10.1.0.0/16 AS65001 best",
"10.1.0.0/16 AS65002",
"10.1.2.0/24 AS65001 best"
]
);
// Only what was chosen; and everything, in address order.
assert_eq!(
routes(&client, &base, "within=10.0.0.0/8&best=true").await?,
[
"10.0.0.0/8 AS65001 best",
"10.1.0.0/16 AS65001 best",
"10.1.2.0/24 AS65001 best"
]
);
assert_eq!(routes(&client, &base, "").await?.len(), 5);
// A page holds whole prefixes: at most the limit, unless one prefix
// alone has more.
assert!((1..=2).contains(&routes(&client, &base, "limit=2").await?.len()));
// One peer's routes, by its address; a peer that is not there has none.
assert_eq!(routes(&client, &base, "peer=127.0.0.1").await?.len(), 5);
assert!(routes(&client, &base, "peer=192.0.2.9").await?.is_empty());
// A query that makes no sense says so.
for (query, complaint) in [
("prefix=10.1.0.0", "is not a prefix"),
("address=example", "is not an IP address"),
("prefix=10.0.0.0/8&address=10.0.0.1", "at most one of"),
] {
let reply = client
.get(format!("{base}/api/routes/received?{query}"))
.send()
.await?;
assert_eq!(reply.status(), 400, "{query}");
let text = reply.text().await?;
assert!(text.contains(complaint), "{query}: {text}");
}
Ok(())
}
/// A refused route is not in the table, so it has to be findable somewhere
/// else: which peer, which prefix, with what path, and why. And there is
/// something to alert on.
#[tokio::test]
async fn test_refused_routes_can_be_seen_and_counted() -> Result<()> {
let ServerWithPeer {
server,
listener: _listener,
} = TestServer::with_peer(1299, 3333, |_, peer| {
peer.name = "refused-demo".into();
peer.import = ImportPolicy {
max_prefixes: Some(100),
..default_policy()
};
})
.await?;
let (base, _http) = server.serve_http().await?;
let client = reqwest::Client::new();
let get = |path: &str| {
let request = client.get(format!("{base}{path}"));
async move { Ok::<_, eyre::Report>(request.send().await?.error_for_status()?) }
};
let refused: Value = get("/api/routes/rejected").await?.json().await?;
assert_eq!(refused, serde_json::json!([]), "nothing refused yet");
let mut peer = server.established(3333, Ipv4Addr::new(10, 0, 0, 2)).await?;
peer.announce(announcement(&["193.0.0.0/21", "10.0.0.0/8"], &[3333]))
.await?;
peer.announce(announcement(
&["194.0.1.0/24", "194.0.2.0/24"],
&[3333, 64512],
))
.await?;
peer.announce(announcement(&["193.0.9.0/25"], &[3333, 2121]))
.await?;
wait_until("four routes are refused", || {
server
.registry
.list()
.first()
.map(|s| s.counters.rejected())
== Some(4)
})
.await?;
let refused: Value = get("/api/routes/rejected").await?.json().await?;
let record = &refused[0];
assert_eq!(record["peer"], "refused-demo");
assert_eq!(record["peer_addr"], "127.0.0.1");
assert_eq!(record["total"], 4);
assert_eq!(
record["by_reason"],
serde_json::json!({"bogon AS in path": 2, "bogon prefix": 1, "prefix too specific": 1})
);
// The latest first, each with the path it came with.
let recent: Vec<(String, String, String)> = record["recent"]
.as_array()
.unwrap()
.iter()
.map(|r| {
(
r["prefix"].as_str().unwrap().to_string(),
r["reason"].as_str().unwrap().to_string(),
r["as_path"].to_string(),
)
})
.collect();
let row = |p: &str, why: &str, path: &str| (p.to_string(), why.to_string(), path.to_string());
assert_eq!(
recent,
[
row("193.0.9.0/25", "prefix too specific", "[3333,2121]"),
row("194.0.2.0/24", "bogon AS in path", "[3333,64512]"),
row("194.0.1.0/24", "bogon AS in path", "[3333,64512]"),
row("10.0.0.0/8", "bogon prefix", "[3333]"),
]
);
// And with all its attributes, as for a route that was taken, to see
// what came along with it.
let latest = &record["recent"][0];
assert_eq!(latest["summary"]["next_hop"], "192.0.2.1");
assert_eq!(latest["summary"]["origin"], "IGP");
assert_eq!(
latest["summary"]["as_path"],
serde_json::json!([3333, 2121])
);
assert_eq!(
latest["attributes"].as_array().map(Vec::len),
Some(3),
"ORIGIN, AS_PATH and NEXT_HOP: {latest}"
);
// Asked for by peer, by name or by address.
for peer in ["refused-demo", "127.0.0.1"] {
let one: Value = get(&format!("/api/routes/rejected?peer={peer}"))
.await?
.json()
.await?;
assert_eq!(one.as_array().unwrap().len(), 1, "{peer}");
}
let none: Value = get("/api/routes/rejected?peer=someone-else")
.await?
.json()
.await?;
assert_eq!(none, serde_json::json!([]));
// The peer's line in the list of peers has the numbers too.
let peers: Value = get("/api/peers").await?.json().await?;
assert_eq!(peers[0]["routes"], 1);
assert_eq!(peers[0]["rejected_routes"], 4);
// What an alert would be made from.
let metrics = get("/metricz").await?.text().await?;
for line in [
r#"bgp_routes_rejected_total{peer="refused-demo",reason="bogon AS in path"} 2"#,
r#"bgp_routes_rejected_total{peer="refused-demo",reason="bogon prefix"} 1"#,
r#"bgp_peer_routes{peer="refused-demo"} 1"#,
r#"bgp_peer_max_prefixes{peer="refused-demo"} 100"#,
] {
assert!(metrics.contains(line), "{line} is not in:\n{metrics}");
}
// The record outlives the session; the count of routes does not.
drop(peer);
server.wait_for_sessions(0).await?;
let refused: Value = get("/api/routes/rejected").await?.json().await?;
assert_eq!(refused[0]["total"], 4);
let metrics = get("/metricz").await?.text().await?;
assert!(!metrics.contains(r#"bgp_peer_routes{peer="refused-demo"}"#));
Ok(())
}
crates/integration-tests/tests/dialer.rs
+147 -112
View File
@@ -5,47 +5,68 @@ use std::net::{Ipv4Addr, SocketAddr};
use std::time::Duration;
use bgp_integration_tests::test_peer::TestPeer;
use bgp_integration_tests::test_server::{ServerWithPeer, TestServer};
use bgp_integration_tests::{TIMEOUT, peer_config, prefix, wait_for, wait_until};
use bgp_integration_tests::test_server::TestServer;
use bgp_packet::errors::{BgpError, OpenMsgSubcode};
use bgp_packet::message::Message;
use bgp_primitives::ip_prefix::IpPrefix;
use bgp_rib::{LocalRoute, LocalRouteAttrs};
use bgp_server::config::PeerConfig;
use bgp_server::policy::ImportPolicy;
use bgp_server::family::FamilySet;
use bgp_server::session::DialState;
use eyre::Result;
use tokio::net::TcpListener;
const LOCAL_ASN: u32 = 65000;
const PEER_ASN: u32 = 65001;
const PEER_ID: Ipv4Addr = Ipv4Addr::new(10, 0, 0, 2);
const TIMEOUT: Duration = Duration::from_secs(5);
/// A server that dials its peer "r1", in `remote_asn`, every 200 ms.
async fn server_dialing(remote_asn: u32) -> Result<ServerWithPeer> {
server_with_peer(remote_asn, false).await
}
/// A server with peer "r1" in `remote_asn`, which it dials every 200 ms
/// unless the peer is `passive`.
async fn server_with_peer(remote_asn: u32, passive: bool) -> Result<ServerWithPeer> {
TestServer::with_peer(LOCAL_ASN, remote_asn, |c, peer| {
/// Starts a server configured to dial `listener`'s address as peer "r1".
async fn server_dialing(listener: &TcpListener, remote_asn: u32) -> Result<TestServer> {
let endpoint: SocketAddr = listener.local_addr()?;
TestServer::start_with(LOCAL_ASN, |c| {
c.connect_retry = Duration::from_millis(200);
peer.import = ImportPolicy::permissive();
peer.passive = passive;
c.peers = vec![PeerConfig {
name: "r1".into(),
endpoint,
remote_asn,
hold_time_secs: 90,
local_address: None,
families: FamilySet::supported(),
add_path: None,
}];
})
.await
}
async fn accept(listener: &TcpListener) -> Result<TestPeer> {
let (stream, _) = tokio::time::timeout(TIMEOUT, listener.accept()).await??;
Ok(TestPeer::from_stream(stream))
}
#[tokio::test]
async fn test_server_dials_configured_peer_and_establishes() -> Result<()> {
let ServerWithPeer { server, listener } = server_dialing(PEER_ASN).await?;
let listener = TcpListener::bind((Ipv4Addr::LOCALHOST, 0)).await?;
let server = server_dialing(&listener, PEER_ASN).await?;
let mut peer = TestPeer::accept(&listener).await?;
let mut peer = accept(&listener).await?;
// Both sides send OPEN on connect; the test peer's handshake order works
// whichever side opened TCP.
let open = peer.handshake(PEER_ASN, PEER_ID, 90).await?;
let open = peer
.handshake(PEER_ASN, Ipv4Addr::new(10, 0, 0, 2), 90)
.await?;
assert_eq!(open.asn, LOCAL_ASN as u16);
let listed = server.wait_for_session().await?;
let mut events = server.registry.subscribe();
// Either the Up event is still queued or the session is already listed.
let listed = tokio::time::timeout(TIMEOUT, async {
loop {
if let Some(s) = server.registry.list().first().cloned() {
return s;
}
let _ = events.recv().await;
}
})
.await?;
assert_eq!(listed.name.as_deref(), Some("r1"));
assert_eq!(listed.peer.asn, PEER_ASN);
assert_eq!(listed.peer.addr, listener.local_addr()?);
@@ -67,26 +88,39 @@ async fn test_server_dials_configured_peer_and_establishes() -> Result<()> {
server
.local_rib
.announce(LocalRoute::new(
prefix("10.100.1.0/24"),
IpPrefix::try_from("10.100.1.0/24")?,
LocalRouteAttrs::default(),
))
.await?;
let update = peer.recv_update().await?;
let update = tokio::time::timeout(TIMEOUT, peer.recv_update()).await??;
assert_eq!(update.announced_routes[0].to_string(), "10.100.1.0/24");
Ok(())
}
#[tokio::test]
async fn test_server_reconnects_after_peer_closes() -> Result<()> {
let ServerWithPeer { server, listener } = server_dialing(PEER_ASN).await?;
let listener = TcpListener::bind((Ipv4Addr::LOCALHOST, 0)).await?;
let server = server_dialing(&listener, PEER_ASN).await?;
let mut first = TestPeer::accept(&listener).await?;
first.handshake(PEER_ASN, PEER_ID, 90).await?;
server.wait_for_session().await?;
let mut first = accept(&listener).await?;
first
.handshake(PEER_ASN, Ipv4Addr::new(10, 0, 0, 2), 90)
.await?;
tokio::time::timeout(TIMEOUT, async {
while server.registry.list().is_empty() {
tokio::time::sleep(Duration::from_millis(10)).await;
}
})
.await?;
drop(first);
// Session goes away, then the dialer comes back.
server.wait_for_sessions(0).await?;
tokio::time::timeout(TIMEOUT, async {
while !server.registry.list().is_empty() {
tokio::time::sleep(Duration::from_millis(10)).await;
}
})
.await?;
let (_, state) = server
.registry
.configured_peers()
@@ -95,30 +129,45 @@ async fn test_server_reconnects_after_peer_closes() -> Result<()> {
.unwrap();
assert!(matches!(state, DialState::Backoff { .. }), "{state:?}");
let mut second = TestPeer::accept(&listener).await?;
second.handshake(PEER_ASN, PEER_ID, 90).await?;
server.wait_for_session().await?;
let mut second = accept(&listener).await?;
second
.handshake(PEER_ASN, Ipv4Addr::new(10, 0, 0, 2), 90)
.await?;
tokio::time::timeout(TIMEOUT, async {
while server.registry.list().is_empty() {
tokio::time::sleep(Duration::from_millis(10)).await;
}
})
.await?;
Ok(())
}
#[tokio::test]
async fn test_wrong_asn_is_refused_with_bad_peer_as() -> Result<()> {
let ServerWithPeer {
server: _server,
listener,
} = server_dialing(PEER_ASN).await?;
let listener = TcpListener::bind((Ipv4Addr::LOCALHOST, 0)).await?;
let _server = server_dialing(&listener, PEER_ASN).await?;
let mut peer = TestPeer::accept(&listener).await?;
peer.send_open(65099, PEER_ID, 90).await?;
let mut peer = accept(&listener).await?;
peer.send_open(65099, Ipv4Addr::new(10, 0, 0, 2), 90)
.await?;
let _ = peer.recv_open().await?;
let notif = peer.expect_notification().await?;
let notif = tokio::time::timeout(TIMEOUT, async {
loop {
match peer.recv().await? {
Message::Notification(n) => return Ok::<_, eyre::Error>(n),
Message::KeepAlive => {}
other => eyre::bail!("unexpected {other:?}"),
}
}
})
.await??;
assert!(
matches!(notif.error, BgpError::OpenMsg(OpenMsgSubcode::BadPeerAs)),
"{notif:?}"
);
// The dialer retries rather than giving up.
let _again = TestPeer::accept(&listener).await?;
let _again = accept(&listener).await?;
Ok(())
}
@@ -130,19 +179,23 @@ async fn test_inbound_from_configured_address_uses_config() -> Result<()> {
// so the dialer never succeeds; then connect inbound from loopback.
let server = TestServer::start_with(LOCAL_ASN, |c| {
c.connect_retry = Duration::from_secs(60);
let at = SocketAddr::new(Ipv4Addr::LOCALHOST.into(), 1);
c.peers = vec![PeerConfig {
import: ImportPolicy::permissive(),
..peer_config("r1", at, PEER_ASN)
name: "r1".into(),
endpoint: SocketAddr::new(Ipv4Addr::LOCALHOST.into(), 1),
remote_asn: PEER_ASN,
hold_time_secs: 90,
local_address: None,
families: FamilySet::supported(),
add_path: None,
}];
})
.await?;
// Wrong ASN inbound → refused.
let mut bad = TestPeer::connect(server.addr).await?;
bad.send_open(65099, PEER_ID, 90).await?;
bad.send_open(65099, Ipv4Addr::new(10, 0, 0, 2), 90).await?;
let _ = bad.recv_open().await?;
let msg = bad.recv().await?;
let msg = tokio::time::timeout(TIMEOUT, bad.recv()).await??;
assert!(
matches!(&msg, Message::Notification(n) if matches!(n.error, BgpError::OpenMsg(OpenMsgSubcode::BadPeerAs))),
"{msg:?}"
@@ -150,8 +203,17 @@ async fn test_inbound_from_configured_address_uses_config() -> Result<()> {
// Right ASN inbound → established, named.
let mut good = TestPeer::connect(server.addr).await?;
good.handshake(PEER_ASN, PEER_ID, 90).await?;
let listed = server.wait_for_session().await?;
good.handshake(PEER_ASN, Ipv4Addr::new(10, 0, 0, 2), 90)
.await?;
let listed = tokio::time::timeout(TIMEOUT, async {
loop {
if let Some(s) = server.registry.list().first().cloned() {
return s;
}
tokio::time::sleep(Duration::from_millis(10)).await;
}
})
.await?;
assert_eq!(listed.name.as_deref(), Some("r1"));
Ok(())
}
@@ -161,33 +223,46 @@ async fn test_inbound_from_configured_address_uses_config() -> Result<()> {
/// reported as the reason for backing off rather than silently ignored.
#[tokio::test]
async fn test_local_address_is_used_as_the_source() -> Result<()> {
let ServerWithPeer {
server: _server,
listener,
} = TestServer::with_peer(LOCAL_ASN, PEER_ASN, |c, peer| {
let listener = TcpListener::bind((Ipv4Addr::LOCALHOST, 0)).await?;
let endpoint: SocketAddr = listener.local_addr()?;
let _server = TestServer::start_with(LOCAL_ASN, |c| {
c.connect_retry = Duration::from_millis(200);
peer.import = ImportPolicy::permissive();
peer.local_address = Some(Ipv4Addr::LOCALHOST.into());
c.peers = vec![PeerConfig {
name: "r1".into(),
endpoint,
remote_asn: PEER_ASN,
hold_time_secs: 90,
local_address: Some(Ipv4Addr::LOCALHOST.into()),
families: FamilySet::supported(),
add_path: None,
}];
})
.await?;
let (_stream, from) = tokio::time::timeout(TIMEOUT, listener.accept()).await??;
assert_eq!(from.ip(), std::net::IpAddr::from(Ipv4Addr::LOCALHOST));
// 192.0.2.1 is documentation space; no host has it.
let endpoint = listener.local_addr()?;
let server = TestServer::start_with(LOCAL_ASN, |c| {
c.connect_retry = Duration::from_millis(200);
c.peers = vec![PeerConfig {
name: "r1".into(),
endpoint,
remote_asn: PEER_ASN,
hold_time_secs: 90,
local_address: Some(Ipv4Addr::new(192, 0, 2, 1).into()),
import: ImportPolicy::permissive(),
..peer_config("r1", endpoint, PEER_ASN)
families: FamilySet::supported(),
add_path: None,
}];
})
.await?;
let reason = wait_for("the dialer is backing off", || {
match server.registry.configured_peers().into_iter().next() {
Some((_, DialState::Backoff { last_error, .. })) => Some(last_error),
_ => None,
let reason = tokio::time::timeout(TIMEOUT, async {
loop {
if let Some((_, DialState::Backoff { last_error, .. })) =
server.registry.configured_peers().into_iter().next()
{
return last_error;
}
tokio::time::sleep(Duration::from_millis(20)).await;
}
})
.await?;
@@ -202,14 +277,16 @@ async fn test_local_address_is_used_as_the_source() -> Result<()> {
/// such a peer was dialled every few seconds for ever.
#[tokio::test]
async fn test_a_peer_that_refuses_the_session_is_dialled_ever_more_slowly() -> Result<()> {
let ServerWithPeer { server, listener } = server_dialing(PEER_ASN).await?;
let listener = TcpListener::bind((Ipv4Addr::LOCALHOST, 0)).await?;
let server = server_dialing(&listener, PEER_ASN).await?;
let mut waits = Vec::new();
for _ in 0..3 {
let mut peer = TestPeer::accept(&listener).await?;
peer.send_open(65099, PEER_ID, 90).await?;
let mut peer = accept(&listener).await?;
peer.send_open(65099, Ipv4Addr::new(10, 0, 0, 2), 90)
.await?;
let _ = peer.recv_open().await?;
let refusal = peer.recv().await?;
let refusal = tokio::time::timeout(TIMEOUT, peer.recv()).await??;
assert!(
matches!(&refusal, Message::Notification(n) if n.error == BgpError::OpenMsg(OpenMsgSubcode::BadPeerAs)),
"{refusal:?}"
@@ -217,17 +294,17 @@ async fn test_a_peer_that_refuses_the_session_is_dialled_ever_more_slowly() -> R
drop(peer);
// How long the dialer says it will wait before trying again.
let wait = wait_for("the dialer is backing off", || {
match server.registry.configured_peers().into_iter().next() {
Some((_, DialState::Backoff { until, last_error })) => {
let wait = tokio::time::timeout(TIMEOUT, async {
loop {
if let Some((_, DialState::Backoff { until, last_error })) =
server.registry.configured_peers().into_iter().next()
{
assert!(last_error.contains("AS65099"), "{last_error}");
Some(
until
.duration_since(std::time::SystemTime::now())
.unwrap_or_default(),
)
return until
.duration_since(std::time::SystemTime::now())
.unwrap_or_default();
}
_ => None,
tokio::time::sleep(Duration::from_millis(5)).await;
}
})
.await?;
@@ -240,45 +317,3 @@ async fn test_a_peer_that_refuses_the_session_is_dialled_ever_more_slowly() -> R
);
Ok(())
}
/// A passive peer is not dialled: its session comes up when it connects,
/// and until then, and after, it is shown as waiting, with how its last
/// session ended.
#[tokio::test]
async fn test_a_passive_peer_is_not_dialled_and_comes_up_when_it_connects() -> Result<()> {
let ServerWithPeer { server, listener } = server_with_peer(PEER_ASN, true).await?;
let state = || server.registry.configured_peers()[0].1.clone();
// Many retry intervals pass, and nothing connects.
assert!(
tokio::time::timeout(Duration::from_secs(1), listener.accept())
.await
.is_err(),
"a passive peer was dialled"
);
assert_eq!(state(), DialState::Passive { last_error: None });
let mut peer = TestPeer::connect(server.addr).await?;
peer.handshake(PEER_ASN, PEER_ID, 90).await?;
let listed = server.wait_for_session().await?;
assert_eq!(listed.name.as_deref(), Some("r1"));
drop(peer);
wait_until("the peer is shown as waiting, with how it ended", || {
matches!(
state(),
DialState::Passive {
last_error: Some(_)
}
)
})
.await?;
assert!(server.registry.list().is_empty());
assert!(
tokio::time::timeout(Duration::from_millis(600), listener.accept())
.await
.is_err(),
"a passive peer was dialled after its session ended"
);
Ok(())
}
crates/integration-tests/tests/export_limits.rs
-147
View File
@@ -1,147 +0,0 @@
//! Export limits, end to end: what a peer above or beside us is told is
//! held to the prefixes they list and to as many routes as they allow,
//! whatever else would send it more.
use std::collections::BTreeSet;
use std::net::Ipv4Addr;
use std::sync::Arc;
use std::time::Duration;
use bgp_integration_tests::test_peer::TestPeer;
use bgp_integration_tests::test_server::{ServerWithPeer, TestServer};
use bgp_integration_tests::{TIMEOUT, prefix};
use bgp_packet::errors::{BgpError, CeaseSubcode};
use bgp_packet::message::Message;
use bgp_packet::open::{BgpRole, Capability};
use bgp_primitives::constants::AfiSafi;
use bgp_rib::LocalRoute;
use bgp_server::policy::{ExportLimits, PeerRole, PrefixSet};
use eyre::{Result, bail};
const LOCAL_ASN: u32 = 1299;
const PEER_ASN: u32 = 3333;
const PEER_ID: Ipv4Addr = Ipv4Addr::new(10, 0, 0, 2);
/// A server that originates `originated`, with one configured peer at the
/// address the test connects from, which is a peer of ours held to
/// `limits`.
async fn server_with(originated: &[&str], limits: ExportLimits) -> Result<ServerWithPeer> {
let originate = originated
.iter()
.map(|p| LocalRoute::new(prefix(p), Default::default()))
.collect();
TestServer::with_peer(LOCAL_ASN, PEER_ASN, |c, peer| {
c.originate = originate;
peer.export_limits = Some(Arc::new(limits));
peer.role = Some(PeerRole::Peer);
})
.await
}
/// A session with the server, as its peer.
async fn established(server: &TestServer) -> Result<TestPeer> {
let capabilities = vec![
Capability::FourByteAsn { asn: PEER_ASN },
Capability::multiprotocol(AfiSafi::IPV4_UNICAST),
Capability::multiprotocol(AfiSafi::IPV6_UNICAST),
Capability::BgpRole {
role: BgpRole::Peer,
},
];
server
.established_with(PEER_ASN, PEER_ID, capabilities)
.await
}
/// What the peer has been told once it has been quiet for a while: the
/// prefixes it holds, and the NOTIFICATION that ended the session, if one
/// did.
async fn told(peer: &mut TestPeer) -> (BTreeSet<String>, Option<(BgpError, Vec<u8>)>) {
let mut routes = BTreeSet::new();
while let Ok(message) = peer.recv_within(Duration::from_millis(500)).await {
match message {
Message::Update(update) => {
for route in &update.withdrawn_routes {
routes.remove(&route.prefix.to_string());
}
for route in &update.announced_routes {
routes.insert(route.prefix.to_string());
}
}
Message::Notification(notification) => {
return (routes, Some((notification.error, notification.data)));
}
_ => {}
}
}
(routes, None)
}
fn allowing(prefixes: &[&str]) -> Option<PrefixSet> {
let mut set = PrefixSet::default();
for p in prefixes {
let p = prefix(p);
set.allow(p, p.length());
}
Some(set)
}
/// A route of ours whose prefix is not in the peer's export limits is not
/// announced to it; the one that is, is.
#[tokio::test]
async fn test_a_route_outside_the_export_limits_is_not_announced() -> Result<()> {
let limits = ExportLimits {
prefixes: allowing(&["193.0.0.0/21"]),
max_prefixes: None,
};
let ServerWithPeer { server, .. } =
server_with(&["193.0.0.0/21", "198.51.100.0/24"], limits).await?;
let mut peer = established(&server).await?;
let (routes, ended) = told(&mut peer).await;
assert_eq!(routes, BTreeSet::from(["193.0.0.0/21".to_string()]));
assert_eq!(ended, None);
Ok(())
}
/// One route more than the export limit allows is not announced: the
/// session ends with an Administrative Shutdown that says why (RFC 9003),
/// and the peer is kept down, so that it is told nothing rather than too
/// much.
#[tokio::test]
async fn test_going_over_the_export_limit_ends_the_session_and_keeps_the_peer_down() -> Result<()> {
let limits = ExportLimits {
prefixes: None,
max_prefixes: Some(2),
};
let ServerWithPeer { server, .. } =
server_with(&["193.0.0.0/21", "193.0.8.0/21"], limits).await?;
let mut peer = established(&server).await?;
let (routes, ended) = told(&mut peer).await;
assert_eq!(routes.len(), 2);
assert_eq!(ended, None);
server
.local_rib
.announce(LocalRoute::new(prefix("193.0.16.0/21"), Default::default()))
.await?;
let (more, ended) = told(&mut peer).await;
assert_eq!(more, BTreeSet::new(), "nothing more is announced");
let Some((error, data)) = ended else {
bail!("the session was not ended");
};
assert_eq!(error, BgpError::Cease(CeaseSubcode::AdminShutdown));
let text = String::from_utf8(data[1..].to_vec())?;
assert!(
text.starts_with("export limit of 2 routes reached"),
"{text}"
);
assert!(peer.closed_within(TIMEOUT).await);
// The peer is kept down: a new connection is closed unanswered.
let mut again = TestPeer::connect(server.addr).await?;
assert!(
again.closed_unanswered().await,
"a held-down peer gets no session"
);
Ok(())
}
crates/integration-tests/tests/failure.rs
+34 -6
View File
@@ -6,24 +6,52 @@
//! stops with an error, for whatever supervises it to restart it.
use std::net::Ipv4Addr;
use std::time::Duration;
use bgp_integration_tests::TIMEOUT;
use bgp_integration_tests::test_peer::TestPeer;
use bgp_integration_tests::test_server::TestServer;
use bgp_packet::errors::{BgpError, CeaseSubcode};
use bgp_packet::message::Message;
use bgp_rib::RibStopped;
use bgp_server::error::ServerError;
use eyre::Result;
use eyre::{Result, bail};
const LOCAL_ASN: u32 = 65000;
const PEER_ASN: u32 = 65001;
const TIMEOUT: Duration = Duration::from_secs(5);
async fn established(server: &TestServer) -> Result<TestPeer> {
let mut peer = TestPeer::connect(server.addr).await?;
peer.handshake(PEER_ASN, Ipv4Addr::new(10, 0, 0, 1), 90)
.await?;
tokio::time::timeout(TIMEOUT, async {
while server.registry.list().is_empty() {
tokio::time::sleep(Duration::from_millis(10)).await;
}
})
.await?;
Ok(peer)
}
/// Reads until the server's NOTIFICATION, skipping KEEPALIVEs.
async fn farewell(peer: &mut TestPeer) -> Result<BgpError> {
tokio::time::timeout(TIMEOUT, async {
loop {
match peer.recv().await? {
Message::KeepAlive => {}
Message::Notification(n) => return Ok(n.error),
other => bail!("expected NOTIFICATION, got {other:?}"),
}
}
})
.await?
}
#[tokio::test]
async fn test_losing_the_local_rib_stops_the_server_and_tells_the_peers() -> Result<()> {
let mut server = TestServer::start(LOCAL_ASN).await?;
let (base, _http) = server.serve_http().await?;
let mut peer = server
.established(PEER_ASN, Ipv4Addr::new(10, 0, 0, 1))
.await?;
let mut peer = established(&server).await?;
server.local_rib.stop_for_testing();
@@ -33,7 +61,7 @@ async fn test_losing_the_local_rib_stops_the_server_and_tells_the_peers() -> Res
"{outcome:?}"
);
assert_eq!(
peer.expect_notification().await?.error,
farewell(&mut peer).await?,
BgpError::Cease(CeaseSubcode::AdminShutdown)
);
crates/integration-tests/tests/graceful_restart.rs
-332
View File
@@ -1,332 +0,0 @@
//! Graceful restart (RFC 4724, RFC 8538): a peer that restarts keeps its
//! routes with us while it does, and we keep ours with it while we do.
use std::net::Ipv4Addr;
use std::time::Duration;
use bgp_integration_tests::announcement::{Announcement, announcement};
use bgp_integration_tests::test_peer::TestPeer;
use bgp_integration_tests::test_server::TestServer;
use bgp_integration_tests::{TIMEOUT, wait_until as until};
use bgp_packet::errors::{BgpError, CeaseSubcode};
use bgp_packet::message::{Message, NotificationMessage, UpdateMessage};
use bgp_packet::open::{Capability, GracefulRestartEntry};
use bgp_primitives::constants::{Afi, AfiSafi, Safi};
use bgp_server::restart::Leave;
use eyre::{Result, bail};
const PEER_ASN: u32 = 65001;
const PEER_ID: Ipv4Addr = Ipv4Addr::new(10, 0, 0, 2);
const PREFIXES: [&str; 3] = ["10.1.0.0/16", "10.2.0.0/16", "10.3.0.0/16"];
/// The graceful restart capability of a peer that speaks IPv4 and IPv6.
fn restart(restarted: bool, notification: bool, forwarding: bool, time: u16) -> Capability {
Capability::GracefulRestart {
restart_state: restarted,
restart_time: time,
notification,
entries: [Afi::IPV4, Afi::IPV6]
.into_iter()
.map(|afi| GracefulRestartEntry {
afi,
safi: Safi::UNICAST,
preserve_forwarding: forwarding,
})
.collect(),
}
}
/// What a dual-stack peer offers, with `more` besides.
fn capabilities(more: impl IntoIterator<Item = Capability>) -> Vec<Capability> {
let mut capabilities = vec![
Capability::FourByteAsn { asn: PEER_ASN },
Capability::MultiProtocol {
afi: Afi::IPV4,
safi: Safi::UNICAST,
},
Capability::MultiProtocol {
afi: Afi::IPV6,
safi: Safi::UNICAST,
},
Capability::RouteRefresh,
];
capabilities.extend(more);
capabilities
}
/// The peer's own routes.
fn own(prefixes: &[&str]) -> Announcement {
announcement(prefixes, &[PEER_ASN])
}
fn end_of_rib(family: AfiSafi) -> UpdateMessage {
UpdateMessage::end_of_rib(family)
}
/// The prefixes in the server's table, and how many of its routes are
/// stale.
fn table(server: &TestServer) -> (Vec<String>, usize) {
let mut prefixes = Vec::new();
let mut stale = 0;
server.rib.for_each_route(|route| {
prefixes.push(route.prefix.to_string());
stale += usize::from(route.stale);
});
prefixes.sort();
(prefixes, stale)
}
/// A peer that has announced `PREFIXES` with graceful restart as `offer`
/// says, and is Established.
async fn peer_with_routes(server: &TestServer, offer: Capability) -> Result<TestPeer> {
let mut peer = TestPeer::connect(server.addr).await?;
peer.handshake_with(PEER_ASN, PEER_ID, 90, capabilities([offer]))
.await?;
peer.announce(own(&PREFIXES)).await?;
until("the routes are in", || server.rib.len() == 3).await?;
Ok(peer)
}
/// The peer's connection fails; its routes are kept, stale, and used as
/// before. It comes back, sends some of them again, and its End-of-RIB:
/// the others go.
#[tokio::test]
async fn test_a_restarting_peer_keeps_its_routes_until_it_has_sent_them_again() -> Result<()> {
let server = TestServer::start(65000).await?;
let peer = peer_with_routes(&server, restart(false, true, true, 90)).await?;
drop(peer);
until("the session is down", || server.registry.list().is_empty()).await?;
until("the routes are stale", || table(&server).1 == 3).await?;
let kept = server.restarts.kept();
assert_eq!(kept.len(), 1);
assert_eq!(kept[0].1.routes, 3);
// Still chosen: the RFC has stale routes used like any others.
assert!(server.received_routes().iter().all(|route| route.best));
// And shown, as the peer's, stale.
let (base, _http) = server.serve_http().await?;
let received: serde_json::Value = reqwest::get(format!("{base}/api/routes/received"))
.await?
.json()
.await?;
let routes = received["routes"].as_array().unwrap();
assert_eq!(routes.len(), 3, "{received}");
for route in routes {
assert_eq!(route["stale"], true, "{route}");
assert_eq!(route["peer_asn"], PEER_ASN, "{route}");
}
let mut peer = TestPeer::connect(server.addr).await?;
peer.handshake_with(
PEER_ASN,
PEER_ID,
90,
capabilities([restart(true, true, true, 90)]),
)
.await?;
until("the routes are taken over", || {
server.registry.list().len() == 1 && server.restarts.kept().is_empty()
})
.await?;
peer.announce(own(&PREFIXES[..2])).await?;
until("two are fresh", || table(&server).1 == 1).await?;
// The peer has them all, as far as its limit goes.
assert_eq!(server.registry.list()[0].counters.routes(), 3);
peer.announce(end_of_rib(AfiSafi::IPV4_UNICAST)).await?;
until("the one not sent again is gone", || {
table(&server) == (vec!["10.1.0.0/16".into(), "10.2.0.0/16".into()], 0)
})
.await?;
until("it is counted out", || {
server
.registry
.list()
.first()
.is_some_and(|session| session.counters.routes() == 2)
})
.await?;
Ok(())
}
/// What ends a session decides whether the peer is restarting: a Hard
/// Reset never is, and a NOTIFICATION is only where both sides said so
/// (RFC 8538). A peer that does not offer graceful restart never is.
#[tokio::test]
async fn test_only_a_restart_keeps_the_routes() -> Result<()> {
let cease = |subcode| NotificationMessage {
error: BgpError::Cease(subcode),
data: vec![],
};
let hard_reset = NotificationMessage {
error: BgpError::Cease(CeaseSubcode::HardReset),
data: vec![6, 2],
};
for (offer, goodbye, kept) in [
(
Some(restart(false, true, true, 90)),
Some(cease(CeaseSubcode::AdminReset)),
true,
),
(
Some(restart(false, true, true, 90)),
Some(hard_reset.clone()),
false,
),
(
Some(restart(false, false, true, 90)),
Some(cease(CeaseSubcode::AdminReset)),
false,
),
(Some(restart(false, false, true, 90)), None, true),
(Some(restart(false, true, true, 0)), None, false),
(None, None, false),
] {
let server = TestServer::start(65000).await?;
let mut peer = TestPeer::connect(server.addr).await?;
peer.handshake_with(PEER_ASN, PEER_ID, 90, capabilities(offer))
.await?;
peer.announce(own(&PREFIXES)).await?;
until("the routes are in", || server.rib.len() == 3).await?;
if let Some(goodbye) = &goodbye {
peer.send(&Message::Notification(goodbye.clone())).await?;
}
drop(peer);
until("the session is down", || server.registry.list().is_empty()).await?;
if kept {
until("the routes are stale", || table(&server).1 == 3).await?;
} else {
until("the routes are gone", || server.rib.is_empty()).await?;
assert!(server.restarts.kept().is_empty());
}
}
Ok(())
}
/// A peer that does not come back within its restart time has its routes
/// removed then.
#[tokio::test]
async fn test_the_routes_go_when_the_peer_does_not_come_back_in_time() -> Result<()> {
let server = TestServer::start(65000).await?;
let peer = peer_with_routes(&server, restart(false, true, true, 1)).await?;
drop(peer);
until("the routes are stale", || table(&server).1 == 3).await?;
until("the routes are gone", || server.rib.is_empty()).await?;
assert!(server.restarts.kept().is_empty());
Ok(())
}
/// A peer that comes back without having kept its forwarding (F bit), or
/// without graceful restart at all, has its stale routes removed at once.
#[tokio::test]
async fn test_routes_whose_forwarding_was_lost_go_when_the_peer_is_back() -> Result<()> {
for back_with in [Some(restart(true, true, false, 90)), None] {
let server = TestServer::start(65000).await?;
let peer = peer_with_routes(&server, restart(false, true, true, 90)).await?;
drop(peer);
until("the routes are stale", || table(&server).1 == 3).await?;
let mut peer = TestPeer::connect(server.addr).await?;
peer.handshake_with(PEER_ASN, PEER_ID, 90, capabilities(back_with.clone()))
.await?;
until("the stale routes are gone", || server.rib.is_empty()).await?;
until("they are counted out", || {
server
.registry
.list()
.first()
.is_some_and(|session| session.counters.routes() == 0)
})
.await?;
drop(peer);
}
Ok(())
}
/// Our OPEN offers graceful restart, as just restarted, and a peer that
/// agreed is sent End-of-RIB markers once it has our routes.
#[tokio::test]
async fn test_we_offer_graceful_restart_and_say_when_our_routes_are_all_sent() -> Result<()> {
let server = TestServer::start(65000).await?;
let mut peer = TestPeer::connect(server.addr).await?;
let open = peer
.handshake_with(
PEER_ASN,
PEER_ID,
90,
capabilities([restart(false, true, true, 90)]),
)
.await?;
let offered = open.capabilities().find_map(|capability| match capability {
Capability::GracefulRestart {
restart_state,
restart_time,
notification,
entries,
} => Some((
*restart_state,
*restart_time,
*notification,
entries.clone(),
)),
_ => None,
});
let Some((restarted, time, notification, entries)) = offered else {
bail!("no graceful restart in {open:?}");
};
assert!(restarted, "started just now");
assert_eq!(time, 120);
assert!(notification);
// Without kernel routes, our forwarding is not ours to lose.
assert!(entries.iter().all(|entry| entry.preserve_forwarding));
// Nothing is originated: the markers are all there is.
let mut markers = Vec::new();
while markers.len() < 2 {
let update = peer.recv_update().await?;
match update.end_of_rib_family() {
Some(family) => markers.push(family),
None => bail!("expected End-of-RIB, got {update:?}"),
}
}
markers.sort();
assert_eq!(markers, [AfiSafi::IPV4_UNICAST, AfiSafi::IPV6_UNICAST]);
Ok(())
}
/// Leaving to restart closes the connection without a NOTIFICATION where
/// graceful restart was agreed, so that the peer keeps our routes; leaving
/// for good is a Hard Reset there, and a Cease elsewhere.
#[tokio::test]
async fn test_leaving_tells_each_peer_as_graceful_restart_has_it() -> Result<()> {
let hard_reset = BgpError::Cease(CeaseSubcode::HardReset);
let shutdown = BgpError::Cease(CeaseSubcode::AdminShutdown);
for (leave, offer, told) in [
(Leave::Restart, Some(restart(false, true, true, 90)), None),
(Leave::Restart, None, Some(shutdown.clone())),
(
Leave::Stop,
Some(restart(false, true, true, 90)),
Some(hard_reset),
),
(Leave::Stop, None, Some(shutdown)),
] {
let server = TestServer::start(65000).await?;
let mut peer = TestPeer::connect(server.addr).await?;
peer.handshake_with(PEER_ASN, PEER_ID, 90, capabilities(offer.clone()))
.await?;
until("the session is up", || server.registry.list().len() == 1).await?;
server.registry.leave(leave, Duration::from_secs(2)).await;
let mut notification = None;
// Whatever came before (End-of-RIB markers) is passed over.
while let Ok(message) = peer.recv().await {
if let Message::Notification(n) = message {
notification = Some(n.error);
break;
}
}
assert_eq!(notification, told, "{leave:?} with {offer:?}");
assert!(peer.closed_within(TIMEOUT).await);
}
Ok(())
}
crates/integration-tests/tests/http_api.rs
+41 -34
View File
@@ -3,22 +3,46 @@
use std::net::Ipv4Addr;
use std::time::Duration;
use bgp_integration_tests::announcement::announcement;
use bgp_integration_tests::test_peer::TestPeer;
use bgp_integration_tests::test_server::TestServer;
use bgp_integration_tests::{TIMEOUT, get_json, prefix};
use bgp_packet::message::LargeCommunity;
use bgp_packet::message::{
AsPathAttribute, CommunitiesPathAttribute, LargeCommunity, Message, NextHopPathAttribute,
OriginPathAttribute, PathAttribute, UpdateMessage,
};
use bgp_primitives::ip_prefix::IpPrefix;
use bgp_rib::{LocalRoute, LocalRouteAttrs};
use eyre::Result;
use serde_json::{Value, json};
const LOCAL_ASN: u32 = 65000;
const PEER_ASN: u32 = 65001;
const TIMEOUT: Duration = Duration::from_secs(5);
fn prefix(s: &str) -> IpPrefix {
IpPrefix::try_from(s).unwrap()
}
async fn established_peer(server: &TestServer) -> Result<TestPeer> {
server
.established(PEER_ASN, Ipv4Addr::new(10, 0, 0, 1))
.await
let mut peer = TestPeer::connect(server.addr).await?;
peer.handshake(PEER_ASN, Ipv4Addr::new(10, 0, 0, 1), 90)
.await?;
tokio::time::timeout(TIMEOUT, async {
while server.registry.list().is_empty() {
tokio::time::sleep(Duration::from_millis(10)).await;
}
})
.await?;
Ok(peer)
}
async fn get_json(client: &reqwest::Client, url: &str) -> Result<Value> {
Ok(client
.get(url)
.send()
.await?
.error_for_status()?
.json()
.await?)
}
#[tokio::test]
@@ -52,27 +76,6 @@ async fn test_status_and_peers() -> Result<()> {
peers[0]["address_families"],
serde_json::json!(["IPv4 unicast", "IPv6 unicast"])
);
// The session's TCP connection, by the address the peer list gives.
// What the kernel says of it depends on the platform; that it was
// asked, and why it had nothing to say if it had not, does not.
let peer_addr = peers[0]["peer_addr"].as_str().unwrap();
let tcp = get_json(&client, &format!("{base}/api/tcp?peer={peer_addr}")).await?;
let available = tcp["available"].as_bool().expect("available is a bool");
if available {
assert!(tcp["connection"].is_object(), "{tcp}");
} else {
assert!(tcp["reason"].is_string(), "{tcp}");
}
// The address alone names the peer too, as it does elsewhere.
let by_address = get_json(&client, &format!("{base}/api/tcp?peer=127.0.0.1")).await?;
assert_eq!(by_address["available"], tcp["available"]);
// A peer there is no session with is a bad request, not an answer.
let nobody = client
.get(format!("{base}/api/tcp?peer=nobody"))
.send()
.await?;
assert_eq!(nobody.status(), 400);
Ok(())
}
@@ -152,17 +155,21 @@ async fn test_received_routes_summary() -> Result<()> {
let client = reqwest::Client::new();
let mut peer = established_peer(&server).await?;
peer.announce(
announcement(&["203.0.113.0/24"], &[PEER_ASN, 65002])
.with_next_hop(Ipv4Addr::new(10, 0, 0, 1))
.with_communities(&[(65001, 7)]),
)
peer.send(&Message::Update(UpdateMessage {
withdrawn_routes: vec![],
path_attributes: vec![
PathAttribute::Origin(OriginPathAttribute::IGP),
AsPathAttribute::from_asns(vec![PEER_ASN, 65002]),
PathAttribute::NextHop(NextHopPathAttribute(Ipv4Addr::new(10, 0, 0, 1))),
PathAttribute::Communities(CommunitiesPathAttribute(vec![(65001, 7)])),
],
announced_routes: vec![prefix("203.0.113.0/24").into()],
}))
.await?;
let received = tokio::time::timeout(TIMEOUT, async {
loop {
let v =
get_json(&client, &format!("{base}/api/routes/received")).await?["routes"].take();
let v = get_json(&client, &format!("{base}/api/routes/received")).await?;
if !v.as_array().unwrap().is_empty() {
return Ok::<_, eyre::Error>(v);
}
crates/integration-tests/tests/import_policy.rs
-456
View File
@@ -1,456 +0,0 @@
//! What is taken from a peer: the checks every route goes through, and the
//! limit on how many a peer may have.
use std::collections::BTreeSet;
use std::net::Ipv4Addr;
use std::time::Duration;
use bgp_integration_tests::announcement::announcement;
use bgp_integration_tests::test_peer::TestPeer;
use bgp_integration_tests::test_server::{ServerWithPeer, TestServer};
use bgp_integration_tests::{default_policy, prefix, wait_for, wait_until};
use bgp_packet::errors::{BgpError, CeaseSubcode};
use bgp_packet::message::{
CommunitiesPathAttribute, LargeCommunitiesPathAttribute, LargeCommunity, PathAttribute,
};
use bgp_server::marks::{Marks, Patterns, Scrub};
use bgp_server::policy::{ImportPolicy, ImportRule, PrefixSet};
use bgp_server::session::DialState;
use eyre::Result;
/// AS numbers as they are on the Internet: the checks refuse the private
/// ones the other tests use.
const LOCAL_ASN: u32 = 1299;
const PEER_ASN: u32 = 3333;
const PEER_ID: Ipv4Addr = Ipv4Addr::new(10, 0, 0, 2);
/// A server with one configured peer, at the address the test connects
/// from, whose routes are judged by `import`. The listener that keeps the
/// server's own dialling occupied comes with it.
async fn server_with(import: ImportPolicy) -> Result<ServerWithPeer> {
TestServer::with_peer(LOCAL_ASN, PEER_ASN, |c, peer| {
c.connect_retry = Duration::from_millis(50);
peer.import = import;
})
.await
}
async fn established(server: &TestServer) -> Result<TestPeer> {
server.established(PEER_ASN, PEER_ID).await
}
/// Announces IPv4 `prefixes` with the AS path `path`.
async fn announce(peer: &mut TestPeer, prefixes: &[&str], path: &[u32]) -> Result<()> {
peer.announce(announcement(prefixes, path)).await
}
/// Waits until the server has dealt with `count` announced routes of the
/// session's, taken or refused, and returns the prefixes in its table.
async fn table_after(server: &TestServer, taken: u64, refused: u64) -> Result<BTreeSet<String>> {
let counters = server.registry.list()[0].counters.clone();
wait_until("the routes are dealt with", || {
(counters.routes(), counters.rejected()) == (taken, refused)
})
.await
.map_err(|_| {
eyre::eyre!(
"the server took {} routes and refused {}; expected {taken} and {refused}",
counters.routes(),
counters.rejected()
)
})?;
Ok(server
.received_routes()
.iter()
.map(|route| route.prefix.to_string())
.collect())
}
fn set(prefixes: &[&str]) -> BTreeSet<String> {
prefixes.iter().map(|p| p.to_string()).collect()
}
/// With nothing configured for a peer but its address and AS, its routes
/// still go through the checks that hold for anyone: only space that is
/// routed on the Internet, nothing too specific, a path that begins with
/// the peer and has no reserved AS in it, and never one that has been
/// through us. Everything a peer sent used to be taken.
#[tokio::test]
async fn test_routes_that_no_peer_should_announce_are_refused() -> Result<()> {
let ServerWithPeer {
server,
listener: _listener,
} = server_with(default_policy()).await?;
let mut peer = established(&server).await?;
// One UPDATE with good and bad prefixes: the good ones are taken.
announce(
&mut peer,
&[
"193.0.0.0/21",
"10.0.0.0/8", // private
"193.0.10.0/25", // more specific than is routed
"0.0.0.0/0", // default
"193.0.16.0/24",
],
&[PEER_ASN],
)
.await?;
assert_eq!(
table_after(&server, 2, 3).await?,
set(&["193.0.0.0/21", "193.0.16.0/24"])
);
// Paths that are refused whatever the prefix: not the peer's own,
// with a private AS in it, and one that has been through us.
announce(&mut peer, &["194.0.1.0/24"], &[174, PEER_ASN]).await?;
announce(&mut peer, &["194.0.2.0/24"], &[PEER_ASN, 64512]).await?;
announce(&mut peer, &["194.0.3.0/24"], &[PEER_ASN, LOCAL_ASN, 174]).await?;
assert_eq!(
table_after(&server, 2, 6).await?,
set(&["193.0.0.0/21", "193.0.16.0/24"])
);
Ok(())
}
/// A route that is refused takes the place of the one the peer had
/// announced before, which goes: the peer no longer has the old route,
/// and the new one is not acceptable.
#[tokio::test]
async fn test_a_refused_route_replaces_the_one_that_was_taken() -> Result<()> {
let ServerWithPeer {
server,
listener: _listener,
} = server_with(default_policy()).await?;
let mut peer = established(&server).await?;
announce(&mut peer, &["193.0.0.0/21", "193.0.16.0/24"], &[PEER_ASN]).await?;
assert_eq!(table_after(&server, 2, 0).await?.len(), 2);
announce(&mut peer, &["193.0.0.0/21"], &[PEER_ASN, 64512]).await?;
assert_eq!(table_after(&server, 1, 1).await?, set(&["193.0.16.0/24"]));
Ok(())
}
/// A peer with a list of what it may announce is held to it: its own
/// prefixes, originated by the AS numbers given.
#[tokio::test]
async fn test_a_peer_is_held_to_its_prefix_list_and_origins() -> Result<()> {
let mut prefixes = PrefixSet::default();
prefixes.allow(prefix("193.0.0.0/21"), 24);
let ServerWithPeer {
server,
listener: _listener,
} = server_with(ImportPolicy {
prefixes: Some(prefixes),
origin_asns: Some([PEER_ASN, 2121].into()),
..default_policy()
})
.await?;
let mut peer = established(&server).await?;
// Its own, and a more specific that the list allows.
announce(&mut peer, &["193.0.0.0/21", "193.0.4.0/24"], &[PEER_ASN]).await?;
// A customer of its that is listed as an origin.
announce(&mut peer, &["193.0.6.0/24"], &[PEER_ASN, 2121]).await?;
// Someone else's prefix, as a hijack or a leak would be.
announce(&mut peer, &["8.8.8.0/24"], &[PEER_ASN]).await?;
// Its own prefix, but passed on from an origin that is not listed.
announce(&mut peer, &["193.0.7.0/24"], &[PEER_ASN, 15169]).await?;
assert_eq!(
table_after(&server, 3, 2).await?,
set(&["193.0.0.0/21", "193.0.4.0/24", "193.0.6.0/24"])
);
Ok(())
}
/// A peer that sends more routes than it is allowed is cut off and told
/// why, and kept down for a while: neither its connections are taken nor
/// is it dialled. A session that came straight back would only fill up
/// again.
#[tokio::test]
async fn test_a_peer_over_its_limit_is_cut_off_and_kept_down() -> Result<()> {
let ServerWithPeer { server, listener } = server_with(ImportPolicy {
max_prefixes: Some(3),
hold_down: Duration::from_millis(1500),
..default_policy()
})
.await?;
// The server dials the peer as well; that connection is left unanswered
// until the end.
let dialled = TestPeer::accept(&listener).await?;
let mut peer = established(&server).await?;
// Refused routes do not count towards the limit, and replacing a
// route does not either.
announce(
&mut peer,
&["193.0.0.0/24", "193.0.1.0/24", "10.0.0.0/8"],
&[PEER_ASN],
)
.await?;
announce(&mut peer, &["193.0.1.0/24", "193.0.2.0/24"], &[PEER_ASN]).await?;
assert_eq!(table_after(&server, 3, 1).await?.len(), 3);
announce(&mut peer, &["193.0.3.0/24"], &[PEER_ASN]).await?;
let notification = peer.expect_notification().await?;
assert_eq!(
notification.error,
BgpError::Cease(CeaseSubcode::MaxPrefixes)
);
drop(peer);
wait_until("the peer's routes are gone", || {
server.received_routes().is_empty()
})
.await?;
// Held down: a new connection gets nothing.
let mut early = TestPeer::connect(server.addr).await?;
assert!(
early.closed_unanswered().await,
"a peer that is held down is not answered"
);
// Nor is it dialled: once the connection the server had open to it is
// gone, the dialer waits out the hold-down and says so.
drop(dialled);
let reason = wait_for("the dialer says the peer is held down", || {
match server.registry.configured_peers().into_iter().next() {
Some((_, DialState::Backoff { last_error, .. }))
if last_error.contains("held down") =>
{
Some(last_error)
}
_ => None,
}
})
.await?;
assert!(reason.contains("limit of routes"), "{reason}");
// When the time is up the peer is dialled again, and taken back.
TestPeer::accept(&listener).await?;
let mut later = TestPeer::connect(server.addr).await?;
later.handshake(PEER_ASN, PEER_ID, 90).await?;
Ok(())
}
/// Routes from a peer are marked with where they were learned and given
/// a preference, as the peer's `import` says; and marks that mean
/// something to us are not taken on the peer's word.
#[tokio::test]
async fn test_a_peers_routes_are_marked_and_ranked_as_configured() -> Result<()> {
use bgp_packet::message::{CommunitiesPathAttribute, LocalPrefPathAttribute};
use bgp_server::marks::Marks;
let ServerWithPeer {
server,
listener: _listener,
} = server_with(ImportPolicy {
marks: Marks::parse(
&["1299:*".to_string()],
&["1299:1001".to_string()],
&["1299:9:7".to_string()],
)
.unwrap(),
local_pref: Some(250),
..default_policy()
})
.await?;
let mut peer = established(&server).await?;
peer.announce(
announcement(&["193.0.0.0/21", "193.0.8.0/22"], &[PEER_ASN])
.with_communities(&[(3333, 5), (1299, 2002)]),
)
.await?;
table_after(&server, 2, 0).await?;
let routes = server.received_routes();
for route in &routes {
assert!(
route
.attributes
.contains(&PathAttribute::Communities(CommunitiesPathAttribute(vec![
(3333, 5),
(1299, 1001)
]))),
"{route:?}"
);
assert!(
route
.attributes
.contains(&PathAttribute::LocalPref(LocalPrefPathAttribute(250))),
"{route:?}"
);
assert!(
route.attributes.iter().any(|attribute| matches!(
attribute,
PathAttribute::LargeCommunities(large) if large.communities.len() == 1
)),
"{route:?}"
);
}
assert_eq!(server.rib.attrs().len(), 1, "they share their attributes");
Ok(())
}
/// The communities of our own AS are taken off what another network
/// sends, before the route is kept, and nothing else is; from a peer that
/// is to steer its routes with them (a customer), they are kept.
#[tokio::test]
async fn test_our_communities_are_not_taken_from_other_networks() -> Result<()> {
let ours_and_theirs = || {
vec![
PathAttribute::Communities(CommunitiesPathAttribute(vec![
(0, 13335),
(3356, 3),
(1299, 5),
])),
PathAttribute::LargeCommunities(LargeCommunitiesPathAttribute {
communities: vec![large(1299, 0, 0), large(1299, 31, 2), large(3356, 1, 2)],
}),
]
};
let kept = |server: &TestServer| -> Vec<PathAttribute> {
let routes = server.received_routes();
routes[0]
.attributes
.iter()
.filter(|attribute| {
matches!(
attribute,
PathAttribute::Communities(_) | PathAttribute::LargeCommunities(_)
)
})
.cloned()
.collect()
};
let scrubbed = ImportPolicy {
scrub: Some(Scrub {
local_asn: LOCAL_ASN,
zero: true,
}),
..default_policy()
};
let ServerWithPeer {
server,
listener: _listener,
} = server_with(scrubbed).await?;
let mut peer = established(&server).await?;
announce_with(&mut peer, &["193.0.0.0/21"], ours_and_theirs()).await?;
table_after(&server, 1, 0).await?;
assert_eq!(
kept(&server),
[
PathAttribute::Communities(CommunitiesPathAttribute(vec![(3356, 3)])),
PathAttribute::LargeCommunities(LargeCommunitiesPathAttribute {
communities: vec![large(3356, 1, 2)],
}),
]
);
let ServerWithPeer {
server,
listener: _listener,
} = server_with(default_policy()).await?;
let mut peer = established(&server).await?;
announce_with(&mut peer, &["193.0.0.0/21"], ours_and_theirs()).await?;
table_after(&server, 1, 0).await?;
assert_eq!(kept(&server), ours_and_theirs());
Ok(())
}
fn large(global_admin: u32, data_1: u32, data_2: u32) -> LargeCommunity {
LargeCommunity {
global_admin,
data_1,
data_2,
}
}
/// Announces IPv4 `prefixes` from the peer itself, with `more` attributes.
async fn announce_with(
peer: &mut TestPeer,
prefixes: &[&str],
more: Vec<PathAttribute>,
) -> Result<()> {
peer.announce(announcement(prefixes, &[PEER_ASN]).with_attributes(more))
.await
}
/// Rules do to some routes what the rest of the policy does to all: by
/// their communities, as the session has marked them, or by their prefix,
/// even when they came in one UPDATE with others. The first rule that
/// holds for a route is the one applied.
#[tokio::test]
async fn test_rules_apply_to_the_routes_they_hold_for() -> Result<()> {
let mut within = PrefixSet::default();
within.allow(prefix("193.0.0.0/21"), 24);
let rule = |communities: &[&str], prefixes: Option<PrefixSet>, local_pref, add: &[&str]| {
let strings = |texts: &[&str]| texts.iter().map(|t| t.to_string()).collect::<Vec<_>>();
ImportRule {
communities: Patterns::parse(&strings(communities)).unwrap(),
prefixes,
local_pref,
marks: Marks::parse(&[], &[], &strings(add)).unwrap(),
}
};
let policy = ImportPolicy {
// The session marks where its routes were learned.
marks: Marks::parse(&[], &[], &["1299:101:1".to_string()]).unwrap(),
rules: vec![
rule(&["3356:3"], None, Some(50), &[]),
rule(&[], Some(within), None, &["1299:31:10"]),
rule(&["1299:101:*"], None, Some(120), &[]),
],
..default_policy()
};
let ServerWithPeer {
server,
listener: _listener,
} = server_with(policy).await?;
let mut peer = established(&server).await?;
announce_with(
&mut peer,
&["194.0.0.0/16"],
vec![PathAttribute::Communities(CommunitiesPathAttribute(vec![
(3356, 3),
]))],
)
.await?;
announce_with(&mut peer, &["193.0.0.0/22", "195.0.0.0/16"], vec![]).await?;
table_after(&server, 3, 0).await?;
let kept = |wanted: &str| {
let routes = server.received_routes();
let route = routes
.iter()
.find(|route| route.prefix.to_string() == wanted)
.unwrap_or_else(|| panic!("{wanted} is in the table"));
let local_pref = route
.attributes
.iter()
.find_map(|attribute| match attribute {
PathAttribute::LocalPref(local_pref) => Some(local_pref.0),
_ => None,
});
let large = route
.attributes
.iter()
.find_map(|attribute| match attribute {
PathAttribute::LargeCommunities(large) => Some(large.communities.clone()),
_ => None,
})
.unwrap_or_default();
(local_pref, large)
};
let learned = large(1299, 101, 1);
// By its community: the first rule, and only that one.
assert_eq!(kept("194.0.0.0/16"), (Some(50), vec![learned.clone()]));
// By its prefix, apart from the route it came with.
assert_eq!(
kept("193.0.0.0/22"),
(None, vec![learned.clone(), large(1299, 31, 10)])
);
// By the mark the session put on.
assert_eq!(kept("195.0.0.0/16"), (Some(120), vec![learned]));
Ok(())
}
crates/integration-tests/tests/interop.rs
+10 -4
View File
@@ -12,6 +12,8 @@ use bgp_primitives::ip_prefix::IpPrefix;
use bgp_rib::{LocalRoute, LocalRouteAttrs};
use eyre::Result;
const TIMEOUT: Duration = Duration::from_secs(5);
/// The OPEN a BIRD 2 router sends for an IPv6-only session from AS210036:
/// one optional parameter carrying MP(IPv6 unicast), route refresh,
/// extended message, graceful restart (aware only), 4-octet AS, enhanced
@@ -75,13 +77,19 @@ async fn test_bird_open_and_end_of_rib_are_accepted() -> Result<()> {
peer.send_raw(BIRD_OPEN).await?;
peer.recv_open().await?;
match peer.recv().await? {
match tokio::time::timeout(TIMEOUT, peer.recv()).await?? {
Message::KeepAlive => {}
other => eyre::bail!("server did not accept BIRD's OPEN: {other:?}"),
}
peer.send(&Message::KeepAlive).await?;
let session = server.wait_for_session().await?;
tokio::time::timeout(TIMEOUT, async {
while server.registry.list().is_empty() {
tokio::time::sleep(Duration::from_millis(10)).await;
}
})
.await?;
let session = server.registry.list().remove(0);
assert_eq!(
session.peer.asn, 210036,
"4-octet ASN must come from the capability, not AS_TRANS"
@@ -118,8 +126,6 @@ async fn test_ipv6_prefix_update_shape() -> Result<()> {
next_hop: nh.into(),
four_octet_asn: true,
ibgp: false,
role: None,
extended_next_hop: false,
},
)
.expect("ipv6 over ipv6");
crates/integration-tests/tests/kernel.rs
-172
View File
@@ -1,172 +0,0 @@
//! The whole daemon against a real kernel: a peer announces routes over a
//! session, and `ip route` shows them.
//!
//! This changes the machine's interfaces and routes, so it is not run by
//! `cargo test`; `scripts/kernel-tests.sh` runs it in a network namespace
//! of its own.
#![cfg(target_os = "linux")]
use std::collections::BTreeSet;
use std::net::Ipv4Addr;
use std::process::Command;
use std::time::{Duration, Instant};
use bgp_fib::FibConfig;
use bgp_integration_tests::announcement::{announcement, withdrawal};
use bgp_integration_tests::prefix;
use bgp_integration_tests::test_peer::TestPeer;
use bgp_integration_tests::test_server::TestServer;
use eyre::Result;
use serde_json::Value;
/// The kernel is given more time than the loopback tests allow.
const TIMEOUT: Duration = Duration::from_secs(10);
const TABLE: u32 = 1100;
const INTERFACE: &str = "ferrix100";
#[track_caller]
fn ip(arguments: &str) {
let output = Command::new("ip")
.args(arguments.split_whitespace())
.output()
.expect("the ip command is there");
assert!(
output.status.success(),
"ip {arguments}: {}",
String::from_utf8_lossy(&output.stderr)
);
}
fn kernel_routes() -> BTreeSet<String> {
let output = Command::new("ip")
.args(["route", "show", "table", &TABLE.to_string()])
.output()
.unwrap();
String::from_utf8_lossy(&output.stdout)
.lines()
.map(|line| line.split_whitespace().collect::<Vec<_>>().join(" "))
// The number of the nexthop object a route names is the kernel's
// to choose.
.map(|line| match line.split_once(" nhid ") {
Some((prefix, rest)) => {
let rest = rest.split_once(' ').map_or("", |(_, rest)| rest);
format!("{prefix} {rest}")
}
None => line,
})
.filter(|line| !line.is_empty())
.collect()
}
#[track_caller]
fn kernel_becomes(expected: &[&str]) {
let expected: BTreeSet<String> = expected.iter().map(|route| route.to_string()).collect();
let start = Instant::now();
while kernel_routes() != expected {
assert!(
start.elapsed() < TIMEOUT,
"the kernel has {:#?}; expected {expected:#?}",
kernel_routes()
);
std::thread::sleep(Duration::from_millis(20));
}
}
/// Removes the test's interface, and its routes with it, when dropped.
struct Interface;
impl Drop for Interface {
fn drop(&mut self) {
let _ = Command::new("ip").args(["link", "del", INTERFACE]).output();
let _ = Command::new("ip")
.args(["route", "flush", "table", &TABLE.to_string()])
.output();
}
}
async fn announce(peer: &mut TestPeer, prefixes: &[&str], next_hop: Ipv4Addr) -> Result<()> {
peer.announce(announcement(prefixes, &[65001]).with_next_hop(next_hop))
.await
}
/// A peer's routes go from the session to the kernel, and out of it again
/// when they are withdrawn and when the session ends.
#[tokio::test(flavor = "multi_thread")]
#[ignore = "changes the machine's routes; run scripts/kernel-tests.sh"]
async fn test_routes_from_a_session_reach_the_kernel() -> Result<()> {
ip(&format!("link add {INTERFACE} type dummy"));
let _interface = Interface;
ip(&format!("addr add 10.100.0.1/24 dev {INTERFACE}"));
ip(&format!("link set {INTERFACE} up"));
let server = TestServer::start_with(65000, |config| {
config.kernel = Some(FibConfig {
table: TABLE,
..FibConfig::default()
});
})
.await?;
let (base, _http) = server.serve_http().await?;
let mut peer = TestPeer::connect(server.addr).await?;
peer.handshake(65001, Ipv4Addr::new(10, 0, 0, 2), 90)
.await?;
// The peer speaks from the loopback and names a router on the test's
// interface as next hop, as a route server would.
let neighbor = Ipv4Addr::new(10, 100, 0, 2);
announce(&mut peer, &["203.0.113.0/24", "198.18.0.0/15"], neighbor).await?;
// One by way of a router that is not a neighbor.
announce(&mut peer, &["192.88.99.0/24"], Ipv4Addr::new(172, 31, 0, 1)).await?;
kernel_becomes(&[
"203.0.113.0/24 via 10.100.0.2 dev ferrix100 proto bgp metric 20",
"198.18.0.0/15 via 10.100.0.2 dev ferrix100 proto bgp metric 20",
]);
let kernel: Value = reqwest::get(format!("{base}/api/kernel"))
.await?
.json()
.await?;
assert_eq!(kernel["enabled"], true);
assert_eq!(kernel["table"], TABLE);
assert_eq!(kernel["installed"], 2);
// The route by way of a router that is no neighbor is not in the
// table for the kernel to be asked for: it is held, in case the
// router becomes one.
assert_eq!(kernel["unresolved"], serde_json::json!({}));
let session = server.registry.list().pop().expect("the session");
assert_eq!(session.counters.held(), 1);
assert_eq!(server.rib.len(), 2);
let refused: Value = reqwest::get(format!("{base}/api/routes/rejected"))
.await?
.json()
.await?;
assert_eq!(
refused[0]["by_reason"],
serde_json::json!({"next hop unreachable": 1})
);
let shown = kernel["interfaces"]
.as_array()
.unwrap()
.iter()
.find(|interface| interface["name"] == INTERFACE)
.expect("the test's interface is listed");
assert_eq!(shown["up"], true);
// Its own address, beside the link-local one the kernel gave it.
assert!(
shown["addresses"]
.as_array()
.unwrap()
.contains(&serde_json::json!("10.100.0.1/24")),
"{shown}"
);
// One is withdrawn.
peer.announce(withdrawal([prefix("198.18.0.0/15").into()]))
.await?;
kernel_becomes(&["203.0.113.0/24 via 10.100.0.2 dev ferrix100 proto bgp metric 20"]);
// The session ends: packets must not go on following its routes.
drop(peer);
kernel_becomes(&[]);
Ok(())
}
crates/integration-tests/tests/negotiation.rs
+77 -35
View File
@@ -4,25 +4,29 @@
use std::net::{Ipv4Addr, Ipv6Addr, SocketAddr};
use std::time::Duration;
use bgp_integration_tests::announcement::announcement;
use bgp_integration_tests::test_peer::TestPeer;
use bgp_integration_tests::test_server::{ReceivedRoute, TestServer};
use bgp_integration_tests::{peer_config, prefix, wait_for};
use bgp_integration_tests::test_server::TestServer;
use bgp_packet::errors::{BgpError, OpenMsgSubcode};
use bgp_packet::message::{
AsPathAttribute, Message, MpReachNlriPathAttribute, NlriNextHop, PathAttribute, UpdateMessage,
AsPathAttribute, Message, MpReachNlriPathAttribute, NextHopPathAttribute, NlriNextHop,
OriginPathAttribute, PathAttribute, UpdateMessage,
};
use bgp_packet::open::Capability;
use bgp_primitives::constants::{Afi, AfiSafi, Safi};
use bgp_primitives::ip_prefix::IpPrefix;
use bgp_rib::{LocalRoute, LocalRouteAttrs};
use bgp_server::config::PeerConfig;
use bgp_server::family::{FamilySet, SUPPORTED};
use bgp_server::policy::ImportPolicy;
use eyre::{Result, bail};
const LOCAL_ASN: u32 = 65000;
const PEER_ASN: u32 = 65001;
const PEER_ID: Ipv4Addr = Ipv4Addr::new(10, 0, 0, 1);
const TIMEOUT: Duration = Duration::from_secs(5);
fn prefix(s: &str) -> IpPrefix {
IpPrefix::try_from(s).unwrap()
}
fn only(family: AfiSafi) -> FamilySet {
[family].into_iter().collect()
@@ -32,26 +36,35 @@ fn four_octet() -> Capability {
Capability::FourByteAsn { asn: PEER_ASN }
}
/// Waits until the server has a route, and returns what it has.
async fn some_routes(server: &TestServer) -> Result<Vec<ReceivedRoute>> {
wait_for("a route is in the table", || {
let routes = server.received_routes();
(!routes.is_empty()).then_some(routes)
async fn wait_for_session(server: &TestServer) -> Result<bgp_server::session::SessionHandle> {
Ok(tokio::time::timeout(TIMEOUT, async {
loop {
if let Some(session) = server.registry.list().into_iter().next() {
return session;
}
tokio::time::sleep(Duration::from_millis(10)).await;
}
})
.await
.await?)
}
/// An UPDATE announcing one IPv4 route in the body and one IPv6 route in
/// MP_REACH_NLRI.
fn dual_stack_update() -> UpdateMessage {
let ipv6 = PathAttribute::MpReachNlri(MpReachNlriPathAttribute {
family: AfiSafi::IPV6_UNICAST,
next_hop: NlriNextHop::Ipv6(Ipv6Addr::new(0x2001, 0xdb8, 0, 0, 0, 0, 0, 1)),
prefixes: vec![prefix("2001:db8:1::/48").into()],
});
announcement(&["192.0.2.0/24"], &[PEER_ASN])
.with_attributes([ipv6])
.into()
UpdateMessage {
withdrawn_routes: vec![],
path_attributes: vec![
PathAttribute::Origin(OriginPathAttribute::IGP),
AsPathAttribute::from_asns(vec![PEER_ASN]),
PathAttribute::NextHop(NextHopPathAttribute(Ipv4Addr::new(192, 0, 2, 1))),
PathAttribute::MpReachNlri(MpReachNlriPathAttribute {
family: AfiSafi::IPV6_UNICAST,
next_hop: NlriNextHop::Ipv6(Ipv6Addr::new(0x2001, 0xdb8, 0, 0, 0, 0, 0, 1)),
prefixes: vec![prefix("2001:db8:1::/48").into()],
}),
],
announced_routes: vec![prefix("192.0.2.0/24").into()],
}
}
/// The session carries the families both OPENs offered, and the server's
@@ -79,7 +92,7 @@ async fn test_families_are_the_ones_both_sides_offered() -> Result<()> {
"the server's OPEN offers {family}"
);
}
let session = server.wait_for_session().await?;
let session = wait_for_session(&server).await?;
assert_eq!(session.families, only(AfiSafi::IPV4_UNICAST));
Ok(())
}
@@ -100,11 +113,20 @@ async fn test_routes_of_a_family_not_negotiated_are_not_accepted() -> Result<()>
],
)
.await?;
server.wait_for_session().await?;
wait_for_session(&server).await?;
peer.announce(dual_stack_update()).await?;
peer.send(&Message::Update(dual_stack_update())).await?;
let routes = some_routes(&server).await?;
let routes = tokio::time::timeout(TIMEOUT, async {
loop {
let routes = server.received_routes();
if !routes.is_empty() {
return routes;
}
tokio::time::sleep(Duration::from_millis(10)).await;
}
})
.await?;
// One UPDATE is applied as a whole, so the IPv6 route is not on its way.
assert_eq!(routes.len(), 1);
assert_eq!(routes[0].prefix, prefix("192.0.2.0/24"));
@@ -136,7 +158,7 @@ async fn test_routes_of_a_family_not_negotiated_are_not_sent() -> Result<()> {
],
)
.await?;
let session = server.wait_for_session().await?;
let session = wait_for_session(&server).await?;
assert_eq!(session.families, only(AfiSafi::IPV6_UNICAST));
// A second peer that takes IPv4 is sent the route. The advertiser
@@ -145,7 +167,7 @@ async fn test_routes_of_a_family_not_negotiated_are_not_sent() -> Result<()> {
let mut dual = TestPeer::connect(server.addr).await?;
dual.handshake(PEER_ASN, Ipv4Addr::new(10, 0, 0, 2), 90)
.await?;
let update = dual.recv_update().await?;
let update = tokio::time::timeout(TIMEOUT, dual.recv_update()).await??;
assert_eq!(update.announced_routes, vec![prefix("10.100.0.0/16")]);
let nothing = tokio::time::timeout(Duration::from_millis(200), v6_only.recv_update()).await;
@@ -159,11 +181,14 @@ async fn test_routes_of_a_family_not_negotiated_are_not_sent() -> Result<()> {
async fn test_no_family_in_common_refuses_the_session() -> Result<()> {
let server = TestServer::start_with(LOCAL_ASN, |c| {
// Inbound connections from this address inherit the peer's settings.
let at = SocketAddr::new(Ipv4Addr::LOCALHOST.into(), 1);
c.peers = vec![PeerConfig {
name: "v6-only".into(),
endpoint: SocketAddr::new(Ipv4Addr::LOCALHOST.into(), 1),
remote_asn: PEER_ASN,
hold_time_secs: 90,
local_address: None,
families: only(AfiSafi::IPV6_UNICAST),
import: ImportPolicy::permissive(),
..peer_config("v6-only", at, PEER_ASN)
add_path: None,
}];
// Keep the dialer out of the way.
c.connect_retry = Duration::from_secs(3600);
@@ -188,7 +213,7 @@ async fn test_no_family_in_common_refuses_the_session() -> Result<()> {
.collect();
assert_eq!(offered, [&Capability::multiprotocol(AfiSafi::IPV6_UNICAST)]);
match peer.recv().await? {
match tokio::time::timeout(TIMEOUT, peer.recv()).await?? {
Message::Notification(n) => {
assert_eq!(
n.error,
@@ -221,7 +246,7 @@ async fn test_a_peer_offering_an_unsupported_family_is_still_accepted() -> Resul
)
.await?;
let session = server.wait_for_session().await?;
let session = wait_for_session(&server).await?;
assert_eq!(session.families, only(AfiSafi::IPV4_UNICAST));
assert!(
session.four_octet_asn,
@@ -246,13 +271,13 @@ async fn test_a_peer_without_capabilities_gets_two_octet_as_paths() -> Result<()
let mut peer = TestPeer::connect(server.addr).await?;
peer.handshake_with(PEER_ASN, PEER_ID, 90, vec![]).await?;
let session = server.wait_for_session().await?;
let session = wait_for_session(&server).await?;
assert!(!session.four_octet_asn);
assert_eq!(session.families, only(AfiSafi::IPV4_UNICAST));
// The test peer now decodes AS_PATH two octets wide, as such a speaker
// would; an UPDATE encoded four octets wide would not parse.
let update = peer.recv_update().await?;
let update = tokio::time::timeout(TIMEOUT, peer.recv_update()).await??;
assert_eq!(update.announced_routes, vec![prefix("10.100.0.0/16")]);
assert!(
update
@@ -267,9 +292,26 @@ async fn test_a_peer_without_capabilities_gets_two_octet_as_paths() -> Result<()
assert_eq!(as4_path, Some(vec![WIDE_ASN]));
// And its own two-octet paths are read correctly.
peer.announce(announcement(&["192.0.2.0/24"], &[PEER_ASN, 64512]))
.await?;
let routes = some_routes(&server).await?;
peer.send(&Message::Update(UpdateMessage {
withdrawn_routes: vec![],
path_attributes: vec![
PathAttribute::Origin(OriginPathAttribute::IGP),
AsPathAttribute::from_asns(vec![PEER_ASN, 64512]),
PathAttribute::NextHop(NextHopPathAttribute(Ipv4Addr::new(192, 0, 2, 1))),
],
announced_routes: vec![prefix("192.0.2.0/24").into()],
}))
.await?;
let routes = tokio::time::timeout(TIMEOUT, async {
loop {
let routes = server.received_routes();
if !routes.is_empty() {
return routes;
}
tokio::time::sleep(Duration::from_millis(10)).await;
}
})
.await?;
assert!(
routes[0]
.attributes
crates/integration-tests/tests/roles.rs
-338
View File
@@ -1,338 +0,0 @@
//! Roles and route leak prevention (RFC 9234), end to end: what the two
//! ends say they are to each other, and what becomes of routes that are
//! marked as only to go to customers.
use std::net::{Ipv4Addr, Ipv6Addr};
use bgp_integration_tests::announcement::announcement;
use bgp_integration_tests::test_peer::TestPeer;
use bgp_integration_tests::test_server::{ServerWithPeer, TestServer};
use bgp_integration_tests::{prefix, wait_until};
use bgp_packet::errors::{BgpError, OpenMsgSubcode};
use bgp_packet::message::{
AsPathAttribute, Message, MpReachNlriPathAttribute, NlriNextHop, OriginPathAttribute,
PathAttribute, UpdateMessage,
};
use bgp_packet::open::{BgpRole, Capability};
use bgp_primitives::constants::AfiSafi;
use bgp_rib::LocalRoute;
use bgp_server::policy::PeerRole;
use eyre::{Result, bail};
use serde_json::Value;
const LOCAL_ASN: u32 = 1299;
const PEER_ASN: u32 = 3333;
const PEER_ID: Ipv4Addr = Ipv4Addr::new(10, 0, 0, 2);
/// A server with one configured peer at the address the test connects
/// from, which is `role` to it, and which originates 193.0.0.0/21.
async fn server_with(role: Option<PeerRole>, require_role: bool) -> Result<ServerWithPeer> {
TestServer::with_peer(LOCAL_ASN, PEER_ASN, |c, peer| {
c.originate = vec![LocalRoute::new(prefix("193.0.0.0/21"), Default::default())];
peer.role = role;
peer.require_role = require_role;
})
.await
}
/// The capabilities of an ordinary peer, and with `role` what it says it
/// is.
fn capabilities(role: Option<BgpRole>) -> Vec<Capability> {
let mut capabilities = vec![
Capability::FourByteAsn { asn: PEER_ASN },
Capability::multiprotocol(AfiSafi::IPV4_UNICAST),
Capability::multiprotocol(AfiSafi::IPV6_UNICAST),
Capability::RouteRefresh,
];
capabilities.extend(role.map(|role| Capability::BgpRole { role }));
capabilities
}
/// A session with the server, on which the peer says it is `says`.
async fn established(server: &TestServer, says: Option<BgpRole>) -> Result<TestPeer> {
server
.established_with(PEER_ASN, PEER_ID, capabilities(says))
.await
}
/// Announces `prefixes` with the AS path `path` and, with `marked_by`, an
/// Only to Customer attribute with that AS number.
async fn announce(
peer: &mut TestPeer,
prefixes: &[&str],
path: &[u32],
marked_by: Option<u32>,
) -> Result<()> {
let mut announcement = announcement(prefixes, path);
if let Some(asn) = marked_by {
announcement = announcement.with_only_to_customer(asn);
}
peer.announce(announcement).await
}
/// Waits until the table holds `count` routes, and returns the mark each
/// has, by prefix.
async fn marks(server: &TestServer, count: usize) -> Result<Vec<(String, Option<u32>)>> {
server.wait_for_routes(count).await.map_err(|_| {
eyre::eyre!(
"expected {count} routes, the table has {:?}",
server.received_routes()
)
})?;
let mut marks: Vec<(String, Option<u32>)> = server
.received_routes()
.iter()
.map(|route| {
let mark = route
.attributes
.iter()
.find_map(|attribute| match attribute {
PathAttribute::OnlyToCustomer(asn) => Some(*asn),
_ => None,
});
(route.prefix.to_string(), mark)
})
.collect();
marks.sort();
Ok(marks)
}
/// The role the server's OPEN announces.
fn announced_role(open: &bgp_packet::open::OpenMessage) -> Option<BgpRole> {
open.capabilities().find_map(|capability| match capability {
Capability::BgpRole { role } => Some(*role),
_ => None,
})
}
/// Both ends say what they are, and it fits: the session comes up, and is
/// shown as one whose peer confirmed its role.
#[tokio::test]
async fn test_roles_that_fit_together_are_confirmed() -> Result<()> {
let ServerWithPeer { server, .. } = server_with(Some(PeerRole::Customer), false).await?;
let (base, _http) = server.serve_http().await?;
let mut peer = TestPeer::connect(server.addr).await?;
let open = peer
.handshake_with(PEER_ASN, PEER_ID, 90, capabilities(Some(BgpRole::Customer)))
.await?;
// The peer is our customer: we are its provider.
assert_eq!(announced_role(&open), Some(BgpRole::Provider));
server.wait_for_session().await?;
let peers: Value = reqwest::get(format!("{base}/api/peers"))
.await?
.json()
.await?;
assert_eq!(peers[0]["role"], "customer");
assert_eq!(peers[0]["role_confirmed"], true);
Ok(())
}
/// The commonest way to a leak is the two ends not agreeing on what they
/// are to each other. A peer that says it is something else than it is
/// configured as is refused, with the error RFC 9234 gives for it.
#[tokio::test]
async fn test_a_peer_that_says_it_is_something_else_is_refused() -> Result<()> {
let ServerWithPeer { server, .. } = server_with(Some(PeerRole::Customer), false).await?;
let mut peer = TestPeer::connect(server.addr).await?;
// It thinks it is our peer; we have it down as a customer.
peer.send_open_with(PEER_ASN, PEER_ID, 90, capabilities(Some(BgpRole::Peer)))
.await?;
peer.recv_open().await?;
match peer.recv().await? {
Message::Notification(n) => {
assert_eq!(n.error, BgpError::OpenMsg(OpenMsgSubcode::RoleMismatch));
// Subcode 11 on the wire.
assert_eq!(n.error.code_subcode(), (2, 11));
}
other => bail!("expected NOTIFICATION, got {other:?}"),
}
assert!(server.registry.list().is_empty());
Ok(())
}
/// A peer that says nothing does not implement RFC 9234. It is taken at
/// our word for what it is, unless the configuration requires it to say.
#[tokio::test]
async fn test_a_peer_that_does_not_say_is_refused_only_where_required() -> Result<()> {
let ServerWithPeer { server, .. } = server_with(Some(PeerRole::Customer), false).await?;
let (base, _http) = server.serve_http().await?;
let _peer = established(&server, None).await?;
let peers: Value = reqwest::get(format!("{base}/api/peers"))
.await?
.json()
.await?;
assert_eq!(peers[0]["role"], "customer");
assert_eq!(peers[0]["role_confirmed"], false);
let ServerWithPeer { server, .. } = server_with(Some(PeerRole::Customer), true).await?;
let mut peer = TestPeer::connect(server.addr).await?;
peer.send_open_with(PEER_ASN, PEER_ID, 90, capabilities(None))
.await?;
peer.recv_open().await?;
match peer.recv().await? {
Message::Notification(n) => {
assert_eq!(n.error, BgpError::OpenMsg(OpenMsgSubcode::RoleMismatch));
}
other => bail!("expected NOTIFICATION, got {other:?}"),
}
Ok(())
}
/// A route marked as only to go to customers has been handed down or
/// across once. Coming up from a customer it is a leak, and is refused
/// whatever else the peer's policy takes.
#[tokio::test]
async fn test_a_marked_route_from_a_customer_is_a_leak() -> Result<()> {
let ServerWithPeer { server, .. } = server_with(Some(PeerRole::Customer), false).await?;
let mut peer = established(&server, Some(BgpRole::Customer)).await?;
// The customer's own route, and one its other provider gave it.
announce(&mut peer, &["194.0.0.0/24"], &[PEER_ASN], None).await?;
announce(
&mut peer,
&["8.8.8.0/24"],
&[PEER_ASN, 3356, 15169],
Some(3356),
)
.await?;
// A customer's routes may go anywhere: they are not marked on the way
// in.
assert_eq!(
marks(&server, 1).await?,
[("194.0.0.0/24".to_string(), None)]
);
wait_until("a route is refused", || {
!server.rejections.peers().is_empty()
})
.await?;
let refused = &server.rejections.peers()[0].1;
assert_eq!(refused.total(), 1);
assert_eq!(refused.recent()[0].prefix, prefix("8.8.8.0/24"));
assert_eq!(
refused.recent()[0].reason.as_str(),
"route leak (Only to Customer)"
);
Ok(())
}
/// From a peer a marked route is in order only if the peer marked it
/// itself, on its way to us. A mark of someone else's means the peer got
/// the route from above or from another peer.
#[tokio::test]
async fn test_a_route_marked_by_someone_else_is_a_leak_from_a_peer() -> Result<()> {
let ServerWithPeer { server, .. } = server_with(Some(PeerRole::Peer), false).await?;
let mut peer = established(&server, Some(BgpRole::Peer)).await?;
announce(&mut peer, &["194.0.0.0/24"], &[PEER_ASN], Some(PEER_ASN)).await?;
announce(
&mut peer,
&["8.8.8.0/24"],
&[PEER_ASN, 3356, 15169],
Some(3356),
)
.await?;
// A peer that does not mark its routes has them marked for it.
announce(&mut peer, &["195.0.0.0/24"], &[PEER_ASN, 2121], None).await?;
assert_eq!(
marks(&server, 2).await?,
[
("194.0.0.0/24".to_string(), Some(PEER_ASN)),
("195.0.0.0/24".to_string(), Some(PEER_ASN)),
]
);
assert_eq!(server.rejections.peers()[0].1.total(), 1);
Ok(())
}
/// A route from a provider is marked on the way in, if it is not already:
/// from here it may only go down. That holds for both address families,
/// and whether or not the peer's attributes could be stored as they came.
#[tokio::test]
async fn test_routes_from_a_provider_are_marked_on_the_way_in() -> Result<()> {
for four_octet in [true, false] {
let ServerWithPeer { server, .. } = server_with(Some(PeerRole::Provider), false).await?;
let mut peer = TestPeer::connect(server.addr).await?;
let mut offered = capabilities(Some(BgpRole::Provider));
if !four_octet {
offered.retain(|capability| !matches!(capability, Capability::FourByteAsn { .. }));
}
peer.handshake_with(PEER_ASN, PEER_ID, 90, offered).await?;
announce(&mut peer, &["194.0.0.0/24"], &[PEER_ASN, 2121], None).await?;
// One that someone further up has marked already keeps that mark.
announce(
&mut peer,
&["8.8.8.0/24"],
&[PEER_ASN, 3356, 15169],
Some(3356),
)
.await?;
peer.send(&Message::Update(UpdateMessage {
withdrawn_routes: vec![],
path_attributes: vec![
PathAttribute::Origin(OriginPathAttribute::IGP),
AsPathAttribute::from_asns(vec![PEER_ASN, 2121]),
PathAttribute::MpReachNlri(MpReachNlriPathAttribute {
family: AfiSafi::IPV6_UNICAST,
next_hop: NlriNextHop::Ipv6(Ipv6Addr::new(0x2001, 0x7f8, 0, 0, 0, 0, 0, 1)),
prefixes: vec![prefix("2a00:1450::/32").into()],
}),
],
announced_routes: vec![],
}))
.await?;
assert_eq!(
marks(&server, 3).await?,
[
("194.0.0.0/24".to_string(), Some(PEER_ASN)),
("2a00:1450::/32".to_string(), Some(PEER_ASN)),
("8.8.8.0/24".to_string(), Some(3356)),
],
"four-octet AS numbers: {four_octet}"
);
assert!(server.rejections.peers().is_empty());
}
Ok(())
}
/// Our own routes are marked on their way down to a customer or across to
/// a peer, and not on their way up to a provider.
#[tokio::test]
async fn test_our_routes_are_marked_on_the_way_down_or_across() -> Result<()> {
for (peer_is, says, marked) in [
(PeerRole::Customer, BgpRole::Customer, true),
(PeerRole::Peer, BgpRole::Peer, true),
(
PeerRole::RouteServerClient,
BgpRole::RouteServerClient,
true,
),
(PeerRole::Provider, BgpRole::Provider, false),
(PeerRole::RouteServer, BgpRole::RouteServer, false),
] {
let ServerWithPeer { server, .. } = server_with(Some(peer_is), false).await?;
let mut peer = established(&server, Some(says)).await?;
let update = peer.recv_update().await?;
assert_eq!(update.announced_routes, [prefix("193.0.0.0/21")]);
let mark = update
.path_attributes
.iter()
.find_map(|attribute| match attribute {
PathAttribute::OnlyToCustomer(asn) => Some(*asn),
_ => None,
});
assert_eq!(mark, marked.then_some(LOCAL_ASN), "to our {peer_is:?}");
}
// Nothing is said to a peer whose role is not known.
let ServerWithPeer { server, .. } = server_with(None, false).await?;
let mut peer = established(&server, None).await?;
let update = peer.recv_update().await?;
assert!(
!update
.path_attributes
.iter()
.any(|attribute| matches!(attribute, PathAttribute::OnlyToCustomer(_)))
);
Ok(())
}
crates/integration-tests/tests/rpki.rs
-291
View File
@@ -1,291 +0,0 @@
//! Route origin validation, end to end: a server that validates against a
//! cache, a peer that announces routes, and the validation data changing
//! under both.
use std::net::Ipv4Addr;
use std::time::Duration;
use bgp_integration_tests::announcement::{announcement, withdrawal};
use bgp_integration_tests::test_cache::TestCache;
use bgp_integration_tests::test_peer::TestPeer;
use bgp_integration_tests::test_server::{ServerWithPeer, TestServer};
use bgp_integration_tests::{default_policy, prefix, wait_until};
use bgp_packet::message::{AsPathAttribute, PathAttribute};
use bgp_rpki::{CacheState, RpkiConfig, Vrp};
use bgp_server::policy::ImportPolicy;
use eyre::Result;
use serde_json::Value;
const LOCAL_ASN: u32 = 1299;
const PEER_ASN: u32 = 3333;
const PEER_ID: Ipv4Addr = Ipv4Addr::new(10, 0, 0, 2);
fn vrp(p: &str, max_length: u8, asn: u32) -> Vrp {
Vrp {
prefix: prefix(p),
max_length,
asn,
}
}
/// A server that validates against `cache`, with one configured peer at
/// the address the test connects from. Returns once the server has the
/// cache's data.
async fn server_with(
cache: &TestCache,
expire: Option<Duration>,
import: ImportPolicy,
) -> Result<ServerWithPeer> {
let cache = cache.address();
let setup = TestServer::with_peer(LOCAL_ASN, PEER_ASN, |c, peer| {
c.rpki = Some(RpkiConfig {
caches: vec![cache],
// Data never expires sooner than a refresh and a retry take.
refresh: expire.map(|expire| expire / 2),
retry: Some(Duration::from_millis(100)),
expire,
});
peer.import = import;
})
.await?;
let rpki = setup.server.rpki.clone().expect("validation is configured");
wait_until("the server has the cache's data", || {
rpki.status().state == CacheState::Synchronised
})
.await?;
Ok(setup)
}
async fn established(server: &TestServer) -> Result<TestPeer> {
server.established(PEER_ASN, PEER_ID).await
}
/// Announces `prefixes` as originated by `origin`, reached through the
/// peer.
async fn announce(peer: &mut TestPeer, prefixes: &[&str], origin: u32) -> Result<()> {
let path = if origin == PEER_ASN {
vec![PEER_ASN]
} else {
vec![PEER_ASN, origin]
};
peer.announce(announcement(prefixes, &path)).await
}
/// A route whose origin is not the one the holder of the addresses
/// authorised is refused. One that is authorised is taken, and so is one
/// that nothing is known about.
#[tokio::test]
async fn test_an_invalid_route_is_refused_and_the_rest_are_taken() -> Result<()> {
let cache = TestCache::start(vec![vrp("193.0.0.0/21", 24, PEER_ASN)]).await;
let ServerWithPeer { server, .. } = server_with(&cache, None, default_policy()).await?;
let (base, _http) = server.serve_http().await?;
let mut peer = established(&server).await?;
announce(&mut peer, &["193.0.0.0/21", "194.0.0.0/24"], PEER_ASN).await?;
// Someone else's AS as the origin of a covered prefix; and the right
// AS, more specific than it may announce.
announce(&mut peer, &["193.0.4.0/24"], 15169).await?;
announce(&mut peer, &["193.0.6.0/25"], PEER_ASN).await?;
server
.table_becomes(&["193.0.0.0/21", "194.0.0.0/24"], 1)
.await?;
let client = reqwest::Client::new();
let get = |path: &str| {
let request = client.get(format!("{base}{path}"));
async move { Ok::<Value, eyre::Report>(request.send().await?.error_for_status()?.json().await?) }
};
// Each route says how it stands.
let routes = get("/api/routes/received").await?["routes"].take();
let validity: Vec<(&str, &str)> = routes
.as_array()
.unwrap()
.iter()
.map(|r| (r["prefix"].as_str().unwrap(), r["rpki"].as_str().unwrap()))
.collect();
assert_eq!(
validity,
[("193.0.0.0/21", "valid"), ("194.0.0.0/24", "not found")]
);
// The refused one is on record with its reason. The too-specific one
// was refused before validation came to it.
let refused = get("/api/routes/rejected").await?;
assert_eq!(
refused[0]["by_reason"],
serde_json::json!({"RPKI invalid": 1, "prefix too specific": 1})
);
let peers = get("/api/peers").await?;
assert_eq!(peers[0]["routes"], 2);
assert_eq!(peers[0]["held_routes"], 1);
let rpki = get("/api/rpki").await?;
assert_eq!(rpki["enabled"], true);
assert_eq!(rpki["state"], "synchronised");
assert_eq!(rpki["vrps_ipv4"], 1);
assert_eq!(rpki["cache"], cache.address().to_string());
Ok(())
}
/// Validity changes with the validation data, and the routes follow: one
/// that was taken goes when it becomes invalid, and one that was refused
/// as invalid is taken when it no longer is. The peer sends nothing more
/// meanwhile; it has said all it has to say.
#[tokio::test]
async fn test_routes_follow_the_validation_data() -> Result<()> {
let ripe = vrp("193.0.0.0/21", 24, PEER_ASN);
let cache = TestCache::start(vec![ripe]).await;
let ServerWithPeer { server, .. } = server_with(&cache, None, default_policy()).await?;
let mut peer = established(&server).await?;
// Nothing is known about 8.8.8.0/24, so it is taken. 193.0.4.0/24 is
// covered, and not for this origin.
announce(&mut peer, &["8.8.8.0/24", "193.0.4.0/24"], 15169).await?;
server.table_becomes(&["8.8.8.0/24"], 1).await?;
// The holder of 8.8.8.0/24 authorises another AS: the route in the
// table is now invalid, and goes.
cache.set(vec![ripe, vrp("8.8.8.0/24", 24, 13335)]);
server.table_becomes(&[], 2).await?;
let refused = &server.rejections.peers()[0].1;
assert_eq!(refused.total(), 2);
assert_eq!(refused.recent()[0].prefix, prefix("8.8.8.0/24"));
assert_eq!(&refused.recent()[0].as_path[..], [PEER_ASN, 15169]);
// Both are authorised for the origin they have: both come back, with
// the attributes they were announced with.
cache.set(vec![
ripe,
vrp("8.8.8.0/24", 24, 15169),
vrp("193.0.4.0/24", 24, 15169),
]);
server
.table_becomes(&["8.8.8.0/24", "193.0.4.0/24"], 0)
.await?;
for route in server.received_routes() {
assert!(
route
.attributes
.contains(&AsPathAttribute::from_asns(vec![PEER_ASN, 15169])),
"{route:?}"
);
}
assert_eq!(server.registry.list()[0].counters.routes(), 2);
Ok(())
}
/// A held route is the peer's to withdraw or replace like any other. One
/// it has withdrawn does not appear when it would have become valid.
#[tokio::test]
async fn test_a_held_route_the_peer_withdrew_does_not_come_back() -> Result<()> {
let ripe = vrp("193.0.0.0/21", 24, PEER_ASN);
let cache = TestCache::start(vec![ripe]).await;
let ServerWithPeer { server, .. } = server_with(&cache, None, default_policy()).await?;
let mut peer = established(&server).await?;
announce(&mut peer, &["193.0.4.0/24", "193.0.5.0/24"], 15169).await?;
server.table_becomes(&[], 2).await?;
// One is withdrawn, the other announced again with the right origin.
peer.announce(withdrawal([prefix("193.0.4.0/24").into()]))
.await?;
announce(&mut peer, &["193.0.5.0/24"], PEER_ASN).await?;
server.table_becomes(&["193.0.5.0/24"], 0).await?;
// Now the old origin would be valid for both. Neither old route is
// there to be brought back.
cache.set(vec![ripe, vrp("193.0.4.0/23", 24, 15169)]);
tokio::time::sleep(Duration::from_millis(500)).await;
server.table_becomes(&["193.0.5.0/24"], 0).await?;
assert!(
server.received_routes()[0]
.attributes
.contains(&AsPathAttribute::from_asns(vec![PEER_ASN])),
"the route the peer has now"
);
Ok(())
}
/// A peer can be exempt: its invalid routes are taken, and shown for what
/// they are.
#[tokio::test]
async fn test_a_peer_can_be_exempt_from_validation() -> Result<()> {
let cache = TestCache::start(vec![vrp("193.0.0.0/21", 24, PEER_ASN)]).await;
let import = ImportPolicy {
accept_rpki_invalid: true,
..default_policy()
};
let ServerWithPeer { server, .. } = server_with(&cache, None, import).await?;
let (base, _http) = server.serve_http().await?;
let mut peer = established(&server).await?;
announce(&mut peer, &["193.0.4.0/24"], 15169).await?;
server.table_becomes(&["193.0.4.0/24"], 0).await?;
let routes: Value = reqwest::get(format!("{base}/api/routes/received"))
.await?
.json()
.await?;
assert_eq!(routes["routes"][0]["rpki"], "invalid");
assert!(server.rejections.peers().is_empty());
Ok(())
}
/// Validation data that no cache has confirmed for too long is dropped,
/// and with it every verdict: routes held as invalid are taken. Refusing
/// routes on the word of data that may be long out of date would be worse.
#[tokio::test]
async fn test_held_routes_are_taken_when_the_validation_data_expires() -> Result<()> {
let cache = TestCache::start(vec![vrp("193.0.0.0/21", 24, PEER_ASN)]).await;
let ServerWithPeer { server, .. } =
server_with(&cache, Some(Duration::from_millis(1500)), default_policy()).await?;
let mut peer = established(&server).await?;
announce(&mut peer, &["193.0.4.0/24"], 15169).await?;
server.table_becomes(&[], 1).await?;
drop(cache);
server.table_becomes(&["193.0.4.0/24"], 0).await?;
assert!(server.rpki.as_ref().unwrap().current().vrps.is_empty());
Ok(())
}
/// A hijack that was possible: on a session with two-octet AS numbers a
/// route has two paths, AS_PATH and AS4_PATH, and a peer can make them
/// say different things. Here AS_PATH is the peer alone, and an AS4_PATH
/// too long to be used ends at the AS that holds the addresses. The route
/// is the peer's and invalid, when it arrives and when it is looked at
/// again; nothing may take the unused path's word for where it comes from.
#[tokio::test]
async fn test_an_unused_as4_path_does_not_make_a_route_valid() -> Result<()> {
use bgp_packet::message::AsPathSegment;
const HOLDER: u32 = 15169;
let cache = TestCache::start(vec![vrp("193.0.0.0/21", 24, HOLDER)]).await;
let ServerWithPeer { server, .. } = server_with(&cache, None, default_policy()).await?;
// No capabilities: AS numbers are two octets wide on this session.
let mut peer = server.established_with(PEER_ASN, PEER_ID, vec![]).await?;
assert!(!server.registry.list()[0].four_octet_asn);
let as4_path = PathAttribute::As4Path(AsPathAttribute {
segments: vec![AsPathSegment {
ordered: true,
path: vec![3356, HOLDER],
}],
});
peer.announce(announcement(&["193.0.4.0/24"], &[PEER_ASN]).with_attributes([as4_path]))
.await?;
server.table_becomes(&[], 1).await?;
// The data changes, and every held route is looked at again.
cache.set(vec![
vrp("193.0.0.0/21", 24, HOLDER),
vrp("198.51.100.0/24", 24, 64500),
]);
let rpki = server.rpki.clone().unwrap();
wait_until("the server has the new data", || {
rpki.current().vrps.len() == 2
})
.await?;
// Time enough for the look, and then some.
tokio::time::sleep(Duration::from_millis(600)).await;
server.table_becomes(&[], 1).await?;
Ok(())
}
crates/integration-tests/tests/transit.rs
-356
View File
@@ -1,356 +0,0 @@
//! Passing routes on: what one peer announced reaches the others, as far
//! as the rules for that let it.
use std::collections::BTreeSet;
use std::net::Ipv4Addr;
use std::sync::Arc;
use std::time::Duration;
use bgp_integration_tests::announcement::{announcement, withdrawal};
use bgp_integration_tests::test_peer::TestPeer;
use bgp_integration_tests::test_server::TestServer;
use bgp_integration_tests::{TIMEOUT, prefix};
use bgp_packet::message::{
CommunitiesPathAttribute, Message, NextHopPathAttribute, PathAttribute, RouteRefreshMessage,
UpdateMessage,
};
use bgp_primitives::constants::AfiSafi;
use bgp_rib::{LocalRoute, LocalRouteAttrs};
use bgp_server::marks::{Control, Marks};
use bgp_server::policy::ExportPolicy;
use eyre::Result;
const LOCAL_ASN: u32 = 64601;
/// A server that passes on to every peer what it learned from the others.
async fn transit(customize: impl FnOnce(&mut bgp_server::config::Config)) -> Result<TestServer> {
TestServer::start_with(LOCAL_ASN, |config| {
config.export = Arc::new(ExportPolicy {
learned_routes: true,
all_paths: false,
..ExportPolicy::default()
});
customize(config);
})
.await
}
/// The peer with AS `64700 + n`, its session up.
async fn peer(server: &TestServer, n: u8) -> Result<TestPeer> {
server
.established(64700 + u32::from(n), Ipv4Addr::new(10, 0, 0, n))
.await
}
async fn announce(
peer: &mut TestPeer,
prefixes: &[&str],
path: &[u32],
communities: &[(u16, u16)],
) -> Result<()> {
peer.announce(announcement(prefixes, path).with_communities(communities))
.await
}
async fn withdraw(peer: &mut TestPeer, prefixes: &[&str]) -> Result<()> {
peer.announce(withdrawal(prefixes.iter().map(|p| prefix(p).into())))
.await
}
/// What a peer has been told so far, as the prefixes it would now have
/// and the path each came with.
#[derive(Default, Debug)]
struct Told {
routes: std::collections::BTreeMap<String, Vec<u32>>,
updates: usize,
}
impl Told {
fn take(&mut self, update: &UpdateMessage) {
self.updates += 1;
for route in &update.withdrawn_routes {
self.routes.remove(&route.prefix.to_string());
}
let path: Vec<u32> = update
.path_attributes
.iter()
.find_map(|attribute| match attribute {
PathAttribute::AsPath(path) => Some(
path.segments
.iter()
.flat_map(|segment| segment.path.clone())
.collect(),
),
_ => None,
})
.unwrap_or_default();
for route in &update.announced_routes {
self.routes.insert(route.prefix.to_string(), path.clone());
}
}
fn prefixes(&self) -> BTreeSet<&str> {
self.routes.keys().map(String::as_str).collect()
}
}
/// Reads what the peer is sent until it has exactly `expected`.
async fn told_becomes(peer: &mut TestPeer, told: &mut Told, expected: &[&str]) -> Result<()> {
let expected: BTreeSet<&str> = expected.iter().copied().collect();
let reading = async {
while told.prefixes() != expected {
let update = peer.recv_update().await?;
told.take(&update);
}
Ok::<(), eyre::Report>(())
};
match tokio::time::timeout(TIMEOUT, reading).await {
Ok(read) => read,
Err(_) => Err(eyre::eyre!(
"the peer has {:?}; expected {expected:?}",
told.prefixes()
)),
}
}
/// Nothing more is sent to the peer for a while.
async fn told_nothing(peer: &mut TestPeer) {
let more = tokio::time::timeout(Duration::from_millis(300), peer.recv_update()).await;
assert!(more.is_err(), "the peer was sent {more:?}");
}
/// What one peer announces the others are told, with our AS on the path
/// and ourselves as next hop; and what it takes back, they are told too.
#[tokio::test]
async fn test_routes_from_one_peer_reach_the_others() -> Result<()> {
let server = transit(|_| {}).await?;
let mut one = peer(&server, 1).await?;
let mut two = peer(&server, 2).await?;
announce(
&mut one,
&["10.1.0.0/16", "10.2.0.0/16", "10.3.0.0/16"],
&[64701, 65000],
&[],
)
.await?;
let mut told = Told::default();
told_becomes(
&mut two,
&mut told,
&["10.1.0.0/16", "10.2.0.0/16", "10.3.0.0/16"],
)
.await?;
assert_eq!(told.routes["10.1.0.0/16"], [LOCAL_ASN, 64701, 65000]);
assert_eq!(
told.updates, 1,
"routes that go out alike go in one message"
);
// Not back to where they came from.
told_nothing(&mut one).await;
// The next hop is ourselves: the address of our end of the session.
announce(&mut one, &["10.4.0.0/16"], &[64701], &[]).await?;
let update = two.recv_update().await?;
assert!(
update
.path_attributes
.contains(&PathAttribute::NextHop(NextHopPathAttribute(
Ipv4Addr::LOCALHOST
))),
"{update:?}"
);
told.take(&update);
// One is taken back; then the peer goes, and its routes with it.
withdraw(&mut one, &["10.2.0.0/16"]).await?;
told_becomes(
&mut two,
&mut told,
&["10.1.0.0/16", "10.3.0.0/16", "10.4.0.0/16"],
)
.await?;
drop(one);
told_becomes(&mut two, &mut told, &[]).await?;
Ok(())
}
/// A peer is told of the best route, and of the next best when that one
/// goes; and a peer that comes later is told what there is.
#[tokio::test]
async fn test_a_peer_is_kept_at_the_best_route() -> Result<()> {
let server = transit(|_| {}).await?;
let mut near = peer(&server, 1).await?;
let mut far = peer(&server, 2).await?;
let mut watcher = peer(&server, 3).await?;
let mut told = Told::default();
announce(&mut far, &["10.1.0.0/16"], &[64702, 65000, 65001], &[]).await?;
told_becomes(&mut watcher, &mut told, &["10.1.0.0/16"]).await?;
assert_eq!(told.routes["10.1.0.0/16"], [LOCAL_ASN, 64702, 65000, 65001]);
// A better route for it comes: that is the one told now.
announce(&mut near, &["10.1.0.0/16"], &[64701, 65001], &[]).await?;
let update = watcher.recv_update().await?;
told.take(&update);
assert_eq!(told.routes["10.1.0.0/16"], [LOCAL_ASN, 64701, 65001]);
assert!(
update.withdrawn_routes.is_empty(),
"replaced, not withdrawn"
);
// And goes again: back to the other, with no gap.
withdraw(&mut near, &["10.1.0.0/16"]).await?;
let update = watcher.recv_update().await?;
told.take(&update);
assert_eq!(told.routes["10.1.0.0/16"], [LOCAL_ASN, 64702, 65000, 65001]);
// The other peer was told of the better route while it was the best,
// and then that it is gone: the one left is its own, which it is not
// told of.
let mut told_far = Told::default();
told_becomes(&mut far, &mut told_far, &["10.1.0.0/16"]).await?;
let gone = far.recv_update().await?;
assert_eq!(gone.withdrawn_routes, vec![prefix("10.1.0.0/16")]);
told_nothing(&mut far).await;
// A peer that comes now is told what there is; and again if it asks.
let mut late = peer(&server, 4).await?;
let mut late_told = Told::default();
told_becomes(&mut late, &mut late_told, &["10.1.0.0/16"]).await?;
late.send(&Message::RouteRefresh(RouteRefreshMessage::request(
AfiSafi::IPV4_UNICAST,
)))
.await?;
let again = late.recv_update().await?;
assert_eq!(again.announced_routes, vec![prefix("10.1.0.0/16")]);
Ok(())
}
/// A route is not sent to a peer whose AS is on its path already, nor
/// anywhere if it says NO_EXPORT; and what we originate goes to everyone
/// beside what was learned.
#[tokio::test]
async fn test_routes_stop_where_they_must() -> Result<()> {
let server = transit(|config| {
config.originate = vec![LocalRoute::new(
prefix("10.99.0.0/16"),
LocalRouteAttrs::default(),
)];
})
.await?;
let mut one = peer(&server, 1).await?;
let mut two = peer(&server, 2).await?;
let mut told = Told::default();
told_becomes(&mut two, &mut told, &["10.99.0.0/16"]).await?;
assert_eq!(told.routes["10.99.0.0/16"], [LOCAL_ASN]);
// Through the second peer's AS already; and not for other networks.
announce(&mut one, &["10.1.0.0/16"], &[64701, 64702, 65000], &[]).await?;
announce(&mut one, &["10.2.0.0/16"], &[64701], &[(0xFFFF, 0xFF01)]).await?;
announce(&mut one, &["10.3.0.0/16"], &[64701], &[]).await?;
told_becomes(&mut two, &mut told, &["10.99.0.0/16", "10.3.0.0/16"]).await?;
told_nothing(&mut two).await;
assert_eq!(server.rib.len(), 3, "all three were taken");
Ok(())
}
/// Communities on a route say which peers it is passed on to, and are
/// gone when it is; and every route to a peer gets the marks of `export`.
#[tokio::test]
async fn test_communities_say_who_is_told() -> Result<()> {
let server = transit(|config| {
config.control = Some(Control {
local_asn: LOCAL_ASN,
});
config.export = Arc::new(ExportPolicy {
learned_routes: true,
all_paths: false,
marks: Marks::parse(&[], &["64601:7".to_string()], &[]).unwrap(),
prepend: 1,
});
config.originate = vec![LocalRoute::new(
prefix("10.99.0.0/16"),
LocalRouteAttrs {
// Our own route, and not for the second peer.
communities: vec![(0, 64702)],
..LocalRouteAttrs::default()
},
)];
})
.await?;
let mut one = peer(&server, 1).await?;
let mut two = peer(&server, 2).await?;
let mut three = peer(&server, 3).await?;
let (mut told_two, mut told_three) = (Told::default(), Told::default());
// To everyone but the second; to the second only; to nobody; and
// with nothing said.
announce(
&mut one,
&["10.1.0.0/16"],
&[64701],
&[(0, 64702), (3356, 3)],
)
.await?;
announce(&mut one, &["10.2.0.0/16"], &[64701], &[(64601, 64702)]).await?;
announce(&mut one, &["10.3.0.0/16"], &[64701], &[(0, 64601)]).await?;
announce(&mut one, &["10.4.0.0/16"], &[64701], &[]).await?;
told_becomes(&mut two, &mut told_two, &["10.2.0.0/16", "10.4.0.0/16"]).await?;
told_becomes(
&mut three,
&mut told_three,
&["10.99.0.0/16", "10.1.0.0/16", "10.4.0.0/16"],
)
.await?;
told_nothing(&mut two).await;
told_nothing(&mut three).await;
// Our AS once for passing it on, and once more by `export`.
assert_eq!(
told_three.routes["10.1.0.0/16"],
[LOCAL_ASN, LOCAL_ASN, 64701]
);
assert_eq!(told_three.routes["10.99.0.0/16"], [LOCAL_ASN, LOCAL_ASN]);
// What was for us to act on is gone, and the router's mark is on.
announce(
&mut one,
&["10.5.0.0/16"],
&[64701],
&[(0, 64702), (3356, 3)],
)
.await?;
let update = three.recv_update().await?;
assert!(
update
.path_attributes
.contains(&PathAttribute::Communities(CommunitiesPathAttribute(vec![
(3356, 3),
(64601, 7)
]))),
"{update:?}"
);
Ok(())
}
/// Without `learned_routes` a peer is told what we originate and nothing
/// else, as before there was such a thing.
#[tokio::test]
async fn test_nothing_learned_is_passed_on_unless_asked_for() -> Result<()> {
let server = TestServer::start_with(LOCAL_ASN, |config| {
config.originate = vec![LocalRoute::new(
prefix("10.99.0.0/16"),
LocalRouteAttrs::default(),
)];
})
.await?;
let mut one = peer(&server, 1).await?;
let mut two = peer(&server, 2).await?;
let mut told = Told::default();
told_becomes(&mut two, &mut told, &["10.99.0.0/16"]).await?;
announce(&mut one, &["10.1.0.0/16"], &[64701], &[]).await?;
server.wait_for_routes(1).await?;
told_nothing(&mut two).await;
Ok(())
}
crates/integration-tests/tests/update_errors.rs
+40 -19
View File
@@ -2,18 +2,19 @@
//! cost the routes in the UPDATE, not the session.
use std::net::Ipv4Addr;
use std::time::Duration;
use bgp_integration_tests::test_peer::TestPeer;
use bgp_integration_tests::test_server::{ReceivedRoute, TestServer};
use bgp_integration_tests::{prefix, wait_for};
use bgp_packet::errors::{BgpError, UpdateMsgSubcode};
use bgp_packet::message::{LocalPrefPathAttribute, PathAttribute};
use bgp_packet::message::{LocalPrefPathAttribute, Message, PathAttribute};
use bgp_primitives::ip_prefix::IpPrefix;
use bgp_server::metrics::UPDATE_ERRORS;
use eyre::Result;
use eyre::{Result, bail};
const LOCAL_ASN: u32 = 65000;
const PEER_ASN: u32 = 65001;
const TIMEOUT: Duration = Duration::from_secs(5);
const WELL_KNOWN: u8 = 0x40;
const OPTIONAL_TRANSITIVE: u8 = 0xc0;
@@ -57,24 +58,31 @@ fn update(attributes: &[Vec<u8>], nlri: &[u8]) -> Vec<u8> {
const ROUTE_A: [u8; 3] = [16, 10, 1];
const ROUTE_B: [u8; 3] = [16, 10, 2];
fn prefix(s: &str) -> IpPrefix {
IpPrefix::try_from(s).unwrap()
}
async fn established(server: &TestServer, asn: u32) -> Result<TestPeer> {
server.established(asn, Ipv4Addr::new(10, 0, 0, 1)).await
let mut peer = TestPeer::connect(server.addr).await?;
peer.handshake(asn, Ipv4Addr::new(10, 0, 0, 1), 90).await?;
Ok(peer)
}
/// Waits until the server's Adj-RIB-In holds exactly `prefixes`.
async fn wait_for_routes(server: &TestServer, prefixes: &[&str]) -> Result<Vec<ReceivedRoute>> {
let mut expected: Vec<IpPrefix> = prefixes.iter().map(|p| prefix(p)).collect();
expected.sort_by_key(|p| p.to_string());
wait_for(&format!("the table holds {prefixes:?}"), || {
let mut routes = server.received_routes();
routes.sort_by_key(|r| r.prefix.to_string());
routes
.iter()
.map(|r| &r.prefix)
.eq(expected.iter())
.then_some(routes)
Ok(tokio::time::timeout(TIMEOUT, async {
loop {
let mut routes = server.received_routes();
routes.sort_by_key(|r| r.prefix.to_string());
if routes.iter().map(|r| &r.prefix).eq(expected.iter()) {
return routes;
}
tokio::time::sleep(Duration::from_millis(10)).await;
}
})
.await
.await?)
}
/// The situation RFC 7606 exists for: a route is in the table, and then an
@@ -197,11 +205,24 @@ async fn test_an_unreadable_prefix_still_resets_the_session() -> Result<()> {
))
.await?;
let notification = peer.expect_notification().await?;
assert_eq!(
notification.error,
BgpError::UpdateMsg(UpdateMsgSubcode::InvalidNetworkField)
);
server.wait_for_sessions(0).await?;
loop {
match tokio::time::timeout(TIMEOUT, peer.recv()).await?? {
Message::KeepAlive => {}
Message::Notification(n) => {
assert_eq!(
n.error,
BgpError::UpdateMsg(UpdateMsgSubcode::InvalidNetworkField)
);
break;
}
other => bail!("expected NOTIFICATION, got {other:?}"),
}
}
tokio::time::timeout(TIMEOUT, async {
while !server.registry.list().is_empty() {
tokio::time::sleep(Duration::from_millis(10)).await;
}
})
.await?;
Ok(())
}
crates/packet/README.md
+3 -2
View File
@@ -2,5 +2,6 @@
This crate implements parsers and serialisers for the Border Gateway Protocol wire format.
It is used by the daemon in `bin/bgp`, and tested with property tests
(`tests/properties.rs`) and fuzz targets (`fuzz/`).
## ⚠️ Current status
This crate is still under development and is not yet ready for use.
crates/packet/src/attributes.rs
-936
View File
@@ -1,936 +0,0 @@
//! Path attributes: what an UPDATE says about the routes it announces
//! (RFC 4271 §5 and those that added to it), each with how it is read
//! from the wire and written to it.
//!
//! Everything here is re-exported from [`crate::message`], which is where
//! the rest of the tree names it from.
use std::net::{Ipv4Addr, Ipv6Addr};
use bitfield::bitfield;
use bytes::{BufMut, BytesMut};
use nom::Parser;
use nom::number::complete::{be_u8, be_u32};
use serde::Serialize;
use serde_repr::Serialize_repr;
use bgp_primitives::constants::{AfiSafi, IpVersion, Safi};
use bgp_primitives::ip_prefix::{IpPrefix, Nlri};
use bgp_primitives::parser::{BgpParserError, EncodeError, ParserContext};
use crate::wire::{
Parsed, list, malformed, parse_afi_safi, parse_asn, put_afi_safi, put_asn, put_length, records,
whole,
};
bitfield! {
#[derive(Clone, Debug, Serialize, PartialEq, Eq)]
pub struct PathAttributeFlags(u8);
impl new;
u8;
pub optional, set_optional: 7;
pub transitive, set_transitive: 6;
pub partial, set_partial: 5;
pub extended_length, set_extended_length: 4;
}
/// Path attribute type codes.
pub mod attr_type {
pub const ORIGIN: u8 = 1;
pub const AS_PATH: u8 = 2;
pub const NEXT_HOP: u8 = 3;
pub const MULTI_EXIT_DISC: u8 = 4;
pub const LOCAL_PREF: u8 = 5;
pub const ATOMIC_AGGREGATE: u8 = 6;
pub const AGGREGATOR: u8 = 7;
pub const COMMUNITIES: u8 = 8;
pub const ORIGINATOR_ID: u8 = 9;
pub const CLUSTER_LIST: u8 = 10;
pub const MP_REACH_NLRI: u8 = 14;
pub const MP_UNREACH_NLRI: u8 = 15;
pub const EXTENDED_COMMUNITIES: u8 = 16;
pub const AS4_PATH: u8 = 17;
pub const AS4_AGGREGATOR: u8 = 18;
pub const LARGE_COMMUNITIES: u8 = 32;
pub const ONLY_TO_CUSTOMER: u8 = 35;
}
#[derive(Debug, Clone, Serialize, PartialEq, Eq)]
pub enum PathAttribute {
Origin(OriginPathAttribute),
AsPath(AsPathAttribute),
NextHop(NextHopPathAttribute),
MultiExitDisc(MultiExitDiscPathAttribute),
LocalPref(LocalPrefPathAttribute),
AtomicAggregate(AtomicAggregatePathAttribute),
Aggregator(AggregatorPathAttribute),
Communities(CommunitiesPathAttribute),
MpReachNlri(MpReachNlriPathAttribute),
MpUnreachNlri(MpUnreachNlriPathAttribute),
ExtendedCommunities(ExtendedCommunitiesPathAttribute),
/// The AS path with four-octet AS numbers, for a route that crosses
/// speakers that only handle two-octet ones (RFC 6793 §3).
As4Path(AsPathAttribute),
LargeCommunities(LargeCommunitiesPathAttribute),
/// Only to Customer (RFC 9234 §5): the route has been handed to a
/// customer, a peer or a route server client by the AS with this
/// number, and from there on may only go down to customers.
OnlyToCustomer(u32),
/// ORIGINATOR_ID (RFC 4456 §8): the BGP Identifier of the router in our
/// AS that learned the route from outside, put on by the first router
/// that passes the route from one router of the AS to another.
OriginatorId(Ipv4Addr),
/// CLUSTER_LIST (RFC 4456 §8): the clusters the route has been passed
/// on by inside our AS, the last first.
ClusterList(Vec<Ipv4Addr>),
/// An attribute this implementation does not interpret, kept as it
/// came so that it can be passed on.
Unknown {
flags: PathAttributeFlags,
type_code: u8,
payload: Vec<u8>,
},
}
/// The Optional and Transitive flags that the attribute with this type code
/// is defined to have, for the attributes whose definition we know. A
/// received attribute with other values is in error (RFC 7606 §3.c).
pub(crate) fn defined_flags(type_code: u8) -> Option<(bool, bool)> {
use attr_type::*;
const WELL_KNOWN: (bool, bool) = (false, true);
const OPTIONAL_TRANSITIVE: (bool, bool) = (true, true);
const OPTIONAL_NON_TRANSITIVE: (bool, bool) = (true, false);
Some(match type_code {
ORIGIN | AS_PATH | NEXT_HOP | LOCAL_PREF | ATOMIC_AGGREGATE => WELL_KNOWN,
MULTI_EXIT_DISC | ORIGINATOR_ID | CLUSTER_LIST | MP_REACH_NLRI | MP_UNREACH_NLRI => {
OPTIONAL_NON_TRANSITIVE
}
AGGREGATOR | COMMUNITIES | EXTENDED_COMMUNITIES | AS4_PATH | AS4_AGGREGATOR
| LARGE_COMMUNITIES | ONLY_TO_CUSTOMER => OPTIONAL_TRANSITIVE,
_ => return None,
})
}
/// One attribute as it lies on the wire, split out but not interpreted.
#[derive(Debug, Clone)]
pub struct RawAttribute<'a> {
pub flags: PathAttributeFlags,
pub type_code: u8,
pub value: &'a [u8],
/// The whole attribute, header and value.
pub wire: &'a [u8],
}
/// Why the bytes at the front of a buffer are not an attribute.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum NotAnAttribute {
/// Too few octets for flags, type and length.
NoHeader,
/// The length says more than there is.
Truncated {
type_code: u8,
length: usize,
available: usize,
},
}
impl std::fmt::Display for NotAnAttribute {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Self::NoHeader => f.write_str("too few octets for an attribute"),
Self::Truncated {
type_code,
length,
available,
} => write!(
f,
"attribute {type_code} has length {length} but only {available} octets remain"
),
}
}
}
/// Splits the attribute at the front of `buf` from what follows it.
pub fn split_attribute(buf: &[u8]) -> Result<(RawAttribute<'_>, &[u8]), NotAnAttribute> {
let [flags, type_code, rest @ ..] = buf else {
return Err(NotAnAttribute::NoHeader);
};
let flags = PathAttributeFlags(*flags);
let (length, rest) = match (flags.extended_length(), rest) {
(true, [hi, lo, rest @ ..]) => (usize::from(u16::from_be_bytes([*hi, *lo])), rest),
(false, [length, rest @ ..]) => (usize::from(*length), rest),
_ => return Err(NotAnAttribute::NoHeader),
};
let Some((value, rest)) = rest.split_at_checked(length) else {
return Err(NotAnAttribute::Truncated {
type_code: *type_code,
length,
available: rest.len(),
});
};
let raw = RawAttribute {
flags,
type_code: *type_code,
value,
wire: &buf[..buf.len() - rest.len()],
};
Ok((raw, rest))
}
impl PathAttribute {
pub fn type_code(&self) -> u8 {
use attr_type::*;
match self {
Self::Origin(_) => ORIGIN,
Self::AsPath(_) => AS_PATH,
Self::NextHop(_) => NEXT_HOP,
Self::MultiExitDisc(_) => MULTI_EXIT_DISC,
Self::LocalPref(_) => LOCAL_PREF,
Self::AtomicAggregate(_) => ATOMIC_AGGREGATE,
Self::Aggregator(_) => AGGREGATOR,
Self::Communities(_) => COMMUNITIES,
Self::MpReachNlri(_) => MP_REACH_NLRI,
Self::MpUnreachNlri(_) => MP_UNREACH_NLRI,
Self::ExtendedCommunities(_) => EXTENDED_COMMUNITIES,
Self::As4Path(_) => AS4_PATH,
Self::LargeCommunities(_) => LARGE_COMMUNITIES,
Self::OnlyToCustomer(_) => ONLY_TO_CUSTOMER,
Self::OriginatorId(_) => ORIGINATOR_ID,
Self::ClusterList(_) => CLUSTER_LIST,
Self::Unknown { type_code, .. } => *type_code,
}
}
/// The flags to send the attribute with, but for Extended Length, which
/// depends on how long the value turns out to be.
fn flags(&self) -> PathAttributeFlags {
match self {
// As they came: the Partial bit has to survive (RFC 4271 §5).
Self::Unknown { flags, .. } => flags.clone(),
known => {
let (optional, transitive) = defined_flags(known.type_code())
.expect("every attribute with a variant has defined flags");
PathAttributeFlags::new(optional, transitive, false, false)
}
}
}
/// Parses one attribute, header and value, from the front of `buf`.
/// The flags are not checked against the attribute's definition;
/// [`crate::message::UpdateMessage::decode`] does that for attributes off a session.
pub fn from_wire<'a>(ctx: &ParserContext, buf: &'a [u8]) -> Parsed<'a, Self> {
let (raw, rest) = split_attribute(buf).map_err(|e| malformed(e.to_string()))?;
let attribute = Self::parse_value(ctx, raw.flags, raw.type_code, raw.value)?;
Ok((rest, attribute))
}
/// Makes the attribute with this type code from its value. `value` is
/// exactly the value, and all of it has to be used. A type code without
/// a variant of its own is kept as bytes.
pub(crate) fn parse_value<'a>(
ctx: &ParserContext,
flags: PathAttributeFlags,
type_code: u8,
value: &'a [u8],
) -> Result<Self, nom::Err<BgpParserError<&'a [u8]>>> {
use attr_type::*;
Ok(match type_code {
ORIGIN => Self::Origin(whole("ORIGIN", OriginPathAttribute::from_wire(value)?)?),
AS_PATH => Self::AsPath(whole("AS_PATH", AsPathAttribute::from_wire(ctx, value)?)?),
NEXT_HOP => Self::NextHop(whole("NEXT_HOP", NextHopPathAttribute::from_wire(value)?)?),
MULTI_EXIT_DISC => Self::MultiExitDisc(whole(
"MULTI_EXIT_DISC",
MultiExitDiscPathAttribute::from_wire(value)?,
)?),
LOCAL_PREF => Self::LocalPref(whole(
"LOCAL_PREF",
LocalPrefPathAttribute::from_wire(value)?,
)?),
ATOMIC_AGGREGATE => Self::AtomicAggregate(whole(
"ATOMIC_AGGREGATE",
(value, AtomicAggregatePathAttribute {}),
)?),
AGGREGATOR => Self::Aggregator(whole(
"AGGREGATOR",
AggregatorPathAttribute::from_wire(ctx, value)?,
)?),
COMMUNITIES => Self::Communities(whole(
"COMMUNITIES",
CommunitiesPathAttribute::from_wire(value)?,
)?),
MP_REACH_NLRI => Self::MpReachNlri(whole(
"MP_REACH_NLRI",
MpReachNlriPathAttribute::from_wire(ctx, value)?,
)?),
MP_UNREACH_NLRI => Self::MpUnreachNlri(whole(
"MP_UNREACH_NLRI",
MpUnreachNlriPathAttribute::from_wire(ctx, value)?,
)?),
EXTENDED_COMMUNITIES => Self::ExtendedCommunities(whole(
"EXTENDED_COMMUNITIES",
ExtendedCommunitiesPathAttribute::from_wire(value)?,
)?),
// Always four octets wide, whatever the session's AS_PATH is.
AS4_PATH => Self::As4Path(whole(
"AS4_PATH",
AsPathAttribute::from_wire_with_width(true, value)?,
)?),
LARGE_COMMUNITIES => Self::LargeCommunities(whole(
"LARGE_COMMUNITIES",
LargeCommunitiesPathAttribute::from_wire(value)?,
)?),
ONLY_TO_CUSTOMER => {
let asn = <[u8; 4]>::try_from(value).map_err(|_| {
malformed(format!(
"Only to Customer is {} octets long, not four",
value.len()
))
})?;
Self::OnlyToCustomer(u32::from_be_bytes(asn))
}
ORIGINATOR_ID => {
let id = <[u8; 4]>::try_from(value).map_err(|_| {
malformed(format!(
"ORIGINATOR_ID is {} octets long, not four",
value.len()
))
})?;
Self::OriginatorId(Ipv4Addr::from(id))
}
CLUSTER_LIST => Self::ClusterList(
records::<4>(value, "CLUSTER_LIST")?
.iter()
.map(|id| Ipv4Addr::from(*id))
.collect(),
),
_ => Self::Unknown {
flags,
type_code,
payload: value.to_vec(),
},
})
}
/// Writes the attribute, header and value, to the end of `out`.
pub fn to_wire(&self, ctx: &ParserContext, out: &mut BytesMut) -> Result<(), EncodeError> {
// A header with a one-octet length first; whether the value needs
// the two-octet one is known once it is written.
let start = out.len();
let mut flags = self.flags();
out.put_u8(0);
out.put_u8(self.type_code());
let length_at = out.len();
out.put_u8(0);
self.value_to_wire(ctx, out)?;
let length = out.len() - length_at - 1;
if let Ok(short) = u8::try_from(length)
&& !flags.extended_length()
{
out[length_at] = short;
} else {
flags.set_extended_length(true);
// Move the value up by one octet to make room.
out.put_u8(0);
out.copy_within(length_at + 1..length_at + 1 + length, length_at + 2);
put_length(out, length_at, 2, length, u16::MAX.into(), "path attribute")?;
}
out[start] = flags.0;
Ok(())
}
fn value_to_wire(&self, ctx: &ParserContext, out: &mut BytesMut) -> Result<(), EncodeError> {
match self {
Self::Origin(a) => a.to_wire(out),
Self::AsPath(a) => a.to_wire(ctx, out)?,
Self::NextHop(a) => a.to_wire(out),
Self::MultiExitDisc(a) => a.to_wire(out),
Self::LocalPref(a) => a.to_wire(out),
Self::AtomicAggregate(_) => {}
Self::Aggregator(a) => a.to_wire(ctx, out),
Self::Communities(a) => a.to_wire(out),
Self::MpReachNlri(a) => a.to_wire(ctx, out),
Self::MpUnreachNlri(a) => a.to_wire(ctx, out),
Self::ExtendedCommunities(a) => a.to_wire(out),
Self::As4Path(a) => a.to_wire_with_width(true, out)?,
Self::LargeCommunities(a) => a.to_wire(out),
Self::OnlyToCustomer(asn) => out.put_u32(*asn),
Self::OriginatorId(id) => out.put_slice(&id.octets()),
Self::ClusterList(ids) => {
for id in ids {
out.put_slice(&id.octets());
}
}
Self::Unknown { payload, .. } => out.put_slice(payload),
}
Ok(())
}
}
/// An Only to Customer attribute with this AS number, as it is on the
/// wire: for adding to attributes that are kept as bytes.
pub fn only_to_customer_wire(asn: u32) -> [u8; 7] {
const OPTIONAL_TRANSITIVE: u8 = 0xc0;
let [a, b, c, d] = asn.to_be_bytes();
[
OPTIONAL_TRANSITIVE,
attr_type::ONLY_TO_CUSTOMER,
4,
a,
b,
c,
d,
]
}
/// ORIGIN: how the route came into BGP (RFC 4271 §5.1.1).
#[derive(Debug, PartialEq, Eq, Copy, Clone, Serialize_repr)]
#[repr(u8)]
pub enum OriginPathAttribute {
IGP = 0,
EGP = 1,
INCOMPLETE = 2,
}
impl OriginPathAttribute {
pub fn from_wire(buf: &[u8]) -> Parsed<'_, Self> {
let (buf, byte) = be_u8(buf)?;
let origin = match byte {
0 => Self::IGP,
1 => Self::EGP,
2 => Self::INCOMPLETE,
other => return Err(malformed(format!("unknown ORIGIN {other}"))),
};
Ok((buf, origin))
}
pub fn to_wire(&self, out: &mut BytesMut) {
out.put_u8(*self as u8);
}
}
/// AS_PATH: the ASes a route has passed through, as a list of segments
/// (RFC 4271 §4.3). Each segment is a type (1 for AS_SET, 2 for
/// AS_SEQUENCE), a count in one octet, and that many AS numbers.
#[derive(Debug, PartialEq, Eq, Clone, Serialize)]
pub struct AsPathAttribute {
pub segments: Vec<AsPathSegment>,
}
#[derive(Debug, PartialEq, Eq, Clone, Serialize)]
pub struct AsPathSegment {
/// True for an AS_SEQUENCE, false for an AS_SET.
pub ordered: bool,
pub path: Vec<u32>,
}
impl AsPathAttribute {
pub fn from_asns(asns: Vec<u32>) -> PathAttribute {
let segment = AsPathSegment {
ordered: true,
path: asns,
};
PathAttribute::AsPath(AsPathAttribute {
segments: vec![segment],
})
}
/// How long the path is, as RFC 4271 §9.1.2.2 counts: one for each AS
/// of a sequence, one for a whole set.
pub fn hops(&self) -> usize {
self.segments
.iter()
.map(|segment| {
if segment.ordered {
segment.path.len()
} else {
1
}
})
.sum()
}
/// The path as it really is, given what a two-octet session carried:
/// this, with AS_TRANS wherever a number did not fit, and `wide`, the
/// AS4_PATH that has the numbers themselves for the path's tail
/// (RFC 6793 §4.2.3). What `wide` does not cover is kept from the
/// front of this.
///
/// `None` if `wide` is longer than the path, which is not to be used:
/// some speaker on the way that knew nothing of AS4_PATH changed the
/// one and not the other.
pub fn with_as4_path(&self, wide: &AsPathAttribute) -> Option<AsPathAttribute> {
let mut keep = self.hops().checked_sub(wide.hops())?;
let mut segments = Vec::with_capacity(self.segments.len() + wide.segments.len());
for segment in &self.segments {
if keep == 0 {
break;
}
let (hops, taken) = if segment.ordered {
let taken = segment.path.len().min(keep);
(taken, &segment.path[..taken])
} else {
(1, &segment.path[..])
};
keep -= hops;
segments.push(AsPathSegment {
ordered: segment.ordered,
path: taken.to_vec(),
});
}
segments.extend(wide.segments.iter().cloned());
Some(AsPathAttribute { segments })
}
/// Parses an AS_PATH with AS numbers as wide as the session negotiated
/// (RFC 6793).
pub fn from_wire<'a>(ctx: &ParserContext, buf: &'a [u8]) -> Parsed<'a, Self> {
Self::from_wire_with_width(ctx.four_octet_asn, buf)
}
fn from_wire_with_width(four_octet: bool, buf: &[u8]) -> Parsed<'_, Self> {
let parse_segment = |buf| -> Parsed<'_, AsPathSegment> {
let (buf, typ) = be_u8(buf)?;
let (buf, len) = be_u8(buf)?;
// AS_SET and AS_SEQUENCE. The confederation segment types of
// RFC 5065 are an error from a peer outside a confederation,
// and this speaker is never inside one.
if typ != 1 && typ != 2 {
return Err(malformed(format!(
"unrecognised AS path segment type {typ}"
)));
}
if len == 0 {
return Err(malformed("AS path segment with no AS numbers"));
}
let count = usize::from(len);
let (buf, path) =
nom::multi::many_m_n(count, count, |buf| parse_asn(four_octet, buf)).parse(buf)?;
Ok((
buf,
AsPathSegment {
ordered: typ == 2,
path,
},
))
};
let (buf, segments) = list(buf, parse_segment)?;
Ok((buf, Self { segments }))
}
/// Writes the path with AS numbers as wide as the session negotiated.
/// On a two-octet session an AS number that does not fit is written as
/// AS_TRANS (RFC 6793 §4.2.2); [`AsPathAttribute::as4_path`] is what
/// carries the real numbers alongside.
pub fn to_wire(&self, ctx: &ParserContext, out: &mut BytesMut) -> Result<(), EncodeError> {
self.to_wire_with_width(ctx.four_octet_asn, out)
}
fn to_wire_with_width(&self, four_octet: bool, out: &mut BytesMut) -> Result<(), EncodeError> {
for segment in &self.segments {
out.put_u8(if segment.ordered { 2 } else { 1 });
let count = u8::try_from(segment.path.len()).map_err(|_| EncodeError::TooLong {
what: "AS path segment",
len: segment.path.len(),
max: u8::MAX.into(),
})?;
out.put_u8(count);
for asn in &segment.path {
put_asn(four_octet, *asn, out);
}
}
Ok(())
}
/// The AS4_PATH to send with this path on a two-octet session, if one is
/// needed: it is when some AS number does not fit in two octets and so
/// goes out as AS_TRANS in the AS_PATH itself (RFC 6793 §4.2.2).
pub fn as4_path(&self) -> Option<PathAttribute> {
self.segments
.iter()
.flat_map(|segment| &segment.path)
.any(|asn| u16::try_from(*asn).is_err())
.then(|| PathAttribute::As4Path(self.clone()))
}
}
#[derive(Debug, PartialEq, Eq, Clone, Serialize)]
pub struct NextHopPathAttribute(pub Ipv4Addr);
impl NextHopPathAttribute {
pub fn from_wire(buf: &[u8]) -> Parsed<'_, Self> {
let (buf, address) = be_u32(buf)?;
Ok((buf, Self(Ipv4Addr::from(address))))
}
pub fn to_wire(&self, out: &mut BytesMut) {
out.put_u32(self.0.into());
}
}
#[derive(Debug, PartialEq, Eq, Clone, Serialize)]
pub struct MultiExitDiscPathAttribute(pub u32);
impl MultiExitDiscPathAttribute {
pub fn from_wire(buf: &[u8]) -> Parsed<'_, Self> {
let (buf, value) = be_u32(buf)?;
Ok((buf, Self(value)))
}
pub fn to_wire(&self, out: &mut BytesMut) {
out.put_u32(self.0);
}
}
#[derive(Debug, PartialEq, Eq, Clone, Serialize)]
pub struct LocalPrefPathAttribute(pub u32);
impl LocalPrefPathAttribute {
pub fn from_wire(buf: &[u8]) -> Parsed<'_, Self> {
let (buf, value) = be_u32(buf)?;
Ok((buf, Self(value)))
}
pub fn to_wire(&self, out: &mut BytesMut) {
out.put_u32(self.0);
}
}
/// ATOMIC_AGGREGATE: a marker with no value (RFC 4271 §5.1.6).
#[derive(Debug, PartialEq, Eq, Clone, Serialize)]
pub struct AtomicAggregatePathAttribute {}
#[derive(Debug, PartialEq, Eq, Clone, Serialize)]
pub struct AggregatorPathAttribute {
pub asn: u32,
pub ip: Ipv4Addr,
}
impl AggregatorPathAttribute {
/// The AS number is as wide as the session's AS numbers (RFC 6793), so
/// the attribute is six or eight octets.
pub fn from_wire<'a>(ctx: &ParserContext, buf: &'a [u8]) -> Parsed<'a, Self> {
let (buf, asn) = parse_asn(ctx.four_octet_asn, buf)?;
let (buf, ip) = be_u32(buf)?;
Ok((
buf,
Self {
asn,
ip: Ipv4Addr::from(ip),
},
))
}
/// On a two-octet session an AS number that does not fit is written as
/// AS_TRANS (RFC 6793 §4.2.2).
pub fn to_wire(&self, ctx: &ParserContext, out: &mut BytesMut) {
put_asn(ctx.four_octet_asn, self.asn, out);
out.put_u32(self.ip.into());
}
}
#[derive(Debug, PartialEq, Eq, Clone, Serialize)]
pub struct CommunitiesPathAttribute(pub Vec<(u16, u16)>);
impl CommunitiesPathAttribute {
/// Reads all of `buf` as communities, four octets each.
pub fn from_wire(buf: &[u8]) -> Parsed<'_, Self> {
let values = records::<4>(buf, "COMMUNITIES")?
.iter()
.map(|[a, b, c, d]| (u16::from_be_bytes([*a, *b]), u16::from_be_bytes([*c, *d])))
.collect();
Ok((&buf[buf.len()..], CommunitiesPathAttribute(values)))
}
pub fn to_wire(&self, out: &mut BytesMut) {
for value in &self.0 {
out.put_u16(value.0);
out.put_u16(value.1);
}
}
}
/// EXTENDED COMMUNITIES (RFC 4360).
#[derive(Debug, PartialEq, Eq, Clone, Serialize)]
pub struct ExtendedCommunitiesPathAttribute {
pub extended_communities: Vec<ExtendedCommunity>,
}
impl ExtendedCommunitiesPathAttribute {
/// Reads all of `buf` as extended communities, eight octets each.
pub fn from_wire(buf: &[u8]) -> Parsed<'_, Self> {
let communities = records::<8>(buf, "EXTENDED_COMMUNITIES")?;
Ok((
&buf[buf.len()..],
Self {
extended_communities: communities.iter().copied().map(ExtendedCommunity).collect(),
},
))
}
pub fn to_wire(&self, out: &mut BytesMut) {
for community in &self.extended_communities {
out.put_slice(&community.0);
}
}
}
/// One extended community: eight octets, of which the first is the type
/// and, for most types, the second a sub-type (RFC 4360 §2). How the rest
/// divides into an administrator and a value depends on the type, and is
/// left to whoever acts on a particular community.
#[derive(Debug, PartialEq, Eq, Clone, Copy, Serialize)]
pub struct ExtendedCommunity(pub [u8; 8]);
impl ExtendedCommunity {
pub fn typ(&self) -> u8 {
self.0[0]
}
pub fn sub_typ(&self) -> u8 {
self.0[1]
}
}
#[derive(Debug, Clone, Serialize, PartialEq, Eq)]
pub struct LargeCommunitiesPathAttribute {
pub communities: Vec<LargeCommunity>,
}
impl LargeCommunitiesPathAttribute {
/// Reads all of `buf` as large communities, twelve octets each.
pub fn from_wire(buf: &[u8]) -> Parsed<'_, Self> {
let communities = records::<12>(buf, "LARGE_COMMUNITIES")?
.iter()
.map(|community| {
let (_, community) =
LargeCommunity::from_wire(community).expect("twelve octets are one");
community
})
.collect();
Ok((&buf[buf.len()..], Self { communities }))
}
pub fn to_wire(&self, out: &mut BytesMut) {
for community in &self.communities {
community.to_wire(out);
}
}
}
#[derive(Debug, Clone, Serialize, PartialEq, Eq)]
pub struct LargeCommunity {
pub global_admin: u32,
pub data_1: u32,
pub data_2: u32,
}
impl LargeCommunity {
pub fn from_wire(buf: &[u8]) -> Parsed<'_, Self> {
let (buf, global_admin) = be_u32(buf)?;
let (buf, data_1) = be_u32(buf)?;
let (buf, data_2) = be_u32(buf)?;
Ok((
buf,
Self {
global_admin,
data_1,
data_2,
},
))
}
pub fn to_wire(&self, out: &mut BytesMut) {
out.put_u32(self.global_admin);
out.put_u32(self.data_1);
out.put_u32(self.data_2);
}
}
/// The next hop of routes in MP_REACH_NLRI.
#[derive(Debug, Clone, Copy, Serialize, PartialEq, Eq, Hash)]
pub enum NlriNextHop {
Ipv4(Ipv4Addr),
/// A global IPv6 address.
Ipv6(Ipv6Addr),
/// A global IPv6 address and a link-local one (RFC 2545 §3).
Ipv6WithLl {
global: Ipv6Addr,
link_local: Ipv6Addr,
},
}
impl std::fmt::Display for NlriNextHop {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
NlriNextHop::Ipv4(addr) => write!(f, "{addr}"),
NlriNextHop::Ipv6(addr) => write!(f, "{addr}"),
NlriNextHop::Ipv6WithLl { global, link_local } => {
write!(f, "{global} (link-local {link_local})")
}
}
}
}
impl NlriNextHop {
/// Reads a next hop that is all of `bytes`. `None` if they are not the
/// length of one.
pub fn from_bytes(bytes: &[u8]) -> Option<Self> {
let v6 = |bytes: &[u8]| <[u8; 16]>::try_from(bytes).ok().map(Ipv6Addr::from);
match bytes.len() {
4 => <[u8; 4]>::try_from(bytes)
.ok()
.map(|a| Self::Ipv4(a.into())),
16 => v6(bytes).map(Self::Ipv6),
32 => Some(Self::Ipv6WithLl {
global: v6(&bytes[..16])?,
link_local: v6(&bytes[16..])?,
}),
_ => None,
}
}
pub fn to_wire(&self, out: &mut BytesMut) {
match self {
NlriNextHop::Ipv4(address) => out.put_slice(&address.octets()),
NlriNextHop::Ipv6(address) => out.put_slice(&address.octets()),
NlriNextHop::Ipv6WithLl { global, link_local } => {
out.put_slice(&global.octets());
out.put_slice(&link_local.octets());
}
}
}
/// How many octets the next hop is on the wire.
pub fn wire_len(&self) -> u8 {
match self {
NlriNextHop::Ipv4(_) => 4,
NlriNextHop::Ipv6(_) => 16,
NlriNextHop::Ipv6WithLl { .. } => 32,
}
}
}
/// Parses one prefix as an UPDATE names it: with a path identifier in
/// front if the session uses them for the family (RFC 7911 §3).
fn parse_nlri(version: IpVersion, with_path_id: bool, buf: &[u8]) -> Parsed<'_, Nlri> {
let (buf, path_id) = if with_path_id { be_u32(buf)? } else { (buf, 0) };
let (buf, prefix_len) = be_u8(buf)?;
let byte_len = usize::from(prefix_len.div_ceil(8));
let (buf, prefix_bytes) = nom::bytes::take(byte_len).parse(buf)?;
let prefix =
IpPrefix::new(version, prefix_bytes, prefix_len).map_err(|e| malformed(e.to_string()))?;
Ok((buf, Nlri { prefix, path_id }))
}
/// Parses a field that is nothing but prefixes, to its end.
pub(crate) fn parse_nlri_list(
version: IpVersion,
with_path_id: bool,
buf: &[u8],
) -> Parsed<'_, Vec<Nlri>> {
list(buf, |buf| parse_nlri(version, with_path_id, buf))
}
/// Writes prefixes as `parse_nlri_list` reads them: for each, its path
/// identifier if the session uses them for the family (RFC 7911 §3), its
/// length in bits, and the bytes that length covers.
pub fn put_nlri_list(nlri: &[Nlri], with_path_id: bool, out: &mut BytesMut) {
for n in nlri {
if with_path_id {
out.put_u32(n.path_id);
}
out.put_u8(n.prefix.length());
out.put(n.prefix.bytes());
}
}
/// The IP version of the prefixes that make up the NLRI of `family`.
///
/// Any family can be named in MP_REACH_NLRI and MP_UNREACH_NLRI, but only
/// unicast and multicast routes of IPv4 and IPv6 are plain prefixes, and
/// those are the only NLRI this parser reads. A peer does not send routes
/// of a family that was not negotiated, so the others do not come up.
fn ip_version_of_nlri<'a>(
family: AfiSafi,
) -> Result<IpVersion, nom::Err<BgpParserError<&'a [u8]>>> {
let unsupported = || malformed(format!("NLRI of {family} is not supported"));
if family.safi != Safi::UNICAST && family.safi != Safi::MULTICAST {
return Err(unsupported());
}
IpVersion::try_from(family.afi).map_err(|_| unsupported())
}
/// MP_REACH_NLRI: routes of a family other than the one in the message
/// body, with their next hop (RFC 4760 §3).
#[derive(Debug, Clone, Serialize, PartialEq, Eq)]
pub struct MpReachNlriPathAttribute {
pub family: AfiSafi,
pub next_hop: NlriNextHop,
pub prefixes: Vec<Nlri>,
}
impl MpReachNlriPathAttribute {
pub fn from_wire<'a>(ctx: &ParserContext, buf: &'a [u8]) -> Parsed<'a, Self> {
let (buf, family) = parse_afi_safi(buf)?;
let version = ip_version_of_nlri(family)?;
let (buf, next_hop_len) = be_u8(buf)?;
let (buf, next_hop) = nom::bytes::take(usize::from(next_hop_len)).parse(buf)?;
// For IPv6 routes, a global address, optionally followed by a
// link-local one (RFC 2545). For IPv4 routes an IPv4 address, or
// the same as for IPv6: routes over a link that has only IPv6
// addresses (RFC 8950). Which of the two it is shows in its length.
let next_hop = NlriNextHop::from_bytes(next_hop)
.filter(|next_hop| {
version == IpVersion::Ipv4 || !matches!(next_hop, NlriNextHop::Ipv4(_))
})
.ok_or_else(|| {
malformed(format!(
"a next hop of {next_hop_len} octets is not valid for {family}"
))
})?;
// Reserved octet (formerly the SNPA count), for every family.
let (buf, _) = be_u8(buf)?;
let (buf, prefixes) = parse_nlri_list(version, ctx.has_add_path(family), buf)?;
Ok((
buf,
Self {
family,
next_hop,
prefixes,
},
))
}
pub fn to_wire(&self, ctx: &ParserContext, out: &mut BytesMut) {
put_afi_safi(self.family, out);
out.put_u8(self.next_hop.wire_len());
self.next_hop.to_wire(out);
out.put_u8(0);
put_nlri_list(&self.prefixes, ctx.has_add_path(self.family), out);
}
}
/// MP_UNREACH_NLRI: withdrawals of routes of a family other than the one
/// in the message body (RFC 4760 §4).
#[derive(Debug, Clone, Serialize, PartialEq, Eq)]
pub struct MpUnreachNlriPathAttribute {
pub family: AfiSafi,
pub prefixes: Vec<Nlri>,
}
impl MpUnreachNlriPathAttribute {
pub fn from_wire<'a>(ctx: &ParserContext, buf: &'a [u8]) -> Parsed<'a, Self> {
let (buf, family) = parse_afi_safi(buf)?;
let version = ip_version_of_nlri(family)?;
let (buf, prefixes) = parse_nlri_list(version, ctx.has_add_path(family), buf)?;
Ok((buf, Self { family, prefixes }))
}
pub fn to_wire(&self, ctx: &ParserContext, out: &mut BytesMut) {
put_afi_safi(self.family, out);
put_nlri_list(&self.prefixes, ctx.has_add_path(self.family), out);
}
}
crates/packet/src/errors.rs
+1 -6
View File
@@ -128,9 +128,8 @@ pub enum OpenMsgSubcode {
#[error("Unsupported capability")]
UnsupportedCapability = 7,
/// RFC 9234 §4.2.
#[error("Role mismatch")]
RoleMismatch = 11,
RoleMismatch = 8,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Error, Serialize, FromRepr)]
@@ -170,10 +169,6 @@ pub enum UpdateMsgSubcode {
#[derive(Debug, Clone, Copy, PartialEq, Eq, Error, Serialize, FromRepr)]
#[repr(u8)]
pub enum FsmSubcode {
/// RFC 6608: what speakers from before it send, and FRR still does.
#[error("unspecified")]
Unspecified = 0,
#[error("Received an unexpected message in OpenSent state")]
UnexpectedOpenSent = 1,
crates/packet/src/lib.rs
-2
View File
@@ -1,7 +1,5 @@
pub mod attributes;
pub mod community;
pub mod errors;
pub mod message;
pub mod open;
pub mod update_decode;
mod wire;
crates/packet/src/message.rs
+916 -183
View File
File diff suppressed because it is too large. Load diff
crates/packet/src/open.rs
+100 -83
View File
@@ -4,7 +4,7 @@ use bitfield::bitfield;
use bytes::{BufMut, BytesMut};
use nom::{
Err::Failure,
Parser,
IResult, Parser,
number::complete::{be_u8, be_u16, be_u24, be_u32},
};
use serde::Serialize;
@@ -16,8 +16,26 @@ use bgp_primitives::{
parser::{BgpParserError, EncodeError, ProtocolErrorValues},
};
use crate::message::BGP4_VERSION;
use crate::wire::{Parsed, fill_length, list, malformed, parse_afi_safi, put_afi_safi, whole};
type Parsed<'a, T> = IResult<&'a [u8], T, BgpParserError<&'a [u8]>>;
macro_rules! fail {
( $($arg:expr),* $(,)? ) => {
Failure(BgpParserError::Malformed(format!($($arg),*)))
};
}
/// Parses `buf` to its end as a list of `item`s. Unlike nom's `many0`,
/// which stops quietly at an item it cannot read, an error in an item is an
/// error of the list.
fn list<'a, T>(mut buf: &'a [u8], item: impl Fn(&'a [u8]) -> Parsed<'a, T>) -> Parsed<'a, Vec<T>> {
let mut items = Vec::new();
while !buf.is_empty() {
let (rest, parsed) = item(buf)?;
items.push(parsed);
buf = rest;
}
Ok((buf, items))
}
/// The optional parameter type that marks an OPEN whose optional parameters
/// have two-octet lengths (RFC 9072).
@@ -44,7 +62,7 @@ impl OpenMessage {
/// in a well-formed OPEN is nothing.
pub fn from_wire(buf: &[u8]) -> Parsed<'_, Self> {
let (buf, version) = be_u8(buf)?;
if version != BGP4_VERSION {
if version != 4 {
return Err(Failure(BgpParserError::ProtocolError(
ProtocolErrorValues::UnsupportedVersion(version),
)));
@@ -237,7 +255,7 @@ pub enum Capability {
OutboundRouteFilter(Vec<OrfFamily>),
/// Extended Next Hop encoding (RFC 8950).
ExtendedNextHop {
entries: Vec<ExtendedNextHopEntry>,
entries: Vec<ExtendedNextHop>,
},
/// Extended Message (RFC 8654).
ExtendedMessage,
@@ -248,7 +266,7 @@ pub enum Capability {
},
/// Multiple labels (RFC 8277 §2.1): for each address family, how many
/// labels the speaker can handle in one NLRI.
MultiLabel {
MultiLabelCompat {
entries: Vec<MultiLabelEntry>,
},
BgpRole {
@@ -262,9 +280,6 @@ pub enum Capability {
/// The estimated time in seconds for the session to be
/// re-established after a restart.
restart_time: u16,
/// The speaker applies graceful restart to sessions that end with
/// a NOTIFICATION too, other than a Hard Reset (RFC 8538).
notification: bool,
/// The address families whose routes are kept over a restart.
entries: Vec<GracefulRestartEntry>,
},
@@ -280,10 +295,10 @@ pub enum Capability {
/// Enhanced Route Refresh (RFC 7313).
EnhancedRouteRefresh,
/// Long Lived Graceful Restart (RFC 9494).
LongLivedGracefulRestart(Vec<LongLivedGracefulRestartEntry>),
LongLivedGracefulRestart(Vec<LongLivedGracefulRestart>),
/// A capability this implementation has no definition for, kept as it
/// came (RFC 5492 §3: what is not understood is ignored).
Unknown {
UnknownCapability {
type_code: u8,
value: Vec<u8>,
},
@@ -298,6 +313,12 @@ fn afi_reserved_safi(buf: &[u8]) -> Parsed<'_, (Afi, Safi)> {
Ok((buf, (Afi(afi), Safi(safi))))
}
fn afi_safi(buf: &[u8]) -> Parsed<'_, (Afi, Safi)> {
let (buf, afi) = be_u16(buf)?;
let (buf, safi) = be_u8(buf)?;
Ok((buf, (Afi(afi), Safi(safi))))
}
impl Capability {
fn type_code(&self) -> u8 {
use cap_type::*;
@@ -308,14 +329,14 @@ impl Capability {
Self::ExtendedNextHop { .. } => EXTENDED_NEXT_HOP,
Self::ExtendedMessage => EXTENDED_MESSAGE,
Self::BgpSec { .. } => BGP_SEC,
Self::MultiLabel { .. } => MULTI_LABEL,
Self::MultiLabelCompat { .. } => MULTI_LABEL,
Self::BgpRole { .. } => BGP_ROLE,
Self::GracefulRestart { .. } => GRACEFUL_RESTART,
Self::FourByteAsn { .. } => FOUR_BYTE_ASN,
Self::AddPath { .. } => ADD_PATH,
Self::EnhancedRouteRefresh => ENHANCED_ROUTE_REFRESH,
Self::LongLivedGracefulRestart(_) => LONG_LIVED_GRACEFUL_RESTART,
Self::Unknown { type_code, .. } => *type_code,
Self::UnknownCapability { type_code, .. } => *type_code,
}
}
@@ -363,10 +384,10 @@ impl Capability {
// else: no SAFI is larger than 255.
let safi = u8::try_from(safi)
.map(Safi)
.map_err(|_| malformed(format!("SAFI {safi} does not fit in one octet")))?;
.map_err(|_| fail!("SAFI {} does not fit in one octet", safi))?;
Ok((
buf,
ExtendedNextHopEntry {
ExtendedNextHop {
afi: Afi(afi),
safi,
nh_afi: Afi(nh_afi),
@@ -374,7 +395,7 @@ impl Capability {
))
})?;
if entries.is_empty() {
return Err(malformed("Extended Next Hop capability with no entries"));
return Err(fail!("Extended Next Hop capability with no entries"));
}
(rest, Self::ExtendedNextHop { entries })
}
@@ -392,14 +413,14 @@ impl Capability {
}
MULTI_LABEL => {
let (rest, entries) = list(value, |buf| {
let (buf, AfiSafi { afi, safi }) = parse_afi_safi(buf)?;
let (buf, (afi, safi)) = afi_safi(buf)?;
let (buf, count) = be_u8(buf)?;
Ok((buf, MultiLabelEntry { afi, safi, count }))
})?;
if entries.is_empty() {
return Err(malformed("Multiple Labels capability with no entries"));
return Err(fail!("Multiple Labels capability with no entries"));
}
(rest, Self::MultiLabel { entries })
(rest, Self::MultiLabelCompat { entries })
}
BGP_ROLE => {
let (rest, role) = be_u8(value)?;
@@ -414,7 +435,7 @@ impl Capability {
// Four bits of flags and twelve of restart time.
let (rest, flags_restart_time) = be_u16(value)?;
let (rest, entries) = list(rest, |buf| {
let (buf, AfiSafi { afi, safi }) = parse_afi_safi(buf)?;
let (buf, (afi, safi)) = afi_safi(buf)?;
let (buf, flags) = be_u8(buf)?;
Ok((
buf,
@@ -429,7 +450,6 @@ impl Capability {
rest,
Self::GracefulRestart {
restart_state: flags_restart_time & (1 << 15) != 0,
notification: flags_restart_time & (1 << 14) != 0,
restart_time: flags_restart_time & 0xfff,
entries,
},
@@ -452,7 +472,7 @@ impl Capability {
Some(AddPathEntry {
afi: Afi(u16::from_be_bytes([entry[0], entry[1]])),
safi: Safi(entry[2]),
send_recv: SendReceive::try_from(entry[3]).ok()?,
send_recv: AddPathSendRecv::try_from(entry[3]).ok()?,
})
})
.collect();
@@ -461,12 +481,12 @@ impl Capability {
ENHANCED_ROUTE_REFRESH => (value, Self::EnhancedRouteRefresh),
LONG_LIVED_GRACEFUL_RESTART => {
let (rest, entries) = list(value, |buf| {
let (buf, AfiSafi { afi, safi }) = parse_afi_safi(buf)?;
let (buf, (afi, safi)) = afi_safi(buf)?;
let (buf, flags) = be_u8(buf)?;
let (buf, stale_time) = be_u24(buf)?;
Ok((
buf,
LongLivedGracefulRestartEntry {
LongLivedGracefulRestart {
afi,
safi,
flags,
@@ -478,13 +498,19 @@ impl Capability {
}
type_code => (
&[][..],
Self::Unknown {
Self::UnknownCapability {
type_code,
value: value.to_vec(),
},
),
};
whole(format_args!("capability {type_code}"), (rest, capability))
if !rest.is_empty() {
return Err(fail!(
"{} octets left over in capability {type_code}",
rest.len()
));
}
Ok(capability)
}
/// Writes the capability, with its type and length, to the end of
@@ -495,7 +521,13 @@ impl Capability {
let length_at = out.len();
out.put_u8(0);
self.value_to_wire(out)?;
fill_length(out, length_at, 1, u8::MAX.into(), "capability")
let length = out.len() - length_at - 1;
out[length_at] = u8::try_from(length).map_err(|_| EncodeError::TooLong {
what: "capability",
len: length,
max: u8::MAX.into(),
})?;
Ok(())
}
fn value_to_wire(&self, out: &mut BytesMut) -> Result<(), EncodeError> {
@@ -524,9 +556,10 @@ impl Capability {
out.put_u8(header.0);
out.put_u16(afi.0);
}
Capability::MultiLabel { entries } => {
Capability::MultiLabelCompat { entries } => {
for entry in entries {
put_afi_safi(AfiSafi::new(entry.afi, entry.safi), out);
out.put_u16(entry.afi.0);
out.put_u8(entry.safi.0);
out.put_u8(entry.count);
}
}
@@ -534,7 +567,6 @@ impl Capability {
Capability::GracefulRestart {
restart_state,
restart_time,
notification,
entries,
} => {
if *restart_time > 0xfff {
@@ -544,29 +576,30 @@ impl Capability {
max: 0xfff,
});
}
out.put_u16(
u16::from(*restart_state) << 15 | u16::from(*notification) << 14 | restart_time,
);
out.put_u16(u16::from(*restart_state) << 15 | restart_time);
for entry in entries {
put_afi_safi(AfiSafi::new(entry.afi, entry.safi), out);
out.put_u16(entry.afi.0);
out.put_u8(entry.safi.0);
out.put_u8(u8::from(entry.preserve_forwarding) << 7);
}
}
Capability::FourByteAsn { asn } => out.put_u32(*asn),
Capability::AddPath { entries } => {
for entry in entries {
put_afi_safi(entry.family(), out);
out.put_u16(entry.afi.0);
out.put_u8(entry.safi.0);
out.put_u8(entry.send_recv as u8);
}
}
Capability::LongLivedGracefulRestart(entries) => {
for entry in entries {
put_afi_safi(AfiSafi::new(entry.afi, entry.safi), out);
out.put_u16(entry.afi.0);
out.put_u8(entry.safi.0);
out.put_u8(entry.flags);
out.put_slice(&entry.stale_time.to_be_bytes()[1..]);
}
}
Capability::Unknown { value, .. } => out.put_slice(value),
Capability::UnknownCapability { value, .. } => out.put_slice(value),
}
Ok(())
}
@@ -589,8 +622,8 @@ impl OrfFamily {
for _ in 0..count {
let (rest, orf_type) = be_u8(buf)?;
let (rest, send_recv) = be_u8(rest)?;
let send_recv = SendReceive::try_from(send_recv)
.map_err(|e| malformed(format!("Failed to parse ORF send/recv: {e}")))?;
let send_recv = OrfSendRecv::try_from(send_recv)
.map_err(|e| fail!("Failed to parse ORF send/recv: {}", e))?;
orfs.push(OutboundRouteFilterEntry {
r#type: OrfType(orf_type),
send_recv,
@@ -622,7 +655,7 @@ impl OrfFamily {
#[derive(Debug, Clone, Serialize, PartialEq, Eq)]
pub struct OutboundRouteFilterEntry {
pub r#type: OrfType,
pub send_recv: SendReceive,
pub send_recv: OrfSendRecv,
}
/// An ORF type. Any number is one: most are unassigned or vendor-specific.
@@ -647,6 +680,9 @@ pub enum SendReceive {
Both = 3,
}
pub type OrfSendRecv = SendReceive;
pub type AddPathSendRecv = SendReceive;
impl SendReceive {
/// Whether the speaker that advertised this sends.
pub fn sends(self) -> bool {
@@ -680,7 +716,7 @@ impl TryFrom<u8> for SendReceive {
pub struct AddPathEntry {
pub afi: Afi,
pub safi: Safi,
pub send_recv: SendReceive,
pub send_recv: AddPathSendRecv,
}
impl AddPathEntry {
@@ -689,11 +725,11 @@ impl AddPathEntry {
}
}
/// `ExtendedNextHopEntry` allows for the nexthop to be for a different address family
/// `ExtendedNextHop` allows for the nexthop to be for a different address family
/// than the one being used to advertise prefixes for. E.g. it allows for sending
/// IPv4 prefixes with an IPv6 nexthop.
#[derive(Debug, Clone, Serialize, PartialEq, Eq)]
pub struct ExtendedNextHopEntry {
pub struct ExtendedNextHop {
/// The Address Family to update the nexthop format for.
pub afi: Afi,
/// The Subsequent Address Family to update the nexthop format for.
@@ -740,7 +776,7 @@ impl From<&BgpRole> for u8 {
}
#[derive(Debug, Clone, Serialize, PartialEq, Eq)]
pub struct LongLivedGracefulRestartEntry {
pub struct LongLivedGracefulRestart {
pub afi: Afi,
pub safi: Safi,
pub flags: u8,
@@ -777,8 +813,9 @@ mod tests {
use eyre::Result;
use crate::open::{
AddPathEntry, Capability, ExtendedNextHopEntry, GracefulRestartEntry, MultiLabelEntry,
OpenMessage, OpenOption, OrfFamily, OrfType, OutboundRouteFilterEntry, SendReceive,
AddPathEntry, AddPathSendRecv, Capability, ExtendedNextHop, GracefulRestartEntry,
MultiLabelEntry, OpenMessage, OpenOption, OrfFamily, OrfSendRecv, OrfType,
OutboundRouteFilterEntry,
};
use bgp_primitives::constants::{Afi, Safi};
@@ -799,7 +836,7 @@ mod tests {
}
test_capability_roundtrip!(
test_multiprotocol,
test_multiptotocol,
&[0x01, 0x04, 0x00, 0x01, 0x00, 0x01],
Capability::MultiProtocol {
afi: Afi::IPV4,
@@ -812,7 +849,7 @@ mod tests {
test_capability_roundtrip!(
test_route_refresh_cisco_unsupported,
&[0x80, 0x00],
Capability::Unknown {
Capability::UnknownCapability {
type_code: 0x80,
value: vec![]
}
@@ -830,7 +867,6 @@ mod tests {
Capability::GracefulRestart {
restart_state: false,
restart_time: 300,
notification: false,
entries: vec![GracefulRestartEntry {
afi: Afi::IPV4,
safi: Safi::UNICAST,
@@ -839,23 +875,6 @@ mod tests {
}
);
// Restarted (R), with Notification support (N, RFC 8538), and the
// forwarding of IPv6 routes kept (F).
test_capability_roundtrip!(
test_graceful_restart_with_flags,
&[0x40, 0x06, 0xc0, 0x78, 0x00, 0x02, 0x01, 0x80],
Capability::GracefulRestart {
restart_state: true,
restart_time: 120,
notification: true,
entries: vec![GracefulRestartEntry {
afi: Afi::IPV6,
safi: Safi::UNICAST,
preserve_forwarding: true,
}]
}
);
test_capability_roundtrip!(
test_extended_nexthop,
&[
@@ -864,17 +883,17 @@ mod tests {
],
Capability::ExtendedNextHop {
entries: vec![
ExtendedNextHopEntry {
ExtendedNextHop {
afi: Afi::IPV4,
safi: Safi::UNICAST,
nh_afi: Afi::IPV6,
},
ExtendedNextHopEntry {
ExtendedNextHop {
afi: Afi::IPV4,
safi: Safi::MULTICAST,
nh_afi: Afi::IPV6,
},
ExtendedNextHopEntry {
ExtendedNextHop {
afi: Afi::IPV4,
safi: Safi::MPLS_VPN,
nh_afi: Afi::IPV6
@@ -902,7 +921,7 @@ mod tests {
afi: Afi::IPV4,
safi: Safi::UNICAST,
}]),
OpenOption::Capabilities(vec![Capability::Unknown {
OpenOption::Capabilities(vec![Capability::UnknownCapability {
type_code: 0x80,
value: vec![],
}]),
@@ -911,7 +930,6 @@ mod tests {
OpenOption::Capabilities(vec![Capability::GracefulRestart {
restart_state: false,
restart_time: 300,
notification: false,
entries: vec![GracefulRestartEntry {
afi: Afi::IPV4,
safi: Safi::UNICAST,
@@ -920,17 +938,17 @@ mod tests {
}]),
OpenOption::Capabilities(vec![Capability::ExtendedNextHop {
entries: vec![
ExtendedNextHopEntry {
ExtendedNextHop {
afi: Afi::IPV4,
safi: Safi::UNICAST,
nh_afi: Afi::IPV6,
},
ExtendedNextHopEntry {
ExtendedNextHop {
afi: Afi::IPV4,
safi: Safi::MULTICAST,
nh_afi: Afi::IPV6,
},
ExtendedNextHopEntry {
ExtendedNextHop {
afi: Afi::IPV4,
safi: Safi::MPLS_VPN,
nh_afi: Afi::IPV6,
@@ -958,7 +976,7 @@ mod tests {
assert!(rest.is_empty());
assert_eq!(
cap,
Capability::MultiLabel {
Capability::MultiLabelCompat {
entries: vec![
MultiLabelEntry {
afi: Afi::IPV4,
@@ -1001,7 +1019,7 @@ mod tests {
entries: vec![AddPathEntry {
afi: evpn.0,
safi: evpn.1,
send_recv: SendReceive::Both,
send_recv: AddPathSendRecv::Both,
}],
},
),
@@ -1010,7 +1028,6 @@ mod tests {
Capability::GracefulRestart {
restart_state: false,
restart_time: 120,
notification: false,
entries: vec![GracefulRestartEntry {
afi: evpn.0,
safi: evpn.1,
@@ -1050,12 +1067,12 @@ mod tests {
AddPathEntry {
afi: Afi::IPV4,
safi: Safi::UNICAST,
send_recv: SendReceive::Receive,
send_recv: AddPathSendRecv::Receive,
},
AddPathEntry {
afi: Afi::IPV6,
safi: Safi::UNICAST,
send_recv: SendReceive::Both,
send_recv: AddPathSendRecv::Both,
},
],
}
@@ -1115,8 +1132,8 @@ mod tests {
assert_eq!(
cap,
Capability::OutboundRouteFilter(vec![
family(Afi::IPV4, SendReceive::Both),
family(Afi::IPV6, SendReceive::Receive),
family(Afi::IPV4, OrfSendRecv::Both),
family(Afi::IPV6, OrfSendRecv::Receive),
])
);
let mut out = BytesMut::new();
@@ -1132,7 +1149,7 @@ mod tests {
#[test]
fn long_optional_parameters_use_two_octet_lengths() {
let capabilities: Vec<Capability> = (0..60)
.map(|i| Capability::Unknown {
.map(|i| Capability::UnknownCapability {
type_code: 200,
value: vec![i; 4],
})
@@ -1190,7 +1207,7 @@ mod tests {
/// used to be written with the length wrapped around.
#[test]
fn a_capability_too_long_for_its_length_octet_is_refused() {
let entry = ExtendedNextHopEntry {
let entry = ExtendedNextHop {
afi: Afi::IPV4,
safi: Safi::UNICAST,
nh_afi: Afi::IPV6,
crates/packet/src/update_decode.rs
+30 -241
View File
@@ -19,7 +19,7 @@
//! be acted on, with treat-as-withdraw and attribute discard already carried
//! out, together with a report of what was wrong, for the log.
use bgp_primitives::constants::IpVersion;
use bgp_primitives::constants::{AfiSafi, IpVersion};
use bgp_primitives::ip_prefix::Nlri;
use bgp_primitives::parser::ParserContext;
@@ -29,7 +29,6 @@ use crate::message::{
MpUnreachNlriPathAttribute, NotAnAttribute, PathAttribute, RawAttribute, UpdateMessage,
defined_flags, describe, parse_nlri_list, split_attribute,
};
use crate::wire::body_has_path_id;
/// How an error in an UPDATE was handled, short of resetting the session.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
@@ -63,9 +62,7 @@ pub struct UpdateError {
pub struct DecodedUpdate {
/// The UPDATE after treat-as-withdraw and attribute discard.
pub update: UpdateMessage,
/// What was wrong, in the order met; empty when the UPDATE was well
/// formed. Where a message of any type was decoded this is
/// [`Decoded::update_errors`](crate::message::Decoded::update_errors).
/// Empty when the UPDATE was well formed.
pub errors: Vec<UpdateError>,
/// The attributes in `update`, as they were on the wire, for a table
/// that keeps attributes in that form and need not write them out
@@ -105,7 +102,9 @@ fn reset(subcode: UpdateMsgSubcode, data: Vec<u8>, detail: String) -> DecodeErro
/// error here cannot be recovered from: the prefixes are what
/// treat-as-withdraw would act on.
fn prefixes(ctx: &ParserContext, field: &str, buf: &[u8]) -> Result<Vec<Nlri>, DecodeError> {
match parse_nlri_list(IpVersion::Ipv4, body_has_path_id(ctx), buf) {
// The prefixes in the body are IPv4 unicast (RFC 4271 §4.3).
let with_path_id = ctx.has_add_path(AfiSafi::IPV4_UNICAST);
match parse_nlri_list(IpVersion::Ipv4, with_path_id, buf) {
Ok((_, prefixes)) => Ok(prefixes),
Err(e) => Err(reset(
UpdateMsgSubcode::InvalidNetworkField,
@@ -186,36 +185,6 @@ impl UpdateMessage {
}
}
/// Makes one path of the two a session with two-octet AS numbers brings:
/// AS_PATH becomes the path as it really is, and AS4_PATH is gone, used or
/// not (RFC 6793 §4.2.3).
///
/// For whoever receives an UPDATE to do once, before anything looks at
/// the path: the checks on it, the choice between routes, validation
/// against the RPKI now and again later. Were each to read the two for
/// itself, a peer could have them disagree about where a route comes
/// from.
pub fn merge_as4_path(attributes: &mut Vec<PathAttribute>) {
let Some(at) = attributes
.iter()
.position(|attribute| matches!(attribute, PathAttribute::As4Path(_)))
else {
return;
};
let wide = match attributes.remove(at) {
PathAttribute::As4Path(wide) => wide,
other => unreachable!("{other:?} is not the AS4_PATH found at {at}"),
};
for attribute in attributes {
if let PathAttribute::AsPath(path) = attribute {
if let Some(merged) = path.with_as4_path(&wide) {
*path = merged;
}
return;
}
}
}
/// What to do with an attribute, once it is clear.
enum Verdict {
/// Use it. `route_property` is false for one that was read only to find
@@ -235,10 +204,7 @@ struct AttributeReader<'a> {
attributes: Vec<PathAttribute>,
errors: Vec<UpdateError>,
/// The bytes of the attributes that are kept and are a property of a
/// route: all but MP_REACH_NLRI and MP_UNREACH_NLRI. Left empty on a
/// session with two-octet AS numbers, whose bytes no table keeps (see
/// [`DecodedUpdate::attributes_wire`]), so that nothing is copied or
/// reserved for them.
/// route: all but MP_REACH_NLRI and MP_UNREACH_NLRI.
kept_wire: Vec<u8>,
/// The type codes met so far, one bit each.
seen: [u64; 4],
@@ -250,7 +216,7 @@ impl<'a> AttributeReader<'a> {
ctx,
attributes: Vec::new(),
errors: Vec::new(),
kept_wire: Vec::with_capacity(if ctx.four_octet_asn { wire_len } else { 0 }),
kept_wire: Vec::with_capacity(wire_len),
seen: [0; 4],
}
}
@@ -295,7 +261,7 @@ impl<'a> AttributeReader<'a> {
route_property,
} = self.judge(&raw)?
{
if route_property && self.ctx.four_octet_asn {
if route_property {
self.kept_wire.extend_from_slice(raw.wire);
}
self.attributes.push(attribute);
@@ -411,6 +377,8 @@ impl<'a> AttributeReader<'a> {
// Known by their definition but not modelled: kept as bytes, once
// their length has been checked.
let length_is_valid = match raw.type_code {
ORIGINATOR_ID => Some(value.len() == 4),
CLUSTER_LIST => Some(!value.is_empty() && value.len().is_multiple_of(4)),
AS4_AGGREGATOR => Some(value.len() == 8),
_ => None,
};
@@ -425,7 +393,7 @@ impl<'a> AttributeReader<'a> {
/// The attribute kept as bytes.
fn opaque(raw: &RawAttribute<'_>) -> PathAttribute {
PathAttribute::Unknown {
PathAttribute::UnknownPathAttribute {
flags: raw.flags.clone(),
type_code: raw.type_code,
payload: raw.value.to_vec(),
@@ -498,7 +466,6 @@ mod tests {
use std::net::Ipv4Addr;
use crate::message::PathAttributeFlags;
use bgp_primitives::constants::AfiSafi;
use bgp_primitives::ip_prefix::IpPrefix;
use super::*;
@@ -507,6 +474,11 @@ mod tests {
OriginPathAttribute,
};
const MED: u8 = 4;
const COMMUNITIES: u8 = 8;
const EXTENDED_COMMUNITIES: u8 = 16;
const LARGE_COMMUNITY: u8 = 32;
const WELL_KNOWN: u8 = 0x40;
const OPTIONAL: u8 = 0x80;
const OPTIONAL_TRANSITIVE: u8 = 0xc0;
@@ -602,7 +574,7 @@ mod tests {
#[test]
fn a_well_formed_update_is_left_alone() {
let decoded = announce_with(&ebgp(), attr(OPTIONAL, MULTI_EXIT_DISC, &[0, 0, 0, 50]));
let decoded = announce_with(&ebgp(), attr(OPTIONAL, MED, &[0, 0, 0, 50]));
assert!(decoded.errors.is_empty());
assert_eq!(decoded.update.announced_routes, [route()]);
assert_eq!(
@@ -632,7 +604,7 @@ mod tests {
// Nothing when the routes were withdrawn instead: there is no route
// for the attributes to belong to.
let decoded = announce_with(&ebgp(), attr(OPTIONAL, MULTI_EXIT_DISC, &[1]));
let decoded = announce_with(&ebgp(), attr(OPTIONAL, MED, &[1]));
assert_eq!(decoded.attributes_wire, None);
let decoded = UpdateMessage::decode(&ebgp(), &body(&NLRI, &[], &[])).unwrap();
assert_eq!(decoded.attributes_wire, None);
@@ -654,31 +626,6 @@ mod tests {
assert_eq!(decoded.attributes_wire, None);
}
#[test]
fn only_to_customer_is_read_and_kept_as_it_came() {
let wire = crate::message::only_to_customer_wire(3333);
assert_eq!(
wire.to_vec(),
attr(OPTIONAL_TRANSITIVE, ONLY_TO_CUSTOMER, &[0, 0, 0x0d, 0x05])
);
let decoded = announce_with(&ebgp(), wire.to_vec());
assert!(decoded.errors.is_empty());
assert_eq!(
decoded.update.path_attributes.last(),
Some(&PathAttribute::OnlyToCustomer(3333))
);
assert_eq!(
decoded.attributes_wire,
Some([mandatory().concat(), wire.to_vec()].concat())
);
// And is written as it is read.
let mut out = bytes::BytesMut::new();
PathAttribute::OnlyToCustomer(3333)
.to_wire(&ebgp(), &mut out)
.unwrap();
assert_eq!(out[..], wire);
}
#[test]
fn an_update_that_announces_nothing_needs_no_attributes() {
// End-of-RIB, and a plain withdrawal.
@@ -691,11 +638,7 @@ mod tests {
#[test]
fn a_malformed_attribute_withdraws_the_routes_and_keeps_the_session() {
let cases: Vec<(&str, u8, Vec<u8>)> = vec![
(
"MED of three octets",
MULTI_EXIT_DISC,
attr(OPTIONAL, MULTI_EXIT_DISC, &[0, 0, 50]),
),
("MED of three octets", MED, attr(OPTIONAL, MED, &[0, 0, 50])),
(
"COMMUNITIES with nothing in it",
COMMUNITIES,
@@ -713,24 +656,13 @@ mod tests {
),
(
"LARGE_COMMUNITY of eleven octets",
LARGE_COMMUNITIES,
attr(OPTIONAL_TRANSITIVE, LARGE_COMMUNITIES, &[0; 11]),
LARGE_COMMUNITY,
attr(OPTIONAL_TRANSITIVE, LARGE_COMMUNITY, &[0; 11]),
),
(
"MED marked transitive",
MULTI_EXIT_DISC,
attr(OPTIONAL_TRANSITIVE, MULTI_EXIT_DISC, &[0, 0, 0, 50]),
),
// RFC 9234 §5.
(
"Only to Customer of two octets",
ONLY_TO_CUSTOMER,
attr(OPTIONAL_TRANSITIVE, ONLY_TO_CUSTOMER, &[0xfd, 0xe8]),
),
(
"Only to Customer of eight octets",
ONLY_TO_CUSTOMER,
attr(OPTIONAL_TRANSITIVE, ONLY_TO_CUSTOMER, &[0; 8]),
MED,
attr(OPTIONAL_TRANSITIVE, MED, &[0, 0, 0, 50]),
),
];
for (case, type_code, extra) in cases {
@@ -807,13 +739,13 @@ mod tests {
fn an_attribute_that_overruns_the_attributes_withdraws_what_was_found() {
// A MED that claims 200 octets.
let mut attributes = mandatory();
attributes.push(vec![OPTIONAL, MULTI_EXIT_DISC, 200, 0, 0]);
attributes.push(vec![OPTIONAL, MED, 200, 0, 0]);
let decoded = UpdateMessage::decode(&ebgp(), &body(&[], &attributes, &NLRI)).unwrap();
assert_withdrawn(&decoded, Some(MULTI_EXIT_DISC), "overrun");
assert_withdrawn(&decoded, Some(MED), "overrun");
// Two stray octets after the last attribute: not even a header.
let mut attributes = mandatory();
attributes.push(vec![OPTIONAL, MULTI_EXIT_DISC]);
attributes.push(vec![OPTIONAL, MED]);
let decoded = UpdateMessage::decode(&ebgp(), &body(&[], &attributes, &NLRI)).unwrap();
assert_withdrawn(&decoded, None, "stray octets");
}
@@ -864,13 +796,10 @@ mod tests {
#[test]
fn a_repeated_attribute_keeps_its_first_value() {
let mut attributes = mandatory();
attributes.push(attr(OPTIONAL, MULTI_EXIT_DISC, &[0, 0, 0, 50]));
attributes.push(attr(OPTIONAL, MULTI_EXIT_DISC, &[0, 0, 0, 99]));
attributes.push(attr(OPTIONAL, MED, &[0, 0, 0, 50]));
attributes.push(attr(OPTIONAL, MED, &[0, 0, 0, 99]));
let decoded = UpdateMessage::decode(&ebgp(), &body(&[], &attributes, &NLRI)).unwrap();
assert_eq!(
handled(&decoded),
[(Handling::AttributeDiscard, Some(MULTI_EXIT_DISC))]
);
assert_eq!(handled(&decoded), [(Handling::AttributeDiscard, Some(MED))]);
assert!(
decoded
.update
@@ -904,21 +833,6 @@ mod tests {
let reflected = UpdateMessage::decode(&ibgp(), &body(&[], &attributes, &NLRI)).unwrap();
assert!(reflected.errors.is_empty());
assert_eq!(reflected.update.path_attributes.len(), 6);
assert!(
reflected
.update
.path_attributes
.contains(&PathAttribute::OriginatorId(Ipv4Addr::new(10, 0, 0, 1)))
);
assert!(
reflected
.update
.path_attributes
.contains(&PathAttribute::ClusterList(vec![
Ipv4Addr::new(10, 0, 0, 2),
Ipv4Addr::new(10, 0, 0, 3)
]))
);
let mut attributes = mandatory();
attributes.push(local_pref());
@@ -957,18 +871,6 @@ mod tests {
.iter()
.any(|a| matches!(a, PathAttribute::As4Path(_)))
);
// The two made one: AS 4200000001 where AS_PATH had AS_TRANS, and
// no AS4_PATH.
let mut one_path = decoded.update.path_attributes;
merge_as4_path(&mut one_path);
assert_eq!(
one_path,
[
PathAttribute::Origin(OriginPathAttribute::IGP),
AsPathAttribute::from_asns(vec![4_200_000_001]),
PathAttribute::NextHop(NextHopPathAttribute(Ipv4Addr::new(192, 0, 2, 1))),
]
);
let bad = attr(OPTIONAL_TRANSITIVE, AS4_PATH, &[2, 1, 0xfa, 0x56]);
let decoded = UpdateMessage::decode(&ctx, &body(&[], &attributes(bad), &NLRI)).unwrap();
@@ -979,119 +881,6 @@ mod tests {
assert_eq!(decoded.update.announced_routes, [route()]);
}
/// AS4_AGGREGATOR (RFC 6793 §4.2.2) is kept as it came, once its
/// length is known to be the four octets of an AS number and the four
/// of an address. One of another length is dropped and the route kept
/// (RFC 6793 §6), like a malformed AGGREGATOR.
#[test]
fn as4_aggregator_is_kept_opaque_and_dropped_if_malformed() {
let ctx = ParserContext {
four_octet_asn: false,
ibgp: false,
add_path: Vec::new(),
};
let attributes = |as4_aggregator: Vec<u8>| {
vec![
origin(),
attr(WELL_KNOWN, AS_PATH, &[2, 1, 0x5b, 0xa0]),
next_hop(),
as4_aggregator,
]
};
// AS 4200000001, 192.0.2.9.
let value = [0xfa, 0x56, 0xea, 0x01, 192, 0, 2, 9];
let good = attr(OPTIONAL_TRANSITIVE, AS4_AGGREGATOR, &value);
let decoded = UpdateMessage::decode(&ctx, &body(&[], &attributes(good), &NLRI)).unwrap();
assert!(decoded.errors.is_empty());
assert_eq!(
decoded.update.path_attributes.last(),
Some(&PathAttribute::Unknown {
flags: PathAttributeFlags(OPTIONAL_TRANSITIVE),
type_code: AS4_AGGREGATOR,
payload: value.to_vec(),
})
);
for (case, bad) in [
("two-octet AS number", &value[2..]),
("no address", &value[..4]),
("nothing", &[][..]),
] {
let bad = attr(OPTIONAL_TRANSITIVE, AS4_AGGREGATOR, bad);
let decoded = UpdateMessage::decode(&ctx, &body(&[], &attributes(bad), &NLRI)).unwrap();
assert_eq!(
handled(&decoded),
[(Handling::AttributeDiscard, Some(AS4_AGGREGATOR))],
"{case}"
);
assert_eq!(decoded.update.announced_routes, [route()], "{case}");
assert_eq!(decoded.update.path_attributes.len(), 3, "{case}");
}
}
/// RFC 6793 §4.2.3. AS4_PATH is the tail of the path; the front is
/// kept from AS_PATH. One longer than the path is not used, and is
/// gone all the same: nothing after the decoder is to see two paths,
/// or a peer could have them disagree about a route's origin.
#[test]
fn as4_path_is_merged_into_the_path() {
use crate::message::AsPathSegment;
let sequence = |path: &[u32]| AsPathSegment {
ordered: true,
path: path.to_vec(),
};
let set = |path: &[u32]| AsPathSegment {
ordered: false,
path: path.to_vec(),
};
let merged = |narrow: Vec<AsPathSegment>, wide: Vec<AsPathSegment>| {
let mut attributes = vec![
PathAttribute::Origin(OriginPathAttribute::IGP),
PathAttribute::As4Path(AsPathAttribute { segments: wide }),
PathAttribute::AsPath(AsPathAttribute { segments: narrow }),
];
merge_as4_path(&mut attributes);
assert_eq!(attributes.len(), 2, "AS4_PATH is gone");
match attributes.pop() {
Some(PathAttribute::AsPath(path)) => path.segments,
other => panic!("{other:?}"),
}
};
assert_eq!(
merged(
vec![sequence(&[3333, 23456, 23456])],
vec![sequence(&[210_036, 4_200_000_001])]
),
[sequence(&[3333]), sequence(&[210_036, 4_200_000_001])]
);
// As long as the path: all of it.
assert_eq!(
merged(vec![sequence(&[23456])], vec![sequence(&[210_036])]),
[sequence(&[210_036])]
);
// Longer than the path: not used. The hijack this would allow is
// a path [3333] that says it ends at someone else's AS.
assert_eq!(
merged(vec![sequence(&[3333])], vec![sequence(&[64500, 1299])]),
[sequence(&[3333])]
);
// A set counts as one, whole.
assert_eq!(
merged(
vec![sequence(&[3333]), set(&[1, 2]), sequence(&[23456])],
vec![sequence(&[210_036])]
),
[sequence(&[3333]), set(&[1, 2]), sequence(&[210_036])]
);
assert_eq!(
merged(
vec![sequence(&[3333, 23456])],
vec![set(&[210_036, 210_037])]
),
[sequence(&[3333]), set(&[210_036, 210_037])]
);
}
/// 2001:db8:1::/48 via 2001:db8::1.
fn mp_reach_v6() -> Vec<u8> {
let mut value = vec![0, 2, 1, 16];
@@ -1209,7 +998,7 @@ mod tests {
assert!(decoded.errors.is_empty());
assert_eq!(
decoded.update.path_attributes[3],
PathAttribute::Unknown {
PathAttribute::UnknownPathAttribute {
flags: PathAttributeFlags(0xe0),
type_code: 99,
payload: vec![1, 2, 3],
crates/packet/src/wire.rs
-198
View File
@@ -1,198 +0,0 @@
//! What the readers and writers of the wire format share: the shape of a
//! parser's result and its errors, the fields that several messages and
//! attributes have in common, and the lengths that are written before what
//! they count.
use bytes::{BufMut, BytesMut};
use nom::Err::Failure;
use nom::IResult;
use nom::number::complete::{be_u8, be_u16, be_u32};
use bgp_primitives::constants::{Afi, AfiSafi, Safi};
use bgp_primitives::parser::{BgpParserError, EncodeError, ParserContext};
use crate::message::AS_TRANS;
/// A parser's result: what is left of the input, and what was read.
pub(crate) type Parsed<'a, T> = IResult<&'a [u8], T, BgpParserError<&'a [u8]>>;
/// The error for input that does not follow the format; `detail` says how.
pub(crate) fn malformed<'a>(detail: impl Into<String>) -> nom::Err<BgpParserError<&'a [u8]>> {
Failure(BgpParserError::Malformed(detail.into()))
}
/// Parses `buf` to its end as a list of `item`s. Unlike nom's `many0`,
/// which stops quietly at an item it cannot read, an error in an item is an
/// error of the list.
pub(crate) fn list<'a, T>(
mut buf: &'a [u8],
item: impl Fn(&'a [u8]) -> Parsed<'a, T>,
) -> Parsed<'a, Vec<T>> {
let mut items = Vec::new();
while !buf.is_empty() {
let (rest, parsed) = item(buf)?;
items.push(parsed);
buf = rest;
}
Ok((buf, items))
}
/// A parsed value, provided the parser used all of the input. `what` names
/// the input in the error.
pub(crate) fn whole<T>(
what: impl std::fmt::Display,
(rest, parsed): (&[u8], T),
) -> Result<T, nom::Err<BgpParserError<&[u8]>>> {
if rest.is_empty() {
Ok(parsed)
} else {
Err(malformed(format!(
"{} octets left over in {what}",
rest.len()
)))
}
}
/// `value` as a list of one or more records of `N` octets each: how the
/// community attributes and CLUSTER_LIST are laid out. An empty list is
/// malformed (RFC 7606 §7.8 for COMMUNITIES, and the others alike). `what`
/// names the attribute in the error.
pub(crate) fn records<'a, const N: usize>(
value: &'a [u8],
what: &str,
) -> Result<&'a [[u8; N]], nom::Err<BgpParserError<&'a [u8]>>> {
let (records, rest) = value.as_chunks::<N>();
if records.is_empty() || !rest.is_empty() {
return Err(malformed(format!(
"{what} is {} octets long, not a non-empty multiple of {N}",
value.len()
)));
}
Ok(records)
}
/// Parses an AS number as wide as the session's: four octets or two
/// (RFC 6793).
pub(crate) fn parse_asn(four_octet: bool, buf: &[u8]) -> Parsed<'_, u32> {
if four_octet {
be_u32(buf)
} else {
be_u16(buf).map(|(buf, asn)| (buf, u32::from(asn)))
}
}
/// Writes an AS number as wide as the session's. On a two-octet session
/// one that does not fit is written as AS_TRANS (RFC 6793 §4.2.2).
pub(crate) fn put_asn(four_octet: bool, asn: u32, out: &mut BytesMut) {
if four_octet {
out.put_u32(asn);
} else {
out.put_u16(u16::try_from(asn).unwrap_or(AS_TRANS));
}
}
/// Whether the prefixes in the body of an UPDATE carry path identifiers
/// on a session with `ctx`. The body holds IPv4 unicast routes and nothing
/// else (RFC 4271 §4.3), so that is the family whose ADD-PATH agreement
/// decides it; every other family's prefixes are in MP_REACH_NLRI and
/// MP_UNREACH_NLRI, which name their family.
pub(crate) fn body_has_path_id(ctx: &ParserContext) -> bool {
ctx.has_add_path(AfiSafi::IPV4_UNICAST)
}
/// Parses an address family as the multiprotocol attributes and most
/// capabilities name it: a two-octet AFI and a one-octet SAFI.
pub(crate) fn parse_afi_safi(buf: &[u8]) -> Parsed<'_, AfiSafi> {
let (buf, afi) = be_u16(buf)?;
let (buf, safi) = be_u8(buf)?;
Ok((buf, AfiSafi::new(Afi(afi), Safi(safi))))
}
/// Writes an address family as [`parse_afi_safi`] reads it.
pub(crate) fn put_afi_safi(family: AfiSafi, out: &mut BytesMut) {
out.put_u16(family.afi.0);
out.put_u8(family.safi.0);
}
/// Writes `length` into the field of `width` octets reserved at `at`, if
/// it is no more than `max`. `what` names the field in the error.
pub(crate) fn put_length(
out: &mut BytesMut,
at: usize,
width: usize,
length: usize,
max: usize,
what: &'static str,
) -> Result<(), EncodeError> {
if length > max {
return Err(EncodeError::TooLong {
what,
len: length,
max,
});
}
let bytes = length.to_be_bytes();
out[at..at + width].copy_from_slice(&bytes[bytes.len() - width..]);
Ok(())
}
/// Fills in the field of `width` octets reserved at `at` with how much
/// has been written after it, if that is no more than `max`: for a length
/// that is known only once what it counts has been written. `what` names
/// the field in the error.
pub(crate) fn fill_length(
out: &mut BytesMut,
at: usize,
width: usize,
max: usize,
what: &'static str,
) -> Result<(), EncodeError> {
let length = out.len() - at - width;
put_length(out, at, width, length, max, what)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn a_length_is_filled_in_after_what_it_counts() {
let mut out = BytesMut::new();
out.put_u8(0xaa);
let at = out.len();
out.put_u16(0);
out.put_slice(&[1, 2, 3]);
fill_length(&mut out, at, 2, u16::MAX.into(), "field").unwrap();
assert_eq!(&out[..], [0xaa, 0, 3, 1, 2, 3]);
let mut out = BytesMut::new();
out.put_u8(0);
out.put_slice(&[0; 300]);
assert_eq!(
fill_length(&mut out, 0, 1, u8::MAX.into(), "field"),
Err(EncodeError::TooLong {
what: "field",
len: 300,
max: 255,
})
);
}
#[test]
fn records_are_a_whole_number_of_them_and_at_least_one() {
assert_eq!(
records::<4>(&[1, 2, 3, 4, 5, 6, 7, 8], "X").unwrap().len(),
2
);
for value in [&[][..], &[1, 2, 3][..], &[1, 2, 3, 4, 5][..]] {
let error = records::<4>(value, "X").unwrap_err();
assert_eq!(
crate::message::describe(&error),
format!(
"X is {} octets long, not a non-empty multiple of 4",
value.len()
)
);
}
}
}
crates/packet/tests/properties.rs
+39 -128
View File
@@ -25,9 +25,9 @@ use bgp_packet::message::{
PathAttributeFlags, RouteRefreshMessage, UpdateMessage,
};
use bgp_packet::open::{
AddPathEntry, BgpRole, BgpSecHeader, Capability, ExtendedNextHopEntry, GracefulRestartEntry,
LongLivedGracefulRestartEntry, MultiLabelEntry, OpenMessage, OpenOption, OrfFamily, OrfType,
OutboundRouteFilterEntry, SendReceive,
AddPathEntry, AddPathSendRecv, BgpRole, BgpSecHeader, Capability, ExtendedNextHop,
GracefulRestartEntry, LongLivedGracefulRestart, MultiLabelEntry, OpenMessage, OpenOption,
OrfFamily, OrfSendRecv, OrfType, OutboundRouteFilterEntry,
};
use bgp_primitives::constants::{Afi, AfiSafi, IpVersion, Safi};
use bgp_primitives::ip_prefix::{IpPrefix, Nlri};
@@ -137,6 +137,19 @@ fn extended_community() -> impl Strategy<Value = ExtendedCommunity> {
fn mp_reach(session: Session) -> impl Strategy<Value = MpReachNlriPathAttribute> {
let ip_safi = || prop_oneof![Just(Safi::UNICAST), Just(Safi::MULTICAST)];
let v4 = ip_safi()
.prop_flat_map(move |safi| {
(
Just(safi),
any::<[u8; 4]>(),
nlri(session, IpVersion::Ipv4, safi),
)
})
.prop_map(|(safi, next_hop, prefixes)| MpReachNlriPathAttribute {
family: AfiSafi::new(Afi::IPV4, safi),
next_hop: NlriNextHop::Ipv4(Ipv4Addr::from(next_hop)),
prefixes,
});
let v6_next_hop = || {
prop_oneof![
any::<[u8; 16]>().prop_map(|a| NlriNextHop::Ipv6(Ipv6Addr::from(a))),
@@ -148,27 +161,6 @@ fn mp_reach(session: Session) -> impl Strategy<Value = MpReachNlriPathAttribute>
}),
]
};
// IPv4 routes have an IPv4 next hop, or an IPv6 one over a link that
// has only IPv6 addresses (RFC 8950).
let v4_next_hop = move || {
prop_oneof![
3 => any::<[u8; 4]>().prop_map(|a| NlriNextHop::Ipv4(Ipv4Addr::from(a))),
1 => v6_next_hop(),
]
};
let v4 = ip_safi()
.prop_flat_map(move |safi| {
(
Just(safi),
v4_next_hop(),
nlri(session, IpVersion::Ipv4, safi),
)
})
.prop_map(|(safi, next_hop, prefixes)| MpReachNlriPathAttribute {
family: AfiSafi::new(Afi::IPV4, safi),
next_hop,
prefixes,
});
let v6 = ip_safi()
.prop_flat_map(move |safi| {
(
@@ -197,7 +189,7 @@ fn mp_unreach(session: Session) -> impl Strategy<Value = MpUnreachNlriPathAttrib
/// Type codes whose definition the decoder knows; an unknown attribute has
/// none of these.
const DEFINED_ATTRIBUTES: [u8; 17] = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 14, 15, 16, 17, 18, 32, 35];
const DEFINED_ATTRIBUTES: [u8; 16] = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 14, 15, 16, 17, 18, 32];
/// Unknown attributes, each with a type code of its own: a repeated
/// attribute is dropped by the decoder.
@@ -220,7 +212,7 @@ fn unknown_attributes() -> impl Strategy<Value = Vec<PathAttribute>> {
if payload.len() > 255 {
flags |= 0x10;
}
PathAttribute::Unknown {
PathAttribute::UnknownPathAttribute {
flags: PathAttributeFlags(flags),
type_code,
payload,
@@ -299,40 +291,11 @@ fn path_attributes(session: Session) -> impl Strategy<Value = Vec<PathAttribute>
.collect(),
})
})),
proptest::option::of(any::<u32>().prop_map(PathAttribute::OnlyToCustomer)),
// Route reflection's, which only an internal peer sends.
proptest::option::of((any::<[u8; 4]>(), vec(any::<[u8; 4]>(), 1..4))).prop_map(
move |reflected| {
reflected
.filter(|_| session.ibgp)
.map(|(originator, clusters)| {
vec![
PathAttribute::OriginatorId(Ipv4Addr::from(originator)),
PathAttribute::ClusterList(
clusters.into_iter().map(Ipv4Addr::from).collect(),
),
]
})
.unwrap_or_default()
},
),
unknown_attributes(),
);
(mandatory, optional).prop_map(|(mut attributes, optional)| {
let (
med,
atomic,
aggregator,
communities,
reach,
unreach,
extended,
as4,
large,
otc,
reflected,
unknown,
) = optional;
let (med, atomic, aggregator, communities, reach, unreach, extended, as4, large, unknown) =
optional;
attributes.extend(
[
med,
@@ -344,12 +307,10 @@ fn path_attributes(session: Session) -> impl Strategy<Value = Vec<PathAttribute>
extended,
as4,
large,
otc,
]
.into_iter()
.flatten(),
);
attributes.extend(reflected);
attributes.extend(unknown);
attributes
})
@@ -377,16 +338,16 @@ fn capability() -> impl Strategy<Value = Capability> {
let orf = (
any::<u8>().prop_map(OrfType),
prop_oneof![
Just(SendReceive::Receive),
Just(SendReceive::Send),
Just(SendReceive::Both)
Just(OrfSendRecv::Receive),
Just(OrfSendRecv::Send),
Just(OrfSendRecv::Both)
],
)
.prop_map(|(r#type, send_recv)| OutboundRouteFilterEntry { r#type, send_recv });
let add_path = prop_oneof![
Just(SendReceive::Receive),
Just(SendReceive::Send),
Just(SendReceive::Both)
Just(AddPathSendRecv::Receive),
Just(AddPathSendRecv::Send),
Just(AddPathSendRecv::Both)
];
prop_oneof![
(afi(), safi()).prop_map(|(afi, safi)| Capability::MultiProtocol { afi, safi }),
@@ -402,7 +363,7 @@ fn capability() -> impl Strategy<Value = Capability> {
vec((afi(), safi(), afi()), 1..4).prop_map(|entries| Capability::ExtendedNextHop {
entries: entries
.into_iter()
.map(|(afi, safi, nh_afi)| ExtendedNextHopEntry { afi, safi, nh_afi })
.map(|(afi, safi, nh_afi)| ExtendedNextHop { afi, safi, nh_afi })
.collect(),
}),
Just(Capability::ExtendedMessage),
@@ -411,7 +372,7 @@ fn capability() -> impl Strategy<Value = Capability> {
afi,
}),
vec((afi(), safi(), any::<u8>()), 1..4).prop_map(|entries| {
Capability::MultiLabel {
Capability::MultiLabelCompat {
entries: entries
.into_iter()
.map(|(afi, safi, count)| MultiLabelEntry { afi, safi, count })
@@ -422,15 +383,13 @@ fn capability() -> impl Strategy<Value = Capability> {
role: BgpRole::from(role),
}),
(
any::<bool>(),
any::<bool>(),
0u16..4096,
vec((afi(), safi(), any::<bool>()), 0..4)
)
.prop_map(|(restart_state, notification, restart_time, entries)| {
.prop_map(|(restart_state, restart_time, entries)| {
Capability::GracefulRestart {
restart_state,
notification,
restart_time,
entries: entries
.into_iter()
@@ -458,14 +417,12 @@ fn capability() -> impl Strategy<Value = Capability> {
Capability::LongLivedGracefulRestart(
entries
.into_iter()
.map(
|(afi, safi, flags, stale_time)| LongLivedGracefulRestartEntry {
afi,
safi,
flags,
stale_time,
},
)
.map(|(afi, safi, flags, stale_time)| LongLivedGracefulRestart {
afi,
safi,
flags,
stale_time,
})
.collect(),
)
}),
@@ -475,7 +432,7 @@ fn capability() -> impl Strategy<Value = Capability> {
}),
vec(any::<u8>(), 0..20)
)
.prop_map(|(type_code, value)| Capability::Unknown { type_code, value }),
.prop_map(|(type_code, value)| Capability::UnknownCapability { type_code, value }),
]
}
@@ -702,9 +659,7 @@ proptest! {
check_decoding(&ctx, &damaged(&wire, &damage))?;
}
/// The parsers below the message level are public too. What they
/// accept must write back as it was read, where the format leaves
/// nothing for the reader to normalise.
/// The parsers below the message level are public too.
#[test]
fn noise_is_never_more_than_an_error_for_the_parts(
session in session(),
@@ -712,52 +667,8 @@ proptest! {
) {
let ctx = session.ctx();
let _ = OpenMessage::from_wire(&bytes);
let _ = Capability::from_wire(&bytes);
let _ = UpdateMessage::decode(&ctx, &bytes);
if let Ok((rest, capability)) = Capability::from_wire(&bytes) {
let consumed = &bytes[..bytes.len() - rest.len()];
let mut rewritten = BytesMut::new();
capability.to_wire(&mut rewritten)
.map_err(|e| TestCaseError::fail(format!("accepted but cannot be written: {e}")))?;
// ADD-PATH families with a Send/Receive value nobody defined
// are left out (RFC 7911 §4), and the reserved octet of
// Multiprotocol and ORF and the reserved bits of Graceful
// Restart are read as zero whatever they were. Everything else
// is written as it was read.
let normalised = matches!(
capability,
Capability::AddPath { .. }
| Capability::MultiProtocol { .. }
| Capability::OutboundRouteFilter(_)
| Capability::GracefulRestart { .. }
);
if !normalised {
prop_assert_eq!(&rewritten[..], consumed, "{:?}", capability);
}
let again = Capability::from_wire(&rewritten)
.map_err(|e| TestCaseError::fail(format!("written but cannot be read: {e:?}")))?;
prop_assert_eq!(again, (&[][..], capability));
}
if let Ok((rest, attribute)) = PathAttribute::from_wire(&ctx, &bytes) {
let consumed = &bytes[..bytes.len() - rest.len()];
let mut rewritten = BytesMut::new();
attribute.to_wire(&ctx, &mut rewritten)
.map_err(|e| TestCaseError::fail(format!("accepted but cannot be written: {e}")))?;
// The flags are not checked against the attribute's definition
// here, and are written as defined; and the bits of a prefix
// past its length are cleared. With the flags as defined and
// no prefixes, the attribute is written as it was read.
let names_prefixes = matches!(
attribute,
PathAttribute::MpReachNlri(_) | PathAttribute::MpUnreachNlri(_)
);
if rewritten[0] == consumed[0] && !names_prefixes {
prop_assert_eq!(&rewritten[..], consumed, "{:?}", attribute);
}
let again = PathAttribute::from_wire(&ctx, &rewritten)
.map_err(|e| TestCaseError::fail(format!("written but cannot be read: {e:?}")))?;
prop_assert_eq!(again, (&[][..], attribute));
}
let _ = PathAttribute::from_wire(&ctx, &bytes);
}
}
crates/primitives/Cargo.toml
+2
View File
@@ -9,11 +9,13 @@ repository.workspace = true
version.workspace = true
[dependencies]
bitfield.workspace = true
bytes.workspace = true
nom.workspace = true
serde.workspace = true
serde_repr.workspace = true
thiserror.workspace = true
strum.workspace = true
[dev-dependencies]
proptest.workspace = true
crates/primitives/src/constants.rs
+1 -1
View File
@@ -11,7 +11,7 @@ pub struct UnknownAfi(pub u16);
/// The IP version of a prefix. For the address family field of a capability
/// or an MP_REACH_NLRI, which can name families that are not IP, see [`Afi`].
#[derive(Debug, Copy, Clone, Hash, PartialEq, Eq, PartialOrd, Ord, Serialize_repr)]
#[derive(Debug, Copy, Clone, Hash, PartialEq, Eq, Serialize_repr)]
#[repr(u16)]
pub enum IpVersion {
Ipv4 = 1,
crates/primitives/src/ip_prefix.rs
+8 -44
View File
@@ -31,7 +31,7 @@ pub enum PrefixError {
/// value. A prefix has one representation: the bits past its length are
/// always zero, so two prefixes are equal exactly when they are the same
/// prefix.
#[derive(Debug, PartialEq, Eq, PartialOrd, Ord, Clone, Copy, Hash, Serialize)]
#[derive(Debug, PartialEq, Eq, Clone, Copy, Hash, Serialize)]
pub struct IpPrefix {
address_family: IpVersion,
length: u8,
@@ -64,7 +64,7 @@ impl IpPrefix {
let mut address = [0u8; 16];
address[..needed].copy_from_slice(&prefix[..needed]);
if let (Some(last), rem @ 1..) = (address[..needed].last_mut(), length % 8) {
*last &= high_bits_mask(rem);
*last &= u8::MAX << (8 - rem);
}
Ok(Self {
address_family,
@@ -88,25 +88,6 @@ impl IpPrefix {
&self.address[..usize::from(self.length.div_ceil(8))]
}
/// The prefix of `length` bits that covers this one: itself, if it is
/// no longer than that.
pub fn shortened(&self, length: u8) -> Self {
if length >= self.length {
return *self;
}
let mut address = [0u8; 16];
let whole = usize::from(length / 8);
address[..whole].copy_from_slice(&self.address[..whole]);
if !length.is_multiple_of(8) {
address[whole] = self.address[whole] & high_bits_mask(length % 8);
}
Self {
address_family: self.address_family,
length,
address,
}
}
/// Returns true if `other` is equal to or more specific than `self`, i.e.
/// `self` covers `other`. Prefixes of different address families never
/// contain each other.
@@ -122,18 +103,11 @@ impl IpPrefix {
if rem_bits == 0 {
return true;
}
let mask = high_bits_mask(rem_bits);
let mask = u8::MAX << (8 - rem_bits);
(self.address[full_bytes] & mask) == (other.address[full_bytes] & mask)
}
}
/// The mask that keeps the first `bits` bits of an octet and clears the
/// rest, for `bits` from 1 to 7: what the last octet of a prefix whose
/// length is not a whole number of octets is masked with.
fn high_bits_mask(bits: u8) -> u8 {
u8::MAX << (8 - bits)
}
/// A prefix as an UPDATE names it: the prefix and, on a session where
/// ADD-PATH is in use for the address family (RFC 7911), the identifier
/// that tells apart the several routes a peer may announce for it.
@@ -151,6 +125,11 @@ impl Nlri {
pub fn plain(prefixes: impl IntoIterator<Item = IpPrefix>) -> Vec<Self> {
prefixes.into_iter().map(Self::from).collect()
}
/// The bytes this takes on the wire, with or without its identifier.
pub fn wire_len(&self, with_path_id: bool) -> usize {
1 + self.prefix.bytes().len() + if with_path_id { 4 } else { 0 }
}
}
impl From<IpPrefix> for Nlri {
@@ -365,21 +344,6 @@ mod tests {
}
}
/// A shortened prefix is the same as one made from the same
/// address with the shorter length, and covers the original.
#[test]
fn shortening_keeps_the_leading_bits(
version: bool, bytes: [u8; 16], length in 0u8..=128, shorter in 0u8..=128,
) {
let version = if version { IpVersion::Ipv6 } else { IpVersion::Ipv4 };
let length = length.min(version.bits());
let prefix = IpPrefix::new(version, bytes, length).unwrap();
let shortened = prefix.shortened(shorter);
let expected = IpPrefix::new(version, bytes, shorter.min(length)).unwrap();
prop_assert_eq!(shortened, expected);
prop_assert!(shortened.contains(&prefix));
}
/// `contains` agrees with comparing the leading bits by hand.
#[test]
fn contains_is_a_comparison_of_leading_bits(
crates/rib/Cargo.toml
+1
View File
@@ -16,6 +16,7 @@ hashbrown.workspace = true
serde.workspace = true
thiserror.workspace = true
tokio = { workspace = true, features = ["full"] }
tracing.workspace = true
[dev-dependencies]
proptest.workspace = true
crates/rib/src/table/attrs.rs
+26 -357
View File
@@ -13,7 +13,7 @@
use std::hash::BuildHasher;
use std::sync::{Arc, Mutex, MutexGuard, PoisonError};
use bgp_packet::message::{NlriNextHop, PathAttribute, RawAttribute, attr_type, split_attribute};
use bgp_packet::message::{NlriNextHop, PathAttribute, attr_type, split_attribute};
use bgp_primitives::parser::{EncodeError, ParserContext};
use bytes::{BufMut, BytesMut};
use foldhash::fast::RandomState;
@@ -21,10 +21,6 @@ use hashbrown::HashTable;
const OPTIONAL: u8 = 0x80;
/// The type of an AS_PATH segment whose AS numbers are in order; the other
/// type, AS_SET, is an unordered handful (RFC 4271 §4.3).
const AS_SEQUENCE: u8 = 2;
/// What route selection reads from a set of attributes (RFC 4271 §9.1.2.2).
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct SelectionKey {
@@ -41,26 +37,12 @@ pub struct SelectionKey {
pub as_path_len: u16,
/// ORIGIN: 0 for IGP, 1 for EGP, 2 for INCOMPLETE.
pub origin: u8,
/// The AS the route originates from: the last AS of the path. Zero,
/// which is no AS's number, if the path is empty or ends in a set and
/// so has no one origin. Not part of selection; kept here because it
/// is what a route's validity is asked about (RFC 6811), for every
/// route, when the validation data changes.
pub origin_as: u32,
}
/// Reads the selection key straight off attributes in the form an
/// [`AttrSet`] keeps, without building anything. `None` if the bytes are
/// not a well-formed list of attributes.
///
/// A list with the same attribute twice is not well formed (RFC 4271
/// §5). The decoder keeps only the first of such, so none comes from a
/// session; were one to get in some other way, the readers of a set, of
/// which this is one, could each take a different one of the two for the
/// route's path or next hop.
fn scan(mut wire: &[u8]) -> Option<SelectionKey> {
// One bit for each type of attribute there can be.
let mut seen = [0u64; 4];
let mut key = SelectionKey {
local_pref: None,
med: None,
@@ -69,29 +51,15 @@ fn scan(mut wire: &[u8]) -> Option<SelectionKey> {
// A route without ORIGIN does not get this far; if one did, it
// would be the least preferred.
origin: 2,
origin_as: 0,
};
let number = |value: &[u8]| Some(u32::from_be_bytes(<[u8; 4]>::try_from(value).ok()?));
while !wire.is_empty() {
let (attribute, rest) = split_attribute(wire).ok()?;
wire = rest;
let value = attribute.value;
let (word, bit) = (
usize::from(attribute.type_code / 64),
1u64 << (attribute.type_code % 64),
);
if seen[word] & bit != 0 {
return None;
}
seen[word] |= bit;
match attribute.type_code {
attr_type::ORIGIN => key.origin = *value.first()?,
attr_type::AS_PATH => {
let path = scan_as_path(value)?;
key.as_path_len = path.hops;
key.neighbor_as = path.first;
key.origin_as = path.last;
}
attr_type::AS_PATH => (key.as_path_len, key.neighbor_as) = scan_as_path(value)?,
attr_type::MULTI_EXIT_DISC => key.med = Some(number(value)?),
attr_type::LOCAL_PREF => key.local_pref = Some(number(value)?),
_ => {}
@@ -100,70 +68,29 @@ fn scan(mut wire: &[u8]) -> Option<SelectionKey> {
Some(key)
}
/// What the selection key takes from an AS path.
struct PathScan {
/// The length as selection counts it.
hops: u16,
/// The first AS, if the path begins with a sequence; else zero.
first: u32,
/// The last AS, if the path ends in a sequence; else zero.
last: u32,
}
impl SelectionKey {
/// The AS the route originates from: the last of its path. `None` if
/// the path is empty or ends in a set, which has no one last AS.
pub fn origin(&self) -> Option<u32> {
Some(self.origin_as).filter(|origin| *origin != 0)
}
}
/// Reads an AS_PATH value with four-octet AS numbers.
fn scan_as_path(mut segments: &[u8]) -> Option<PathScan> {
/// The length of an AS_PATH as selection counts it, and the neighbouring
/// AS, from the attribute's value with four-octet AS numbers.
fn scan_as_path(mut segments: &[u8]) -> Option<(u16, u32)> {
const AS_SEQUENCE: u8 = 2;
let mut hops = 0usize;
let (mut first, mut last) = (0, 0);
let mut at_start = true;
let mut neighbor_as = 0;
let mut first = true;
while let [kind, count, rest @ ..] = segments {
let (asns, rest) = rest.split_at_checked(usize::from(*count) * 4)?;
let ordered = *kind == AS_SEQUENCE;
hops += if ordered { usize::from(*count) } else { 1 };
let asn = |bytes: Option<&[u8; 4]>| bytes.map_or(0, |asn| u32::from_be_bytes(*asn));
if at_start && ordered {
first = asn(asns.first_chunk::<4>());
if first && ordered {
neighbor_as = asns
.first_chunk::<4>()
.map_or(0, |asn| u32::from_be_bytes(*asn));
}
last = if ordered {
asn(asns.last_chunk::<4>())
} else {
0
};
at_start = false;
first = false;
segments = rest;
}
if !segments.is_empty() {
return None;
}
Some(PathScan {
hops: u16::try_from(hops).unwrap_or(u16::MAX),
first,
last,
})
}
/// Appends the AS numbers of an AS_PATH or AS4_PATH value with four-octet
/// AS numbers to `out`, and returns whether any segment is not a sequence.
/// Stops at what is not a segment.
fn read_as_path(mut segments: &[u8], out: &mut Vec<u32>) -> bool {
let mut has_set = false;
while let [kind, count, rest @ ..] = segments {
let Some((asns, rest)) = rest.split_at_checked(usize::from(*count) * 4) else {
break;
};
has_set |= *kind != AS_SEQUENCE;
let (asns, _) = asns.as_chunks::<4>();
out.extend(asns.iter().map(|asn| u32::from_be_bytes(*asn)));
segments = rest;
}
has_set
Some((u16::try_from(hops).unwrap_or(u16::MAX), neighbor_as))
}
/// One distinct list of path attributes, with the next hop that goes with
@@ -189,77 +116,6 @@ impl AttrSet {
&self.wire
}
/// The attributes one by one, as they lie in the stored bytes. The
/// bytes were checked when the set was made, so this is all of them;
/// it stops at the first that does not split as an attribute all the
/// same, rather than go on from the middle of one.
fn raw_attributes(&self) -> impl Iterator<Item = RawAttribute<'_>> {
let mut wire = &*self.wire;
std::iter::from_fn(move || {
let (attribute, rest) = split_attribute(wire).ok()?;
wire = rest;
Some(attribute)
})
}
/// The value of the attribute of type `type_code`, if the set has one.
fn attribute(&self, type_code: u8) -> Option<&[u8]> {
self.raw_attributes()
.find(|attribute| attribute.type_code == type_code)
.map(|attribute| attribute.value)
}
/// The route's next hop: the one that came in MP_REACH_NLRI, or else
/// the NEXT_HOP attribute. Read off the stored bytes without building
/// anything else, for those who ask it of every route.
pub fn next_hop(&self) -> Option<NlriNextHop> {
let mut plain = None;
for attribute in self.raw_attributes() {
match attribute.type_code {
attr_type::MP_REACH_NLRI => return decode_next_hop(attribute.value),
attr_type::NEXT_HOP => plain = NlriNextHop::from_bytes(attribute.value),
_ => {}
}
}
plain
}
/// Whether the route carries the OTC attribute (RFC 9234): it came
/// down from a provider or across from a peer, and is to go on down
/// only. Read off the stored bytes without building anything else,
/// for those who ask it of every route: a session to a peer or a
/// provider asks it of most of the table, to pass over them.
pub fn is_only_to_customer(&self) -> bool {
self.attribute(attr_type::ONLY_TO_CUSTOMER).is_some()
}
/// Whether the route carries the BLACKHOLE community (RFC 7999), which
/// asks for the packets to its prefix to be dropped. Read off the
/// stored bytes without building anything else, for those who ask it
/// of every route.
pub fn is_blackhole(&self) -> bool {
const BLACKHOLE: [u8; 4] = [0xFF, 0xFF, 0x02, 0x9A];
self.attribute(attr_type::COMMUNITIES)
.is_some_and(|communities| communities.as_chunks::<4>().0.contains(&BLACKHOLE))
}
/// Puts the AS numbers of the route's path in `out`, in place of what
/// was there: in the order of the path, the neighbor's first and the
/// origin's last, sets and all. Returns whether the path has a set in
/// it.
///
/// This is the whole path: what a session with two-octet AS numbers
/// brought in AS4_PATH was put in its place when the UPDATE was read
/// (RFC 6793 §4.2.3). It reads the stored bytes and builds nothing
/// else, for those who ask it of every route.
pub fn as_path(&self, out: &mut Vec<u32>) -> bool {
out.clear();
match self.attribute(attr_type::AS_PATH) {
Some(segments) => read_as_path(segments, out),
None => false,
}
}
/// The attributes, parsed, and the next hop if it came in
/// MP_REACH_NLRI.
///
@@ -304,8 +160,7 @@ fn encode(
Ok(())
}
/// Writes a next hop as an MP_REACH_NLRI that holds nothing else: the
/// next hop's length, and the next hop.
/// Writes a next hop as an MP_REACH_NLRI that holds nothing else.
fn encode_next_hop(next_hop: &NlriNextHop, out: &mut BytesMut) {
out.put_u8(OPTIONAL);
out.put_u8(attr_type::MP_REACH_NLRI);
@@ -314,13 +169,6 @@ fn encode_next_hop(next_hop: &NlriNextHop, out: &mut BytesMut) {
next_hop.to_wire(out);
}
/// Reads the next hop out of the value of an MP_REACH_NLRI as
/// [`encode_next_hop`] writes it. `None` if the value is not one.
fn decode_next_hop(value: &[u8]) -> Option<NlriNextHop> {
let (length, address) = value.split_first()?;
NlriNextHop::from_bytes(address).filter(|_| usize::from(*length) == address.len())
}
/// Parses path attributes in the form [`AttrSet`] keeps and table dumps
/// use (RFC 6396 §4.3.4): four-octet AS numbers, and an MP_REACH_NLRI that
/// holds only a next hop. `None` if they are malformed.
@@ -331,7 +179,11 @@ fn parse(mut wire: &[u8]) -> Option<(Vec<PathAttribute>, Option<NlriNextHop>)> {
while !wire.is_empty() {
let (raw, rest) = split_attribute(wire).ok()?;
if raw.type_code == attr_type::MP_REACH_NLRI {
next_hop = Some(decode_next_hop(raw.value)?);
let (length, address) = raw.value.split_first()?;
if usize::from(*length) != address.len() {
return None;
}
next_hop = Some(NlriNextHop::from_bytes(address)?);
} else {
let (_, attribute) = PathAttribute::from_wire(&ctx, raw.wire).ok()?;
attributes.push(attribute);
@@ -415,28 +267,13 @@ impl AttrStore {
wire: &[u8],
next_hop: Option<&NlriNextHop>,
) -> Option<Arc<AttrSet>> {
self.intern_received_with(wire, &[], next_hop)
}
/// As [`AttrStore::intern_received`], with attributes of our own after
/// the peer's: `added` is one or more whole attributes in the same
/// form, of types that `wire` does not have.
pub fn intern_received_with(
&self,
wire: &[u8],
added: &[u8],
next_hop: Option<&NlriNextHop>,
) -> Option<Arc<AttrSet>> {
if added.is_empty() && next_hop.is_none() {
let Some(next_hop) = next_hop else {
return self.intern_wire(wire);
}
let mut whole = BytesMut::with_capacity(wire.len() + added.len() + 36);
whole.put_slice(wire);
whole.put_slice(added);
if let Some(next_hop) = next_hop {
encode_next_hop(next_hop, &mut whole);
}
self.intern_wire(&whole)
};
let mut with_next_hop = BytesMut::with_capacity(wire.len() + 36);
with_next_hop.put_slice(wire);
encode_next_hop(next_hop, &mut with_next_hop);
self.intern_wire(&with_next_hop)
}
/// How many distinct sets are stored.
@@ -559,8 +396,6 @@ mod tests {
neighbor_as: 65001,
as_path_len: 3,
origin: 2,
// The path ends in a set: it has no one origin.
origin_as: 0,
}
);
@@ -568,171 +403,6 @@ mod tests {
assert_eq!(store.intern(&attrs, None).unwrap().key().med, Some(7));
}
#[test]
fn the_next_hop_is_read_off_a_set() {
let store = AttrStore::new();
let plain = store.intern(&attributes(vec![3333]), None).unwrap();
assert_eq!(
plain.next_hop(),
Some(NlriNextHop::Ipv4(Ipv4Addr::new(192, 0, 2, 1)))
);
// The one that came with the routes counts, where there is one.
let with_link_local = NlriNextHop::Ipv6WithLl {
global: "2001:db8::1".parse().unwrap(),
link_local: "fe80::1".parse().unwrap(),
};
let v6 = store
.intern(&attributes(vec![3333]), Some(&with_link_local))
.unwrap();
assert_eq!(v6.next_hop(), Some(with_link_local));
// A route of our own making may have none.
let none = store
.intern(&[PathAttribute::Origin(OriginPathAttribute::IGP)], None)
.unwrap();
assert_eq!(none.next_hop(), None);
}
#[test]
fn the_path_is_read_off_a_set_as_it_really_is() {
use bgp_packet::message::{AsPathAttribute, AsPathSegment};
let store = AttrStore::new();
let mut path = Vec::new();
let set = store
.intern(&attributes(vec![3333, 3333, 1299]), None)
.unwrap();
assert!(!set.as_path(&mut path));
assert_eq!(path, [3333, 3333, 1299]);
// A set is said to be there.
let sequence = |path: Vec<u32>| AsPathSegment {
ordered: true,
path,
};
let mut with_set = attributes(vec![]);
with_set[1] = PathAttribute::AsPath(AsPathAttribute {
segments: vec![
sequence(vec![3333]),
AsPathSegment {
ordered: false,
path: vec![64500, 64501],
},
],
});
let set = store.intern(&with_set, None).unwrap();
assert!(set.as_path(&mut path));
assert_eq!(path, [3333, 64500, 64501]);
// No path at all.
let set = store.intern(&attributes(vec![]), None).unwrap();
assert!(!set.as_path(&mut path));
assert!(path.is_empty());
}
/// The BLACKHOLE community is read off the stored bytes.
#[test]
fn a_blackhole_request_is_told_from_the_bytes() {
let store = AttrStore::new();
let with = |communities: Vec<(u16, u16)>| {
let mut attributes = attributes(vec![3333]);
attributes.retain(|attribute| !matches!(attribute, PathAttribute::Communities(_)));
attributes.push(PathAttribute::Communities(CommunitiesPathAttribute(
communities,
)));
store.intern(&attributes, None).unwrap()
};
assert!(with(vec![(3333, 100), (0xFFFF, 0x029A)]).is_blackhole());
assert!(!with(vec![(3333, 666), (0xFFFF, 0xFF01)]).is_blackhole());
assert!(
!store
.intern(&attributes(vec![3333]), None)
.unwrap()
.is_blackhole()
);
}
/// The OTC attribute is read off the stored bytes, wherever in them it
/// lies.
#[test]
fn otc_is_told_from_the_bytes() {
let store = AttrStore::new();
let without = store.intern(&attributes(vec![3333]), None).unwrap();
assert!(!without.is_only_to_customer());
let mut with = attributes(vec![3333]);
with.insert(1, PathAttribute::OnlyToCustomer(3333));
assert!(store.intern(&with, None).unwrap().is_only_to_customer());
// After COMMUNITIES, which is the last of the usual attributes.
let mut after = attributes(vec![3333]);
after.push(PathAttribute::OnlyToCustomer(3333));
assert!(store.intern(&after, None).unwrap().is_only_to_customer());
}
/// What a session received, with attributes of our own added after it,
/// is one set with the parsed form of the whole; the bytes are not
/// parsed to find that out, so the bytes must say the same thing.
#[test]
fn received_attributes_with_ours_added_are_the_same_set_as_the_parsed_form() {
let store = AttrStore::new();
let ctx = four_octet();
let received = attributes(vec![65001, 65002]);
let mut wire = BytesMut::new();
encode(&received, None, &mut wire).unwrap();
let ours = [
PathAttribute::OnlyToCustomer(65001),
PathAttribute::LocalPref(LocalPrefPathAttribute(150)),
];
let mut added = BytesMut::new();
for attribute in &ours {
attribute.to_wire(&ctx, &mut added).unwrap();
}
let next_hop = NlriNextHop::Ipv6("2001:db8::1".parse().unwrap());
let mut whole = received.clone();
whole.extend(ours.iter().cloned());
let parsed = store.intern(&whole, Some(&next_hop)).unwrap();
let from_bytes = store
.intern_received_with(&wire, &added, Some(&next_hop))
.unwrap();
assert!(Arc::ptr_eq(&parsed, &from_bytes));
assert!(from_bytes.is_only_to_customer());
assert_eq!(from_bytes.next_hop(), Some(next_hop));
// Without a next hop too.
let parsed = store.intern(&whole, None).unwrap();
let from_bytes = store.intern_received_with(&wire, &added, None).unwrap();
assert!(Arc::ptr_eq(&parsed, &from_bytes));
// An attribute of ours of a type the peer sent is not a set.
let mut repeated = BytesMut::new();
PathAttribute::MultiExitDisc(MultiExitDiscPathAttribute(1))
.to_wire(&ctx, &mut repeated)
.unwrap();
assert!(store.intern_received_with(&wire, &repeated, None).is_none());
}
/// Two of the same attribute are not a set: its readers could each
/// take a different one.
#[test]
fn the_same_attribute_twice_is_not_a_set() {
let store = AttrStore::new();
let origin = [0x40, 1, 1, 0];
let path = |asn: u8| [0x40, 2, 6, 2, 1, 0, 0, 0, asn];
let once = [&origin[..], &path(1)].concat();
assert!(store.intern_wire(&once).is_some());
let twice = [&origin[..], &path(1), &path(2)].concat();
assert!(store.intern_wire(&twice).is_none());
// Whatever the type, known here or not.
let unknown = [0xc0, 200, 1, 7];
assert!(store.intern_wire(&[&once[..], &unknown].concat()).is_some());
assert!(
store
.intern_wire(&[&once[..], &unknown, &unknown].concat())
.is_none()
);
}
#[test]
fn attributes_on_the_wire_are_interned_without_parsing_them() {
let store = AttrStore::new();
@@ -752,7 +422,6 @@ mod tests {
assert_eq!(fresh.key(), parsed.key());
assert_eq!(fresh.key().as_path_len, 3);
assert_eq!(fresh.key().neighbor_as, 65001);
assert_eq!(fresh.key().origin_as, 65003);
assert_eq!(fresh.attributes(), Some((attrs, Some(next_hop))));
// Bytes that are not attributes are refused.
crates/rib/src/table/journal.rs
+179 -374
View File
@@ -87,15 +87,10 @@ impl Default for Limits {
struct Entry {
seq: u64,
prefix: IpPrefix,
/// When, by the table's clock: the same for every change one call
/// made, in whatever shard. A reader gives out all the changes up to
/// a time or none of them, so that what was changed together is read
/// together.
stamp: u64,
}
/// One journal of one shard.
pub(super) struct Journal {
struct Journal {
kind: Kind,
/// The sequence number of the latest change. Counts every change this
/// journal is for, whether or not anyone was reading.
@@ -116,8 +111,6 @@ pub(super) struct Journal {
/// How many reader slots are in use.
readers: usize,
limits: Limits,
/// The time the entries written now are given; see [`Entry::stamp`].
stamp: u64,
}
impl Journal {
@@ -131,7 +124,6 @@ impl Journal {
gone: HashMap::default(),
cursors: Vec::new(),
readers: 0,
stamp: 0,
}
}
@@ -143,7 +135,6 @@ impl Journal {
self.entries.push_back(Entry {
seq: self.seq,
prefix,
stamp: self.stamp,
});
}
self.seq
@@ -165,41 +156,30 @@ impl Journal {
self.entries.push_back(Entry {
seq: self.seq,
prefix,
stamp: self.stamp,
});
self.gone.insert(prefix, self.seq);
}
}
/// Voids the note that `entry`'s prefix had gone, if `entry` is what
/// said so: an entry that is dropped must not leave the note behind,
/// or the prefix would be given out as gone on the strength of a later
/// entry that says something else.
fn void_gone_note(gone: &mut HashMap<IpPrefix, u64, RandomState>, entry: &Entry) {
/// Forgets that `entry`'s prefix had gone, if `entry` is what said so.
fn drop_entry(gone: &mut HashMap<IpPrefix, u64, RandomState>, entry: &Entry) {
if gone.get(&entry.prefix) == Some(&entry.seq) {
gone.remove(&entry.prefix);
}
}
/// What `entry` says: something only if it is the latest change of its
/// prefix.
fn says<'a>(&self, entry: &Entry, rows: &'a HashTable<Row>, hasher: &RandomState) -> Says<'a> {
/// What `entry` says, if it still says something: it is the latest
/// change of its prefix. The row, or `None` for a prefix that has gone.
fn live<'a>(
&self,
entry: &Entry,
rows: &'a HashTable<Row>,
hasher: &RandomState,
) -> Option<Option<&'a Row>> {
let hash = hasher.hash_one(entry.prefix);
match rows.find(hash, |row| row.prefix == entry.prefix) {
Some(row) if *row.seqs.of(self.kind) == entry.seq => Says::Row(row),
None if self.gone.get(&entry.prefix) == Some(&entry.seq) => Says::Gone,
_ => Says::Nothing,
}
}
/// Where a reader whose cursor is `cursor` is to read from.
fn place_of(&self, cursor: u64) -> Place {
if cursor < self.floor {
Place::Lagged
} else if cursor == self.seq {
Place::Nothing
} else {
Place::From(self.entries.partition_point(|entry| entry.seq <= cursor))
Some(row) => (row.seqs.of(self.kind) == entry.seq).then_some(Some(row)),
None => (self.gone.get(&entry.prefix) == Some(&entry.seq)).then_some(None),
}
}
@@ -209,26 +189,7 @@ impl Journal {
if self.entries.len() <= self.limits.tidy_at {
return;
}
self.drop_passed();
// Entries for prefixes that have a row are at most as many as the
// rows once the dead ones are out. Entries for prefixes that have
// gone are not counted in: they are what a reader that has stopped
// reading makes pile up. Twice that, so that the pass over every
// entry that finding the dead ones takes is made only once the
// journal has doubled, and the cost per change stays small.
let limit = 2 * (rows.len() + self.limits.slack);
if self.entries.len() <= limit {
return;
}
self.drop_dead(rows, hasher);
if self.entries.len() <= limit {
return;
}
self.drop_oldest_half();
}
/// Drops what every reader has passed.
fn drop_passed(&mut self) {
// What every reader has passed can go.
let passed = self
.cursors
.iter()
@@ -239,57 +200,40 @@ impl Journal {
if front.seq > passed {
break;
}
Self::void_gone_note(&mut self.gone, front);
Self::drop_entry(&mut self.gone, front);
self.entries.pop_front();
}
}
/// Drops the dead entries: those whose prefix has changed again since,
/// so that a later entry says what they said. This looks every entry
/// up.
fn drop_dead(&mut self, rows: &HashTable<Row>, hasher: &RandomState) {
// Entries for prefixes that have a row are at most as many as the
// rows once the dead ones are out. Entries for prefixes that have
// gone are not counted in: they are what a reader that has stopped
// reading makes pile up.
let limit = 2 * (rows.len() + self.limits.slack);
if self.entries.len() <= limit {
return;
}
// Dead entries: their prefix has changed again since. This looks
// every entry up, but only once the journal has doubled, so the
// cost per change stays small.
let mut entries = std::mem::take(&mut self.entries);
entries.retain(|entry| !matches!(self.says(entry, rows, hasher), Says::Nothing));
entries.retain(|entry| self.live(entry, rows, hasher).is_some());
self.entries = entries;
}
/// Drops the older half of the entries without regard to the readers,
/// for a journal that is too long even without its dead entries: a
/// reader is not reading while prefixes come and go, or more prefixes
/// went at once than are left. A reader that had not read what goes
/// loses its place. Half, so that readers that are only a little
/// behind keep theirs, and so that getting a long journal down to size
/// takes few passes.
fn drop_oldest_half(&mut self) {
if self.entries.len() <= limit {
return;
}
// Still too long: a reader is not reading while prefixes come and
// go, or more prefixes went at once than are left. The older half
// goes, and a reader that had not read it loses its place. Half,
// so that readers that are only a little behind keep theirs, and
// so that getting a long journal down to size takes few passes.
for entry in self.entries.drain(..self.entries.len() / 2) {
Self::void_gone_note(&mut self.gone, &entry);
Self::drop_entry(&mut self.gone, &entry);
self.floor = entry.seq;
}
}
}
/// What an entry says; see [`Journal::says`].
enum Says<'a> {
/// The prefix's row, whose latest change the entry is.
Row(&'a Row),
/// The prefix has no route any more, and the entry is what says so.
Gone,
/// Nothing: the prefix has changed again since, and a later entry has
/// it.
Nothing,
}
/// Where a reader is to read a journal from; see [`Journal::place_of`].
enum Place {
/// Entries the reader had not read were dropped; it has lost its
/// place.
Lagged,
/// The reader has read everything up to the latest change.
Nothing,
/// The index of the first entry the reader has not read.
From(usize),
}
/// Which of a shard's two journals.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub(super) enum Kind {
@@ -299,36 +243,29 @@ pub(super) enum Kind {
All,
}
/// One of something for each [`Kind`] of journal.
#[derive(Debug, Clone, Copy, Default)]
pub(super) struct ByKind<T> {
best: T,
all: T,
}
impl<T> ByKind<T> {
pub(super) fn of(&self, kind: Kind) -> &T {
match kind {
Kind::Best => &self.best,
Kind::All => &self.all,
}
}
pub(super) fn of_mut(&mut self, kind: Kind) -> &mut T {
match kind {
Kind::Best => &mut self.best,
Kind::All => &mut self.all,
}
}
}
/// The sequence numbers a row remembers, one for each journal: of the
/// latest change of its best route, and of any of its routes.
pub(super) type Seqs = ByKind<u64>;
#[derive(Debug, Clone, Copy)]
pub(super) struct Seqs {
best: u64,
all: u64,
}
impl Seqs {
fn of(self, kind: Kind) -> u64 {
match kind {
Kind::Best => self.best,
Kind::All => self.all,
}
}
}
/// The two journals of a shard, written together: every change to a row is
/// a change of one of its routes, and some are a change of its best route.
pub(super) type Journals = ByKind<Journal>;
pub(super) struct Journals {
best: Journal,
all: Journal,
}
impl Journals {
pub(super) fn new(limits: Limits) -> Self {
@@ -338,20 +275,18 @@ impl Journals {
}
}
/// Copies the journals' sequence numbers to `latest`, which the table
/// keeps outside the shard's lock for readers to look at
/// ([`RibTable::latest_seq`]). Called with the lock held, after every
/// change to the shard.
pub(super) fn publish(&self, latest: &ByKind<AtomicU64>) {
latest.best.store(self.best.seq, Ordering::Release);
latest.all.store(self.all.seq, Ordering::Release);
fn of(&self, kind: Kind) -> &Journal {
match kind {
Kind::Best => &self.best,
Kind::All => &self.all,
}
}
/// The time by the table's clock that the changes made from now on
/// are given; see [`Entry::stamp`].
pub(super) fn set_stamp(&mut self, stamp: u64) {
self.best.stamp = stamp;
self.all.stamp = stamp;
fn of_mut(&mut self, kind: Kind) -> &mut Journal {
match kind {
Kind::Best => &mut self.best,
Kind::All => &mut self.all,
}
}
/// Notes that `prefix` has its first route, and so a row. Returns the
@@ -410,11 +345,18 @@ pub(super) struct Mark(u64, u64);
pub struct Routes<'a>(&'a Row);
impl<'a> Routes<'a> {
/// The routes: at least one, since a prefix with none has no row, and
/// is given out as [`Event::Withdrawn`] instead.
pub fn iter(&self) -> impl Iterator<Item = RouteRef<'a>> + use<'a> {
self.0.routes()
}
/// How many there are: at least one.
pub fn len(&self) -> usize {
self.0.paths.len()
}
pub fn is_empty(&self) -> bool {
self.len() == 0
}
}
impl std::fmt::Debug for Routes<'_> {
@@ -428,8 +370,9 @@ impl std::fmt::Debug for Routes<'_> {
pub enum Event<'a> {
/// Forget what you know. The whole table follows, as [`Event::Best`]
/// or [`Event::Routes`] for every prefix, and then [`Event::Synced`].
/// The first thing a new reader is given, and what a reader that fell
/// too far behind is given in place of what it missed.
/// The first thing a new
/// reader is given, and what a reader that fell too far behind is
/// given in place of what it missed.
Reset,
/// This is now the best route for `prefix`. Given to a follower of the
/// best routes.
@@ -465,30 +408,19 @@ pub struct Progress {
}
enum Phase {
/// Giving out the whole table; `next` is the shard to do next. `upto`
/// is the time on the table's clock the table is given out as of, and
/// is taken when [`Event::Reset`] is given, so `None` until then: the
/// shards are gone through one at a time while the table changes, and
/// the place taken in each shard's journal is as of that one time, so
/// that what follows is consistent from shard to shard. A change after
/// it that a later shard's rows already show is read again from the
/// journal; that is harmless.
Resync { next: usize, upto: Option<u64> },
/// Following the journals; `next` is the shard to read next, and
/// `upto` the time by the table's clock up to which this pass reads
/// (see [`Entry::stamp`]).
Follow { next: usize, upto: u64 },
/// Giving out the whole table; `next` is the shard to do next, and
/// `announced` whether [`Event::Reset`] has been given.
Resync { next: usize, announced: bool },
/// Following the journals; `next` is the shard to read next.
Follow { next: usize },
}
impl Phase {
/// The beginning of a resynchronisation.
const RESYNC: Self = Self::Resync {
next: 0,
upto: None,
announced: false,
};
/// The beginning of a pass over the journals.
const FOLLOW: Self = Self::Follow { next: 0, upto: 0 };
}
/// Follows the routes of a table: the best of each prefix, or all of them.
@@ -505,10 +437,7 @@ pub struct Reader {
/// is for the shard's benefit.
cursors: Vec<u64>,
phase: Phase,
/// The table's version as it was when the current pass over the
/// journals began. [`Reader::changed`] waits while the version is still
/// this one: whatever was written since the pass began has moved it
/// on, and is there to be read.
/// The table's version when the reader was last found caught up.
seen: u64,
caught_up: bool,
}
@@ -524,7 +453,7 @@ impl RibTable {
/// those who want the news and not the table. If it falls too far
/// behind it is reset like any other, and is then given the table.
pub fn reader_from_now(self: &Arc<Self>) -> Reader {
let mut reader = self.register(Kind::Best, Phase::FOLLOW);
let mut reader = self.register(Kind::Best, Phase::Follow { next: 0 });
for (index, shard) in self.shards.iter().enumerate() {
let shard = read(shard);
reader.set_cursor(index, &shard, shard.journals.best.seq);
@@ -618,14 +547,8 @@ impl Drop for Reader {
enum Drained {
/// Entries were dropped that this reader had not read.
Lagged,
/// `events` were given out; `consumed` if the reader's place in the
/// journal moved at all; `more` if entries were left past the time
/// read up to.
Read {
events: usize,
consumed: bool,
more: bool,
},
/// `events` were given out; `more` if the budget ran out first.
Read { events: usize, more: bool },
}
impl Reader {
@@ -640,128 +563,77 @@ impl Reader {
pub fn poll(&mut self, budget: usize, mut visit: impl FnMut(Event<'_>)) -> Progress {
// A call that may give nothing out would never get anywhere.
let budget = budget.max(1);
// Held once for the call: a count taken on the table for every
// shard of every pass, by every reader, was the most of what a
// router with many peers did.
let table = Arc::clone(&self.table);
let shards = self.table.shards.len();
let mut events = 0;
loop {
let done = match self.phase {
Phase::Resync { next, upto } => {
self.resync_step(&table, next, upto, budget, &mut events, &mut visit)
match self.phase {
Phase::Resync { next, announced } => {
if !announced {
self.forget_place();
visit(Event::Reset);
events += 1;
}
if next == shards {
visit(Event::Synced);
events += 1;
self.phase = Phase::Follow { next: 0 };
continue;
}
events += self.scan(next, &mut visit);
self.phase = Phase::Resync {
next: next + 1,
announced: true,
};
if events >= budget {
return self.progress(events, false);
}
}
Phase::Follow { next, upto } => {
self.follow_pass(&table, next, upto, budget, &mut events, &mut visit)
}
};
if let Some(progress) = done {
return progress;
}
}
}
/// One step of a resynchronisation: the shard `next`, or the end of it
/// once every shard has been given out. Returns what to tell the
/// caller if the budget is spent, and `None` to go on.
fn resync_step(
&mut self,
table: &RibTable,
next: usize,
upto: Option<u64>,
budget: usize,
events: &mut usize,
visit: &mut impl FnMut(Event<'_>),
) -> Option<Progress> {
let upto = upto.unwrap_or_else(|| {
self.forget_place();
visit(Event::Reset);
*events += 1;
table.settled()
});
if next == table.shards.len() {
visit(Event::Synced);
*events += 1;
self.phase = Phase::FOLLOW;
return None;
}
*events += self.scan(table, next, upto, visit);
self.phase = Phase::Resync {
next: next + 1,
upto: Some(upto),
};
(*events >= budget).then(|| self.progress(*events, false))
}
/// A pass over the journals from shard `next` on, or the rest of one
/// that an earlier call cut short. Returns what to tell the caller if
/// the pass is to be the call's last, and `None` to go round again.
fn follow_pass(
&mut self,
table: &RibTable,
next: usize,
upto: u64,
budget: usize,
events: &mut usize,
visit: &mut impl FnMut(Event<'_>),
) -> Option<Progress> {
// Read before the pass: a change made during it shows as a newer
// version, and `changed` then returns at once. And the pass reads
// up to one time in every shard alike, chosen so that about the
// budget is read.
let (version, upto) = if next == 0 {
(
table.version.load(Ordering::Acquire),
self.time_for(budget - *events),
)
} else {
(self.seen, upto)
};
self.seen = version;
let mut consumed_any = false;
let mut more_somewhere = false;
for shard in next..table.shards.len() {
if *events >= budget {
self.phase = Phase::Follow { next: shard, upto };
return Some(self.progress(*events, false));
}
// Nothing new in this shard, by the number that is read
// without its lock: it is left alone, and so are the sessions
// writing to it.
if self.cursors[shard] == table.latest_seq(shard, self.kind) {
continue;
}
match self.drain(table, shard, upto, visit) {
Drained::Lagged => {
self.phase = Phase::RESYNC;
return None;
}
Drained::Read {
events: read,
consumed,
more,
} => {
*events += read;
consumed_any |= consumed;
more_somewhere |= more;
Phase::Follow { next } => {
// Read before the pass: a change made during it shows as
// a newer version, and `changed` then returns at once.
let version = if next == 0 {
self.table.version.load(Ordering::Acquire)
} else {
self.seen
};
self.seen = version;
let mut shard = next;
let mut lagged = false;
while shard < shards {
if events >= budget {
self.phase = Phase::Follow { next: shard };
return self.progress(events, false);
}
match self.drain(shard, budget - events, &mut visit) {
Drained::Lagged => {
lagged = true;
break;
}
Drained::Read { events: read, more } => {
events += read;
if more {
self.phase = Phase::Follow { next: shard };
return self.progress(events, false);
}
}
}
shard += 1;
}
if lagged {
self.phase = Phase::RESYNC;
continue;
}
self.phase = Phase::Follow { next: 0 };
if next != 0 {
// This pass took up where an earlier call stopped.
// The shards before that point have not been looked
// at in this call, so go round again from the first.
continue;
}
return self.progress(events, true);
}
}
}
self.phase = Phase::FOLLOW;
if next == 0 && !consumed_any && more_somewhere {
// A whole pass got nowhere, though there is more: it is all
// beyond the earliest change still under way, which no pass
// can read past until it ends. Asking again at once would spin
// until then. The change wakes the readers when it ends.
return Some(self.progress(*events, true));
}
if next != 0 || more_somewhere {
// This pass took up where an earlier call stopped, so the
// shards before that point have not been looked at in this
// call; or some shard had more than its share, or changes
// later than this pass's time. Go round again from the first.
return (*events >= budget).then(|| self.progress(*events, false));
}
Some(self.progress(*events, true))
}
fn progress(&mut self, events: usize, caught_up: bool) -> Progress {
@@ -769,11 +641,6 @@ impl Reader {
Progress { events, caught_up }
}
/// Whether the last [`Reader::poll`] read everything there was.
pub fn caught_up(&self) -> bool {
self.caught_up
}
/// Waits until there may be something to read. Returns at once if the
/// last [`Reader::poll`] did not catch up.
pub async fn changed(&mut self) {
@@ -824,109 +691,52 @@ impl Reader {
shard.journals.of(self.kind).cursors[self.slot].store(cursor, Ordering::Release);
}
/// Gives out every prefix in a shard and takes a place in its journal
/// as of `upto` on the table's clock, in one hold of the shard: what
/// changed after that time is in the journal after that place, whether
/// or not the rows given out show it already.
fn scan(
&mut self,
table: &RibTable,
index: usize,
upto: u64,
visit: &mut impl FnMut(Event<'_>),
) -> usize {
/// Gives out every prefix in a shard and takes a place at the end of
/// its journal, in one hold of the shard: what changes afterwards is in
/// the journal after that place.
fn scan(&mut self, index: usize, visit: &mut impl FnMut(Event<'_>)) -> usize {
let table = Arc::clone(&self.table);
let shard = read(&table.shards[index]);
for row in &shard.rows {
visit(self.event_for(row));
}
let journal = shard.journals.of(self.kind);
let beyond = journal.entries.partition_point(|entry| entry.stamp <= upto);
let cursor = match (beyond.checked_sub(1), journal.entries.get(beyond)) {
// Nothing in the journal is later than `upto`.
(_, None) => journal.seq,
// The last entry that is not later; or, if every entry is
// later, just before the first.
(Some(last), Some(_)) => journal.entries[last].seq,
(None, Some(first)) => first.seq - 1,
};
self.set_cursor(index, &shard, cursor);
self.set_cursor(index, &shard, shard.journals.of(self.kind).seq);
shard.rows.len()
}
/// The time on the table's clock up to which a pass over the shards
/// is to read, for about `budget` events in all: changes made together
/// are read together, in whatever shards they are (see
/// [`Entry::stamp`]). Taken shard by shard to the budget, the first
/// shards would be read far ahead of the last, and the routes of one
/// UPDATE, spread over the shards by their prefixes, given out far
/// apart.
fn time_for(&self, budget: usize) -> u64 {
let shards = self.table.shards.len();
let share = (budget / shards).max(1);
// No further than the changes that are in every shard already.
let settled = self.table.settled();
let mut upto = settled;
// The earliest change not yet read: at least that one is, however
// many changes were made with it.
let mut earliest = None;
for (index, shard) in self.table.shards.iter().enumerate() {
let cursor = self.cursors[index];
if cursor == self.table.latest_seq(index, self.kind) {
continue;
}
let shard = read(shard);
let journal = shard.journals.of(self.kind);
let Place::From(start) = journal.place_of(cursor) else {
continue;
};
if let Some(next) = journal.entries.get(start) {
earliest = Some(earliest.map_or(next.stamp, |e: u64| e.min(next.stamp)));
}
// Everything before the change that would go over the share.
if let Some(beyond) = journal.entries.get(start + share) {
upto = upto.min(beyond.stamp - 1);
}
}
upto.max(earliest.unwrap_or(upto)).min(settled)
}
/// Reads a shard's journal from this reader's place, no further than
/// the changes made by `upto` on the table's clock.
fn drain(
&mut self,
table: &RibTable,
index: usize,
upto: u64,
visit: &mut impl FnMut(Event<'_>),
) -> Drained {
/// Reads a shard's journal from this reader's place, for at most `max`
/// events.
fn drain(&mut self, index: usize, max: usize, visit: &mut impl FnMut(Event<'_>)) -> Drained {
let table = Arc::clone(&self.table);
let shard = read(&table.shards[index]);
let journal = shard.journals.of(self.kind);
let mut cursor = self.cursors[index];
let start = match journal.place_of(cursor) {
Place::Lagged => return Drained::Lagged,
Place::Nothing => {
return Drained::Read {
events: 0,
consumed: false,
more: false,
};
}
Place::From(start) => start,
};
if cursor < journal.floor {
return Drained::Lagged;
}
if cursor == journal.seq {
return Drained::Read {
events: 0,
more: false,
};
}
let start = journal.entries.partition_point(|entry| entry.seq <= cursor);
let mut events = 0;
let mut more = false;
for entry in journal.entries.range(start..) {
if entry.stamp > upto {
if events == max {
more = true;
break;
}
cursor = entry.seq;
let event = match journal.says(entry, &shard.rows, &shard.hasher) {
Says::Row(row) => self.event_for(row),
Says::Gone => Event::Withdrawn {
let event = match journal.live(entry, &shard.rows, &shard.hasher) {
Some(Some(row)) => self.event_for(row),
Some(None) => Event::Withdrawn {
prefix: entry.prefix,
},
Says::Nothing => continue,
// The prefix has changed again; a later entry has it.
None => continue,
};
visit(event);
events += 1;
@@ -935,13 +745,8 @@ impl Reader {
// Past the last entry there is nothing up to the latest change.
cursor = journal.seq;
}
let consumed = cursor != self.cursors[index];
self.set_cursor(index, &shard, cursor);
Drained::Read {
events,
consumed,
more,
}
Drained::Read { events, more }
}
}
crates/rib/src/table/journal_tests.rs
+5 -198
View File
@@ -8,7 +8,7 @@ use proptest::prelude::*;
use super::super::tests::{Fixture, Route, external, internal, prefix, route};
use super::super::{AttrSet, PeerIndex, PeerInfo, RibTable};
use super::{Event, Kind, Limits, Reader, read};
use super::{Event, Limits, Reader};
/// What a follower would keep: its own copy of the best routes, brought
/// up to date from the events it is given.
@@ -53,10 +53,7 @@ impl Mirror {
}
Event::Routes { prefix, routes } => {
self.routes_events += 1;
assert!(
routes.iter().next().is_some(),
"a prefix with no routes is withdrawn"
);
assert!(!routes.is_empty(), "a prefix with no routes is withdrawn");
self.routes.insert(
prefix,
sorted(
@@ -118,6 +115,9 @@ impl Mirror {
);
}
}
}
impl Mirror {
/// For the mirror of a follower of all routes: it holds every route in
/// the table, and knows which is best.
#[track_caller]
@@ -213,107 +213,6 @@ fn a_reader_from_now_is_given_the_changes_and_not_the_table() {
assert_eq!((best, withdrawn), (vec![new], vec![old]));
}
/// The number a reader checks without a shard's lock must be the journal's
/// own after every kind of change, whether or not anyone is reading: a
/// reader skips a shard on the strength of it, and a shard with news that
/// said it had none would never be read until the next change.
#[test]
fn the_lock_free_number_is_the_journals_after_every_kind_of_change() {
let mut fixture = Fixture::new(&[external(1), external(2)]);
let prefixes: Vec<IpPrefix> = (0..16).map(nth_prefix).collect();
let agree = |table: &RibTable| {
for (index, shard) in table.shards.iter().enumerate() {
let shard = read(shard);
for kind in [Kind::Best, Kind::All] {
assert_eq!(
table.latest_seq(index, kind),
shard.journals.of(kind).seq,
"shard {index}, {kind:?}"
);
}
}
};
// Without a reader, with one of each kind, and with both.
for readers in 0..4 {
let _best = (readers & 1 != 0).then(|| fixture.table.reader());
let _all = (readers & 2 != 0).then(|| fixture.table.reader_of_all_routes());
agree(&fixture.table);
for p in &prefixes {
fixture.announce(0, p, &Route::via(&[1]));
}
agree(&fixture.table);
// The same prefixes again, with other attributes.
for p in &prefixes {
fixture.announce(0, p, &Route::via(&[1, 2]));
}
agree(&fixture.table);
// A second route for each, then the last route of one taken away.
for p in &prefixes {
fixture.announce(1, p, &Route::via(&[2, 3]));
}
agree(&fixture.table);
fixture.withdraw(0, &prefixes[0]);
agree(&fixture.table);
fixture.withdraw(1, &prefixes[0]);
agree(&fixture.table);
fixture.table.remove_peer(fixture.peers[1]);
agree(&fixture.table);
fixture.peers[1] = fixture.table.add_peer(external(2)).unwrap();
fixture
.table
.mark_stale(fixture.peers[0], &[IpVersion::Ipv4]);
agree(&fixture.table);
fixture
.table
.sweep_stale(fixture.peers[0], &[IpVersion::Ipv4]);
agree(&fixture.table);
assert!(
fixture.table.is_empty(),
"round {readers} left routes behind"
);
}
}
/// A shard that had nothing new for a reader is passed over without its
/// lock. The next change to it must bring the reader back.
#[test]
fn a_shard_passed_over_is_read_again_once_it_has_news() {
let fixture = Fixture::on(RibTable::new(1), &[external(1)]);
let mut reader = fixture.table.reader();
let mut mirror = Mirror::default();
mirror.catch_up(&mut reader, 100);
assert_eq!(reader.poll(100, |_| panic!("no event expected")).events, 0);
fixture.announce(0, &prefix("10.0.0.0/8"), &Route::via(&[1]));
let mut events = Vec::new();
let progress = reader.poll(100, |event| events.push(format!("{event:?}")));
assert!(progress.caught_up);
assert_eq!(events.len(), 1, "{events:?}");
assert!(events[0].starts_with("Best"), "{events:?}");
}
/// As [`a_shard_passed_over_is_read_again_once_it_has_news`], for a reader
/// that begins where the journals are rather than with the table.
#[test]
fn a_shard_passed_over_by_a_reader_from_now_is_read_again_once_it_has_news() {
let fixture = Fixture::on(RibTable::new(1), &[external(1)]);
fixture.announce(0, &prefix("192.0.2.0/24"), &Route::via(&[1]));
let mut reader = fixture.table.reader_from_now();
assert!(reader.poll(100, |_| panic!("no event expected")).caught_up);
assert_eq!(reader.poll(100, |_| panic!("no event expected")).events, 0);
fixture.announce(0, &prefix("10.0.0.0/8"), &Route::via(&[1]));
let mut events = Vec::new();
let progress = reader.poll(100, |event| events.push(format!("{event:?}")));
assert!(progress.caught_up);
assert_eq!(events.len(), 1, "{events:?}");
assert!(events[0].starts_with("Best"), "{events:?}");
}
/// A change to a route that is not the best is nothing to a follower of
/// the best routes, and is what a follower of all routes is there for.
#[test]
@@ -618,45 +517,6 @@ async fn a_waiting_reader_is_woken_by_a_change_of_best_route() {
.expect("the change was already there");
}
/// A reader does not read past a change that is still under way, which
/// a change to every shard (a peer's whole table going) is for a while.
/// Changes made by others meanwhile are beyond it. A reader that finds
/// only those must say it is caught up and wait, not ask again at once
/// for as long as the change takes; and when the change ends, it is
/// woken, though the change itself wrote nothing for it.
#[tokio::test]
async fn a_change_under_way_does_not_keep_a_reader_spinning() {
let fixture = Fixture::new(&[external(1), external(2)]);
let mut reader = fixture.table.reader();
let mut mirror = Mirror::default();
mirror.catch_up(&mut reader, 100);
// A long change begins, and another writer announces meanwhile.
let change = fixture.table.begin_change();
fixture.announce(1, &prefix("10.0.0.0/8"), &Route::via(&[2]));
let progress = reader.poll(100, |_| {});
assert_eq!(progress.events, 0, "nothing can be read past the change");
assert!(
progress.caught_up,
"and the reader is to wait, not ask again"
);
let waiting = tokio::time::timeout(std::time::Duration::from_millis(50), reader.changed());
assert!(
waiting.await.is_err(),
"nothing to be had until the change ends"
);
// The change ends without writing anything itself.
drop(change);
tokio::time::timeout(std::time::Duration::from_secs(5), reader.changed())
.await
.expect("woken when the change ended");
mirror.catch_up(&mut reader, 100);
mirror.assert_matches(&fixture.table);
assert_eq!(mirror.best.len(), 1);
}
/// Writers on several threads, a reader on another, reading while they
/// write. When the writers are done and the reader has caught up, it has
/// what the table has.
@@ -851,56 +711,3 @@ proptest! {
}
}
}
/// Routes that were announced together, with one set of attributes, are
/// spread over the shards by their prefixes. A reader that takes its
/// budget from the shards evenly is given them together all the same,
/// which is what lets whoever passes them on send them in one message.
#[test]
fn changes_made_together_are_read_together() {
let fixture = Fixture::on(RibTable::new(128), &[external(1)]);
let mut reader = fixture.table.reader();
assert!(reader.poll(10, |_| {}).caught_up);
// 6000 UPDATEs of six prefixes each, every one with attributes of its
// own.
let groups = 6000u32;
for group in 0..groups {
let attrs = fixture
.table
.attrs()
.intern(&Route::via(&[1, 100_000 + group]).attributes(), None)
.unwrap();
let prefixes: Vec<IpPrefix> = (0..6).map(|n| nth_prefix(group * 6 + n)).collect();
fixture.table.announce(fixture.peers[0], &attrs, &prefixes);
}
// Read in batches a tenth of that size: each should hold whole groups,
// but for the ones cut by its edges.
let mut batches = 0;
let mut split_groups = 0;
let mut seen_total = 0;
while seen_total < groups * 6 {
let mut groups_in_batch: HashMap<usize, usize> = HashMap::new();
let progress = reader.poll(3600, |event| {
if let Event::Best { attrs, .. } = event {
*groups_in_batch
.entry(Arc::as_ptr(attrs) as usize)
.or_default() += 1;
}
});
seen_total += groups_in_batch.values().sum::<usize>() as u32;
split_groups += groups_in_batch.values().filter(|n| **n < 6).count();
batches += 1;
if progress.events == 0 {
break;
}
}
assert_eq!(seen_total, groups * 6);
// A few are, where a batch's time falls inside one: the reader cannot
// know what was written together until it is all there.
assert!(
split_groups < 100,
"{split_groups} of {groups} groups were split over {batches} batches"
);
}
crates/rib/src/table/mod.rs
+59 -384
View File
@@ -22,7 +22,7 @@
//! - **Readers do not stop the table.** Reading takes a shard's lock for
//! reading, one shard at a time, and looks at the rows in place.
//! - **Followers go at their own pace.** What acts on the best routes reads
//! the changes from a `journal`, and a slow reader holds nothing up.
//! the changes from a [`journal`], and a slow reader holds nothing up.
pub mod attrs;
mod journal;
@@ -30,13 +30,11 @@ mod select;
#[cfg(test)]
mod tests;
use std::collections::BTreeSet;
use std::hash::BuildHasher;
use std::net::{IpAddr, Ipv4Addr};
use std::sync::atomic::{AtomicU64, AtomicUsize};
use std::sync::{Arc, Mutex, PoisonError, RwLock, RwLockReadGuard, RwLockWriteGuard};
use bgp_primitives::constants::IpVersion;
use bgp_primitives::ip_prefix::{IpPrefix, Nlri};
use foldhash::fast::RandomState;
use hashbrown::HashTable;
@@ -53,15 +51,8 @@ pub struct PeerInfo {
/// The peer's BGP Identifier.
pub router_id: Ipv4Addr,
pub address: IpAddr,
/// The number of the interface the peer is on, if its address is a
/// link-local one; zero otherwise.
pub interface: u32,
/// Whether the peer is in our own AS.
pub ibgp: bool,
/// Whether routes learned from our other routers are passed on to the
/// peer, and its routes to them: whether it is a route reflector
/// client of ours (RFC 4456).
pub reflector_client: bool,
}
/// A peer's number in a table, given out by [`RibTable::add_peer`]. Small,
@@ -90,9 +81,6 @@ pub struct Outcome {
pub removed: usize,
/// Prefixes whose best route is now another one, or none.
pub best_changed: usize,
/// Stale routes that are no longer: replaced or withdrawn by their
/// peer (graceful restart; see [`RibTable::mark_stale`]).
pub stale_cleared: usize,
}
impl std::ops::AddAssign for Outcome {
@@ -101,31 +89,20 @@ impl std::ops::AddAssign for Outcome {
self.replaced += other.replaced;
self.removed += other.removed;
self.best_changed += other.best_changed;
self.stale_cleared += other.stale_cleared;
}
}
/// One route of a prefix: sixteen bytes.
struct Path {
/// Three things in one word. The high 48 bits are the first steps of
/// Two things in one word. The high 48 bits are the first steps of
/// route selection as one number, the larger the better; see
/// [`select::preference`]. The low 16 are the peer the route is from.
/// And one bit that the number never uses marks the route stale; see
/// [`STALE`].
word: u64,
attrs: Arc<AttrSet>,
}
/// The bit of a [`Path`]'s word that marks a route stale: kept from a
/// session that ended, until the peer announces it again or it is swept
/// away (graceful restart, RFC 4724 §4.2). It is the lowest bit of the
/// preference, which is always zero ([`select::preference`]), and is left
/// out of it: a stale route is chosen like any other, as the RFC has it.
const STALE: u64 = 1 << 16;
impl Path {
fn new(preference: u64, peer: PeerIndex, attrs: &Arc<AttrSet>) -> Self {
debug_assert_eq!(preference & 1, 0, "the bit for stale routes is used");
Self {
word: (preference << 16) | u64::from(peer.0),
attrs: Arc::clone(attrs),
@@ -133,11 +110,7 @@ impl Path {
}
fn preference(&self) -> u64 {
(self.word & !STALE) >> 16
}
fn is_stale(&self) -> bool {
self.word & STALE != 0
self.word >> 16
}
fn peer(&self) -> PeerIndex {
@@ -193,16 +166,18 @@ impl Row {
.position(|(index, path)| path.peer() == peer && self.path_id(index) == path_id)
}
/// Adds a route. `peers` is how many peers the table has: a prefix
/// that one of them announces, the others mostly announce too, so a
/// row is given room for that many at its first growth.
fn push(&mut self, path: Path, path_id: u32, peers: usize) {
/// Where a route from `peer` is, whatever its identifier.
fn position_of_peer(&self, peer: PeerIndex) -> Option<usize> {
self.paths.iter().position(|path| path.peer() == peer)
}
fn push(&mut self, path: Path, path_id: u32) {
if path_id != 0 && self.path_ids.is_empty() {
self.path_ids = vec![0; self.paths.len()];
}
push_tight(&mut self.paths, path, peers);
push_tight(&mut self.paths, path);
if !self.path_ids.is_empty() {
push_tight(&mut self.path_ids, path_id, peers);
push_tight(&mut self.path_ids, path_id);
}
}
@@ -232,7 +207,6 @@ impl Row {
path_id: self.path_id(index),
attrs: &path.attrs,
best: index == self.best(),
stale: path.is_stale(),
}
}
@@ -241,23 +215,6 @@ impl Row {
}
}
/// A prefix in the table with its routes, looked at in place.
#[derive(Clone, Copy)]
pub struct PrefixRef<'a> {
row: &'a Row,
}
impl<'a> PrefixRef<'a> {
pub fn prefix(&self) -> &'a IpPrefix {
&self.row.prefix
}
/// The routes there are for the prefix: at least one.
pub fn routes(&self) -> impl Iterator<Item = RouteRef<'a>> + use<'a> {
self.row.routes()
}
}
/// A route in the table, looked at in place.
#[derive(Debug, Clone, Copy)]
pub struct RouteRef<'a> {
@@ -270,9 +227,6 @@ pub struct RouteRef<'a> {
pub attrs: &'a Arc<AttrSet>,
/// Whether it is the route chosen for its prefix.
pub best: bool,
/// Whether it is kept from a session of the peer's that ended, and has
/// not been announced again since (graceful restart).
pub stale: bool,
}
impl RouteRef<'_> {
@@ -284,7 +238,6 @@ impl RouteRef<'_> {
path_id: self.path_id,
attrs: Arc::clone(self.attrs),
best: self.best,
stale: self.stale,
}
}
}
@@ -297,7 +250,6 @@ pub struct Route {
pub path_id: u32,
pub attrs: Arc<AttrSet>,
pub best: bool,
pub stale: bool,
}
struct Shard {
@@ -313,21 +265,10 @@ struct Shard {
journals: journal::Journals,
}
/// A shard, and beside its lock what is read without taking it.
struct ShardSlot {
lock: RwLock<Shard>,
/// The sequence number of each of the shard's journals as of the last
/// change to the shard, stored by the change while it still holds the
/// lock ([`RibTable::change`]). A reader looks at it first, and passes
/// over a shard with nothing new for it without stopping the shard's
/// writers.
latest: journal::ByKind<AtomicU64>,
}
/// The routing table. Shared between threads by reference: nothing takes
/// `&mut self`, and the readers ([`Reader`]) keep an `Arc` of their own.
/// The routing table. Shared between threads by reference; every method
/// takes `&self`.
pub struct RibTable {
shards: Box<[ShardSlot]>,
shards: Box<[RwLock<Shard>]>,
hasher: RandomState,
attrs: AttrStore,
/// Which peer numbers are in use. Changing the peer list also takes
@@ -340,13 +281,6 @@ pub struct RibTable {
/// Goes up whenever something is written to a journal that has
/// readers.
version: AtomicU64,
/// Counts the calls that change the table, so that a change can say
/// when it was made relative to others (see the journal's `Entry`).
clock: AtomicU64,
/// The changes being made right now, by their time on the clock: a
/// reader does not read past the earliest of them, which may not have
/// reached every shard yet.
in_progress: Mutex<BTreeSet<u64>>,
wake: Notify,
}
@@ -368,16 +302,15 @@ impl RibTable {
let hasher = RandomState::default();
Self {
shards: (0..shards.max(1))
.map(|_| ShardSlot {
lock: RwLock::new(Shard {
.map(|_| {
RwLock::new(Shard {
rows: HashTable::new(),
routes: 0,
peers: Vec::new(),
hasher: hasher.clone(),
scratch: select::Scratch::default(),
journals: journal::Journals::new(limits),
}),
latest: journal::ByKind::default(),
})
})
.collect(),
hasher,
@@ -386,8 +319,6 @@ impl RibTable {
reader_slots: Mutex::default(),
readers: AtomicUsize::new(0),
version: AtomicU64::new(0),
clock: AtomicU64::new(0),
in_progress: Mutex::default(),
wake: Notify::new(),
}
}
@@ -397,13 +328,6 @@ impl RibTable {
&self.attrs
}
/// What is known of the peer with this number, if it is in use. Not to
/// be called from a visitor that is given routes with a shard held.
pub fn peer(&self, index: PeerIndex) -> Option<PeerInfo> {
let peers = self.peers.lock().unwrap_or_else(PoisonError::into_inner);
peers.get(index.as_usize()).copied().flatten()
}
/// Makes room for a peer's routes and returns the number to announce
/// them under.
///
@@ -443,105 +367,30 @@ impl RibTable {
/// held while it does, so other peers can come and go meanwhile.
pub fn remove_peer(&self, peer: PeerIndex) -> Outcome {
let mut outcome = Outcome::default();
self.change(self.shards.iter().map(|shard| (shard, ())), |shard, ()| {
let mut news = false;
for shard in &self.shards {
let mut shard = write(shard);
let before = shard.journals.mark();
outcome += shard.remove_peer(peer);
// From here on the shard takes no more routes from the peer.
if let Some(slot) = shard.peers.get_mut(peer.as_usize()) {
*slot = None;
}
});
shard.tidy_journals();
news |= shard.journals.has_news_since(before);
}
// Only now can the number be given out again: no shard knows it.
let mut peers = self.peers.lock().unwrap_or_else(PoisonError::into_inner);
if let Some(slot) = peers.get_mut(peer.as_usize()) {
*slot = None;
}
drop(peers);
if news {
self.wake_readers();
}
outcome
}
/// Gives a peer number that is in use to a new session of the same
/// peer, whose identifier or address may have changed: what route
/// selection knows of the peer is replaced. Returns whether the number
/// was in use.
///
/// This is how a peer that restarts finds the routes kept from its last
/// session (graceful restart): they are still under its number.
pub fn rebind_peer(&self, peer: PeerIndex, info: PeerInfo) -> bool {
let mut peers = self.peers.lock().unwrap_or_else(PoisonError::into_inner);
match peers.get_mut(peer.as_usize()) {
Some(slot @ Some(_)) => *slot = Some(info),
_ => return false,
}
for shard in &self.shards {
if let Some(slot) = write(shard).peers.get_mut(peer.as_usize()) {
*slot = Some(info);
}
}
true
}
/// Marks every route of `peer` for prefixes of the IP versions in
/// `versions` stale, and returns how many there are. They stay in the
/// table and are chosen as before; announcing one again makes it fresh,
/// and [`RibTable::sweep_stale`] takes out those that are still stale.
///
/// For graceful restart (RFC 4724 §4.2): the routes of a peer that is
/// restarting are kept while it does. Like [`RibTable::remove_peer`]
/// this looks at every prefix in the table. It changes no route, so
/// the readers are not told of anything.
pub fn mark_stale(&self, peer: PeerIndex, versions: &[IpVersion]) -> usize {
let mut marked = 0;
for shard in &self.shards {
let mut shard = write(shard);
for row in shard.rows.iter_mut() {
if !versions.contains(&row.prefix.address_family()) {
continue;
}
for path in &mut row.paths {
if path.peer() == peer {
path.word |= STALE;
marked += 1;
}
}
}
}
marked
}
/// Takes out the routes of `peer` for prefixes of the IP versions in
/// `versions` that are still stale; see [`RibTable::mark_stale`]. The
/// peer keeps its number.
///
/// Safe to run while the peer announces routes: one that is announced
/// again, before or during this, is fresh and stays.
pub fn sweep_stale(&self, peer: PeerIndex, versions: &[IpVersion]) -> Outcome {
let mut outcome = Outcome::default();
self.change(self.shards.iter().map(|shard| (shard, ())), |shard, ()| {
outcome += shard.remove_where(|row, path| {
path.peer() == peer
&& path.is_stale()
&& versions.contains(&row.prefix.address_family())
});
});
outcome
}
/// How many routes of `peer` are stale; see [`RibTable::mark_stale`].
/// Looks at every prefix in the table.
#[cfg(test)]
fn count_stale(&self, peer: PeerIndex) -> usize {
self.shards
.iter()
.map(|shard| {
read(shard)
.rows
.iter()
.flat_map(|row| &row.paths)
.filter(|path| path.peer() == peer && path.is_stale())
.count()
})
.sum()
}
/// Announces `routes` from `peer`, all with the attributes `attrs`:
/// what one UPDATE does. A route is known by its prefix and path
/// identifier together ([`Nlri`]; a plain [`IpPrefix`] is the route
@@ -554,11 +403,11 @@ impl RibTable {
{
let mut outcome = Outcome::default();
self.by_shard(routes, |shard, placed| {
let Some(_) = shard.peers.get(peer.as_usize()).copied().flatten() else {
let Some(info) = shard.peers.get(peer.as_usize()).copied().flatten() else {
// The peer was removed while this was on its way.
return;
};
let preference = select::preference(attrs.key());
let preference = select::preference(attrs.key(), &info);
for (hash, nlri) in placed {
outcome += shard.announce(*hash, *nlri, peer, preference, attrs);
}
@@ -588,7 +437,7 @@ impl RibTable {
/// Calls `apply` once for each shard that some of `routes` fall in,
/// with that shard locked and those routes. Wakes the readers if a
/// journal one of them follows was written to.
fn by_shard<N>(&self, routes: &[N], apply: impl FnMut(&mut Shard, &[(u64, Nlri)]))
fn by_shard<N>(&self, routes: &[N], mut apply: impl FnMut(&mut Shard, &[(u64, Nlri)]))
where
N: Copy + Into<Nlri>,
{
@@ -614,81 +463,19 @@ impl RibTable {
};
placed.sort_unstable_by_key(|(hash, _)| self.shard_of(*hash));
let runs = placed
.chunk_by(|a, b| self.shard_of(a.0) == self.shard_of(b.0))
.map(|run| (&self.shards[self.shard_of(run[0].0)], run));
self.change(runs, apply);
}
/// Makes one change to the table, part by part: `apply` is called with
/// each shard that `parts` names locked for writing, and the part's
/// own item. All the parts are one change, at one time on the clock.
///
/// Readers are woken once the change is complete in every shard, if a
/// journal one of them follows was written to.
fn change<'a, T>(
&'a self,
parts: impl IntoIterator<Item = (&'a ShardSlot, T)>,
mut apply: impl FnMut(&mut Shard, T),
) {
let change = self.begin_change();
let mut news = false;
for (slot, part) in parts {
let mut shard = write(slot);
shard.journals.set_stamp(change.stamp);
for run in placed.chunk_by(|a, b| self.shard_of(a.0) == self.shard_of(b.0)) {
let mut shard = write(&self.shards[self.shard_of(run[0].0)]);
let before = shard.journals.mark();
apply(&mut shard, part);
apply(&mut shard, run);
shard.tidy_journals();
news |= shard.journals.has_news_since(before);
// Once per change, not once per route; and before the lock is
// let go, so that a reader that sees the new number and takes
// the lock finds the entries behind it.
shard.journals.publish(&slot.latest);
}
drop(change);
if news {
self.wake_readers();
}
}
/// Takes a time on the clock for a change that is about to be made,
/// and notes that it is under way until the guard drops.
fn begin_change(&self) -> Change<'_> {
let version = self.version.load(std::sync::atomic::Ordering::Acquire);
let stamp = self.clock.fetch_add(1, std::sync::atomic::Ordering::AcqRel) + 1;
self.in_progress
.lock()
.unwrap_or_else(PoisonError::into_inner)
.insert(stamp);
Change {
table: self,
stamp,
version,
}
}
/// The sequence number of shard `shard`'s journal of `kind` as of the
/// last change to the shard, without the shard's lock.
fn latest_seq(&self, shard: usize, kind: journal::Kind) -> u64 {
self.shards[shard]
.latest
.of(kind)
.load(std::sync::atomic::Ordering::Acquire)
}
/// The time on the clock up to which every change is complete in every
/// shard: everything before the earliest one still under way.
fn settled(&self) -> u64 {
let in_progress = self
.in_progress
.lock()
.unwrap_or_else(PoisonError::into_inner);
match in_progress.first() {
Some(earliest) => earliest - 1,
None => self.clock.load(std::sync::atomic::Ordering::Acquire),
}
}
/// How many routes the table holds, from all peers.
pub fn len(&self) -> usize {
self.shards.iter().map(|shard| read(shard).routes).sum()
@@ -698,31 +485,17 @@ impl RibTable {
self.len() == 0
}
/// Makes room for `prefixes` prefixes, spread over the shards, so that
/// a table of that size goes in without the shards growing as it
/// does. Growing a shard moves every row it has into a bigger table;
/// over a full table that was a seventh of what a session spent
/// taking it in. Room that is already there is kept.
pub fn expect_prefixes(&self, prefixes: usize) {
let each = prefixes.div_ceil(self.shards.len());
for shard in &self.shards {
let mut shard = write(shard);
let hasher = shard.hasher.clone();
let additional = each.saturating_sub(shard.rows.len());
shard
.rows
.reserve(additional, |row| hasher.hash_one(row.prefix));
}
}
/// How many prefixes have at least one route.
pub fn prefix_count(&self) -> usize {
self.shards.iter().map(|shard| read(shard).rows.len()).sum()
}
/// The best route for `prefix`: who it is from, and its attributes.
pub fn best(&self, prefix: &IpPrefix) -> Option<Route> {
self.with_row(prefix, |row| row.route(row.best()).to_owned())
pub fn best(&self, prefix: &IpPrefix) -> Option<(PeerIndex, Arc<AttrSet>)> {
self.with_row(prefix, |row| {
let best = &row.paths[row.best()];
(best.peer(), Arc::clone(&best.attrs))
})
}
/// Every route for `prefix`.
@@ -731,23 +504,6 @@ impl RibTable {
.unwrap_or_default()
}
/// Every route for `prefix`, from the best down: after the best, the
/// route that would be chosen were the ones before it gone, and so on.
pub fn ranked_routes(&self, prefix: &IpPrefix) -> Vec<Route> {
let hash = self.hasher.hash_one(prefix);
let shard = read(&self.shards[self.shard_of(hash)]);
shard
.rows
.find(hash, |row| row.prefix == *prefix)
.map(|row| {
select::ranked(row, &shard.peers)
.into_iter()
.map(|index| row.route(index).to_owned())
.collect()
})
.unwrap_or_default()
}
fn with_row<T>(&self, prefix: &IpPrefix, read_row: impl FnOnce(&Row) -> T) -> Option<T> {
let hash = self.hasher.hash_one(prefix);
let shard = read(&self.shards[self.shard_of(hash)]);
@@ -773,16 +529,14 @@ impl RibTable {
}
}
/// Up to `limit` of the routes that `wanted` picks, in no particular
/// order. This looks at routes until it has enough, which for a choosy
/// `wanted` is every route in the table: it is for looking into the
/// table by hand, not for anything done often. `wanted` is called with
/// a shard held and must not call into the table.
pub fn routes_where(&self, limit: usize, wanted: impl Fn(&RouteRef<'_>) -> bool) -> Vec<Route> {
/// Up to `limit` routes: every route of as many prefixes as fit, in no
/// particular order. For showing some of a table that is too large to
/// show all of.
pub fn some_routes(&self, limit: usize) -> Vec<Route> {
let mut routes = Vec::new();
for shard in &self.shards {
for row in &read(shard).rows {
for route in row.routes().filter(&wanted) {
for route in row.routes() {
if routes.len() == limit {
return routes;
}
@@ -793,21 +547,6 @@ impl RibTable {
routes
}
/// Calls `visit` with every prefix that has routes, in no particular
/// order. As [`RibTable::for_each_best`]: the table is not stopped,
/// and `visit` must not call into it.
///
/// The prefix's routes are there to be looked at, and cost nothing if
/// they are not: this is how to pick a few prefixes out of a table
/// with a great many routes for each.
pub fn for_each_prefix(&self, mut visit: impl FnMut(PrefixRef<'_>)) {
for shard in &self.shards {
for row in &read(shard).rows {
visit(PrefixRef { row });
}
}
}
/// Calls `visit` with every route in the table. As
/// [`RibTable::for_each_best`]: the table is not stopped, and `visit`
/// must not call into it.
@@ -820,44 +559,14 @@ impl RibTable {
}
}
fn read(shard: &ShardSlot) -> RwLockReadGuard<'_, Shard> {
fn read(shard: &RwLock<Shard>) -> RwLockReadGuard<'_, Shard> {
// A panic under the lock cannot leave a row half changed in a way that
// is unsafe to read; the poison is ignored, as for the server's tables.
shard.lock.read().unwrap_or_else(PoisonError::into_inner)
shard.read().unwrap_or_else(PoisonError::into_inner)
}
fn write(shard: &ShardSlot) -> RwLockWriteGuard<'_, Shard> {
shard.lock.write().unwrap_or_else(PoisonError::into_inner)
}
/// A change to the table that is under way; see [`RibTable::begin_change`].
struct Change<'a> {
table: &'a RibTable,
stamp: u64,
/// The readers' version when it began, to tell whether they were
/// woken while it was under way.
version: u64,
}
impl Drop for Change<'_> {
fn drop(&mut self) {
self.table
.in_progress
.lock()
.unwrap_or_else(PoisonError::into_inner)
.remove(&self.stamp);
// A reader does not read past a change still under way. One that
// another change woke meanwhile found nothing it could read yet,
// and is woken again now that it can.
if self
.table
.version
.load(std::sync::atomic::Ordering::Acquire)
!= self.version
{
self.table.wake_readers();
}
}
fn write(shard: &RwLock<Shard>) -> RwLockWriteGuard<'_, Shard> {
shard.write().unwrap_or_else(PoisonError::into_inner)
}
/// Picks one of `count` buckets by a hash, for spreading keys over shards
@@ -865,45 +574,20 @@ impl Drop for Change<'_> {
///
/// That table takes a key's slot from the low bits of the hash and, from
/// the top seven, the tag it compares before it looks at the key itself
/// (hashbrown). Taken from either end, the keys of one bucket would all
/// share those bits, and the table inside would lose them. Nor is it
/// enough to take the bits in between: the hash is one multiplication,
/// which for keys that are numbers in a row (prefixes are) can leave
/// neighbouring bits alike. So the hash is stirred once more, and the
/// bucket taken from the top of that, which every bit of the hash has a
/// part in.
/// (hashbrown). The bucket is taken from bits in between: taken from
/// either end, the keys of one bucket would all share those bits, and the
/// table inside would lose them.
fn spread(hash: u64, count: usize) -> usize {
// 2^64 divided by the golden ratio: the multiplier of Fibonacci
// hashing.
const STIR: u64 = 0x9E37_79B9_7F4A_7C15;
let stirred = hash.wrapping_mul(STIR) >> 32;
((stirred * count as u64) >> 32) as usize
let middle = (hash << 7) >> 32;
((middle * count as u64) >> 32) as usize
}
/// The most room a row is given at its first growth, however many peers
/// there are: a route server's clients do not all announce every prefix.
const FIRST_ROOM: usize = 8;
/// Adds to a vector without the doubling that `push` does: a row of a few
/// hundred routes should not hold room for as many again. The first
/// growth is to `expected` (at most [`FIRST_ROOM`]), since with several
/// full tables nearly every row gets that many routes, and growing a row
/// one route at a time was a twelfth of what taking in four tables cost.
/// After that it grows by a quarter, which keeps adding cheap on average.
fn push_tight<T>(vec: &mut Vec<T>, value: T, expected: usize) {
/// hundred routes should not hold room for as many again. Grows by a
/// quarter, which keeps adding cheap on average.
fn push_tight<T>(vec: &mut Vec<T>, value: T) {
if vec.len() == vec.capacity() {
let first = expected.min(FIRST_ROOM);
let more = if vec.capacity() < first {
first - vec.len()
} else if vec.capacity() < FIRST_ROOM {
// A peer that sends several routes for a prefix (ADD-PATH)
// takes a row past the peers' count: doubled while small,
// and from there by a quarter.
vec.capacity()
} else {
(vec.len() / 4).max(1)
};
vec.reserve_exact(more);
vec.reserve_exact((vec.len() / 4).max(1));
}
vec.push(value);
}
@@ -945,7 +629,7 @@ impl Shard {
let (outcome, best_changed) = match row.position(peer, path_id) {
None => {
row.push(path, path_id, self.peers.len());
row.push(path, path_id);
self.routes += 1;
let outcome = Outcome {
added: 1,
@@ -960,7 +644,6 @@ impl Shard {
let before = std::mem::replace(&mut row.paths[at], path);
let outcome = Outcome {
replaced: 1,
stale_cleared: usize::from(before.is_stale()),
..Outcome::default()
};
if Arc::ptr_eq(&before.attrs, attrs) && before.preference() == preference {
@@ -994,14 +677,12 @@ impl Shard {
return Outcome::default();
};
self.routes -= 1;
let stale_cleared = usize::from(row.paths[at].is_stale());
if row.paths.len() == 1 {
entry.remove();
self.journals.disappeared(prefix);
return Outcome {
removed: 1,
best_changed: 1,
stale_cleared,
..Outcome::default()
};
}
@@ -1010,17 +691,11 @@ impl Shard {
Outcome {
removed: 1,
best_changed: best_changed.into(),
stale_cleared,
..Outcome::default()
}
}
fn remove_peer(&mut self, peer: PeerIndex) -> Outcome {
self.remove_where(|_, path| path.peer() == peer)
}
/// Takes out every route that `doomed` picks, given its row.
fn remove_where(&mut self, doomed: impl Fn(&Row, &Path) -> bool) -> Outcome {
let mut outcome = Outcome::default();
let peers = &self.peers;
let scratch = &mut self.scratch;
@@ -1030,7 +705,7 @@ impl Shard {
let mut best_changed = false;
// A peer that sends several routes for a prefix has several
// here.
while let Some(at) = row.paths.iter().position(|path| doomed(row, path)) {
while let Some(at) = row.position_of_peer(peer) {
removed += 1;
if row.paths.len() == 1 {
outcome.removed += removed;
crates/rib/src/table/select.rs
+9 -41
View File
@@ -27,7 +27,7 @@
//! of all its routes.
use super::attrs::SelectionKey;
use super::{DEFAULT_LOCAL_PREF, Path, PeerInfo, Row};
use super::{DEFAULT_LOCAL_PREF, PeerInfo, Row};
/// Steps 1 to 3 as one number of 48 bits: the larger, the better.
///
@@ -35,12 +35,14 @@ use super::{DEFAULT_LOCAL_PREF, Path, PeerInfo, Row};
/// 12 bits and the ORIGIN in 2, each turned upside down so that shorter and
/// lower compare as larger. Twelve bits are enough for any path: a BGP
/// message of 4096 octets has room for about a thousand AS numbers.
pub(super) fn preference(key: &SelectionKey) -> u64 {
// LOCAL_PREF is a matter between routers of one AS. What an external
// peer sent as one never gets as far as the table (the decoder drops
// it, RFC 7606 §7.5); a route from such a peer that has one was given
// it by our own import policy, which is who is to say.
let local_pref = key.local_pref.unwrap_or(DEFAULT_LOCAL_PREF);
pub(super) fn preference(key: &SelectionKey, peer: &PeerInfo) -> u64 {
// LOCAL_PREF is a matter between routers of one AS. A route from an
// external peer has the default until import policy gives it another.
let local_pref = if peer.ibgp {
key.local_pref.unwrap_or(DEFAULT_LOCAL_PREF)
} else {
DEFAULT_LOCAL_PREF
};
const LONGEST: u16 = (1 << 12) - 1;
(u64::from(local_pref) << 16)
| (u64::from(LONGEST - key.as_path_len.min(LONGEST)) << 4)
@@ -187,40 +189,6 @@ fn choose(row: &mut Row, peers: &[Option<PeerInfo>], scratch: &mut Scratch) -> b
row.best != before
}
/// The routes of `row` from the best down, as indices into its routes:
/// after the best, the one that would be chosen were the best gone, and
/// so on. What an operator asks when looking at a prefix: which route
/// would take over.
pub(super) fn ranked(row: &Row, peers: &[Option<PeerInfo>]) -> Vec<usize> {
// A copy to take the winners out of one by one, and where each of its
// routes is in the row itself.
let mut rest = Row {
prefix: row.prefix,
best: 0,
seqs: row.seqs,
paths: row
.paths
.iter()
.map(|path| Path {
word: path.word,
attrs: std::sync::Arc::clone(&path.attrs),
})
.collect(),
path_ids: row.path_ids.clone(),
};
let mut places: Vec<usize> = (0..row.paths.len()).collect();
let mut scratch = Scratch::default();
let mut order = Vec::with_capacity(row.paths.len());
while !rest.paths.is_empty() {
choose(&mut rest, peers, &mut scratch);
let at = rest.best();
order.push(places[at]);
rest.swap_remove(at);
places.swap_remove(at);
}
order
}
/// For a route just added at `added` that ties with the best in steps 1 to
/// 3: whether it is the new best, if that can be said from the two routes
/// alone. `None` if the whole row has to be looked at.
crates/rib/src/table/tests.rs
+18 -291
View File
@@ -61,9 +61,7 @@ pub(super) fn external(number: u8) -> PeerInfo {
asn: 65000 + u32::from(number),
router_id: Ipv4Addr::new(10, 0, 0, number),
address: IpAddr::from(Ipv4Addr::new(192, 0, 2, number)),
interface: 0,
ibgp: false,
reflector_client: false,
}
}
@@ -71,7 +69,6 @@ pub(super) fn internal(number: u8) -> PeerInfo {
PeerInfo {
asn: 64999,
ibgp: true,
reflector_client: false,
..external(number)
}
}
@@ -162,7 +159,7 @@ impl Fixture {
/// Which peer (by order added) the best route for `prefix` is from.
fn best(&self, prefix: &IpPrefix) -> Option<usize> {
let peer = self.table.best(prefix)?.peer;
let (peer, _) = self.table.best(prefix)?;
self.peers.iter().position(|p| *p == peer)
}
}
@@ -172,7 +169,7 @@ fn each_step_of_selection_decides_when_the_ones_before_tie() {
let p = prefix("10.0.0.0/8");
// Peer 0 has the lower identifier, so it wins whenever nothing else
// decides; each case makes peer 1's route better in one step.
let cases: [(&str, [PeerInfo; 2], Route, Route); 7] = [
let cases: [(&str, [PeerInfo; 2], Route, Route); 6] = [
(
"higher LOCAL_PREF, despite a longer path",
[internal(1), internal(2)],
@@ -185,18 +182,6 @@ fn each_step_of_selection_decides_when_the_ones_before_tie() {
..Route::via(&[1, 2, 3])
},
),
(
"higher LOCAL_PREF from an external peer too, where policy set one",
[external(1), external(2)],
Route {
local_pref: Some(100),
..Route::via(&[1])
},
Route {
local_pref: Some(200),
..Route::via(&[1, 2, 3])
},
),
(
"shorter AS path",
[external(1), external(2)],
@@ -261,62 +246,6 @@ fn each_step_of_selection_decides_when_the_ones_before_tie() {
assert_eq!(fixture.best(&p), Some(1));
}
/// The ranking of a prefix's routes is selection run again without the
/// winners: each route in it is what would be chosen were the ones before
/// it gone.
#[test]
fn routes_are_ranked_as_they_would_be_chosen_one_after_another() {
let p = prefix("10.0.0.0/8");
// Peer 0 would win on identifier alone; peer 2 has the highest
// LOCAL_PREF, peer 1 the shortest path of the rest, peer 3 a MED
// that loses to peer 1's within their shared neighbouring AS.
let peers = [external(1), external(2), internal(3), external(4)];
let fixture = Fixture::new(&peers);
fixture.announce(0, &p, &Route::via(&[7, 1, 2]));
fixture.announce(1, &p, &Route::via(&[7, 3]));
fixture.announce(
2,
&p,
&Route {
local_pref: Some(200),
..Route::via(&[8, 1, 2, 3])
},
);
fixture.announce(
3,
&p,
&Route {
med: Some(5),
..Route::via(&[7, 4])
},
);
let ranked: Vec<usize> = fixture
.table
.ranked_routes(&p)
.iter()
.map(|route| {
fixture
.peers
.iter()
.position(|peer| *peer == route.peer)
.unwrap()
})
.collect();
assert_eq!(ranked, vec![2, 1, 3, 0]);
assert_eq!(fixture.best(&p), Some(2));
// Taking the best away makes the second the best, and so on down.
fixture.withdraw(2, &p);
assert_eq!(fixture.best(&p), Some(1));
fixture.withdraw(1, &p);
assert_eq!(fixture.best(&p), Some(3));
assert!(
fixture
.table
.ranked_routes(&prefix("10.1.0.0/16"))
.is_empty()
);
}
/// MEDs are compared only within a neighbouring AS. Taken pair by pair
/// that is not an order: A beats B, B beats C, C beats A. Whatever the
/// order of arrival, the answer must be the same.
@@ -503,129 +432,6 @@ fn removing_a_peer_takes_its_routes_and_frees_its_number() {
assert_eq!(fixture.table.add_peer(external(3)), Ok(fixture.peers[0]));
}
/// Graceful restart (RFC 4724 §4.2): a restarting peer's routes are kept,
/// marked stale, and used as before; what it announces again is fresh, and
/// what it does not is swept away at the end.
#[test]
fn stale_routes_are_chosen_as_before_and_swept_unless_announced_again() {
let fixture = Fixture::new(&[external(1), external(2)]);
let v4 = [
prefix("10.1.0.0/16"),
prefix("10.2.0.0/16"),
prefix("10.3.0.0/16"),
];
let v6 = prefix("2001:db8::/32");
for p in v4.iter().chain([&v6]) {
fixture.announce(0, p, &Route::via(&[1]));
}
// Peer 1 has a worse route for the first prefix, and loses.
fixture.announce(1, &v4[0], &Route::via(&[2, 3]));
let mut reader = fixture.table.reader_from_now();
let peer = fixture.peers[0];
let both = [IpVersion::Ipv4, IpVersion::Ipv6];
assert_eq!(fixture.table.mark_stale(peer, &both), 4);
assert_eq!(fixture.table.count_stale(peer), 4);
// Nothing changed for anyone who follows the table, and the stale
// route is still the best.
let mut events = 0;
reader.poll(usize::MAX, |_| events += 1);
assert_eq!(events, 0);
assert_eq!(fixture.best(&v4[0]), Some(0));
assert!(fixture.table.routes(&v4[0]).iter().any(|r| r.stale));
// The peer comes back and announces two of them again, one with the
// attributes it had: they are fresh.
assert_eq!(
fixture.announce(0, &v4[0], &Route::via(&[1])).stale_cleared,
1
);
assert_eq!(
fixture
.announce(0, &v4[1], &Route::via(&[1, 4]))
.stale_cleared,
1
);
// Again, and it was not stale any more.
assert_eq!(
fixture
.announce(0, &v4[1], &Route::via(&[1, 4]))
.stale_cleared,
0
);
assert_eq!(fixture.table.count_stale(peer), 2);
// Its End-of-RIB for IPv4: the IPv4 route it did not announce goes,
// the IPv6 one waits for the IPv6 marker.
let outcome = fixture.table.sweep_stale(peer, &[IpVersion::Ipv4]);
assert_eq!((outcome.removed, outcome.best_changed), (1, 1));
assert_eq!(fixture.best(&v4[2]), None);
assert_eq!(fixture.best(&v4[0]), Some(0));
assert_eq!(fixture.table.count_stale(peer), 1);
let outcome = fixture.table.sweep_stale(peer, &[IpVersion::Ipv6]);
assert_eq!(outcome.removed, 1);
assert_eq!(fixture.table.count_stale(peer), 0);
assert_eq!(fixture.table.len(), 3);
// Sweeping again finds nothing.
assert_eq!(fixture.table.sweep_stale(peer, &both), Outcome::default());
}
/// Stale routes are swept from one peer only, and only for the versions
/// asked: the other peer's routes, stale or not, stay.
#[test]
fn sweeping_takes_only_the_stale_routes_of_one_peer() {
let fixture = Fixture::new(&[external(1), external(2)]);
let p = prefix("10.0.0.0/8");
fixture.announce(0, &p, &Route::via(&[1]));
fixture.announce(1, &p, &Route::via(&[2, 3]));
fixture
.table
.mark_stale(fixture.peers[1], &[IpVersion::Ipv4]);
assert_eq!(
fixture
.table
.sweep_stale(fixture.peers[0], &[IpVersion::Ipv4]),
Outcome::default()
);
assert_eq!(
fixture
.table
.sweep_stale(fixture.peers[1], &[IpVersion::Ipv6]),
Outcome::default()
);
assert_eq!(fixture.held(&p), vec![(0, 0), (1, 0)]);
let outcome = fixture
.table
.sweep_stale(fixture.peers[1], &[IpVersion::Ipv4]);
assert_eq!((outcome.removed, outcome.best_changed), (1, 0));
assert_eq!(fixture.held(&p), vec![(0, 0)]);
}
/// A new session of a restarting peer takes over its number, with what
/// selection knows of it brought up to date: here a new identifier, which
/// is what breaks the tie.
#[test]
fn a_peer_number_can_be_given_to_the_next_session_of_the_peer() {
let fixture = Fixture::new(&[external(1), external(2)]);
let p = prefix("10.0.0.0/8");
fixture.announce(0, &p, &Route::via(&[1]));
fixture.announce(1, &p, &Route::via(&[2]));
assert_eq!(fixture.best(&p), Some(0));
let renumbered = PeerInfo {
router_id: Ipv4Addr::new(10, 0, 0, 9),
..external(1)
};
assert!(fixture.table.rebind_peer(fixture.peers[0], renumbered));
assert_eq!(fixture.table.peer(fixture.peers[0]), Some(renumbered));
// Chosen again the next time the row changes.
fixture.announce(1, &p, &Route::via(&[5]));
assert_eq!(fixture.best(&p), Some(1));
// A number nobody has cannot be taken over.
fixture.table.remove_peer(fixture.peers[1]);
assert!(!fixture.table.rebind_peer(fixture.peers[1], external(3)));
}
#[test]
fn readers_see_every_route_and_which_is_best() {
let fixture = Fixture::new(&[external(1), external(2)]);
@@ -727,9 +533,13 @@ fn chosen_plainly(
routes: &BTreeMap<(usize, u32), Route>,
peers: &[PeerInfo],
) -> Option<(usize, u32)> {
// Whoever the route is from: an external peer's own LOCAL_PREF never
// gets into a table, and one that is there was put there by policy.
let local_pref = |_peer: usize, route: &Route| route.local_pref.unwrap_or(DEFAULT_LOCAL_PREF);
let local_pref = |peer: usize, route: &Route| {
if peers[peer].ibgp {
route.local_pref.unwrap_or(DEFAULT_LOCAL_PREF)
} else {
DEFAULT_LOCAL_PREF
}
};
let mut left: Vec<((usize, u32), &Route)> =
routes.iter().map(|(k, route)| (*k, route)).collect();
let most = left.iter().map(|(k, r)| local_pref(k.0, r)).max()?;
@@ -888,35 +698,12 @@ proptest! {
/// differ where that table looks.
#[test]
fn keys_of_one_shard_differ_in_the_bits_their_table_uses() {
// With keys of the hash that are the same every time: a test that
// fails one run in a thousand tells nobody anything.
for seed in 0..32u64 {
keys_of_one_shard_differ(foldhash::fast::FixedState::with_seed(seed));
}
// Always one of the buckets there are.
for hash in [0, 1, u64::MAX, u64::MAX / 2, 1 << 63] {
for count in [1, 3, 64, 1000] {
assert!(spread(hash, count) < count, "{hash} of {count}");
}
}
}
fn keys_of_one_shard_differ(hasher: impl std::hash::BuildHasher) {
let hasher = RandomState::default();
let shards = 64;
let mut counts = vec![0usize; shards];
let mut tags = vec![std::collections::HashSet::new(); shards];
// Keys as the table has them, and as regular as they come: every /24
// of a /8.
for n in 0..64_000u32 {
let [_, a, b, c] = n.to_be_bytes();
let key: Nlri = IpPrefix::new(
bgp_primitives::constants::IpVersion::Ipv4,
[100 + a, b, c, 0],
24,
)
.unwrap()
.into();
let hash = hasher.hash_one(key);
let hash = hasher.hash_one(n);
let shard = spread(hash, shards);
counts[shard] += 1;
// The seven bits hashbrown keeps as a tag.
@@ -928,14 +715,10 @@ fn keys_of_one_shard_differ(hasher: impl std::hash::BuildHasher) {
"shard {shard} has {} of 64000 keys",
counts[shard]
);
// Nearly all of the 128 there are; taken from the bits the shard
// is picked by, there would be two.
assert!(
tags[shard].len() > 100,
"shard {shard} has {} tags",
tags[shard].len()
);
assert_eq!(tags[shard].len(), 128, "shard {shard}");
}
assert_eq!(spread(u64::MAX, 3), 2);
assert_eq!(spread(0, 3), 0);
}
/// A peer that sends several routes for a prefix can send more than fit a
@@ -950,20 +733,21 @@ fn a_row_can_hold_more_routes_than_a_sixteen_bit_index_counts() {
let info = external(1);
let peers = [Some(info)];
let peer = PeerIndex(0);
let path = |attrs: &Arc<AttrSet>| Path::new(select::preference(attrs.key()), peer, attrs);
let path =
|attrs: &Arc<AttrSet>| Path::new(select::preference(attrs.key(), &info), peer, attrs);
let seqs = journal::Journals::new(journal::Limits::default()).appeared(prefix("10.0.0.0/8"));
let mut row = Row::new(prefix("10.0.0.0/8"), path(&worse), 1, seqs);
let mut scratch = select::Scratch::default();
let count = usize::from(u16::MAX) + 10;
for path_id in 2..=count as u32 {
row.push(path(&worse), path_id, peers.len());
row.push(path(&worse), path_id);
}
// Equal routes from one peer: the lowest identifier is the best.
assert!(!select::after_add(&mut row, &peers, &mut scratch));
assert_eq!(row.best(), 0);
row.push(path(&better), count as u32 + 1, peers.len());
row.push(path(&better), count as u32 + 1);
assert!(select::after_add(&mut row, &peers, &mut scratch));
assert_eq!(row.best(), count);
assert_eq!(row.route(row.best()).path_id, count as u32 + 1);
@@ -973,60 +757,3 @@ fn a_row_can_hold_more_routes_than_a_sixteen_bit_index_counts() {
assert!(select::after_remove(&mut row, count, &peers, &mut scratch));
assert_eq!(row.route(row.best()).path_id, 1);
}
#[test]
fn routes_can_be_picked_out_of_the_table() {
let fixture = Fixture::new(&[external(1), external(2)]);
for n in 0..50u8 {
let p = prefix(&format!("10.{n}.0.0/16"));
fixture.announce(0, &p, &Route::via(&[1]));
if n % 2 == 0 {
fixture.announce(1, &p, &Route::via(&[2, 3]));
}
}
let table = &fixture.table;
let second = fixture.peers[1];
assert_eq!(table.routes_where(1000, |_| true).len(), 75);
assert_eq!(
table.routes_where(1000, |route| route.peer == second).len(),
25
);
assert_eq!(table.routes_where(1000, |route| route.best).len(), 50);
let inside = prefix("10.16.0.0/12");
let found = table.routes_where(1000, |route| inside.contains(&route.prefix));
assert_eq!(found.len(), 16 + 8);
// No more than asked for, however many there are.
assert_eq!(
table.routes_where(10, |route| route.peer == second).len(),
10
);
assert!(table.routes_where(0, |_| true).is_empty());
}
#[test]
fn a_table_told_what_to_expect_has_the_room_before_the_routes_come() {
let table = RibTable::new(8);
table.expect_prefixes(100_000);
let room: usize = table
.shards
.iter()
.map(|shard| read(shard).rows.capacity())
.sum();
assert!(room >= 100_000, "room for {room} prefixes");
// Asking again for less, or for the same, takes nothing away.
table.expect_prefixes(10);
let still: usize = table
.shards
.iter()
.map(|shard| read(shard).rows.capacity())
.sum();
assert_eq!(still, room);
// And it is a table like any other.
let peer = table.add_peer(external(1)).unwrap();
let attrs = table
.attrs()
.intern(&Route::via(&[64512]).attributes(), None)
.unwrap();
table.announce(peer, &attrs, &[Nlri::from(prefix("192.0.2.0/24"))]);
assert_eq!(table.prefix_count(), 1);
}
crates/rpki/Cargo.toml
-22
View File
@@ -1,22 +0,0 @@
[package]
name = "bgp-rpki"
authors.workspace = true
edition.workspace = true
homepage.workspace = true
license.workspace = true
repository.workspace = true
version.workspace = true
[dependencies]
bgp-primitives.workspace = true
bytes.workspace = true
foldhash.workspace = true
thiserror.workspace = true
tokio = { workspace = true, features = ["full"] }
tracing.workspace = true
[dev-dependencies]
proptest.workspace = true
[lints]
workspace = true
crates/rpki/src/aspa.rs
-426
View File
@@ -1,426 +0,0 @@
//! Autonomous System Provider Authorisations, and what they say about a
//! route's AS path (draft-ietf-sidrops-aspa-verification).
//!
//! An ASPA is a signed statement by an AS of who its providers are: the
//! networks it buys transit from. With those statements a router can tell
//! a path that goes the way traffic is paid to go (up from the origin
//! through providers, across at most one peering, and down through
//! customers) from one that does not. A path that climbs again after it
//! has come down, or crosses a second peering, is a route leak or a forged
//! path, and this is the check that finds them.
use std::collections::HashMap;
use std::sync::Arc;
/// What the ASPAs say about a path.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub enum PathValidity {
/// Some AS on the path has no ASPA, and nothing on the path is known
/// to be wrong. Most paths are in this state while few networks have
/// an ASPA, and are used.
Unknown,
/// Every hop that has to be from a customer to its provider is
/// attested to be one.
Valid,
/// Some AS on the path has said who its providers are, and the path
/// went from it to an AS that is not one of them where only a provider
/// could come next.
Invalid,
}
impl PathValidity {
pub fn as_str(self) -> &'static str {
match self {
Self::Unknown => "unknown",
Self::Valid => "valid",
Self::Invalid => "invalid",
}
}
}
/// Which of the two procedures verifies a path. That depends on what the
/// neighbor it came from is to us.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub enum Algorithm {
/// For a route from a customer, a peer, or across a route server: all
/// of its path must lead upwards, from each AS to a provider of it
/// (§6.1 of the draft).
Upstream,
/// For a route from a provider: its path may lead up from the origin
/// and then down to us, and nothing else (§6.2).
Downstream,
}
/// Whether one AS is a provider of another, as far as the ASPAs say.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum Hop {
/// The customer has no ASPA.
NoAttestation,
/// The customer's ASPA lists the other AS.
Provider,
/// The customer has an ASPA, and the other AS is not on it.
NotProvider,
}
/// The ASPAs there are: for each customer AS that has one, its providers.
///
/// Cloning copies the list of customers and shares the lists of providers.
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct AspaTable {
providers: HashMap<u32, Arc<[u32]>, foldhash::fast::FixedState>,
}
impl AspaTable {
/// How many customer ASes have an ASPA.
pub fn len(&self) -> usize {
self.providers.len()
}
pub fn is_empty(&self) -> bool {
self.providers.is_empty()
}
/// Records that `customer`'s providers are `providers`, in place of
/// what was recorded for it before, which is returned.
pub fn set(
&mut self,
customer: u32,
providers: impl IntoIterator<Item = u32>,
) -> Option<Arc<[u32]>> {
let mut providers: Vec<u32> = providers.into_iter().collect();
providers.sort_unstable();
providers.dedup();
self.providers.insert(customer, providers.into())
}
/// Forgets `customer`'s ASPA, and returns the providers it listed.
pub fn remove(&mut self, customer: u32) -> Option<Arc<[u32]>> {
self.providers.remove(&customer)
}
/// Puts back what [`AspaTable::set`] or [`AspaTable::remove`] returned.
pub(crate) fn restore(&mut self, customer: u32, providers: Option<Arc<[u32]>>) {
match providers {
Some(providers) => self.providers.insert(customer, providers),
None => self.providers.remove(&customer),
};
}
/// The providers `customer` lists, in increasing order, if it has an
/// ASPA.
pub fn providers(&self, customer: u32) -> Option<&[u32]> {
self.providers.get(&customer).map(|providers| &**providers)
}
/// The customers and their providers, in no particular order.
pub fn iter(&self) -> impl Iterator<Item = (u32, &[u32])> {
self.providers
.iter()
.map(|(customer, providers)| (*customer, &**providers))
}
fn hop(&self, customer: u32, provider: u32) -> Hop {
match self.providers.get(&customer) {
None => Hop::NoAttestation,
Some(providers) if providers.binary_search(&provider).is_ok() => Hop::Provider,
Some(_) => Hop::NotProvider,
}
}
/// Verifies `path`: the AS numbers of an AS_PATH in the order they are
/// in it, the neighbor's first and the origin's last, with whatever
/// repeats it has.
///
/// This is the part of the draft's procedures that the ASPAs decide.
/// What comes before it there is the caller's to check: that the path
/// has no AS_SET and begins with the neighbor's AS. An empty path is
/// invalid.
pub fn verify(&self, path: &[u32], algorithm: Algorithm) -> PathValidity {
// The draft numbers the path's distinct ASes from the origin,
// A(1), to the neighbor, A(N). Here they are `hops[0]` to
// `hops[n - 1]`.
let mut hops: Vec<u32> = Vec::with_capacity(path.len());
for asn in path.iter().rev() {
if hops.last() != Some(asn) {
hops.push(*asn);
}
}
let n = hops.len();
if n == 0 {
return PathValidity::Invalid;
}
// How far up from the origin the path can be a chain of customers
// and their providers at most (as far as nothing says otherwise),
// and at least (as far as it is attested).
let (mut max_up, mut min_up) = (n, n);
for (at, pair) in hops.windows(2).enumerate() {
match self.hop(pair[0], pair[1]) {
Hop::Provider => continue,
Hop::NoAttestation => min_up = min_up.min(at + 1),
Hop::NotProvider => {
min_up = min_up.min(at + 1);
max_up = at + 1;
break;
}
}
}
if algorithm == Algorithm::Upstream {
return match (max_up < n, min_up < n) {
(true, _) => PathValidity::Invalid,
(false, true) => PathValidity::Unknown,
(false, false) => PathValidity::Valid,
};
}
// The same from the neighbor's end, looking down: each AS is to be
// the customer of the one after it.
let (mut max_down, mut min_down) = (n, n);
for (at, pair) in hops.windows(2).rev().enumerate() {
match self.hop(pair[1], pair[0]) {
Hop::Provider => continue,
Hop::NoAttestation => min_down = min_down.min(at + 1),
Hop::NotProvider => {
min_down = min_down.min(at + 1);
max_down = at + 1;
break;
}
}
}
// The way up and the way down have to meet, or overlap.
if max_up + max_down < n {
PathValidity::Invalid
} else if min_up + min_down < n {
PathValidity::Unknown
} else {
PathValidity::Valid
}
}
}
impl FromIterator<(u32, Vec<u32>)> for AspaTable {
fn from_iter<T: IntoIterator<Item = (u32, Vec<u32>)>>(aspas: T) -> Self {
let mut table = Self::default();
for (customer, providers) in aspas {
table.set(customer, providers);
}
table
}
}
#[cfg(test)]
mod tests {
use proptest::prelude::*;
use super::Algorithm::{Downstream, Upstream};
use super::PathValidity::{Invalid, Unknown, Valid};
use super::*;
fn table(aspas: &[(u32, &[u32])]) -> AspaTable {
aspas
.iter()
.map(|(customer, providers)| (*customer, providers.to_vec()))
.collect()
}
/// A small Internet. 10 and 20 are the large networks, which peer and
/// have no providers (an ASPA that lists AS 0). 100 buys from 10, 200
/// from 20, 300 from both. 1000 buys from 100; 2000 from 200 and has
/// no ASPA.
fn internet() -> AspaTable {
table(&[
(10, &[0]),
(20, &[0]),
(100, &[10]),
(200, &[20]),
(300, &[10, 20]),
(1000, &[100]),
])
}
#[test]
fn a_path_that_only_climbs_is_valid_from_a_customer_or_peer() {
let aspas = internet();
// 1000 -> 100 -> 10, as 10 hears it from its customer 100.
assert_eq!(aspas.verify(&[100, 1000], Upstream), Valid);
// As 20 hears it from its peer 10.
assert_eq!(aspas.verify(&[10, 100, 1000], Upstream), Valid);
// A route the neighbor originates is as far up as it has got.
assert_eq!(aspas.verify(&[1000], Upstream), Valid);
// Prepending changes nothing.
assert_eq!(
aspas.verify(&[10, 10, 100, 100, 100, 1000, 1000], Upstream),
Valid
);
}
#[test]
fn a_leak_is_invalid_from_a_customer_or_peer() {
let aspas = internet();
// 300 buys from 10 and 20, and passes 10's routes on to 20: the
// classic leak. 20 hears the path 300 10 100 1000 from its
// customer; 10 says it has no providers, so 300 is not one.
assert_eq!(aspas.verify(&[300, 10, 100, 1000], Upstream), Invalid);
// 20 passes a route of its peer 10's on to another peer: 20 is not
// a provider of 10.
assert_eq!(aspas.verify(&[20, 10, 100, 1000], Upstream), Invalid);
// An origin that is not where the path says: 1000 lists only 100.
assert_eq!(aspas.verify(&[200, 1000], Upstream), Invalid);
}
#[test]
fn a_hop_nobody_has_attested_leaves_the_path_unknown() {
let aspas = internet();
// 2000 has no ASPA.
assert_eq!(aspas.verify(&[200, 2000], Upstream), Unknown);
assert_eq!(aspas.verify(&[20, 200, 2000], Upstream), Unknown);
// But what is known to be wrong is invalid whatever else is not
// known.
assert_eq!(aspas.verify(&[300, 20, 200, 2000], Upstream), Invalid);
// With no ASPAs at all, nothing is known.
assert_eq!(AspaTable::default().verify(&[1, 2, 3], Upstream), Unknown);
assert_eq!(AspaTable::default().verify(&[1, 2, 3], Downstream), Unknown);
// Except of the path that is no path.
assert_eq!(AspaTable::default().verify(&[], Upstream), Invalid);
assert_eq!(AspaTable::default().verify(&[], Downstream), Invalid);
}
#[test]
fn from_a_provider_a_path_may_climb_and_then_descend() {
let aspas = internet();
// 1000's route, as 200 hears it from its provider 20: up through
// 100 to 10, across to 20.
assert_eq!(aspas.verify(&[20, 10, 100, 1000], Downstream), Valid);
// As 300 hears it from 10: only up.
assert_eq!(aspas.verify(&[10, 100, 1000], Downstream), Valid);
// As 1000 would hear 300's route from 100: up to 10, down to 100.
assert_eq!(aspas.verify(&[100, 10, 300], Downstream), Valid);
// Short paths have no room to be wrong in.
assert_eq!(aspas.verify(&[20, 1000], Downstream), Valid);
assert_eq!(aspas.verify(&[20], Downstream), Valid);
}
#[test]
fn from_a_provider_a_path_that_descends_and_climbs_again_is_invalid() {
let aspas = internet();
// 300 leaks 10's routes to 20, and 20 passes them down to 200,
// which hears the path 20 300 10 100 1000. The way up from 1000
// ends at 10, three ASes in: 300 is not a provider of 10. The way
// down to 200 begins no earlier than 20, one AS: 20 has no
// providers, so it did not get the route from one. Three and one
// do not make five.
assert_eq!(aspas.verify(&[20, 300, 10, 100, 1000], Downstream), Invalid);
// Two peerings in a row: 30 is another large network, which hands
// its customer a route it got from its peer 20, which got it from
// its peer 10.
let mut aspas = internet();
aspas.set(30, [0]);
assert_eq!(aspas.verify(&[30, 20, 10, 100, 1000], Downstream), Invalid);
}
#[test]
fn from_a_provider_what_is_not_attested_leaves_the_path_unknown() {
let aspas = internet();
// 2000 (no ASPA) up to 200, to 20, across to 10, down to 100: the
// first hop is not attested either way.
assert_eq!(aspas.verify(&[10, 20, 200, 2000], Downstream), Unknown);
// With 2000's ASPA it is valid.
let mut with = aspas.clone();
with.set(2000, [200]);
assert_eq!(with.verify(&[10, 20, 200, 2000], Downstream), Valid);
}
#[test]
fn an_aspa_is_replaced_and_removed_whole() {
let mut aspas = internet();
assert_eq!(aspas.providers(300), Some(&[10, 20][..]));
assert_eq!(
aspas.set(300, [20, 10, 20, 5]).as_deref(),
Some(&[10, 20][..])
);
assert_eq!(aspas.providers(300), Some(&[5, 10, 20][..]));
assert_eq!(aspas.remove(300).as_deref(), Some(&[5, 10, 20][..]));
assert_eq!(aspas.providers(300), None);
assert_eq!(aspas.remove(300), None);
assert_eq!(aspas.len(), 5);
}
/// The procedures as the draft words them, with its numbering: A(1) is
/// the origin, A(N) the neighbor.
fn by_the_draft(aspas: &AspaTable, path: &[u32], algorithm: Algorithm) -> PathValidity {
let mut a: Vec<u32> = path.iter().rev().copied().collect();
a.dedup();
let n = a.len();
if n == 0 {
return Invalid;
}
// One-based, as written.
let hop = |i: usize, j: usize| aspas.hop(a[i - 1], a[j - 1]);
let first_up = |wanted: &[Hop]| {
(1..n)
.find(|i| wanted.contains(&hop(*i, *i + 1)))
.unwrap_or(n)
};
let last_down = |wanted: &[Hop]| {
(2..=n)
.rev()
.find(|j| wanted.contains(&hop(*j, *j - 1)))
.map_or(n, |j| n - j + 1)
};
let not = [Hop::NotProvider];
let not_attested = [Hop::NoAttestation, Hop::NotProvider];
let (max_up, min_up) = (first_up(&not), first_up(&not_attested));
match algorithm {
Upstream if max_up < n => Invalid,
Upstream if min_up < n => Unknown,
Upstream => Valid,
Downstream => {
let (max_down, min_down) = (last_down(&not), last_down(&not_attested));
if max_up + max_down < n {
Invalid
} else if min_up + min_down < n {
Unknown
} else {
Valid
}
}
}
}
proptest! {
/// Over a handful of AS numbers, so that paths and ASPAs meet.
#[test]
fn verification_agrees_with_the_draft_as_written(
aspas in proptest::collection::vec(
(0u32..8, proptest::collection::vec(0u32..8, 1..4)), 0..8),
path in proptest::collection::vec(0u32..8, 0..10),
downstream: bool,
) {
let aspas: AspaTable = aspas.into_iter().collect();
let algorithm = if downstream { Downstream } else { Upstream };
prop_assert_eq!(
aspas.verify(&path, algorithm),
by_the_draft(&aspas, &path, algorithm)
);
}
/// What is invalid from a provider is invalid from anyone, and
/// what is valid from a customer is valid from anyone: the
/// procedure for providers allows more.
#[test]
fn the_procedure_for_providers_is_the_more_lenient(
aspas in proptest::collection::vec(
(0u32..8, proptest::collection::vec(0u32..8, 1..4)), 0..8),
path in proptest::collection::vec(0u32..8, 1..10),
) {
let aspas: AspaTable = aspas.into_iter().collect();
let (up, down) = (aspas.verify(&path, Upstream), aspas.verify(&path, Downstream));
if down == Invalid {
prop_assert_eq!(up, Invalid);
}
if up == Valid {
prop_assert_eq!(down, Valid);
}
}
}
}
crates/rpki/src/client.rs
-953
View File
@@ -1,953 +0,0 @@
//! Keeps the VRPs and ASPAs in step with a validator, over the
//! RPKI-to-Router protocol (RFC 8210 and draft-ietf-sidrops-8210bis).
//!
//! The client connects to a cache (the validator's RTR server), asks for
//! everything once, and from then on for what has changed: when the cache
//! says there is something new, and otherwise every so often. Each time the
//! data changes an [`Update`] is published for whoever validates routes.
//!
//! The newest version of the protocol is asked for first, and an older one
//! if that is what the cache speaks. Only version 2 carries ASPAs: with an
//! older cache there are none, and paths are not verified.
//!
//! A cache that cannot be reached is tried again, and the other caches in
//! turn. Meanwhile the data last received is used, but not for ever: a list
//! that has not been confirmed for the expire interval is dropped, and
//! routes are then neither valid nor invalid. Using stale data would be
//! worse: it would go on refusing routes whose holders have since
//! authorised them.
use std::io;
use std::net::SocketAddr;
use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::{Arc, Mutex, PoisonError};
use std::time::{Duration, SystemTime};
use bgp_primitives::constants::IpVersion;
use bytes::BytesMut;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::TcpStream;
use tokio::sync::watch;
use tokio::task::JoinHandle;
use tokio::time::{Instant, sleep_until, timeout};
use tracing::{info, warn};
use crate::aspa::AspaTable;
use crate::rtr::{Pdu, RtrError, Timers, VERSION_0, VERSION_2, error_code};
use crate::vrp::{Vrp, VrpTable};
// Hashed by the hundred thousand with every answer, and told apart from
// what there was with every reset: a quick hash matters here.
type HashSet<K> = std::collections::HashSet<K, foldhash::fast::RandomState>;
const CONNECT_TIMEOUT: Duration = Duration::from_secs(10);
/// How long a cache may take to send the next part of an answer.
const RESPONSE_TIMEOUT: Duration = Duration::from_secs(60);
/// How long a whole answer may take. Everything a validator has takes it
/// seconds to send.
const ANSWER_TIMEOUT: Duration = Duration::from_secs(600);
/// The most records taken in one answer: several times what the whole
/// RPKI holds. A cache that sends more is not sending the RPKI, and
/// would otherwise be listened to until there is no memory left.
const MAX_RECORDS: usize = 8_000_000;
/// How long to wait before trying the next cache after one has failed.
const NEXT_CACHE_DELAY: Duration = Duration::from_secs(1);
/// How long to wait before asking a cache again that said it has no data
/// yet, rather than the retry interval: it is up and validating, which
/// takes minutes after it starts, and the retry interval is ten unless it
/// has said otherwise, which it has not had the chance to.
const NO_DATA_RETRY: Duration = Duration::from_secs(30);
/// How long a cache that is being left is given to take the reason.
const ERROR_REPORT_TIMEOUT: Duration = Duration::from_secs(1);
/// Where the VRPs come from, and how patiently.
#[derive(Debug, Clone, Default)]
pub struct RpkiConfig {
/// The caches, in order of preference. One is used at a time; the next
/// is tried when it fails.
pub caches: Vec<SocketAddr>,
/// If set, used in place of what the cache says (or of the defaults of
/// RFC 8210 §6, with a cache that says nothing).
pub refresh: Option<Duration>,
pub retry: Option<Duration>,
pub expire: Option<Duration>,
}
/// The VRPs and ASPAs as they are after a change.
#[derive(Debug, Clone)]
pub struct Update {
/// Goes up by one with every update.
pub version: u64,
pub vrps: Arc<VrpTable>,
pub aspas: Arc<AspaTable>,
/// What was added or removed to get here from the version before.
/// `None` if any route's validity may have changed.
pub changed: Option<Changed>,
}
/// What differs between one [`Update`] and the next.
#[derive(Debug, Clone)]
pub struct Changed {
/// The VRPs that came or went: only routes that these cover can have
/// another origin validity than before.
pub vrps: Arc<VrpTable>,
/// The customer ASes whose ASPA came, went or lists other providers:
/// only paths with one of these on them can verify otherwise than
/// before.
pub aspa_customers: Arc<HashSet<u32>>,
}
/// How things stand with the caches.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum CacheState {
/// No cache has been reached yet, or the one in use was lost.
Connecting,
/// Connected and waiting for the data.
Synchronising,
/// The data is complete as of the cache's serial.
Synchronised,
}
/// What there is to say about the client, for showing.
#[derive(Debug, Clone)]
pub struct Status {
/// The cache in use, or last tried.
pub cache: Option<SocketAddr>,
pub state: CacheState,
/// The cache's session and the serial the data is at.
pub session_id: Option<u16>,
pub serial: Option<u32>,
pub protocol_version: u8,
pub vrps_ipv4: usize,
pub vrps_ipv6: usize,
/// How many ASes have an ASPA. None do if the cache speaks a version
/// of the protocol before 2.
pub aspas: usize,
/// When the data was last confirmed by a cache.
pub updated_at: Option<SystemTime>,
/// When it will be dropped unless confirmed again.
pub expires_at: Option<SystemTime>,
/// Why the last attempt failed, if it did.
pub last_error: Option<String>,
}
struct Shared {
updates: watch::Sender<Update>,
/// The version of the latest update, for a look that costs nothing.
version: AtomicU64,
status: Mutex<Status>,
}
/// Handle to the VRPs and to the client that keeps them. Cheap to clone.
#[derive(Clone)]
pub struct Rpki {
shared: Arc<Shared>,
}
impl Rpki {
/// Starts a client for `config` and returns the handle to its data,
/// with the task that runs it. Until the first cache has answered
/// there are no VRPs and no ASPAs: every route's origin is
/// [`crate::Validity::NotFound`] and every path
/// [`crate::PathValidity::Unknown`].
pub fn spawn(config: RpkiConfig) -> (Self, JoinHandle<()>) {
let (updates, _) = watch::channel(Update {
version: 0,
vrps: Arc::default(),
aspas: Arc::default(),
changed: None,
});
let rpki = Self {
shared: Arc::new(Shared {
updates,
version: AtomicU64::new(0),
status: Mutex::new(Status {
cache: None,
state: CacheState::Connecting,
session_id: None,
serial: None,
protocol_version: VERSION_2,
vrps_ipv4: 0,
vrps_ipv6: 0,
aspas: 0,
updated_at: None,
expires_at: None,
last_error: None,
}),
}),
};
let client = Client {
shared: Arc::clone(&rpki.shared),
config,
data: Data::default(),
place: None,
protocol_version: VERSION_2,
timers: Timers::default(),
expires: None,
};
(rpki, tokio::spawn(client.run()))
}
/// The VRPs and ASPAs as they are now.
pub fn current(&self) -> Update {
self.shared.updates.borrow().clone()
}
/// The version of the latest [`Update`]. Costs a load of one number,
/// for those who ask before every use whether theirs is still current.
pub fn version(&self) -> u64 {
self.shared.version.load(Ordering::Acquire)
}
/// Follows the updates. A follower that is busy sees the latest only;
/// [`Update::version`] tells it whether it skipped any.
pub fn subscribe(&self) -> watch::Receiver<Update> {
self.shared.updates.subscribe()
}
pub fn status(&self) -> Status {
self.shared
.status
.lock()
.unwrap_or_else(PoisonError::into_inner)
.clone()
}
}
/// Why a session with a cache ended.
#[derive(Debug, thiserror::Error)]
enum SessionError {
#[error("{0}")]
Io(#[from] io::Error),
#[error("the cache sent something that is not RTR: {0}")]
Malformed(#[from] RtrError),
#[error("the cache closed the connection")]
Closed,
#[error("the cache did not accept the connection within {0:?}")]
ConnectTimeout(Duration),
#[error("the cache sent nothing for {0:?} in the middle of an answer")]
ResponseTimeout(Duration),
#[error("the cache did not finish its answer within {0:?}")]
AnswerTimeout(Duration),
#[error("the cache confirmed nothing for so long that the data expired")]
Expired,
#[error("the cache reported an error: {text} (code {code})")]
Reported { code: u16, text: String },
#[error("the cache sent {0} where it has no place")]
Unexpected(&'static str),
#[error("the cache {0}")]
Inconsistent(Inconsistency),
/// Not a failure of the cache: it speaks an older version, and is to
/// be asked again in that.
#[error("the cache speaks protocol version {0}")]
Downgrade(u8),
}
impl SessionError {
/// The error code that tells the cache why it is being left, if that
/// is something it did and can be told (RFC 8210 §10). An Error Report
/// is never answered with another.
fn report_code(&self) -> Option<u16> {
match self {
Self::Malformed(e) => Some(e.code()),
Self::Unexpected(_) => Some(error_code::CORRUPT_DATA),
Self::Inconsistent(problem) => Some(problem.code),
Self::Io(_)
| Self::Closed
| Self::ConnectTimeout(_)
| Self::ResponseTimeout(_)
| Self::AnswerTimeout(_)
| Self::Expired
| Self::Reported { .. }
| Self::Downgrade(_) => None,
}
}
}
/// Something a cache said that does not fit with what it said before.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
struct Inconsistency {
what: &'static str,
/// The error code it is reported to the cache with.
code: u16,
}
impl std::fmt::Display for Inconsistency {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str(self.what)
}
}
const fn inconsistent(what: &'static str, code: u16) -> Inconsistency {
Inconsistency { what, code }
}
const DUPLICATE_VRP: Inconsistency =
inconsistent("announced a VRP twice", error_code::DUPLICATE_ANNOUNCEMENT);
const UNKNOWN_VRP: Inconsistency = inconsistent(
"withdrew a VRP it had not announced",
error_code::WITHDRAWAL_OF_UNKNOWN_RECORD,
);
const UNKNOWN_ASPA: Inconsistency = inconsistent(
"withdrew an ASPA it had not announced",
error_code::WITHDRAWAL_OF_UNKNOWN_RECORD,
);
const OTHER_SESSION: Inconsistency = inconsistent(
"ended an answer in another session than it began it",
error_code::CORRUPT_DATA,
);
const OTHER_VERSION: Inconsistency = inconsistent(
"changed protocol version in the middle of a session",
error_code::UNEXPECTED_VERSION,
);
const INTERNAL: Inconsistency = inconsistent(
"could not be followed (a bug here)",
error_code::INTERNAL_ERROR,
);
/// Where the data is at: which of the cache's sessions, and which serial.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
struct Place {
cache: SocketAddr,
/// The version of the protocol the data came in. What another version
/// carries is not the same data.
protocol_version: u8,
session_id: u16,
serial: u32,
}
/// What a cache has given: the data validation rests on.
#[derive(Debug, Clone, Default)]
struct Data {
vrps: HashSet<Vrp>,
aspas: AspaTable,
}
/// One thing an answer says: a VRP or an ASPA, come or gone.
#[derive(Debug, Clone)]
enum Record {
Vrp {
announce: bool,
vrp: Vrp,
},
/// With providers, the ASPA that is now `customer`'s; without, that
/// it has none any more.
Aspa {
customer: u32,
providers: Option<Vec<u32>>,
},
}
struct Client {
shared: Arc<Shared>,
config: RpkiConfig,
/// The data as last received.
data: Data,
/// Where `data` is at, if it came from a cache that can be asked for
/// what has changed since.
place: Option<Place>,
protocol_version: u8,
/// As the cache last gave them, or the defaults.
timers: Timers,
/// When `data` is to be dropped unless a cache confirms it first.
expires: Option<Instant>,
}
/// A connection to a cache, read a PDU at a time.
struct Connection {
stream: TcpStream,
buf: BytesMut,
}
impl Connection {
async fn send(&mut self, version: u8, pdu: &Pdu) -> io::Result<()> {
let mut out = BytesMut::new();
pdu.encode(version, &mut out);
self.stream.write_all(&out).await
}
/// The next PDU, and the protocol version it was sent with.
/// Cancel-safe: what has arrived of a PDU stays in the buffer.
async fn next(&mut self) -> Result<(u8, Pdu), SessionError> {
loop {
if let Some((version, pdu, used)) = Pdu::decode(&self.buf)? {
let _ = self.buf.split_to(used);
return Ok((version, pdu));
}
if self.stream.read_buf(&mut self.buf).await? == 0 {
return Err(SessionError::Closed);
}
}
}
}
impl Client {
fn status(&self, change: impl FnOnce(&mut Status)) {
change(
&mut self
.shared
.status
.lock()
.unwrap_or_else(PoisonError::into_inner),
);
}
fn refresh(&self) -> Duration {
self.config.refresh.unwrap_or(Duration::from_secs(
self.timers.refresh.clamp(1, 86400).into(),
))
}
fn retry(&self) -> Duration {
self.config
.retry
.unwrap_or(Duration::from_secs(self.timers.retry.clamp(1, 7200).into()))
}
/// Never sooner than a refresh that failed and was tried once more
/// would take, whatever the cache or the configuration says: data that
/// ran out between two refreshes would leave nothing validated for a
/// part of every round (RFC 8210 §6 wants it longer than either).
fn expire(&self) -> Duration {
self.config
.expire
.unwrap_or(Duration::from_secs(
self.timers.expire.clamp(600, 172_800).into(),
))
.max(self.refresh() + self.retry())
}
async fn run(mut self) {
if self.config.caches.is_empty() {
return std::future::pending().await;
}
let mut attempt = 0usize;
loop {
let cache = self.config.caches[attempt % self.config.caches.len()];
self.status(|status| {
status.cache = Some(cache);
status.state = CacheState::Connecting;
});
let ended = self.session(cache).await;
if let SessionError::Downgrade(version) = ended {
info!(%cache, version, "RPKI cache speaks an older RTR version; asking again in that");
self.protocol_version = version;
continue;
}
let no_data = matches!(
ended,
SessionError::Reported {
code: error_code::NO_DATA_AVAILABLE,
..
}
);
if no_data {
info!(%cache, "RPKI cache has no data yet; asking again soon");
} else {
warn!(%cache, err = %ended, "RPKI cache session ended");
}
self.status(|status| {
status.state = CacheState::Connecting;
status.last_error = Some(ended.to_string());
});
attempt += 1;
// The next cache, or this one when it is back, may speak a
// newer version.
self.protocol_version = VERSION_2;
let all_tried = attempt.is_multiple_of(self.config.caches.len());
self.wait(rest(all_tried, no_data, self.retry())).await;
}
}
/// Waits for `duration`, dropping the data on the way if its time runs
/// out.
async fn wait(&mut self, duration: Duration) {
let until = Instant::now() + duration;
if let Some(expires) = self.expires
&& expires <= until
{
sleep_until(expires).await;
self.expired();
}
sleep_until(until).await;
}
/// The data has gone unconfirmed for too long to be used.
fn expired(&mut self) {
warn!(
vrps = self.data.vrps.len(),
aspas = self.data.aspas.len(),
"RPKI data has expired without a cache to confirm it; dropping it"
);
self.data = Data::default();
self.place = None;
self.expires = None;
self.publish(Arc::default(), None);
self.status(|status| {
status.session_id = None;
status.serial = None;
status.expires_at = None;
});
}
/// Runs a session with `cache` until it ends, which it does only by
/// failing.
async fn session(&mut self, cache: SocketAddr) -> SessionError {
let stream = match timeout(CONNECT_TIMEOUT, TcpStream::connect(cache)).await {
Ok(Ok(stream)) => stream,
Ok(Err(e)) => return e.into(),
Err(_) => return SessionError::ConnectTimeout(CONNECT_TIMEOUT),
};
let mut connection = Connection {
stream,
buf: BytesMut::with_capacity(64 * 1024),
};
info!(%cache, "Connected to RPKI cache");
self.status(|status| status.state = CacheState::Synchronising);
loop {
if let Err(e) = self.synchronise(cache, &mut connection).await {
return self.leave(&mut connection, e).await;
}
// Until the cache says there is something new, or it is time
// to ask anyway. If the data runs out first, it is the cache
// that has stopped confirming it.
let ask = Instant::now() + self.refresh();
let expires = self.expires.unwrap_or(ask);
tokio::select! {
pdu = connection.next() => {
let problem = match pdu {
Ok((_, Pdu::ErrorReport { code, text })) => {
SessionError::Reported { code, text }
}
Ok((version, _)) if version != self.protocol_version => {
SessionError::Inconsistent(OTHER_VERSION)
}
Ok((_, Pdu::SerialNotify { .. })) => continue,
Ok(_) => SessionError::Unexpected("an answer nobody asked for"),
Err(e) => e,
};
return self.leave(&mut connection, problem).await;
}
() = sleep_until(ask) => {}
() = sleep_until(expires), if expires < ask => {
self.expired();
return SessionError::Expired;
}
}
}
}
/// Tells the cache why the session is ending, if it is for something
/// the cache did, and gives the reason back.
async fn leave(&self, connection: &mut Connection, why: SessionError) -> SessionError {
if let Some(code) = why.report_code() {
let report = Pdu::ErrorReport {
code,
text: why.to_string(),
};
// If it cannot be told, it cannot.
let _ = timeout(
ERROR_REPORT_TIMEOUT,
connection.send(self.protocol_version, &report),
)
.await;
}
why
}
/// Asks the cache for what has changed, or for everything, and takes
/// the answer.
async fn synchronise(
&mut self,
cache: SocketAddr,
connection: &mut Connection,
) -> Result<(), SessionError> {
// Changes can be asked for only of the cache the data came from,
// in the version of the protocol it came in.
let mut since = self.place.filter(|place| {
place.cache == cache && place.protocol_version == self.protocol_version
});
loop {
let query = match since {
Some(place) => Pdu::SerialQuery {
session_id: place.session_id,
serial: place.serial,
},
None => Pdu::ResetQuery,
};
connection.send(self.protocol_version, &query).await?;
match self.answer(cache, connection, since).await? {
Answer::Complete => return Ok(()),
// The cache cannot say what changed: ask for everything.
Answer::Reset => since = None,
}
}
}
/// Reads the answer to a query. `since` is the place a Serial Query
/// asked about; `None` after a Reset Query.
async fn answer(
&mut self,
cache: SocketAddr,
connection: &mut Connection,
since: Option<Place>,
) -> Result<Answer, SessionError> {
// Each PDU in good time, and all of them too: a cache that sends
// one a minute would keep the old data in use for ever.
let deadline = Instant::now() + ANSWER_TIMEOUT;
let mut next = async || {
let left = deadline.saturating_duration_since(Instant::now());
match timeout(RESPONSE_TIMEOUT.min(left), connection.next()).await {
Ok(pdu) => pdu,
Err(_) if left <= RESPONSE_TIMEOUT => {
Err(SessionError::AnswerTimeout(ANSWER_TIMEOUT))
}
Err(_) => Err(SessionError::ResponseTimeout(RESPONSE_TIMEOUT)),
}
};
// The answer begins with a Cache Response, or is a Cache Reset.
let session_id = loop {
let (version, pdu) = next().await?;
if version < self.protocol_version {
// An older cache answers in its own version, with an error
// or as if it had been asked in that; the query has to be
// put again in that version (RFC 8210 §7).
return Err(SessionError::Downgrade(version));
}
if version > self.protocol_version {
return Err(SessionError::Inconsistent(OTHER_VERSION));
}
match pdu {
Pdu::CacheResponse { session_id } => break session_id,
Pdu::CacheReset if since.is_some() => return Ok(Answer::Reset),
// News of what is being asked for anyway.
Pdu::SerialNotify { .. } => {}
// A cache that refuses the version in the version it
// refuses: the next one down, then.
Pdu::ErrorReport { code, .. }
if code == error_code::UNSUPPORTED_VERSION
&& self.protocol_version != VERSION_0 =>
{
return Err(SessionError::Downgrade(self.protocol_version - 1));
}
Pdu::ErrorReport { code, text } => {
return Err(SessionError::Reported { code, text });
}
_ => return Err(SessionError::Unexpected("data before a Cache Response")),
}
};
// The cache has answered in the version asked for: that is the
// version of the session.
let protocol_version = self.protocol_version;
self.status(|status| status.protocol_version = protocol_version);
if since.is_some_and(|place| place.session_id != session_id) {
// The cache has started over; its serials mean something else
// now.
return Ok(Answer::Reset);
}
// Then the VRPs and ASPAs, and End of Data. Nothing takes effect
// until the answer is complete (RFC 8210 §8.1), so what it says is
// kept aside in the order it came.
let mut records: Vec<Record> = Vec::new();
let (serial, timers) = loop {
let (version, pdu) = next().await?;
if version != self.protocol_version && !matches!(pdu, Pdu::ErrorReport { .. }) {
return Err(SessionError::Inconsistent(OTHER_VERSION));
}
if records.len() >= MAX_RECORDS {
return Err(SessionError::Unexpected("more data than any cache has"));
}
match pdu {
Pdu::Prefix { announce, vrp } => records.push(Record::Vrp { announce, vrp }),
Pdu::Aspa {
announce,
customer,
providers,
} => records.push(Record::Aspa {
customer,
providers: announce.then_some(providers),
}),
Pdu::RouterKey => {}
Pdu::EndOfData {
session_id: ended,
serial,
timers,
} => {
if ended != session_id {
return Err(SessionError::Inconsistent(OTHER_SESSION));
}
break (serial, timers);
}
Pdu::ErrorReport { code, text } => {
return Err(SessionError::Reported { code, text });
}
_ => {
return Err(SessionError::Unexpected(
"a PDU that is not part of an answer",
));
}
}
};
// The first data there is can change any route's validity; after
// that, only where something came or went.
let first = self.place.is_none() && self.expires.is_none();
let (before, replaced) = match since {
// Changes, to what there is.
Some(_) => (std::mem::take(&mut self.data), None),
// Everything anew.
None => (Data::default(), Some(std::mem::take(&mut self.data))),
};
let vrps = Arc::clone(&self.shared.updates.borrow().vrps);
// Building the tables is work for a thread that may take its time.
let built = tokio::task::spawn_blocking(move || apply(before, replaced, &records, vrps))
.await
.map_err(|_| SessionError::Inconsistent(INTERNAL))?;
let built = match built {
Ok(built) => built,
Err((data, problem)) => {
// What there was stays as it was.
self.data = data;
return Err(SessionError::Inconsistent(problem));
}
};
self.data = built.data;
self.place = Some(Place {
cache,
protocol_version: self.protocol_version,
session_id,
serial,
});
self.timers = timers.unwrap_or_default();
let expires_in = self.expire();
self.expires = Some(Instant::now() + expires_in);
let changes = built.changed.vrps.len() + built.changed.aspa_customers.len();
if first || changes > 0 {
self.publish(built.vrps, (!first).then_some(built.changed));
}
info!(
%cache,
serial,
vrps = self.data.vrps.len(),
aspas = self.data.aspas.len(),
changes,
"RPKI data is up to date"
);
let updated_at = SystemTime::now();
self.status(|status| {
status.state = CacheState::Synchronised;
status.session_id = Some(session_id);
status.serial = Some(serial);
status.updated_at = Some(updated_at);
status.expires_at = Some(updated_at + expires_in);
status.last_error = None;
});
Ok(Answer::Complete)
}
/// Publishes the data as it now is. `vrps` holds its VRPs.
fn publish(&self, vrps: Arc<VrpTable>, changed: Option<Changed>) {
let count = |version| {
self.data
.vrps
.iter()
.filter(|vrp| vrp.prefix.address_family() == version)
.count()
};
let (ipv4, ipv6) = (count(IpVersion::Ipv4), count(IpVersion::Ipv6));
self.status(|status| {
status.vrps_ipv4 = ipv4;
status.vrps_ipv6 = ipv6;
status.aspas = self.data.aspas.len();
});
let aspas = Arc::new(self.data.aspas.clone());
self.shared.updates.send_modify(|update| {
update.version += 1;
update.vrps = vrps;
update.aspas = aspas;
update.changed = changed;
self.shared.version.store(update.version, Ordering::Release);
});
}
}
/// The data after an answer, ready to be used.
struct Built {
data: Data,
/// The VRPs of `data`, arranged for validating.
vrps: Arc<VrpTable>,
changed: Changed,
}
/// Applies the `records` of a complete answer, in order, to `data`.
/// `replaced` is the data that an answer to a Reset Query takes the place
/// of, and `vrps` the table of the VRPs as they were before the answer,
/// which is kept if the answer changes none.
///
/// If the answer contradicts itself or what there was, what there was is
/// given back with the reason.
fn apply(
mut data: Data,
replaced: Option<Data>,
records: &[Record],
vrps: Arc<VrpTable>,
) -> Result<Built, (Data, Inconsistency)> {
// What each record displaced, to put back if a later one cannot be
// applied.
let mut displaced: Vec<Option<Arc<[u32]>>> = Vec::new();
let mut problem = None;
for record in records {
let ok = match record {
Record::Vrp {
announce: true,
vrp,
} => data.vrps.insert(*vrp).then_some(None).ok_or(DUPLICATE_VRP),
Record::Vrp {
announce: false,
vrp,
} => data.vrps.remove(vrp).then_some(None).ok_or(UNKNOWN_VRP),
// An ASPA that is announced takes the place of the one there
// was (draft-ietf-sidrops-8210bis §5.12).
Record::Aspa {
customer,
providers: Some(providers),
} => Ok(data.aspas.set(*customer, providers.iter().copied())),
Record::Aspa {
customer,
providers: None,
} => data.aspas.remove(*customer).map(Some).ok_or(UNKNOWN_ASPA),
};
match ok {
Ok(was) => displaced.push(was),
Err(e) => {
problem = Some(e);
break;
}
}
}
if let Some(problem) = problem {
return Err(match replaced {
Some(replaced) => (replaced, problem),
None => {
// Undo, last first.
for (record, was) in records.iter().zip(displaced).rev() {
match record {
Record::Vrp {
announce: true,
vrp,
} => {
data.vrps.remove(vrp);
}
Record::Vrp {
announce: false,
vrp,
} => {
data.vrps.insert(*vrp);
}
Record::Aspa { customer, .. } => data.aspas.restore(*customer, was),
}
}
(data, problem)
}
});
}
// What came or went: against the data replaced, or else whatever the
// answer touched.
let (changed_vrps, aspa_customers): (Vec<Vrp>, HashSet<u32>) = match &replaced {
Some(replaced) => (
data.vrps
.symmetric_difference(&replaced.vrps)
.copied()
.collect(),
differing_customers(&data.aspas, &replaced.aspas),
),
None => (
records
.iter()
.filter_map(|record| match record {
Record::Vrp { vrp, .. } => Some(*vrp),
Record::Aspa { .. } => None,
})
.collect(),
records
.iter()
.filter_map(|record| match record {
Record::Aspa { customer, .. } => Some(*customer),
Record::Vrp { .. } => None,
})
.collect(),
),
};
Ok(Built {
vrps: match changed_vrps.is_empty() {
true => vrps,
false => Arc::new(VrpTable::new(data.vrps.iter().copied())),
},
changed: Changed {
vrps: Arc::new(VrpTable::new(changed_vrps)),
aspa_customers: Arc::new(aspa_customers),
},
data,
})
}
/// The customers whose ASPA is not the same in `a` as in `b`.
fn differing_customers(a: &AspaTable, b: &AspaTable) -> HashSet<u32> {
let differs = |(customer, providers): (u32, &[u32]), other: &AspaTable| {
(other.providers(customer) != Some(providers)).then_some(customer)
};
a.iter()
.filter_map(|aspa| differs(aspa, b))
.chain(b.iter().filter_map(|aspa| differs(aspa, a)))
.collect()
}
/// How a query was answered.
enum Answer {
/// With data, which is now in use.
Complete,
/// With a request to ask for everything instead.
Reset,
}
/// How long to rest after a session ended: the next cache at once; when
/// all have been tried, the retry interval, or less if the last one is
/// only still validating.
fn rest(all_tried: bool, no_data: bool, retry: Duration) -> Duration {
match (all_tried, no_data) {
(false, _) => NEXT_CACHE_DELAY,
(true, true) => NO_DATA_RETRY.min(retry),
(true, false) => retry,
}
}
#[cfg(test)]
mod tests {
use super::*;
/// A cache that is still validating after it started is asked again
/// in half a minute, not the ten the retry interval is by default.
#[test]
fn a_cache_without_data_yet_is_asked_again_soon() {
let ten_minutes = Duration::from_secs(600);
assert_eq!(rest(true, true, ten_minutes), NO_DATA_RETRY);
assert_eq!(rest(true, false, ten_minutes), ten_minutes);
// Never later than the retry interval says.
let short = Duration::from_secs(5);
assert_eq!(rest(true, true, short), short);
// Another cache first, whatever this one said.
assert_eq!(rest(false, true, ten_minutes), NEXT_CACHE_DELAY);
}
}
crates/rpki/src/lib.rs
-29
View File
@@ -1,29 +0,0 @@
//! Route origin validation (RFC 6811) and AS path verification
//! (draft-ietf-sidrops-aspa-verification) with the RPKI.
//!
//! The holder of an address block can sign a statement of which AS may
//! originate routes for it. Validators collect and check those statements
//! and hand routers the result, a list of validated ROA payloads (VRPs),
//! over the RPKI-to-Router protocol (RFC 8210). With that list a router can
//! tell a route whose origin is authorised from one whose origin is not:
//! the commonest kind of hijack and of mistake.
//!
//! The same way, a network can sign a statement of who its providers are
//! (an ASPA). With those a router can tell a path that could have come
//! about by networks passing routes on as they are paid to from one that
//! could not: a route leak, or a path made up to get past origin
//! validation.
//!
//! - [`vrp`] holds the VRPs and answers whether a route's origin is valid.
//! - [`aspa`] holds the ASPAs and answers whether a path is.
//! - [`rtr`] is the protocol's messages.
//! - [`client`] keeps both in step with a validator.
pub mod aspa;
pub mod client;
pub mod rtr;
pub mod vrp;
pub use aspa::{Algorithm, AspaTable, PathValidity};
pub use client::{CacheState, Changed, Rpki, RpkiConfig, Status, Update};
pub use vrp::{Validity, Vrp, VrpTable};
crates/rpki/src/rtr.rs
-674
View File
@@ -1,674 +0,0 @@
//! The messages of the RPKI-to-Router protocol, as bytes and back: version
//! 0 (RFC 6810), version 1 (RFC 8210) and version 2
//! (draft-ietf-sidrops-8210bis). No I/O here.
//!
//! Every message (a PDU) begins with the same eight octets: the protocol
//! version, the type, a field whose meaning depends on the type, and the
//! length of the whole PDU.
use bgp_primitives::constants::IpVersion;
use bgp_primitives::ip_prefix::IpPrefix;
use bytes::{BufMut, BytesMut};
use crate::vrp::Vrp;
/// The newest protocol version, which this client asks for first. Version
/// 2 added ASPAs.
pub const VERSION_2: u8 = 2;
/// Version 1 added the timers in End of Data and router keys.
pub const VERSION_1: u8 = 1;
/// What the oldest validators speak.
pub const VERSION_0: u8 = 0;
const HEADER_LEN: usize = 8;
/// The longest PDU this client reads. Every PDU but Error Report, Router
/// Key and ASPA has a fixed, small length. An ASPA of this length lists
/// sixteen thousand providers, more than validators let an AS have.
pub const MAX_PDU_LEN: usize = 64 * 1024;
mod pdu_type {
pub const SERIAL_NOTIFY: u8 = 0;
pub const SERIAL_QUERY: u8 = 1;
pub const RESET_QUERY: u8 = 2;
pub const CACHE_RESPONSE: u8 = 3;
pub const IPV4_PREFIX: u8 = 4;
pub const IPV6_PREFIX: u8 = 6;
pub const END_OF_DATA: u8 = 7;
pub const CACHE_RESET: u8 = 8;
pub const ROUTER_KEY: u8 = 9;
pub const ERROR_REPORT: u8 = 10;
pub const ASPA: u8 = 11;
}
/// Error codes of an Error Report (RFC 8210 §12; from 9 on, version 2).
pub mod error_code {
pub const CORRUPT_DATA: u16 = 0;
pub const INTERNAL_ERROR: u16 = 1;
pub const NO_DATA_AVAILABLE: u16 = 2;
pub const INVALID_REQUEST: u16 = 3;
pub const UNSUPPORTED_VERSION: u16 = 4;
pub const UNSUPPORTED_PDU_TYPE: u16 = 5;
pub const WITHDRAWAL_OF_UNKNOWN_RECORD: u16 = 6;
pub const DUPLICATE_ANNOUNCEMENT: u16 = 7;
pub const UNEXPECTED_VERSION: u16 = 8;
pub const ASPA_PROVIDER_LIST: u16 = 9;
pub const TRANSPORT_FAILURE: u16 = 10;
pub const ORDERING: u16 = 11;
}
/// The timers a validator tells its routers to use (RFC 8210 §6), in
/// seconds.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Timers {
/// How long to wait before asking for changes unprompted.
pub refresh: u32,
/// How long to wait before trying again after a failure.
pub retry: u32,
/// How long the data may be used without a successful update.
pub expire: u32,
}
impl Default for Timers {
/// What RFC 8210 §6 recommends, and what a version 0 session uses.
fn default() -> Self {
Self {
refresh: 3600,
retry: 600,
expire: 7200,
}
}
}
/// One PDU.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum Pdu {
/// The validator has new data, up to `serial`.
SerialNotify { session_id: u16, serial: u32 },
/// Send what has changed since `serial`.
SerialQuery { session_id: u16, serial: u32 },
/// Send everything.
ResetQuery,
/// The answer to a query begins: prefixes follow, then End of Data.
CacheResponse { session_id: u16 },
/// A VRP, announced or withdrawn.
Prefix { announce: bool, vrp: Vrp },
/// The answer is complete, and the data is now at `serial`. `timers`
/// are there from version 1 on.
EndOfData {
session_id: u16,
serial: u32,
timers: Option<Timers>,
},
/// The validator cannot say what changed since the serial asked about;
/// ask for everything.
CacheReset,
/// A BGPsec router key. Not used here; read so as to be skipped.
RouterKey,
/// The ASPA of `customer`, whole (version 2). Announced, it takes the
/// place of any the cache gave for that customer before, and lists at
/// least one provider; withdrawn, it lists none.
Aspa {
announce: bool,
customer: u32,
providers: Vec<u32>,
},
/// Something went wrong at the sender's end, or with what we sent.
ErrorReport { code: u16, text: String },
}
/// Bytes that are not a PDU this client can read.
#[derive(Debug, Clone, PartialEq, Eq, thiserror::Error)]
pub enum RtrError {
#[error("PDU of type {pdu_type} has length {length}, which is not valid for it")]
Length { pdu_type: u8, length: u32 },
#[error("unknown PDU type {0}")]
UnknownType(u8),
#[error("protocol version {0} is not supported")]
Version(u8),
#[error("a prefix PDU with prefix length {length} and maximum length {max_length}")]
Prefix { length: u8, max_length: u8 },
#[error("an ASPA of AS{customer} is announced with no providers")]
AspaWithoutProviders { customer: u32 },
}
impl RtrError {
/// The error code that reports this to the sender (RFC 8210 §12).
pub fn code(&self) -> u16 {
match self {
Self::Length { .. } | Self::Prefix { .. } => error_code::CORRUPT_DATA,
Self::UnknownType(_) => error_code::UNSUPPORTED_PDU_TYPE,
Self::Version(_) => error_code::UNSUPPORTED_VERSION,
Self::AspaWithoutProviders { .. } => error_code::ASPA_PROVIDER_LIST,
}
}
}
fn be_u16(bytes: &[u8]) -> u16 {
u16::from_be_bytes([bytes[0], bytes[1]])
}
fn be_u32(bytes: &[u8]) -> u32 {
u32::from_be_bytes([bytes[0], bytes[1], bytes[2], bytes[3]])
}
impl Pdu {
/// Reads the PDU at the front of `buf`, if all of it has arrived:
/// the protocol version it was sent with, the PDU, and how many octets
/// it took. `Ok(None)` means more octets are needed.
pub fn decode(buf: &[u8]) -> Result<Option<(u8, Self, usize)>, RtrError> {
let Some(header) = buf.first_chunk::<HEADER_LEN>() else {
return Ok(None);
};
let (version, pdu_type, flags) = (header[0], header[1], header[2]);
let field = be_u16(&header[2..]);
let length = be_u32(&header[4..]);
let bad_length = || RtrError::Length { pdu_type, length };
if version > VERSION_2 {
return Err(RtrError::Version(version));
}
let total = usize::try_from(length).map_err(|_| bad_length())?;
if !(HEADER_LEN..=MAX_PDU_LEN).contains(&total) {
return Err(bad_length());
}
// The fixed lengths are checked before waiting for the rest, so
// that a wrong one is not waited for.
let fixed = match pdu_type {
pdu_type::SERIAL_NOTIFY | pdu_type::SERIAL_QUERY => Some(12),
pdu_type::RESET_QUERY | pdu_type::CACHE_RESPONSE | pdu_type::CACHE_RESET => Some(8),
pdu_type::IPV4_PREFIX => Some(20),
pdu_type::IPV6_PREFIX => Some(32),
pdu_type::END_OF_DATA => Some(if version == VERSION_0 { 12 } else { 24 }),
pdu_type::ROUTER_KEY | pdu_type::ERROR_REPORT => None,
pdu_type::ASPA if version >= VERSION_2 => {
// The customer, and whole AS numbers after it; none of
// them in a withdrawal.
let announce = flags & 1 == 1;
if total < 12 || !total.is_multiple_of(4) || (!announce && total != 12) {
return Err(bad_length());
}
None
}
other => return Err(RtrError::UnknownType(other)),
};
if fixed.is_some_and(|fixed| fixed != total) {
return Err(bad_length());
}
let Some(body) = buf.get(HEADER_LEN..total) else {
return Ok(None);
};
let pdu = match pdu_type {
pdu_type::SERIAL_NOTIFY => Self::SerialNotify {
session_id: field,
serial: be_u32(body),
},
pdu_type::SERIAL_QUERY => Self::SerialQuery {
session_id: field,
serial: be_u32(body),
},
pdu_type::RESET_QUERY => Self::ResetQuery,
pdu_type::CACHE_RESPONSE => Self::CacheResponse { session_id: field },
pdu_type::IPV4_PREFIX | pdu_type::IPV6_PREFIX => {
let version = if pdu_type == pdu_type::IPV4_PREFIX {
IpVersion::Ipv4
} else {
IpVersion::Ipv6
};
let (flags, length, max_length) = (body[0], body[1], body[2]);
let address = &body[4..body.len() - 4];
let bad_prefix = || RtrError::Prefix { length, max_length };
if max_length < length || max_length > version.bits() {
return Err(bad_prefix());
}
Self::Prefix {
announce: flags & 1 == 1,
vrp: Vrp {
prefix: IpPrefix::new(version, address, length)
.map_err(|_| bad_prefix())?,
max_length,
asn: be_u32(&body[body.len() - 4..]),
},
}
}
pdu_type::END_OF_DATA => Self::EndOfData {
session_id: field,
serial: be_u32(body),
timers: (version != VERSION_0).then(|| Timers {
refresh: be_u32(&body[4..]),
retry: be_u32(&body[8..]),
expire: be_u32(&body[12..]),
}),
},
pdu_type::CACHE_RESET => Self::CacheReset,
pdu_type::ROUTER_KEY => Self::RouterKey,
pdu_type::ASPA => {
let announce = flags & 1 == 1;
let (customer, providers) = body.split_at(4);
let customer = be_u32(customer);
if announce && providers.is_empty() {
return Err(RtrError::AspaWithoutProviders { customer });
}
Self::Aspa {
announce,
customer,
providers: providers
.as_chunks::<4>()
.0
.iter()
.map(|asn| u32::from_be_bytes(*asn))
.collect(),
}
}
pdu_type::ERROR_REPORT => {
// The length of the PDU that was at fault, that PDU, the
// length of the text, the text.
let text = (|| {
let pdu_len = usize::try_from(be_u32(body.get(..4)?)).ok()?;
let rest = body.get(4usize.checked_add(pdu_len)?..)?;
let text_len = usize::try_from(be_u32(rest.get(..4)?)).ok()?;
let text = rest.get(4..)?;
(text.len() == text_len).then(|| String::from_utf8_lossy(text).into_owned())
})()
.ok_or_else(bad_length)?;
Self::ErrorReport { code: field, text }
}
_ => unreachable!("unknown types were refused above"),
};
Ok(Some((version, pdu, total)))
}
/// Writes the PDU, for a session that speaks protocol version
/// `version`, to the end of `out`. Whether that version has such a PDU
/// is the caller's to know.
pub fn encode(&self, version: u8, out: &mut BytesMut) {
let start = out.len();
let header = |out: &mut BytesMut, pdu_type: u8, field: u16| {
out.put_u8(version);
out.put_u8(pdu_type);
out.put_u16(field);
// The length, filled in below.
out.put_u32(0);
};
match self {
Self::SerialNotify { session_id, serial } => {
header(out, pdu_type::SERIAL_NOTIFY, *session_id);
out.put_u32(*serial);
}
Self::SerialQuery { session_id, serial } => {
header(out, pdu_type::SERIAL_QUERY, *session_id);
out.put_u32(*serial);
}
Self::ResetQuery => header(out, pdu_type::RESET_QUERY, 0),
Self::CacheResponse { session_id } => {
header(out, pdu_type::CACHE_RESPONSE, *session_id);
}
Self::Prefix { announce, vrp } => {
let (pdu_type, address_len) = match vrp.prefix.address_family() {
IpVersion::Ipv4 => (pdu_type::IPV4_PREFIX, 4),
IpVersion::Ipv6 => (pdu_type::IPV6_PREFIX, 16),
};
header(out, pdu_type, 0);
out.put_u8(u8::from(*announce));
out.put_u8(vrp.prefix.length());
out.put_u8(vrp.max_length);
out.put_u8(0);
let mut address = [0u8; 16];
address[..vrp.prefix.bytes().len()].copy_from_slice(vrp.prefix.bytes());
out.put_slice(&address[..address_len]);
out.put_u32(vrp.asn);
}
Self::EndOfData {
session_id,
serial,
timers,
} => {
header(out, pdu_type::END_OF_DATA, *session_id);
out.put_u32(*serial);
if version != VERSION_0 {
let timers = timers.unwrap_or_default();
out.put_u32(timers.refresh);
out.put_u32(timers.retry);
out.put_u32(timers.expire);
}
}
Self::CacheReset => header(out, pdu_type::CACHE_RESET, 0),
Self::RouterKey => header(out, pdu_type::ROUTER_KEY, 0),
Self::Aspa {
announce,
customer,
providers,
} => {
header(out, pdu_type::ASPA, u16::from(*announce) << 8);
out.put_u32(*customer);
for provider in providers {
out.put_u32(*provider);
}
}
Self::ErrorReport { code, text } => {
header(out, pdu_type::ERROR_REPORT, *code);
// No copy of the PDU at fault.
out.put_u32(0);
out.put_u32(text.len() as u32);
out.put_slice(text.as_bytes());
}
}
let length = (out.len() - start) as u32;
out[start + 4..start + 8].copy_from_slice(&length.to_be_bytes());
}
}
#[cfg(test)]
mod tests {
use proptest::prelude::*;
use super::*;
fn decode_one(bytes: &[u8]) -> (u8, Pdu) {
let (version, pdu, used) = Pdu::decode(bytes).unwrap().expect("a whole PDU");
assert_eq!(used, bytes.len());
(version, pdu)
}
/// The PDUs as RFC 8210 §5 draws them.
#[test]
fn pdus_are_read_as_the_rfc_lays_them_out() {
#[rustfmt::skip]
let ipv4 = [
1, 4, 0, 0, 0, 0, 0, 20,
1, 21, 24, 0,
193, 0, 0, 0,
0, 0, 0x0d, 0x05,
];
assert_eq!(
decode_one(&ipv4),
(
1,
Pdu::Prefix {
announce: true,
vrp: Vrp {
prefix: IpPrefix::try_from("193.0.0.0/21").unwrap(),
max_length: 24,
asn: 3333,
},
}
)
);
#[rustfmt::skip]
let ipv6 = [
1, 6, 0, 0, 0, 0, 0, 32,
0, 48, 48, 0,
0x20, 0x01, 0x06, 0x7c, 0x02, 0xe8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0x0d, 0x05,
];
assert_eq!(
decode_one(&ipv6),
(
1,
Pdu::Prefix {
announce: false,
vrp: Vrp {
prefix: IpPrefix::try_from("2001:67c:2e8::/48").unwrap(),
max_length: 48,
asn: 3333,
},
}
)
);
#[rustfmt::skip]
let end_of_data = [
1, 7, 0x12, 0x34, 0, 0, 0, 24,
0, 0, 0, 99,
0, 0, 0x0e, 0x10, 0, 0, 0x02, 0x58, 0, 0, 0x1c, 0x20,
];
assert_eq!(
decode_one(&end_of_data),
(
1,
Pdu::EndOfData {
session_id: 0x1234,
serial: 99,
timers: Some(Timers::default()),
}
)
);
// Version 0 has no timers.
assert_eq!(
decode_one(&[0, 7, 0x12, 0x34, 0, 0, 0, 12, 0, 0, 0, 99]),
(
0,
Pdu::EndOfData {
session_id: 0x1234,
serial: 99,
timers: None,
}
)
);
// Version 2's ASPA, as draft-ietf-sidrops-8210bis §5.12 draws it.
#[rustfmt::skip]
let aspa = [
2, 11, 1, 0, 0, 0, 0, 20,
0, 0, 0x0d, 0x05,
0, 0, 0x04, 0x4d, 0, 0, 0x0c, 0xb9,
];
assert_eq!(
decode_one(&aspa),
(
2,
Pdu::Aspa {
announce: true,
customer: 3333,
providers: vec![1101, 3257],
}
)
);
assert_eq!(
decode_one(&[2, 11, 0, 0, 0, 0, 0, 12, 0, 0, 0x0d, 0x05]),
(
2,
Pdu::Aspa {
announce: false,
customer: 3333,
providers: vec![],
}
)
);
#[rustfmt::skip]
let error = [
1, 10, 0, 2, 0, 0, 0, 30,
0, 0, 0, 8, 1, 1, 0, 0, 0, 0, 0, 12,
0, 0, 0, 6, b'n', b'o', b' ', b'd', b'a', b't',
];
assert_eq!(
decode_one(&error),
(
1,
Pdu::ErrorReport {
code: error_code::NO_DATA_AVAILABLE,
text: "no dat".into(),
}
)
);
}
#[test]
fn a_pdu_that_has_not_all_arrived_is_waited_for() {
let mut wire = BytesMut::new();
Pdu::SerialNotify {
session_id: 7,
serial: 42,
}
.encode(VERSION_1, &mut wire);
for cut in 0..wire.len() {
assert_eq!(Pdu::decode(&wire[..cut]), Ok(None), "{cut} octets");
}
assert!(Pdu::decode(&wire).unwrap().is_some());
}
#[test]
fn what_is_not_a_pdu_is_refused_from_its_header() {
// A version from the future.
assert_eq!(
Pdu::decode(&[3, 2, 0, 0, 0, 0, 0, 8]),
Err(RtrError::Version(3))
);
assert_eq!(
Pdu::decode(&[1, 99, 0, 0, 0, 0, 0, 8]),
Err(RtrError::UnknownType(99))
);
// A Reset Query that claims to be long, and a length no PDU has:
// refused without waiting for the octets.
for length in [9u32, 7, 0, 1 << 30] {
let mut header = vec![1, 2, 0, 0];
header.extend_from_slice(&length.to_be_bytes());
assert_eq!(
Pdu::decode(&header),
Err(RtrError::Length {
pdu_type: 2,
length
}),
);
}
// A maximum length shorter than the prefix, and longer than an
// address.
for (length, max_length) in [(24, 21), (24, 33)] {
#[rustfmt::skip]
let prefix = [
1, 4, 0, 0, 0, 0, 0, 20,
1, length, max_length, 0, 193, 0, 0, 0, 0, 0, 0, 1,
];
assert_eq!(
Pdu::decode(&prefix),
Err(RtrError::Prefix { length, max_length })
);
}
// An ASPA before version 2 had them.
assert_eq!(
Pdu::decode(&[1, 11, 0, 0, 0, 0, 0, 12, 0, 0, 0, 1]),
Err(RtrError::UnknownType(11))
);
// ASPAs with half an AS number, without a customer, withdrawn
// with providers: refused from the header.
for (flags, length) in [(1u8, 18u32), (1, 8), (0, 16)] {
let mut header = vec![2, 11, flags, 0];
header.extend_from_slice(&length.to_be_bytes());
assert_eq!(
Pdu::decode(&header),
Err(RtrError::Length {
pdu_type: 11,
length
}),
);
}
// Announced with no providers.
assert_eq!(
Pdu::decode(&[2, 11, 1, 0, 0, 0, 0, 12, 0, 0, 0xfd, 0xe8]),
Err(RtrError::AspaWithoutProviders { customer: 65000 })
);
// An Error Report whose inner lengths do not add up.
#[rustfmt::skip]
let error = [1, 10, 0, 2, 0, 0, 0, 16, 0, 0, 0, 9, 0, 0, 0, 0];
assert!(matches!(Pdu::decode(&error), Err(RtrError::Length { .. })));
}
fn any_prefix() -> impl Strategy<Value = Vrp> {
(
any::<bool>(),
any::<[u8; 16]>(),
0u8..=128,
0u8..=128,
any::<u32>(),
)
.prop_map(|(v6, bytes, length, extra, asn)| {
let version = if v6 { IpVersion::Ipv6 } else { IpVersion::Ipv4 };
let length = length.min(version.bits());
Vrp {
prefix: IpPrefix::new(version, bytes, length).unwrap(),
max_length: length.saturating_add(extra).min(version.bits()),
asn,
}
})
}
fn any_pdu() -> impl Strategy<Value = Pdu> {
prop_oneof![
(any::<u16>(), any::<u32>())
.prop_map(|(session_id, serial)| Pdu::SerialNotify { session_id, serial }),
(any::<u16>(), any::<u32>())
.prop_map(|(session_id, serial)| Pdu::SerialQuery { session_id, serial }),
Just(Pdu::ResetQuery),
any::<u16>().prop_map(|session_id| Pdu::CacheResponse { session_id }),
(any::<bool>(), any_prefix()).prop_map(|(announce, vrp)| Pdu::Prefix { announce, vrp }),
(any::<u16>(), any::<u32>(), any::<[u32; 3]>()).prop_map(
|(session_id, serial, [refresh, retry, expire])| Pdu::EndOfData {
session_id,
serial,
timers: Some(Timers {
refresh,
retry,
expire
}),
}
),
Just(Pdu::CacheReset),
(any::<u16>(), "[ -~]{0,60}").prop_map(|(code, text)| Pdu::ErrorReport { code, text }),
]
}
proptest! {
#[test]
fn a_pdu_that_is_written_reads_back_equal(pdu in any_pdu(), version in 0u8..=2) {
let mut wire = BytesMut::new();
pdu.encode(version, &mut wire);
// Version 0 does not carry the timers.
let expected = match (&pdu, version) {
(Pdu::EndOfData { session_id, serial, .. }, VERSION_0) => Pdu::EndOfData {
session_id: *session_id,
serial: *serial,
timers: None,
},
_ => pdu.clone(),
};
prop_assert_eq!(Pdu::decode(&wire), Ok(Some((version, expected, wire.len()))));
}
#[test]
fn an_aspa_that_is_written_reads_back_equal(
customer: u32,
providers in proptest::collection::vec(any::<u32>(), 0..40),
) {
let pdu = Pdu::Aspa { announce: !providers.is_empty(), customer, providers };
let mut wire = BytesMut::new();
pdu.encode(VERSION_2, &mut wire);
prop_assert_eq!(Pdu::decode(&wire), Ok(Some((VERSION_2, pdu, wire.len()))));
}
/// Whatever arrives, decoding returns: a PDU, a wait for more, or
/// an error. What it reads is within what arrived.
#[test]
fn noise_is_never_more_than_an_error(bytes in proptest::collection::vec(any::<u8>(), 0..80)) {
if let Ok(Some((_, _, used))) = Pdu::decode(&bytes) {
prop_assert!(used <= bytes.len());
}
}
/// Noise behind a header that could be a PDU's, to get past it.
#[test]
fn noise_behind_a_plausible_header_is_never_more_than_an_error(
version in 0u8..=2,
pdu_type in 0u8..13,
field: u16,
length in 8u32..64,
body in proptest::collection::vec(any::<u8>(), 0..64),
) {
let mut bytes = vec![version, pdu_type];
bytes.extend_from_slice(&field.to_be_bytes());
bytes.extend_from_slice(&length.to_be_bytes());
bytes.extend_from_slice(&body);
if let Ok(Some((_, _, used))) = Pdu::decode(&bytes) {
prop_assert!(used <= bytes.len());
}
}
}
}
crates/rpki/src/vrp.rs
-429
View File
@@ -1,429 +0,0 @@
//! The validated ROA payloads, and what they say about a route.
use bgp_primitives::constants::IpVersion;
use bgp_primitives::ip_prefix::IpPrefix;
/// A validated ROA payload: `asn` may originate `prefix`, and any prefix
/// inside it of at most `max_length` bits.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Ord)]
pub struct Vrp {
pub prefix: IpPrefix,
pub max_length: u8,
pub asn: u32,
}
/// What the VRPs say about a route (RFC 6811 §2).
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub enum Validity {
/// No VRP covers the route's prefix: nothing is known about it. Most
/// of the Internet's routes are in this state, and are used.
NotFound,
/// A VRP covers the prefix and allows this origin at this length.
Valid,
/// VRPs cover the prefix and none of them allows this origin at this
/// length: the holder of the addresses has said who may announce
/// them, and this is not it.
Invalid,
}
impl Validity {
pub fn as_str(self) -> &'static str {
match self {
Self::NotFound => "not found",
Self::Valid => "valid",
Self::Invalid => "invalid",
}
}
}
/// A set of VRPs, arranged for asking about routes: this is asked for
/// every route received.
///
/// It is built once and not changed. When the VRPs change, another is built
/// and takes its place.
#[derive(Debug, Default)]
pub struct VrpTable {
v4: Trie,
v6: Trie,
len: usize,
}
/// A binary trie over prefixes: a node for each prefix that is a VRP's or
/// lies on the way to one.
#[derive(Debug)]
struct Trie {
nodes: Vec<Node>,
/// What the VRPs of each node allow, in runs by node: (maximum length,
/// origin AS).
allowed: Vec<(u8, u32)>,
/// A short cut past the first sixteen levels, which every lookup of a
/// longer prefix would otherwise walk: for each value of an address's
/// first sixteen bits, the node those bits lead to, or zero if there
/// is none. [`WALK`] is set where a VRP lies on the way there, which
/// is rare; such lookups walk from the root.
jump: Box<[u32]>,
}
/// How many leading bits [`Trie::jump`] is indexed by.
const JUMP_BITS: u8 = 16;
/// Set in a [`Trie::jump`] entry: walk from the root.
const WALK: u32 = 1 << 31;
#[derive(Debug, Clone, Copy, Default)]
struct Node {
/// The nodes for this prefix extended by a zero bit and by a one bit;
/// zero where there is none (the root is nobody's child).
children: [u32; 2],
/// Where in `allowed` this prefix's VRPs are, and how many.
first: u32,
count: u32,
}
impl Default for Trie {
fn default() -> Self {
Self {
nodes: vec![Node::default()],
allowed: Vec::new(),
jump: vec![0; 1 << JUMP_BITS].into(),
}
}
}
/// The bit of `bytes` at `index`, counting from the most significant bit
/// of the first byte.
fn bit(bytes: &[u8], index: u8) -> usize {
usize::from(bytes[usize::from(index / 8)] >> (7 - index % 8) & 1)
}
impl Trie {
/// `vrps` must be sorted, so that those of one prefix are together.
fn build(vrps: &[Vrp]) -> Self {
let mut trie = Self::default();
for group in vrps.chunk_by(|a, b| a.prefix == b.prefix) {
let prefix = &group[0].prefix;
let mut at = 0usize;
for index in 0..prefix.length() {
let side = bit(prefix.bytes(), index);
at = match trie.nodes[at].children[side] {
0 => {
trie.nodes.push(Node::default());
let new = trie.nodes.len() - 1;
trie.nodes[at].children[side] = new as u32;
new
}
child => child as usize,
};
}
trie.nodes[at].first = trie.allowed.len() as u32;
trie.nodes[at].count = group.len() as u32;
trie.allowed
.extend(group.iter().map(|vrp| (vrp.max_length, vrp.asn)));
}
assert!(
trie.nodes.len() < WALK as usize,
"more VRPs than a table can hold"
);
for slot in 0..=u16::MAX {
trie.jump[usize::from(slot)] = trie.jump_for(slot);
}
trie
}
/// The [`Trie::jump`] entry for addresses that begin with `bits`.
fn jump_for(&self, bits: u16) -> u32 {
let bytes = bits.to_be_bytes();
let mut at = 0usize;
for index in 0..JUMP_BITS {
if self.nodes[at].count > 0 {
return WALK;
}
match self.nodes[at].children[bit(&bytes, index)] {
0 => return 0,
child => at = child as usize,
}
}
at as u32
}
/// Calls `visit` with what the VRPs of each prefix that covers `prefix`
/// allow, from the shortest; stops when `visit` returns true, and
/// returns whether it did.
fn covering(&self, prefix: &IpPrefix, mut visit: impl FnMut(&[(u8, u32)]) -> bool) -> bool {
let bytes = prefix.bytes();
let mut at = 0usize;
let mut index = 0;
if let [first, second, ..] = bytes
&& prefix.length() >= JUMP_BITS
{
match self.jump[usize::from(u16::from_be_bytes([*first, *second]))] {
WALK => {}
// Nothing covers an address that begins like this.
0 => return false,
node => (at, index) = (node as usize, JUMP_BITS),
}
}
loop {
let node = &self.nodes[at];
if node.count > 0 {
let first = node.first as usize;
if visit(&self.allowed[first..first + node.count as usize]) {
return true;
}
}
if index == prefix.length() {
return false;
}
match node.children[bit(bytes, index)] {
0 => return false,
child => at = child as usize,
}
index += 1;
}
}
}
impl VrpTable {
/// A table of `vrps`. Listing one more than once is the same as listing
/// it once.
pub fn new(vrps: impl IntoIterator<Item = Vrp>) -> Self {
let mut vrps: Vec<Vrp> = vrps.into_iter().collect();
vrps.sort_unstable();
vrps.dedup();
let v6_from = vrps.partition_point(|vrp| vrp.prefix.address_family() == IpVersion::Ipv4);
debug_assert!(
vrps[..v6_from]
.iter()
.all(|vrp| vrp.prefix.address_family() == IpVersion::Ipv4)
);
Self {
v4: Trie::build(&vrps[..v6_from]),
v6: Trie::build(&vrps[v6_from..]),
len: vrps.len(),
}
}
/// How many VRPs the table holds.
pub fn len(&self) -> usize {
self.len
}
pub fn is_empty(&self) -> bool {
self.len == 0
}
fn trie(&self, prefix: &IpPrefix) -> &Trie {
match prefix.address_family() {
IpVersion::Ipv4 => &self.v4,
IpVersion::Ipv6 => &self.v6,
}
}
/// The validity of a route for `prefix` that originates from `origin`
/// (RFC 6811 §2). `None` is a route with no one origin, as when its AS
/// path ends in a set: no VRP can allow it.
pub fn validate(&self, prefix: &IpPrefix, origin: Option<u32>) -> Validity {
let mut covered = false;
let valid = self.trie(prefix).covering(prefix, |allowed| {
covered = true;
// AS 0 in a VRP says that nobody may originate the prefix
// (RFC 6483 §4); it is the origin of no route.
origin.is_some_and(|origin| {
origin != 0
&& allowed
.iter()
.any(|(max_length, asn)| *asn == origin && prefix.length() <= *max_length)
})
});
match (valid, covered) {
(true, _) => Validity::Valid,
(false, true) => Validity::Invalid,
(false, false) => Validity::NotFound,
}
}
/// Whether some VRP covers `prefix`: whether the VRPs say anything
/// about routes for it.
pub fn covers(&self, prefix: &IpPrefix) -> bool {
self.trie(prefix).covering(prefix, |_| true)
}
}
#[cfg(test)]
mod tests {
use proptest::prelude::*;
use super::*;
fn prefix(s: &str) -> IpPrefix {
IpPrefix::try_from(s).unwrap()
}
fn vrp(p: &str, max_length: u8, asn: u32) -> Vrp {
Vrp {
prefix: prefix(p),
max_length,
asn,
}
}
/// The cases of RFC 6811 §2.
#[test]
fn a_route_is_valid_invalid_or_not_found() {
let table = VrpTable::new([
vrp("193.0.0.0/21", 24, 3333),
vrp("193.0.0.0/21", 21, 2121),
vrp("2001:67c:2e8::/48", 48, 3333),
]);
let validate = |p: &str, origin| table.validate(&prefix(p), Some(origin));
// Covered, by the right AS, no longer than allowed.
assert_eq!(validate("193.0.0.0/21", 3333), Validity::Valid);
assert_eq!(validate("193.0.4.0/24", 3333), Validity::Valid);
assert_eq!(validate("193.0.0.0/21", 2121), Validity::Valid);
assert_eq!(validate("2001:67c:2e8::/48", 3333), Validity::Valid);
// The right AS, but more specific than it may announce.
assert_eq!(validate("193.0.4.0/24", 2121), Validity::Invalid);
assert_eq!(validate("193.0.4.0/25", 3333), Validity::Invalid);
// Someone else's AS: a hijack, or a mistake.
assert_eq!(validate("193.0.0.0/21", 15169), Validity::Invalid);
assert_eq!(validate("2001:67c:2e8::/48", 15169), Validity::Invalid);
// Nothing covers these: a shorter prefix, another block, another
// family.
assert_eq!(validate("193.0.0.0/20", 3333), Validity::NotFound);
assert_eq!(validate("194.0.0.0/21", 3333), Validity::NotFound);
assert_eq!(validate("2001:67c:2e9::/48", 3333), Validity::NotFound);
// A route with no one origin matches no VRP.
assert_eq!(
table.validate(&prefix("193.0.0.0/21"), None),
Validity::Invalid
);
assert_eq!(
table.validate(&prefix("194.0.0.0/21"), None),
Validity::NotFound
);
assert_eq!(table.len(), 3);
assert!(table.covers(&prefix("193.0.7.0/24")));
assert!(!table.covers(&prefix("193.0.8.0/24")));
}
/// A VRP for AS 0 says the prefix is not to be routed at all.
#[test]
fn as_zero_allows_nobody() {
let table = VrpTable::new([vrp("203.0.113.0/24", 24, 0)]);
assert_eq!(
table.validate(&prefix("203.0.113.0/24"), Some(0)),
Validity::Invalid
);
assert_eq!(
table.validate(&prefix("203.0.113.0/24"), Some(64500)),
Validity::Invalid
);
}
/// A VRP covers a route however many other VRPs cover it too: one that
/// allows it is enough.
#[test]
fn any_covering_vrp_can_make_a_route_valid() {
let table = VrpTable::new([
vrp("10.0.0.0/8", 8, 1),
vrp("10.1.0.0/16", 24, 2),
vrp("0.0.0.0/0", 0, 3),
]);
assert_eq!(
table.validate(&prefix("10.1.2.0/24"), Some(2)),
Validity::Valid
);
assert_eq!(
table.validate(&prefix("10.1.2.0/24"), Some(1)),
Validity::Invalid
);
assert_eq!(
table.validate(&prefix("10.0.0.0/8"), Some(1)),
Validity::Valid
);
assert_eq!(
table.validate(&prefix("0.0.0.0/0"), Some(3)),
Validity::Valid
);
// The VRP for the default route covers everything.
assert_eq!(
table.validate(&prefix("192.0.2.0/24"), Some(9)),
Validity::Invalid
);
}
#[test]
fn an_empty_table_knows_nothing() {
let table = VrpTable::default();
assert!(table.is_empty());
assert_eq!(
table.validate(&prefix("193.0.0.0/21"), Some(3333)),
Validity::NotFound
);
assert_eq!(table.validate(&prefix("::/0"), None), Validity::NotFound);
}
fn any_vrp() -> impl Strategy<Value = Vrp> {
// Few distinct addresses and AS numbers, so that routes and VRPs
// meet.
(any::<bool>(), 0u8..4, 0u8..=24, 0u8..=8, 0u32..4).prop_map(
|(v6, address, length, extra, asn)| {
let version = if v6 { IpVersion::Ipv6 } else { IpVersion::Ipv4 };
let bytes = [
address << 6 | address,
address,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
];
Vrp {
prefix: IpPrefix::new(version, bytes, length).unwrap(),
max_length: length + extra,
asn,
}
},
)
}
proptest! {
/// The table agrees with reading RFC 6811 out over the list.
#[test]
fn the_table_agrees_with_looking_through_the_list(
vrps in proptest::collection::vec(any_vrp(), 0..30),
routes in proptest::collection::vec((any_vrp(), any::<bool>()), 1..40),
) {
let table = VrpTable::new(vrps.iter().copied());
for (route, has_origin) in routes {
let origin = has_origin.then_some(route.asn);
let covering: Vec<&Vrp> = vrps
.iter()
.filter(|vrp| vrp.prefix.contains(&route.prefix))
.collect();
let expected = if covering.is_empty() {
Validity::NotFound
} else if covering.iter().any(|vrp| {
origin.is_some_and(|origin| origin != 0 && origin == vrp.asn)
&& route.prefix.length() <= vrp.max_length
}) {
Validity::Valid
} else {
Validity::Invalid
};
prop_assert_eq!(table.validate(&route.prefix, origin), expected, "{:?}", route);
prop_assert_eq!(table.covers(&route.prefix), !covering.is_empty());
}
}
}
}
crates/rpki/tests/client.rs
-737
View File
@@ -1,737 +0,0 @@
//! The RTR client against a cache played by the test: what it asks for,
//! what it makes of the answers, and what it does when the cache misbehaves
//! or goes away.
use std::net::{Ipv4Addr, SocketAddr};
use std::time::Duration;
use bgp_primitives::ip_prefix::IpPrefix;
use bgp_rpki::rtr::{Pdu, Timers, error_code};
use bgp_rpki::{Algorithm, CacheState, PathValidity, Rpki, RpkiConfig, Update, Validity, Vrp};
use bytes::BytesMut;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::{TcpListener, TcpStream};
use tokio::sync::watch;
const TIMEOUT: Duration = Duration::from_secs(5);
const SESSION: u16 = 0x1234;
/// The version of the protocol the client asks for first.
const V: u8 = 2;
fn prefix(s: &str) -> IpPrefix {
IpPrefix::try_from(s).unwrap()
}
fn vrp(p: &str, max_length: u8, asn: u32) -> Vrp {
Vrp {
prefix: prefix(p),
max_length,
asn,
}
}
fn announce(vrp: Vrp) -> Pdu {
Pdu::Prefix {
announce: true,
vrp,
}
}
fn withdraw(vrp: Vrp) -> Pdu {
Pdu::Prefix {
announce: false,
vrp,
}
}
fn end_of_data(serial: u32) -> Pdu {
Pdu::EndOfData {
session_id: SESSION,
serial,
timers: Some(Timers::default()),
}
}
/// A cache played by the test.
struct Cache {
listener: TcpListener,
}
/// One connection from the client to the cache.
struct Connection {
stream: TcpStream,
buf: BytesMut,
}
impl Cache {
async fn start() -> Self {
Self {
listener: TcpListener::bind((Ipv4Addr::LOCALHOST, 0)).await.unwrap(),
}
}
fn address(&self) -> SocketAddr {
self.listener.local_addr().unwrap()
}
async fn accept(&self) -> Connection {
let (stream, _) = tokio::time::timeout(TIMEOUT, self.listener.accept())
.await
.expect("the client connects")
.unwrap();
Connection {
stream,
buf: BytesMut::new(),
}
}
}
impl Connection {
/// The next PDU from the client and the version it was sent in.
async fn received(&mut self) -> (u8, Pdu) {
let reading = async {
loop {
if let Some((version, pdu, used)) = Pdu::decode(&self.buf).unwrap() {
let _ = self.buf.split_to(used);
return (version, pdu);
}
assert_ne!(
self.stream.read_buf(&mut self.buf).await.unwrap(),
0,
"the client closed the connection"
);
}
};
tokio::time::timeout(TIMEOUT, reading)
.await
.expect("the client sends something")
}
async fn send(&mut self, version: u8, pdus: &[Pdu]) {
let mut out = BytesMut::new();
for pdu in pdus {
pdu.encode(version, &mut out);
}
self.stream.write_all(&out).await.unwrap();
}
/// Answers a query with everything: `vrps`, at `serial`.
async fn send_all(&mut self, vrps: &[Vrp], serial: u32) {
let mut pdus = vec![Pdu::CacheResponse {
session_id: SESSION,
}];
pdus.extend(vrps.iter().copied().map(announce));
pdus.push(end_of_data(serial));
self.send(V, &pdus).await;
}
}
fn client_of(caches: &[&Cache]) -> (Rpki, watch::Receiver<Update>) {
let (rpki, _task) = Rpki::spawn(RpkiConfig {
caches: caches.iter().map(|cache| cache.address()).collect(),
retry: Some(Duration::from_millis(50)),
..RpkiConfig::default()
});
let updates = rpki.subscribe();
(rpki, updates)
}
async fn next_update(updates: &mut watch::Receiver<Update>) -> Update {
tokio::time::timeout(TIMEOUT, updates.changed())
.await
.expect("the VRPs change")
.unwrap();
updates.borrow_and_update().clone()
}
/// The ordinary life of a session: everything once, then what changes.
#[tokio::test]
async fn the_client_takes_everything_once_and_then_the_changes() {
let cache = Cache::start().await;
let (rpki, mut updates) = client_of(&[&cache]);
// Nothing is known until a cache has answered.
assert_eq!(rpki.version(), 0);
assert!(rpki.current().vrps.is_empty());
let mut connection = cache.accept().await;
assert_eq!(connection.received().await, (V, Pdu::ResetQuery));
let ripe = vrp("193.0.0.0/21", 24, 3333);
let other = vrp("2001:67c:2e8::/48", 48, 3333);
connection.send_all(&[ripe, other], 7).await;
let update = next_update(&mut updates).await;
assert_eq!(update.version, 1);
assert_eq!(rpki.version(), 1);
assert!(
update.changed.is_none(),
"the first data can change anything"
);
assert_eq!(update.vrps.len(), 2);
assert_eq!(
update.vrps.validate(&prefix("193.0.4.0/24"), Some(3333)),
Validity::Valid
);
assert_eq!(
update.vrps.validate(&prefix("193.0.4.0/24"), Some(15169)),
Validity::Invalid
);
let status = rpki.status();
assert_eq!(status.state, CacheState::Synchronised);
assert_eq!((status.session_id, status.serial), (Some(SESSION), Some(7)));
assert_eq!((status.vrps_ipv4, status.vrps_ipv6), (1, 1));
assert_eq!(status.cache, Some(cache.address()));
// The cache has news: the client asks for what changed since serial 7.
connection
.send(
V,
&[Pdu::SerialNotify {
session_id: SESSION,
serial: 8,
}],
)
.await;
assert_eq!(
connection.received().await,
(
V,
Pdu::SerialQuery {
session_id: SESSION,
serial: 7
}
)
);
let new = vrp("8.8.8.0/24", 24, 15169);
connection
.send(
V,
&[
Pdu::CacheResponse {
session_id: SESSION,
},
announce(new),
withdraw(other),
end_of_data(8),
],
)
.await;
let update = next_update(&mut updates).await;
assert_eq!(update.version, 2);
assert_eq!(update.vrps.len(), 2);
assert_eq!(
update.vrps.validate(&prefix("8.8.8.0/24"), Some(15169)),
Validity::Valid
);
assert_eq!(
update
.vrps
.validate(&prefix("2001:67c:2e8::/48"), Some(3333)),
Validity::NotFound,
"withdrawn"
);
// Only routes that what came and went covers can have changed.
let changed = update.changed.expect("a change to what there was");
assert!(changed.vrps.covers(&prefix("8.8.8.0/24")));
assert!(changed.vrps.covers(&prefix("2001:67c:2e8::/48")));
assert!(!changed.vrps.covers(&prefix("193.0.0.0/21")));
assert_eq!(rpki.status().serial, Some(8));
}
/// A cache that cannot say what changed tells the client to start over.
/// What the client then publishes is the difference to what it had.
#[tokio::test]
async fn a_cache_reset_makes_the_client_ask_for_everything() {
let cache = Cache::start().await;
let (rpki, mut updates) = client_of(&[&cache]);
let mut connection = cache.accept().await;
connection.received().await;
let kept = vrp("193.0.0.0/21", 24, 3333);
let dropped = vrp("8.8.8.0/24", 24, 15169);
connection.send_all(&[kept, dropped], 7).await;
next_update(&mut updates).await;
connection
.send(
V,
&[Pdu::SerialNotify {
session_id: SESSION,
serial: 20,
}],
)
.await;
assert!(matches!(
connection.received().await.1,
Pdu::SerialQuery { serial: 7, .. }
));
connection.send(V, &[Pdu::CacheReset]).await;
assert_eq!(connection.received().await.1, Pdu::ResetQuery);
let added = vrp("1.1.1.0/24", 24, 13335);
connection.send_all(&[kept, added], 20).await;
let update = next_update(&mut updates).await;
assert_eq!(update.vrps.len(), 2);
let changed = update.changed.expect("known what changed");
assert!(changed.vrps.covers(&prefix("8.8.8.0/24")));
assert!(changed.vrps.covers(&prefix("1.1.1.0/24")));
assert!(!changed.vrps.covers(&prefix("193.0.0.0/21")), "it stayed");
assert_eq!(rpki.status().serial, Some(20));
}
/// An answer takes effect whole or not at all: one that contradicts itself
/// changes nothing, and the session is given up.
#[tokio::test]
async fn an_answer_that_contradicts_itself_changes_nothing() {
let cache = Cache::start().await;
let (rpki, mut updates) = client_of(&[&cache]);
let mut connection = cache.accept().await;
connection.received().await;
let held = vrp("193.0.0.0/21", 24, 3333);
connection.send_all(&[held], 7).await;
next_update(&mut updates).await;
connection
.send(
V,
&[Pdu::SerialNotify {
session_id: SESSION,
serial: 8,
}],
)
.await;
connection.received().await;
// A good change, then the withdrawal of something never announced.
connection
.send(
V,
&[
Pdu::CacheResponse {
session_id: SESSION,
},
announce(vrp("1.1.1.0/24", 24, 13335)),
withdraw(vrp("9.9.9.0/24", 24, 19281)),
end_of_data(8),
],
)
.await;
// The client says what was wrong,
assert!(matches!(
connection.received().await,
(
V,
Pdu::ErrorReport {
code: error_code::WITHDRAWAL_OF_UNKNOWN_RECORD,
..
}
)
));
// gives the session up and comes back; what it has is what it had,
// and it asks from where it was.
let mut again = cache.accept().await;
assert_eq!(
again.received().await.1,
Pdu::SerialQuery {
session_id: SESSION,
serial: 7
}
);
let current = rpki.current();
assert_eq!(current.version, 1);
assert_eq!(current.vrps.len(), 1);
assert!(
rpki.status()
.last_error
.is_some_and(|e| e.contains("withdrew a VRP it had not announced"))
);
}
fn aspa(customer: u32, providers: &[u32]) -> Pdu {
Pdu::Aspa {
announce: true,
customer,
providers: providers.to_vec(),
}
}
/// ASPAs come with the VRPs, in version 2. One that is announced again
/// takes the place of the one before.
#[tokio::test]
async fn aspas_are_taken_replaced_and_withdrawn() {
let cache = Cache::start().await;
let (rpki, mut updates) = client_of(&[&cache]);
let mut connection = cache.accept().await;
assert_eq!(connection.received().await, (V, Pdu::ResetQuery));
connection
.send(
V,
&[
Pdu::CacheResponse {
session_id: SESSION,
},
announce(vrp("193.0.0.0/21", 24, 3333)),
aspa(3333, &[1101, 3257]),
aspa(1101, &[0]),
end_of_data(7),
],
)
.await;
let update = next_update(&mut updates).await;
assert_eq!(update.vrps.len(), 1);
assert_eq!(update.aspas.len(), 2);
assert_eq!(update.aspas.providers(3333), Some(&[1101, 3257][..]));
assert_eq!(rpki.status().aspas, 2);
assert_eq!(rpki.status().protocol_version, 2);
assert_eq!(
update.aspas.verify(&[174, 3333], Algorithm::Upstream),
PathValidity::Invalid
);
connection
.send(
V,
&[Pdu::SerialNotify {
session_id: SESSION,
serial: 8,
}],
)
.await;
connection.received().await;
connection
.send(
V,
&[
Pdu::CacheResponse {
session_id: SESSION,
},
// Another list for 3333, a new ASPA, and one gone.
aspa(3333, &[174, 1101]),
aspa(2121, &[3333]),
Pdu::Aspa {
announce: false,
customer: 1101,
providers: vec![],
},
end_of_data(8),
],
)
.await;
let update = next_update(&mut updates).await;
assert_eq!(update.aspas.len(), 2);
assert_eq!(update.aspas.providers(3333), Some(&[174, 1101][..]));
assert_eq!(update.aspas.providers(1101), None);
assert_eq!(
update.aspas.verify(&[174, 3333], Algorithm::Upstream),
PathValidity::Valid
);
let changed = update.changed.expect("a change to what there was");
assert_eq!(
*changed.aspa_customers,
[3333, 2121, 1101].into_iter().collect()
);
assert!(changed.vrps.is_empty(), "no VRP came or went");
assert_eq!(update.vrps.len(), 1);
// The withdrawal of an ASPA there is not is a contradiction: nothing
// of the answer is used.
connection
.send(
V,
&[Pdu::SerialNotify {
session_id: SESSION,
serial: 9,
}],
)
.await;
connection.received().await;
connection
.send(
V,
&[
Pdu::CacheResponse {
session_id: SESSION,
},
aspa(3333, &[6939]),
Pdu::Aspa {
announce: false,
customer: 1101,
providers: vec![],
},
end_of_data(9),
],
)
.await;
assert!(matches!(
connection.received().await.1,
Pdu::ErrorReport {
code: error_code::WITHDRAWAL_OF_UNKNOWN_RECORD,
..
}
));
let mut again = cache.accept().await;
assert!(matches!(
again.received().await.1,
Pdu::SerialQuery { serial: 8, .. }
));
let current = rpki.current();
assert_eq!(current.version, 2);
assert_eq!(current.aspas.providers(3333), Some(&[174, 1101][..]));
}
/// A cache that speaks version 1 is spoken to in it. It has no ASPAs to
/// give: those that came from a newer cache go when its data takes their
/// place.
#[tokio::test]
async fn a_cache_that_speaks_version_one_has_no_aspas() {
let cache = Cache::start().await;
let (rpki, mut updates) = client_of(&[&cache]);
let mut connection = cache.accept().await;
connection.received().await;
let ripe = vrp("193.0.0.0/21", 24, 3333);
connection
.send(
V,
&[
Pdu::CacheResponse {
session_id: SESSION,
},
announce(ripe),
aspa(3333, &[1101]),
end_of_data(7),
],
)
.await;
assert_eq!(next_update(&mut updates).await.aspas.len(), 1);
// The validator is replaced by an older one at the same address.
drop(connection);
let mut connection = cache.accept().await;
assert!(matches!(
connection.received().await,
(V, Pdu::SerialQuery { serial: 7, .. })
));
connection
.send(
1,
&[Pdu::ErrorReport {
code: error_code::UNSUPPORTED_VERSION,
text: "version 1 at most".into(),
}],
)
.await;
// Asked again in version 1, and for everything: the serial was of
// another version's data.
let mut connection = cache.accept().await;
assert_eq!(connection.received().await, (1, Pdu::ResetQuery));
connection
.send(
1,
&[
Pdu::CacheResponse {
session_id: SESSION,
},
announce(ripe),
end_of_data(3),
],
)
.await;
let update = next_update(&mut updates).await;
assert!(update.aspas.is_empty());
assert_eq!(update.vrps.len(), 1);
let changed = update.changed.expect("the difference to what there was");
assert_eq!(*changed.aspa_customers, [3333].into_iter().collect());
assert!(changed.vrps.is_empty());
let status = rpki.status();
assert_eq!((status.protocol_version, status.aspas), (1, 0));
// An ASPA in a version that has none is not RTR.
connection
.stream
.write_all(&[1, 11, 1, 0, 0, 0, 0, 16, 0, 0, 0, 1, 0, 0, 0, 2])
.await
.unwrap();
assert!(matches!(
connection.received().await,
(
1,
Pdu::ErrorReport {
code: error_code::UNSUPPORTED_PDU_TYPE,
..
}
)
));
}
/// A validator that only speaks version 0 says so, by an error or by
/// answering in version 0. The client asks again in version 0.
#[tokio::test]
async fn a_cache_that_speaks_version_zero_is_spoken_to_in_it() {
for refuses_with_an_error in [true, false] {
let cache = Cache::start().await;
let (rpki, mut updates) = client_of(&[&cache]);
let mut connection = cache.accept().await;
assert_eq!(connection.received().await, (V, Pdu::ResetQuery));
if refuses_with_an_error {
connection
.send(
0,
&[Pdu::ErrorReport {
code: error_code::UNSUPPORTED_VERSION,
text: "version 0 only".into(),
}],
)
.await;
} else {
connection
.send(
0,
&[Pdu::CacheResponse {
session_id: SESSION,
}],
)
.await;
}
let mut connection = cache.accept().await;
assert_eq!(connection.received().await, (0, Pdu::ResetQuery));
connection
.send(
0,
&[
Pdu::CacheResponse {
session_id: SESSION,
},
announce(vrp("193.0.0.0/21", 24, 3333)),
Pdu::EndOfData {
session_id: SESSION,
serial: 3,
timers: None,
},
],
)
.await;
assert_eq!(next_update(&mut updates).await.vrps.len(), 1);
assert_eq!(rpki.status().protocol_version, 0);
}
}
/// When the cache goes away the data is kept and used while the client
/// tries again, and the next cache is tried. A cache the data did not come
/// from is asked for everything.
#[tokio::test]
async fn a_lost_cache_is_replaced_by_the_next_and_the_data_is_kept() {
let first = Cache::start().await;
let second = Cache::start().await;
let (rpki, mut updates) = client_of(&[&first, &second]);
let mut connection = first.accept().await;
connection.received().await;
connection
.send_all(&[vrp("193.0.0.0/21", 24, 3333)], 7)
.await;
next_update(&mut updates).await;
drop(connection);
drop(first);
let mut connection = second.accept().await;
assert_eq!(connection.received().await.1, Pdu::ResetQuery);
// Still in use meanwhile.
assert_eq!(rpki.current().vrps.len(), 1);
connection
.send_all(
&[vrp("193.0.0.0/21", 24, 3333), vrp("1.1.1.0/24", 24, 13335)],
100,
)
.await;
let update = next_update(&mut updates).await;
assert_eq!(update.vrps.len(), 2);
let changed = update.changed.expect("the difference to what there was");
assert!(changed.vrps.covers(&prefix("1.1.1.0/24")));
assert!(!changed.vrps.covers(&prefix("193.0.0.0/21")));
assert_eq!(rpki.status().cache, Some(second.address()));
}
/// Data that no cache has confirmed for the expire interval is dropped:
/// routes are then neither valid nor invalid.
#[tokio::test]
async fn data_that_is_not_confirmed_in_time_is_dropped() {
let cache = Cache::start().await;
let (rpki, _task) = Rpki::spawn(RpkiConfig {
caches: vec![cache.address()],
// Data never expires sooner than a refresh and a retry take.
refresh: Some(Duration::from_millis(200)),
retry: Some(Duration::from_millis(50)),
expire: Some(Duration::from_millis(300)),
});
let mut updates = rpki.subscribe();
let mut connection = cache.accept().await;
connection.received().await;
connection
.send_all(&[vrp("193.0.0.0/21", 24, 3333)], 7)
.await;
assert_eq!(next_update(&mut updates).await.vrps.len(), 1);
// The cache goes and does not come back.
drop(connection);
drop(cache);
let update = next_update(&mut updates).await;
assert!(update.vrps.is_empty());
assert!(update.changed.is_none(), "anything may have changed");
assert_eq!(
update.vrps.validate(&prefix("193.0.0.0/21"), Some(15169)),
Validity::NotFound
);
let status = rpki.status();
assert_eq!(status.state, CacheState::Connecting);
assert_eq!(status.serial, None);
}
/// A cache that reports an error, or sends what is not RTR, is given up
/// on and tried again; nothing of what it sent is used.
#[tokio::test]
async fn a_cache_that_misbehaves_is_given_up_on() {
let cache = Cache::start().await;
let (rpki, _updates) = client_of(&[&cache]);
let mut connection = cache.accept().await;
connection.received().await;
connection
.send(
V,
&[Pdu::ErrorReport {
code: error_code::NO_DATA_AVAILABLE,
text: "still validating".into(),
}],
)
.await;
let mut connection = cache.accept().await;
assert_eq!(connection.received().await.1, Pdu::ResetQuery);
assert!(
rpki.status()
.last_error
.is_some_and(|e| e.contains("still validating"))
);
// Half an answer, then noise.
connection
.send(
V,
&[
Pdu::CacheResponse {
session_id: SESSION,
},
announce(vrp("193.0.0.0/21", 24, 3333)),
],
)
.await;
connection
.stream
.write_all(&[1, 99, 0, 0, 0, 0, 0, 8])
.await
.unwrap();
let mut connection = cache.accept().await;
assert_eq!(connection.received().await.1, Pdu::ResetQuery);
assert!(rpki.current().vrps.is_empty());
assert_eq!(rpki.version(), 0);
}
crates/server/Cargo.toml
+2 -4
View File
@@ -8,21 +8,19 @@ repository.workspace = true
version.workspace = true
[dependencies]
foldhash.workspace = true
bgp-packet.workspace = true
bgp-primitives.workspace = true
bgp-rib.workspace = true
bgp-fib.workspace = true
bgp-rpki.workspace = true
bytes.workspace = true
clap.workspace = true
futures.workspace = true
prometheus.workspace = true
socket2.workspace = true
tokio.workspace = true
tracing-subscriber.workspace = true
tracing.workspace = true
protobuf.workspace = true
thiserror.workspace = true
nix.workspace = true
[build-dependencies]
protobuf-codegen.workspace = true
crates/server/proto/config.proto
+5 -527
View File
@@ -37,237 +37,10 @@ message BgpConfig {
// restarted take over from a session we have not yet noticed is dead,
// instead of being refused until the hold timer expires.
bool collision_detect_established_state = 8;
// Connections are accepted only from the addresses of configured peers;
// from anywhere else they are closed before a byte is read. When true,
// any host that can reach a listening endpoint may open a session, as
// any AS it cares to name, and have its routes taken. That is for
// testing on a closed network and nothing else.
bool accept_unconfigured_peers = 9;
// Route origin validation (RFC 6811) and AS path verification with the
// RPKI. Left out, routes are not validated.
Rpki rpki = 10;
// Put the best route of every prefix in the kernel's routing table, so
// that the machine forwards packets by them. Left out, the daemon only
// speaks BGP and the machine's forwarding is not touched.
Kernel kernel = 11;
// What is announced, and how, to every peer that has no `export` of its
// own.
ExportPolicy export = 12;
// Let routes say, with communities, which peers they are passed on to
// and how far away we look to them, as the route servers of Internet
// exchanges and transit networks do. With L our own AS number, P a
// peer's, and G a group of sessions (`groups` of a peer):
//
// 0:P or L:0:P not to the peer with AS P
// L:P or L:1:P to the peer with AS P, and then only to peers
// named so
// 0:L or L:0:0 to nobody
// L:11:P with our AS once more on the path to the peer
// L:12:P with AS P; twice, three times. P 0 is every
// L:13:P peer in another AS.
// L:20:G not to the sessions of group G
// L:21:G to the sessions of group G, and then only to
// sessions named so, by group or AS
// L:31:G with our AS once more on the path to the
// L:32:G sessions of group G; twice, three times
// L:33:G
//
// A route with none of these goes to everyone it otherwise would. "Not"
// wins over "to". Of several prepends that apply, the most counts, on
// top of the session's export.as_path_prepend. The two-part form is a
// standard community, with room for AS numbers below 65536; the
// three-part forms are large communities and work for any.
//
// They are acted on where a route leaves our AS, and taken off there:
// to a peer in our own AS a route goes with them, for that router to act
// on. They are honoured on routes we originate too (`communities` and
// `large_communities` of `originate`), on what import.add_communities
// puts on, and on what a peer in our own AS, or one with
// import.accept_control_communities, sends. From any other peer the
// communities of our own AS are taken off as its routes come in, so that
// nobody outside can say what is done with our routes.
//
// With this on, standard communities L:n are taken for "to the peer
// with AS n": mark routes with something else (large communities
// L:100:n and up, say; functions 2 to 9 are left for marks too).
bool control_communities = 13;
// Graceful restart, for every peer that has no `graceful_restart` of its
// own. Left out, it is on, as the defaults below have it.
GracefulRestart graceful_restart = 14;
// Where this router passes on routes from one router of our AS to
// another (Peer.route_reflector_client), the cluster it does so for, as
// a dotted-quad: what it puts on such routes, and what it does not take
// back (RFC 4456). Give routers that serve the same clients the same
// cluster ID. Defaults to router_id.
string cluster_id = 15;
// Limits on what goes to other networks that are not our customers: to
// every peer whose role is PROVIDER, PEER or ROUTE_SERVER, and to one
// in another AS that has no role. Not to customers, route server
// clients or routers of our own AS. A peer's own `export_limits` takes
// the place of these.
//
// They hold whatever the routes are and wherever they came from, and
// whatever `export` says: a check on everything else, in case it goes
// wrong. With the roles, a route learned from a provider or a peer is
// passed on to customers only; these keep it from going further even
// if a route is not marked as it should be.
ExportLimits export_limits = 16;
// Names for communities, for the status page to show in place of the
// numbers: what the marks on routes mean to a reader. A name for one
// community ("64601:100:2" is "comrack"), or for a kind of them, with
// `*` for the part left open ("64601:102:*" is "interconnection"), which
// is shown with the open part after the name. The daemon itself does
// nothing with them.
repeated CommunityName community_names = 17;
}
// A name for a community, or for a kind of them.
message CommunityName {
// The community, standard ("64601:100") or large ("64601:100:2"), or a
// pattern of them with `*` for a part left open ("64601:102:*").
string community = 1;
// What it is shown as.
string name = 2;
}
// Limits on what is announced to a peer, which hold whatever `export`
// and the routes say.
message ExportLimits {
// Only routes for these prefixes are announced: our own address space
// and our customers', say, with max_length as long as the longest
// prefix announced from it. Routes for other prefixes, learned or
// originated, are not. Empty allows any.
repeated PrefixRange prefixes = 1;
// The most routes that are announced to the peer at a time. If more
// were to be, none of them is: the session is ended with a Cease /
// Administrative Shutdown NOTIFICATION that says why (RFC 9003), and the
// peer is kept down for its import.hold_down_seconds, as when it sends
// us too many. Set it somewhat above what is expected. Zero is no
// limit.
uint32 max_prefixes = 2;
}
// Graceful restart (RFC 4724): BGP restarting without the routes going.
//
// When the daemon is restarted in good order for an upgrade (on SIGUSR1),
// it leaves its routes in the kernel, and its peers keep the routes it
// announced to them, for as long as they were told a restart takes. When a
// peer restarts, its routes are kept and used as before, marked stale,
// until it has sent them again; those it does not send again are removed.
// Either way, nothing is withdrawn across the network for a restart.
//
// It is in use with a peer that offers it too. A session that ends with a
// Hard Reset (RFC 8538), or because of an error, has its routes removed at
// once, as without graceful restart.
message GracefulRestart {
// True does not offer graceful restart: a peer's routes are removed as
// soon as its session ends, and it removes ours as soon as it sees ours
// end.
optional bool disabled = 1;
// How long, in seconds, peers are told a restart of this daemon takes:
// how long they keep our routes after our session with them ends. At
// most 4095. Defaults to 120.
optional uint32 restart_time_seconds = 2;
// The longest time, in seconds, a restarting peer's routes are kept
// stale: while it is gone, whatever restart time it gave, and again once
// it is back, until it has sent all its routes. Zero keeps none.
// Defaults to 120.
optional uint32 stale_routes_seconds = 3;
}
// Installing routes in the kernel. Linux only; the daemon needs the
// CAP_NET_ADMIN capability.
//
// The best route of a prefix is installed if its next hop is a neighbor:
// an address in the network of one of the machine's interfaces. Routes
// are marked as BGP's ("proto bgp"), and every route with that mark, the
// table and the metric below is taken to be this daemon's: what it finds
// at startup and no longer wants, it removes. When the daemon stops in
// good order it removes its routes.
message Kernel {
// The routing table. Defaults to 254, the main table, which is the one
// packets are forwarded by unless rules say otherwise.
uint32 table = 1;
// The metric of the routes. Defaults to 20. Of several routes for a
// prefix the kernel uses the one with the lowest metric.
optional uint32 metric = 2;
// By default, where the kernel has nexthop objects (Linux 5.3 and
// later), a gateway is put there once as such an object and routes name
// it, which makes installing a full table about a third faster. `ip
// route` then shows routes with "nhid". True has every route carry its
// own gateway, as on older kernels.
bool without_nexthop_objects = 3;
// How long, after the daemon starts, routes of its own that an earlier
// run left in the kernel are kept though no peer has announced them
// (again). Defaults to 120. The daemon's table is empty until its peers
// have sent theirs; removing the old routes at once would stop the
// machine forwarding for as long as that takes. After this time, what
// was not announced again is removed.
// Zero removes them at once.
optional uint32 stale_seconds = 4;
// The kernel's table whose routes say how a next hop that is not a
// neighbour is reached: its connected and static routes, and an IGP's
// if there is one. Defaults to 254, the main table. A route whose next
// hop is covered by such a route is installed with that route's gateway
// and interface, the kernel not looking gateways up in turn. Routes put
// in the table by BGP, this daemon's or another's, never count: a next
// hop reached only by way of a BGP route rests on what may itself rest
// on a next hop.
uint32 resolution_table = 5;
// Whether the default route in that table counts as the way to a next
// hop. Off by default: on a router the default route points at a
// transit provider, and a next hop "reached" only by sending the
// packets to the Internet at large is not reached.
bool resolve_via_default_route = 6;
}
// Where to get the validated ROA payloads that say which AS may originate
// which prefixes. With this set, a route whose origin is not authorised
// for its prefix ("invalid") is refused, from every peer that is not in
// our own AS and does not have accept_rpki_invalid set. Routes that
// nothing is known about are taken as before.
//
// A validator that speaks version 2 of the protocol also gives the ASPAs,
// in which networks say who their providers are. With those, a route
// whose AS path cannot have come about by networks passing routes on as
// they should (a route leak, or a made-up path) is refused, from every
// peer that has a role set and does not have accept_aspa_invalid set.
message Rpki {
// The RPKI-to-Router servers (RFC 8210, draft-ietf-sidrops-8210bis) of
// one or more validators, such as Routinator, rpki-client with StayRTR,
// or FORT, in order of preference. The port is required. One is used at
// a time, over plain TCP: keep it on a network you trust.
repeated TcpEndpoint caches = 1;
// How often to ask the cache for changes unprompted, how long to wait
// before trying again after a failure, and how long the data may be used
// without a cache confirming it, after which nothing is invalid any
// more. Each defaults to what the cache says, or to 3600, 600 and 7200.
uint32 refresh_seconds = 2;
uint32 retry_seconds = 3;
uint32 expire_seconds = 4;
}
// A route originated by this router. It is announced with an AS_PATH of the
// local AS (empty, to a peer in our own AS) and, on each session, that
// session's local address as next hop.
// local AS and, on each session, that session's local address as next hop.
message OriginatedRoute {
// The prefix in CIDR notation, e.g. "192.0.2.0/24" or "2001:db8::/32".
string prefix = 1;
@@ -291,8 +64,7 @@ message OriginatedRoute {
// An IP address and port pair.
message TcpEndpoint {
// IP address (V4 or V6). A peer at a link-local IPv6 address has the
// interface it is on after it: "fe80::1%eth0".
// IP address (V4 or V6).
string address = 1;
// TCP port. Defaults to 179 for BGP when unset.
uint32 port = 2;
@@ -310,9 +82,8 @@ message Peer {
uint32 remote_asn = 3;
// Hold time to advertise in the OPEN message, in seconds (RFC 4271 §4.2).
// At least 3, or 0 for a session without keepalives that is never timed
// out. Defaults to 90 when unset.
optional uint32 hold_time_seconds = 4;
// Must be 0 or >= 3. Defaults to 90 when unset.
uint32 hold_time_seconds = 4;
// Source IP address for connections we open to this peer. It is also the
// next hop of the routes we send it. Required when the peer identifies us
@@ -332,298 +103,6 @@ message Peer {
// direction and address family, only if the peer agrees. Defaults to
// off: one route for each prefix, as in plain BGP.
AddPath add_path = 7;
// Which of the peer's routes are taken. Left out, the general checks
// apply and there is no limit and no list.
ImportPolicy import = 8;
// What the peer is to us. Not for a peer in our own AS. With it set:
//
// - Our OPEN says what we are to the peer, and a peer that says it is
// something else than configured here is refused (RFC 9234). One side
// having the other down as a customer and the other as a peer is how
// leaks come about.
// - Routes are marked "only to customer" as they come in from a provider,
// a peer or a route server, and as they go out to a customer, a peer
// or a route server client. A route with that mark that comes back up
// from a customer, or across from a second peer, is a leak and is
// refused. This cannot be turned off.
// - Where rpki is configured, the AS paths of the peer's routes are
// verified against the ASPAs: a route from a customer, a peer or a
// route server must have come up through providers all the way; one
// from a provider may have come up and then down.
//
// Left out, none of this is done for the peer.
Role role = 9;
// Refuse the peer unless its OPEN says what it is. By default a peer
// that says nothing (one that does not implement RFC 9234) is taken at
// this configuration's word for it. Needs role.
bool require_role = 10;
// What is announced to the peer, and how. Left out, the top-level
// `export` holds, and without that the routes we originate are
// announced as they are.
ExportPolicy export = 11;
// Graceful restart with this peer. What is set here takes the place of
// what the top-level `graceful_restart` says.
GracefulRestart graceful_restart = 12;
// The groups the session is in, by number, for control communities to
// name sessions by (see control_communities): those at one Internet
// exchange, say, or those over one link, whatever AS the peers are. Use
// the same numbers on all routers of the AS: a route marked on one is
// acted on by another. From 1.
repeated uint32 groups = 13;
// Pass on to this peer, a router of our own AS, the routes learned from
// our other routers, and pass on the routes it sends to all of them: we
// are its route reflector (RFC 4456). Without it, as BGP has it, a route
// learned from one router of our AS goes to no other, and every router
// needs a session with every other. Routes go with this router as next
// hop, so that the peer can reach what lies behind another router
// through this one. Only for a peer in our own AS: other networks are
// told of learned routes by `export` and the roles alone. The peer's
// `export` still has to pass on learned routes.
bool route_reflector_client = 14;
// Limits on what is announced to the peer, in the place of the
// top-level export_limits. They apply to the same peers as those do: to
// one whose role is PROVIDER, PEER or ROUTE_SERVER, or that is in
// another AS and has no role.
ExportLimits export_limits = 15;
// Do not open connections to the peer: wait for it to connect. The
// session is configured in full, and comes up when the peer connects
// from `endpoint`'s address; the port of `endpoint` is not used. For a
// new peer whose side is not ready yet, or one that is to open the
// connections itself.
bool passive = 16;
// About how many routes the peer sends when all is well: a full table,
// a few hundred from an exchange peer, one from a customer. Nothing is
// decided by it. The status page compares the routes the peer has in
// the table with it, and shows an established session with fewer than
// half as needing attention: a peer that is up but has sent next to
// nothing looks, without this, exactly like one that is fine. Zero, the
// default, compares nothing. The routing table is also made with room
// for the largest of these from the start, so that it need not grow,
// and so use twice its memory for a moment, while the tables come in.
uint32 expected_routes = 17;
}
// What is announced to a peer.
//
// The routes we originate are always announced. Routes learned from other
// peers are passed on only if learned_routes says so, and then only as
// far as these let them:
//
// - Never back to the peer a route came from, nor to a peer whose AS is
// on its path already.
// - A route from a peer in our own AS is not passed to another such peer,
// unless one of them is a route reflector client of ours
// (Peer.route_reflector_client).
// - Where peers have roles (RFC 9234): a route learned from a provider, a
// peer or a route server goes on to customers only. So with roles on
// all peers, turning this on makes us transit for our customers and for
// nobody else. Without roles there is no such rule, and every route
// goes to every peer: do not turn this on for a peer without a role
// unless that is what is wanted.
// - The communities NO_EXPORT and NO_ADVERTISE, and control_communities.
//
// A route is passed on with our AS added to its path and ourselves as
// next hop, without the MED it came with. To a peer in our own AS the
// path is left as it is and LOCAL_PREF goes along. To a peer whose role
// is ROUTE_SERVER_CLIENT the route goes as a route server sends it
// (RFC 7947): path, MED and, where the client can use it, next hop
// unchanged.
message ExportPolicy {
// Pass on the best routes learned from other peers.
bool learned_routes = 1;
// Pass on every route learned from other peers, not only the best of
// each prefix, each with a path identifier (ADD-PATH, RFC 7911). For a
// route collector, which wants to see every path. Needs learned_routes,
// and the peer's add_path to offer sending; for a family the peer did
// not agree to receive several routes for, the best goes as usual. The
// export policy and the limits apply to every route alike.
bool all_paths = 6;
// Communities put on every route announced to the peer: which router
// it went through, say. "64601:100", or a well-known name.
repeated string add_communities = 2;
// Large communities put on, as "64601:1:2".
repeated string add_large_communities = 3;
// Communities taken off first: one ("64601:100"), all of an AS
// ("64601:*"), or large ones with three parts ("64601:1:*").
repeated string remove_communities = 4;
// How many more times our AS is put on the path of every route
// announced to the peer, to make them look further away. At most 16.
uint32 as_path_prepend = 5;
}
// What a peer is to us.
enum Role {
ROLE_UNSPECIFIED = 0;
// We buy transit from the peer.
PROVIDER = 1;
// The peer buys transit from us.
CUSTOMER = 2;
// The peer and we exchange our own and our customers' routes.
PEER = 3;
// The peer is an Internet exchange's route server.
ROUTE_SERVER = 4;
// We are a route server and the peer is one of our clients.
ROUTE_SERVER_CLIENT = 5;
}
// What is taken from a peer. Every route is judged before it goes into
// the routing table. One that is refused is dropped, and so is the route
// it would have replaced.
//
// A route whose AS path holds our own AS number is always refused.
message ImportPolicy {
// The most routes the peer may have at a time. One more ends the session
// with Cease / Maximum Number of Prefixes Reached, and the peer is kept
// down for hold_down_seconds. Set it somewhat above what the peer is
// expected to send. Zero is no limit.
uint32 max_prefixes = 1;
// How long the peer is kept down after going over max_prefixes. Neither
// side's connections are taken meanwhile. Defaults to 600.
uint32 hold_down_seconds = 2;
// By default, prefixes that are not routed on the Internet are refused
// (private, loopback, link-local, documentation, multicast and reserved
// space), and so are routes with a reserved, private or documentation AS
// number anywhere in their path. True takes them.
bool accept_bogons = 3;
// By default a default route (0.0.0.0/0 or ::/0) is refused. True takes
// one, as from an upstream that is asked to send it.
bool accept_default_route = 4;
// The longest prefixes taken. Defaults to 24 and 48, the longest that are
// routed between networks.
uint32 max_ipv4_prefix_length = 5;
uint32 max_ipv6_prefix_length = 6;
// By default the AS path of a route has to begin with the peer's own AS
// number. True takes any, as is needed for an Internet exchange's route
// server, which does not put its AS on the path.
bool accept_any_first_as = 7;
// The longest AS path taken, counted in AS numbers. Defaults to 64.
uint32 max_as_path_length = 8;
// The prefixes the peer may announce. Empty allows any. This is the
// list generated for a peer from the routing registries (with bgpq4,
// say).
repeated PrefixRange prefixes = 9;
// The AS numbers the peer's routes may originate from: the last AS of
// the path. Empty allows any.
repeated uint32 origin_asns = 10;
// By default, where route origin validation is configured (rpki), a
// route that is invalid is refused. True takes it all the same.
bool accept_rpki_invalid = 11;
// By default, where the validator gives ASPAs and the peer has a role,
// a route whose AS path they find invalid is refused. True takes it all
// the same.
bool accept_aspa_invalid = 12;
// By default a route whose AS path has an AS_SET in it is refused: sets
// are deprecated (RFC 9774), and with one nobody can say where a route
// comes from. True takes such routes.
bool accept_as_sets = 13;
// By default the next hop of a route from another network has to be the
// peer itself, so that a peer can only draw traffic to itself and not
// send it to a third party. True takes any next hop packets can be sent
// to. Not needed for a peer whose role is ROUTE_SERVER, whose routes
// have its clients as next hops.
bool accept_third_party_next_hop = 14;
// Communities put on every route taken from the peer: where it was
// learned, say, for other routers of ours and for `export` to go by.
// "64601:100", or a well-known name.
repeated string add_communities = 15;
// Large communities put on, as "64601:1:2".
repeated string add_large_communities = 16;
// Communities taken off first: one ("64601:100"), all of an AS
// ("64601:*"), or large ones with three parts ("64601:1:*"). Marks that
// mean something to us are not to be taken on a peer's word.
repeated string remove_communities = 17;
// The LOCAL_PREF of the peer's routes: of two routes for a prefix the
// one with the higher is chosen, whatever else they have. Left out, 100.
optional uint32 local_pref = 18;
// By default the communities of our own AS (L:n and L:n:m; and 0:n,
// with control_communities on) are taken off the routes of a peer in
// another AS as they come in, before add_communities puts ours on: what
// they ask of us is not for others to ask. True keeps them, for a
// customer that is to steer its own routes with control communities, or
// a route server's client. Not for a peer in our own AS, whose routes
// keep them in any case.
bool accept_control_communities = 19;
// What is done to some of the routes, beyond what is done to all of
// them above. Each route that is taken is looked at by the rules in
// turn, and the first that holds for it is applied; the rest are not
// looked at. A rule sees the route as the statements above leave it:
// with our communities taken off and ours put on.
repeated ImportRule rules = 20;
// By default a route with the BLACKHOLE community (65535:666, RFC
// 7999), which asks for the traffic to its prefix to be dropped, is
// refused ("blackhole request"). True takes such a route if it is for
// one address (a /32 or /128), whatever the longest prefix taken
// otherwise, and inside what the peer may announce; its next hop is
// not looked at. The traffic to the address is then dropped by this
// router (a blackhole route in the kernel), and the route goes on to
// the peers the export policy sends it to, community and all, for them
// to do the same. For a customer under attack, and for a route server
// whose blackhole routes are to be honoured.
bool accept_blackholes = 21;
}
// What is done to the routes with some communities, or for some prefixes:
// those learned at one exchange, say, by the mark another router of ours
// put on them, to be preferred less; or one prefix of a customer's, to be
// marked for a prepend.
message ImportRule {
// The rule holds for routes with one of these communities: one
// ("64601:100"), all of an AS ("64601:*"), or large ones with three
// parts ("64601:101:*"). Left out, for routes with any.
repeated string match_communities = 1;
// And for routes for a prefix in these. Left out, for any prefix.
repeated PrefixRange match_prefixes = 2;
// The LOCAL_PREF of the routes, in place of import.local_pref.
optional uint32 local_pref = 3;
// Communities put on the routes.
repeated string add_communities = 4;
repeated string add_large_communities = 5;
}
// A prefix, and optionally the more specific prefixes inside it.
message PrefixRange {
// In CIDR notation, e.g. "193.0.0.0/21".
string prefix = 1;
// The longest prefix inside it that is also allowed. Left out, only the
// prefix itself is.
uint32 max_length = 2;
}
// The directions ADD-PATH can be offered for.
@@ -631,8 +110,7 @@ enum AddPath {
ADD_PATH_OFF = 0;
// Accept several routes for a prefix from the peer.
ADD_PATH_RECEIVE = 1;
// Offer to send the peer several routes for a prefix. As yet only the
// best is sent all the same.
// Send the peer several routes for a prefix.
ADD_PATH_SEND = 2;
ADD_PATH_BOTH = 3;
}
crates/server/src/address.rs
-366
View File
@@ -1,366 +0,0 @@
//! Where a peer is.
//!
//! Most peers are told apart by their IP address alone. A link-local
//! address (`fe80::/10`) means something only on one link, and the same
//! one is often used on every link a router has, so such a peer is the
//! address together with the interface it is reached on: `fe80::1%eth0`.
//!
//! The kernel knows an interface by a number that changes when the
//! interface is made anew, as a tunnel is when it is restarted. So the
//! configuration keeps the name, and the number is looked up whenever it
//! is needed.
use std::fmt;
use std::io;
use std::net::{IpAddr, Ipv6Addr, SocketAddr, SocketAddrV6};
/// The longest name the kernel gives an interface.
const MAX_INTERFACE_NAME: usize = 15;
/// A peer's IP address, and for a link-local address the interface it is
/// on.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Ord)]
pub struct PeerAddress {
pub ip: IpAddr,
/// The number of the interface, for a link-local address; zero for
/// any other.
pub interface: u32,
}
impl From<SocketAddr> for PeerAddress {
fn from(address: SocketAddr) -> Self {
let interface = match address {
SocketAddr::V6(v6) if v6.ip().is_unicast_link_local() => v6.scope_id(),
_ => 0,
};
Self {
ip: address.ip(),
interface,
}
}
}
impl PeerAddress {
/// Whether `text` is this address as someone might type it: as it is
/// written out, or without the interface.
pub fn is_called(&self, text: &str) -> bool {
text.parse() == Ok(self.ip) || self.to_string() == text
}
}
impl From<IpAddr> for PeerAddress {
fn from(ip: IpAddr) -> Self {
Self { ip, interface: 0 }
}
}
/// Whether `wanted`, as someone typed it, names the peer at `address`
/// that is configured as `name`, if it is: by that name, or by the address
/// as [`PeerAddress::is_called`] reads it. The one rule for every place a
/// peer is asked for by name.
pub fn names_peer(wanted: &str, name: Option<&str>, address: PeerAddress) -> bool {
name == Some(wanted) || address.is_called(wanted)
}
/// The address, and the interface by name if there is one: `fe80::1%eth0`.
/// An interface that is no longer there goes by its number.
impl fmt::Display for PeerAddress {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "{}", self.ip)?;
match self.interface {
0 => Ok(()),
index => match interface_name(index) {
Some(name) => write!(f, "%{name}"),
None => write!(f, "%{index}"),
},
}
}
}
/// `address` as text, with an interface by its name: `[fe80::1%eth0]:179`.
pub fn socket_text(address: SocketAddr) -> String {
match PeerAddress::from(address) {
PeerAddress { interface: 0, ip } => SocketAddr::new(ip, address.port()).to_string(),
scoped => format!("[{scoped}]:{}", address.port()),
}
}
/// The number of the interface called `name`, if there is one now.
pub fn interface_index(name: &str) -> Option<u32> {
nix::net::if_::if_nametoindex(name).ok()
}
/// The name of the interface with this number, if there is one now.
pub fn interface_name(index: u32) -> Option<String> {
nix::net::if_::if_indextoname(index)
.ok()?
.into_string()
.ok()
}
/// A peer's address as the configuration has it.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Endpoint {
pub ip: IpAddr,
pub port: u16,
/// The interface a link-local address is on; `None` for any other
/// address.
pub interface: Option<String>,
}
/// What is wrong with a peer's address.
#[derive(Debug, Clone, PartialEq, Eq, thiserror::Error)]
pub enum EndpointProblem {
#[error("{0:?} is not an IP address")]
NotAnAddress(String),
#[error(
"{0} is a link-local address, which needs the interface it is on: {0}%eth0, for example"
)]
NoInterface(Ipv6Addr),
#[error("{0} is not a link-local address, and takes no interface")]
NotLinkLocal(IpAddr),
#[error("{0:?} is not the name of an interface")]
InterfaceName(String),
}
impl Endpoint {
/// Reads `text`: an IP address, with `%` and the name of an interface
/// after it if, and only if, it is link-local.
pub fn parse(text: &str, port: u16) -> Result<Self, EndpointProblem> {
let (address, interface) = match text.split_once('%') {
Some((address, interface)) => (address, Some(interface)),
None => (text, None),
};
let ip: IpAddr = address
.parse()
.map_err(|_| EndpointProblem::NotAnAddress(text.to_string()))?;
let link_local = match ip {
IpAddr::V6(v6) if v6.is_unicast_link_local() => Some(v6),
_ => None,
};
let interface = match (link_local, interface) {
(None, None) => None,
(Some(v6), None) => return Err(EndpointProblem::NoInterface(v6)),
(None, Some(_)) => return Err(EndpointProblem::NotLinkLocal(ip)),
(Some(_), Some(name)) => {
let fine = !name.is_empty()
&& name.len() <= MAX_INTERFACE_NAME
&& name
.chars()
.all(|c| c.is_ascii_graphic() && c != '/' && c != '%');
if !fine {
return Err(EndpointProblem::InterfaceName(name.to_string()));
}
Some(name.to_string())
}
};
Ok(Self {
ip,
port,
interface,
})
}
/// The number of the peer's interface as things are now: zero if the
/// address needs none, and `None` if the interface is not there.
fn scope(&self) -> Option<u32> {
match &self.interface {
None => Some(0),
Some(name) => interface_index(name),
}
}
/// What a connection from the peer comes from, as things are now.
/// `None` if the peer's interface is not there.
pub fn address(&self) -> Option<PeerAddress> {
Some(PeerAddress {
ip: self.ip,
interface: self.scope()?,
})
}
/// Whether a connection from `address` is one from this peer.
pub fn is_at(&self, address: PeerAddress) -> bool {
// The interface is looked up only for a peer whose address it is.
self.ip == address.ip && self.scope() == Some(address.interface)
}
/// Where to connect to reach the peer.
pub fn to_dial(&self) -> io::Result<SocketAddr> {
self.socket(self.ip, self.port)
}
/// `ip` and `port` as an address on the peer's link: for the peer
/// itself, and for our own address there.
pub fn socket(&self, ip: IpAddr, port: u16) -> io::Result<SocketAddr> {
let scope = self.scope().ok_or_else(|| {
io::Error::new(
io::ErrorKind::NotFound,
format!(
"there is no interface {}",
self.interface.as_deref().unwrap_or_default()
),
)
})?;
Ok(match ip {
IpAddr::V6(v6) if v6.is_unicast_link_local() => {
SocketAddrV6::new(v6, port, 0, scope).into()
}
ip => SocketAddr::new(ip, port),
})
}
}
/// The address and the interface: `fe80::1%eth0`. Without the port.
impl fmt::Display for Endpoint {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match &self.interface {
None => write!(f, "{}", self.ip),
Some(interface) => write!(f, "{}%{interface}", self.ip),
}
}
}
impl Endpoint {
/// The address and the port, as [`SocketAddr`] is written:
/// `[fe80::1%eth0]:179`.
pub fn with_port(&self) -> String {
match self.ip {
IpAddr::V4(v4) => format!("{v4}:{}", self.port),
IpAddr::V6(_) => format!("[{self}]:{}", self.port),
}
}
}
impl From<SocketAddr> for Endpoint {
/// An address that needs no interface. A link-local one keeps none
/// and so reaches nothing; those are read with [`Endpoint::parse`].
fn from(address: SocketAddr) -> Self {
Self {
ip: address.ip(),
port: address.port(),
interface: None,
}
}
}
#[cfg(test)]
mod tests {
use super::*;
/// An interface every machine has, and its number.
fn loopback() -> (String, u32) {
["lo", "lo0"]
.iter()
.find_map(|name| Some((name.to_string(), interface_index(name)?)))
.expect("a loopback interface")
}
#[test]
fn a_link_local_address_is_read_with_its_interface() {
let peer = Endpoint::parse("fe80:1234::1%eno3", 179).unwrap();
assert_eq!(peer.ip, "fe80:1234::1".parse::<IpAddr>().unwrap());
assert_eq!(peer.interface.as_deref(), Some("eno3"));
assert_eq!(peer.to_string(), "fe80:1234::1%eno3");
assert_eq!(peer.with_port(), "[fe80:1234::1%eno3]:179");
let plain = Endpoint::parse("2001:db8::1", 1179).unwrap();
assert_eq!(plain.interface, None);
assert_eq!(plain.with_port(), "[2001:db8::1]:1179");
assert_eq!(
Endpoint::parse("192.0.2.1", 179).unwrap().with_port(),
"192.0.2.1:179"
);
}
#[test]
fn an_address_and_an_interface_that_do_not_go_together_are_refused() {
let problem = |text: &str| Endpoint::parse(text, 179).unwrap_err();
assert_eq!(
problem("fe80::1"),
EndpointProblem::NoInterface("fe80::1".parse().unwrap())
);
assert_eq!(
problem("febf::1"),
EndpointProblem::NoInterface("febf::1".parse().unwrap())
);
assert_eq!(
problem("2001:db8::1%eth0"),
EndpointProblem::NotLinkLocal("2001:db8::1".parse().unwrap())
);
assert_eq!(
problem("192.0.2.1%eth0"),
EndpointProblem::NotLinkLocal("192.0.2.1".parse().unwrap())
);
// fec0::/10 was site-local: not link-local.
assert!(Endpoint::parse("fec0::1", 179).is_ok());
for name in ["", "a/b", "an interface", "sixteen-letters-", "a%b"] {
assert_eq!(
problem(&format!("fe80::1%{name}")),
EndpointProblem::InterfaceName(name.to_string()),
);
}
assert_eq!(
problem("example"),
EndpointProblem::NotAnAddress("example".into())
);
}
/// The interface's number is whatever it is when it is asked for.
#[test]
fn the_interface_is_looked_up_when_it_is_needed() {
let (name, index) = loopback();
let peer = Endpoint::parse(&format!("fe80::1%{name}"), 179).unwrap();
let ip: IpAddr = "fe80::1".parse().unwrap();
let there = PeerAddress {
ip,
interface: index,
};
assert_eq!(peer.address(), Some(there));
assert!(peer.is_at(there));
// The same address on another link is another peer.
assert!(!peer.is_at(PeerAddress {
ip,
interface: index + 1000
}));
assert!(!peer.is_at(ip.into()));
match peer.to_dial().unwrap() {
SocketAddr::V6(v6) => assert_eq!((v6.scope_id(), v6.port()), (index, 179)),
SocketAddr::V4(_) => panic!("an IPv6 peer"),
}
assert_eq!(there.to_string(), format!("fe80::1%{name}"));
assert!(there.is_called(&format!("fe80::1%{name}")));
assert!(there.is_called("fe80::1"));
assert!(!there.is_called("fe80::1%other"));
assert!(!there.is_called("fe80::2"));
// By its configured name as well, where it has one.
assert!(names_peer("upstream", Some("upstream"), there));
assert!(names_peer("fe80::1", Some("upstream"), there));
assert!(names_peer("fe80::1", None, there));
assert!(!names_peer("upstream", None, there));
assert!(!names_peer("other", Some("upstream"), there));
assert_eq!(
socket_text(peer.to_dial().unwrap()),
format!("[fe80::1%{name}]:179")
);
let gone = Endpoint::parse("fe80::1%not-there", 179).unwrap();
assert_eq!(gone.address(), None);
assert!(!gone.is_at(there));
assert_eq!(
gone.to_dial().unwrap_err().to_string(),
"there is no interface not-there"
);
}
/// Only a link-local address keeps the interface it came in on.
#[test]
fn only_a_link_local_address_has_an_interface() {
let global: SocketAddr =
SocketAddrV6::new("2001:db8::1".parse().unwrap(), 179, 0, 7).into();
assert_eq!(PeerAddress::from(global).interface, 0);
let local: SocketAddr = SocketAddrV6::new("fe80::1".parse().unwrap(), 179, 0, 7).into();
assert_eq!(PeerAddress::from(local).interface, 7);
assert_eq!(socket_text(global), "[2001:db8::1]:179");
}
}
crates/server/src/advertise.rs
+102 -2385
View File
File diff suppressed because it is too large. Load diff
crates/server/src/collision.rs
+17 -41
View File
@@ -24,13 +24,12 @@
//! by the same side the newer wins: the initiator has abandoned the older.
use std::collections::HashMap;
use std::net::Ipv4Addr;
use std::net::{IpAddr, Ipv4Addr};
use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::{Arc, Mutex};
use tokio::sync::oneshot;
use crate::address::PeerAddress;
use crate::metrics::COLLISIONS;
use crate::sync::IgnorePoison;
@@ -84,7 +83,7 @@ fn new_loses(
/// Who a connection is with: the address at the other end, and the BGP
/// Identifier announced from it.
type Peer = (PeerAddress, Ipv4Addr);
type Peer = (IpAddr, Ipv4Addr);
/// Shared by every connection of one server.
#[derive(Clone)]
@@ -103,33 +102,19 @@ impl CollisionTable {
}
}
/// Call when a connection receives the peer's OPEN. `one_identifier`
/// says that the address is that of one router, which holds for a
/// configured peer. `None` means this connection lost and must be
/// closed with Cease / Connection Collision Resolution. Otherwise any
/// rival has been told to close and this connection holds the slot
/// until the guard drops.
/// Call when a connection receives the peer's OPEN. `None` means this
/// connection lost and must be closed with Cease / Connection Collision
/// Resolution. Otherwise any rival has been told to close and this
/// connection holds the slot until the guard drops.
pub fn admit(
&self,
local_id: Ipv4Addr,
remote_addr: PeerAddress,
remote_addr: IpAddr,
remote_id: Ipv4Addr,
direction: Direction,
one_identifier: bool,
) -> Option<ConnectionGuard> {
let peer = (remote_addr, remote_id);
let mut entries = self.entries.locked();
// A configured peer is one router, with one identifier. A second
// connection from its address under another would be a second
// session, with a second allowance of routes.
if one_identifier
&& entries
.keys()
.any(|(address, id)| *address == remote_addr && *id != remote_id)
{
COLLISIONS.with_label_values(&["new"]).inc();
return None;
}
if let Some(existing) = entries.get_mut(&peer) {
if new_loses(
local_id,
@@ -143,7 +128,6 @@ impl CollisionTable {
}
if let Some(kill) = existing.kill.take() {
COLLISIONS.with_label_values(&["existing"]).inc();
// Fails only if the rival has gone already.
let _ = kill.send(());
}
}
@@ -279,34 +263,26 @@ mod tests {
#[tokio::test]
async fn table_signals_loser_and_releases_slots() {
let table = CollisionTable::new(false);
let addr = PeerAddress::from(std::net::IpAddr::from(Ipv4Addr::new(192, 0, 2, 1)));
let addr: IpAddr = Ipv4Addr::new(192, 0, 2, 1).into();
// We are LOW: outbound first, then the peer's inbound wins.
let mut out = table.admit(LOW, addr, HIGH, Outbound, false).unwrap();
let inb = table.admit(LOW, addr, HIGH, Inbound, false).unwrap();
let mut out = table.admit(LOW, addr, HIGH, Outbound).unwrap();
let inb = table.admit(LOW, addr, HIGH, Inbound).unwrap();
assert!(out.killed.try_recv().is_ok());
// The loser's guard dropping must not free the winner's slot.
drop(out);
inb.set_established();
assert!(table.admit(LOW, addr, HIGH, Outbound, false).is_none());
assert!(table.admit(LOW, addr, HIGH, Outbound).is_none());
// A different identifier never collides.
assert!(
table
.admit(LOW, addr, Ipv4Addr::new(192, 0, 2, 2), Inbound, false)
.admit(LOW, addr, Ipv4Addr::new(192, 0, 2, 2), Inbound)
.is_some()
);
// Unless the address is one router's, which has one identifier: a
// second would be a second session, with its own limit of routes.
assert!(
table
.admit(LOW, addr, Ipv4Addr::new(192, 0, 2, 3), Inbound, true)
.is_none()
);
// Once the survivor goes away the identifier is free again.
drop(inb);
assert!(table.admit(LOW, addr, HIGH, Outbound, false).is_some());
assert!(table.admit(LOW, addr, HIGH, Outbound).is_some());
}
/// A neighbour reached over IPv4 and over IPv6 announces the same
@@ -316,12 +292,12 @@ mod tests {
#[tokio::test]
async fn the_same_identifier_from_another_address_is_another_peer() {
let table = CollisionTable::new(true);
let v4 = PeerAddress::from(std::net::IpAddr::from(Ipv4Addr::new(192, 0, 2, 1)));
let v6 = PeerAddress::from("2001:db8::1".parse::<std::net::IpAddr>().unwrap());
let v4: IpAddr = Ipv4Addr::new(192, 0, 2, 1).into();
let v6: IpAddr = "2001:db8::1".parse().unwrap();
let mut over_v4 = table.admit(LOW, v4, HIGH, Outbound, false).unwrap();
let mut over_v4 = table.admit(LOW, v4, HIGH, Outbound).unwrap();
over_v4.set_established();
let over_v6 = table.admit(LOW, v6, HIGH, Inbound, false);
let over_v6 = table.admit(LOW, v6, HIGH, Inbound);
assert!(over_v6.is_some());
assert!(over_v4.killed.try_recv().is_err(), "the first is untouched");
}
crates/server/src/config.rs
+126 -1215
View File
File diff suppressed because it is too large. Load diff
crates/server/src/connection.rs
+217 -1458
View File
File diff suppressed because it is too large. Load diff
crates/server/src/dialer.rs
+40 -95
View File
@@ -8,7 +8,6 @@
//! connection is the one closed, the next pass finds the surviving session
//! and goes idle.
use std::hash::{BuildHasher, Hasher, RandomState};
use std::sync::Arc;
use std::time::{Duration, SystemTime};
@@ -32,38 +31,26 @@ pub fn spawn_dialers(
) -> Vec<JoinHandle<()>> {
peers
.iter()
// A passive peer is left to connect to us.
.filter(|peer| !peer.passive)
.map(|peer| tokio::spawn(dial_loop(peer.clone(), Arc::clone(server), connect_retry)))
.collect()
}
/// Opens the TCP connection, from the configured source address if any.
async fn connect(peer: &PeerConfig) -> std::io::Result<TcpStream> {
// Worked out for each attempt: an interface that was made anew has a
// new number.
let endpoint = peer.endpoint.to_dial()?;
let Some(local) = peer.local_address else {
return TcpStream::connect(endpoint).await;
return TcpStream::connect(peer.endpoint).await;
};
let socket = if endpoint.is_ipv4() {
let socket = if peer.endpoint.is_ipv4() {
TcpSocket::new_v4()?
} else {
TcpSocket::new_v6()?
};
// A link-local address of our own is on the peer's link.
socket.bind(peer.endpoint.socket(local, 0)?).map_err(|e| {
std::io::Error::new(e.kind(), format!("cannot use local_address {local}: {e}"))
})?;
socket.connect(endpoint).await
}
/// `wait`, less up to a quarter of it, by chance (RFC 4271 §10).
fn jittered(wait: Duration) -> Duration {
// The standard library's source of chance, which is what keys its
// hash tables: good enough to tell dialers apart.
let chance = RandomState::new().build_hasher().finish();
wait.mul_f64(0.75 + 0.25 * (chance as f64 / u64::MAX as f64))
socket
.bind(std::net::SocketAddr::new(local, 0))
.map_err(|e| {
std::io::Error::new(e.kind(), format!("cannot use local_address {local}: {e}"))
})?;
socket.connect(peer.endpoint).await
}
async fn dial_loop(peer: PeerConfig, server: Arc<ServerContext>, connect_retry: Duration) {
@@ -74,98 +61,56 @@ async fn dial_loop(peer: PeerConfig, server: Arc<ServerContext>, connect_retry:
if registry.is_shutting_down() {
return;
}
// Where the peer is now; nowhere if its interface is not there.
let address = peer.endpoint.address();
if address.is_some_and(|address| registry.find(address, peer.remote_asn).is_some()) {
if registry.find(peer.endpoint.ip(), peer.remote_asn).is_some() {
// Peer dialled us; nothing to do until that session goes away.
registry.set_dial_state(&peer.name, DialState::Idle);
tokio::time::sleep(connect_retry).await;
continue;
}
if let Some((remaining, why)) =
address.and_then(|address| server.hold_downs.remaining(address))
{
registry.set_dial_state(
&peer.name,
DialState::Backoff {
until: SystemTime::now() + remaining,
last_error: why.as_str().into(),
},
);
tokio::time::sleep(remaining).await;
// The wait starts over: the peer has had time to be put right.
backoff = connect_retry;
continue;
}
registry.set_dial_state(&peer.name, DialState::Connecting);
info!(peer = %peer.name, endpoint = %peer.endpoint, "Connecting to peer");
// Why the attempt ended, and whether that is a failure worth a
// line in the log.
let (reason, failed) = match tokio::time::timeout(CONNECT_TIMEOUT, connect(&peer)).await {
Ok(Ok(stream)) => match stream.peer_addr() {
Ok(peer_addr) => {
info!(peer = %peer.name, endpoint = %peer.endpoint, "TCP connected, starting session");
let end = run_session(
stream,
peer_addr,
Direction::Outbound,
Arc::clone(&server),
PeerSettings::from(&peer),
)
.await;
// A session that came up is a peer worth dialling again
// soon. One that connects and is then refused, for its AS
// number say, will be refused again: it is retried ever
// more slowly, like one that cannot be reached at all.
if end.established {
backoff = connect_retry;
}
let failed = end.result.is_err();
(end.reason(), failed)
let ended = match tokio::time::timeout(CONNECT_TIMEOUT, connect(&peer)).await {
Ok(Ok(stream)) => {
info!(peer = %peer.name, endpoint = %peer.endpoint, "TCP connected, starting session");
let end = run_session(
stream,
peer.endpoint,
Direction::Outbound,
Arc::clone(&server),
PeerSettings::from(&peer),
)
.await;
// A session that came up is a peer worth dialling again
// soon. One that connects and is then refused, for its AS
// number say, will be refused again: it is retried ever
// more slowly, like one that cannot be reached at all.
if end.established {
backoff = connect_retry;
}
Err(e) => (format!("connection lost at once: {e}"), true),
},
Ok(Err(e)) => (format!("connect failed: {e}"), true),
Err(_) => (format!("connect timed out after {CONNECT_TIMEOUT:?}"), true),
end.result.map_err(|e| e.to_string())
}
Ok(Err(e)) => Err(format!("connect failed: {e}")),
Err(_) => Err(format!("connect timed out after {CONNECT_TIMEOUT:?}")),
};
let reason = match ended {
Ok(()) => "session closed at our request".to_string(),
Err(reason) => {
warn!(peer = %peer.name, endpoint = %peer.endpoint, err = %reason, "Session attempt ended");
reason
}
};
if failed {
warn!(peer = %peer.name, endpoint = %peer.endpoint, err = %reason, "Session attempt ended");
}
// Not all at the same moment, when many peers went away together.
let wait = jittered(backoff);
registry.set_dial_state(
&peer.name,
DialState::Backoff {
until: SystemTime::now() + wait,
until: SystemTime::now() + backoff,
last_error: reason,
},
);
tokio::time::sleep(wait).await;
tokio::time::sleep(backoff).await;
backoff = (backoff * 2).min(MAX_BACKOFF);
}
}
#[cfg(test)]
mod tests {
use super::*;
/// The wait before dialling again is shortened by up to a quarter, by
/// chance, so that peers that went away together are not all dialled
/// at the same moment (RFC 4271 §10).
#[test]
fn the_wait_is_jittered_by_up_to_a_quarter() {
let wait = Duration::from_secs(100);
let mut shortest = wait;
for _ in 0..200 {
let jittered = jittered(wait);
assert!(jittered <= wait, "{jittered:?}");
assert!(jittered >= wait.mul_f64(0.75), "{jittered:?}");
shortest = shortest.min(jittered);
}
assert!(shortest < wait, "the wait varies");
}
}
crates/server/src/error.rs
+5 -203
View File
@@ -2,13 +2,10 @@
use std::io;
use std::net::SocketAddr;
use crate::address::EndpointProblem;
use std::time::Duration;
use bgp_packet::errors::{BgpError, CeaseSubcode, DecodeError, FsmSubcode, OpenMsgSubcode};
use bgp_packet::message::NotificationMessage;
use bgp_packet::open::BgpRole;
use bgp_primitives::parser::EncodeError;
use bgp_rib::RibStopped;
use bgp_rib::table::TooManyPeers;
@@ -16,21 +13,6 @@ use thiserror::Error;
use crate::family::FamilySet;
use crate::fsm::State;
use crate::policy::PeerRole;
/// Says what is wrong between what a peer is configured as and what it
/// says it is.
fn role_mismatch(configured: Option<PeerRole>, theirs: Option<BgpRole>) -> String {
let configured = configured.map_or("no role is configured".to_string(), |role| {
format!("the peer is configured as our {}", role.as_str())
});
match theirs {
Some(theirs) => format!("role mismatch: {configured} and says it is {theirs:?}"),
None => format!(
"role mismatch: {configured} and does not say what it is, or says more than one thing"
),
}
}
/// Why a session ended abnormally.
#[derive(Debug, Error)]
@@ -41,9 +23,6 @@ pub enum SessionError {
Decode(#[from] DecodeError),
#[error("could not encode a message: {0}")]
Encode(#[from] EncodeError),
/// A received UPDATE's attributes could not be kept in the table.
#[error(transparent)]
Unstorable(#[from] crate::routes::Unstorable),
/// The peer sent nothing for as long as the pre-negotiation hold time.
#[error("timed out waiting for a message in the {0} state")]
Timeout(State),
@@ -61,13 +40,6 @@ pub enum SessionError {
OpenRefused(BgpError),
#[error("peer announced AS{actual} but AS{expected} was configured")]
WrongPeerAs { expected: u32, actual: u32 },
/// RFC 9234 §4.2. `theirs` is `None` if the peer did not say, or said
/// more than one thing.
#[error("{}", role_mismatch(*.configured, *.theirs))]
RoleMismatch {
configured: Option<PeerRole>,
theirs: Option<BgpRole>,
},
#[error("closed by connection collision resolution")]
Collision,
/// The routing table has no room for another peer.
@@ -81,17 +53,6 @@ pub enum SessionError {
/// the session could carry nothing.
#[error("no address family in common: we offer {ours}, the peer offers {theirs}")]
NoCommonAddressFamily { ours: FamilySet, theirs: FamilySet },
/// The peer has more routes than it is allowed (RFC 4486). It is kept
/// down for a while after.
#[error("the peer went over its limit of {limit} routes")]
PrefixLimitExceeded { limit: u32 },
/// We were to announce the peer more routes than its export limit
/// allows, and did not. It is kept down for a while after.
#[error(
"{routes} routes were to be advertised to the peer, over its export limit of {limit}; \
none of them was"
)]
ExportLimitExceeded { limit: u32, routes: usize },
/// The peer took nothing we sent for this long (RFC 9687): it has
/// stopped reading, and there is no telling it so.
#[error("the peer accepted no data for {0:?}")]
@@ -104,10 +65,9 @@ impl SessionError {
pub fn notification(&self) -> Option<NotificationMessage> {
let error = match self {
Self::Io(_) | Self::SendHoldTimerExpired(_) | Self::NotifiedByPeer(_) => return None,
// Something of ours could not be written out, or the peer's
// routes could not be kept, which is a bug here: as for a
// panic, the session is reset.
Self::Encode(_) | Self::Unstorable(_) => BgpError::Cease(CeaseSubcode::AdminReset),
// Something of ours could not be written out, which is a bug
// here: as for a panic, the session is reset.
Self::Encode(_) => BgpError::Cease(CeaseSubcode::AdminReset),
Self::Decode(e) => {
return Some(NotificationMessage {
error: e.notification.clone(),
@@ -124,21 +84,8 @@ impl SessionError {
}),
Self::OpenRefused(error) => error.clone(),
Self::WrongPeerAs { .. } => BgpError::OpenMsg(OpenMsgSubcode::BadPeerAs),
Self::RoleMismatch { .. } => BgpError::OpenMsg(OpenMsgSubcode::RoleMismatch),
Self::Collision => BgpError::Cease(CeaseSubcode::ConnCollisionResolution),
Self::TableFull(_) => BgpError::Cease(CeaseSubcode::OutOfResources),
Self::PrefixLimitExceeded { .. } => BgpError::Cease(CeaseSubcode::MaxPrefixes),
// Nothing is wrong at the peer's end, as far as we know; its
// operators are told what is at ours.
Self::ExportLimitExceeded { limit, .. } => {
return Some(NotificationMessage {
error: BgpError::Cease(CeaseSubcode::AdminShutdown),
data: shutdown_communication(&format!(
"export limit of {limit} routes reached; no more routes were sent, \
and the session is held down for a while"
)),
});
}
// The session will be set up again; nothing is wrong at the
// peer's end.
Self::Panicked(_) => BgpError::Cease(CeaseSubcode::AdminReset),
@@ -158,19 +105,6 @@ impl SessionError {
}
}
/// A Shutdown Communication (RFC 9003 §2): text for the peer's operators,
/// at most 255 octets of UTF-8, after its length.
fn shutdown_communication(text: &str) -> Vec<u8> {
let mut end = text.len().min(255);
while !text.is_char_boundary(end) {
end -= 1;
}
let mut data = Vec::with_capacity(end + 1);
data.push(end as u8);
data.extend_from_slice(&text.as_bytes()[..end]);
data
}
/// Why the server could not start, or could not go on.
#[derive(Debug, Error)]
pub enum ServerError {
@@ -189,9 +123,6 @@ pub enum ServerError {
/// they panic.
#[error("the {task} task stopped unexpectedly: {reason}")]
TaskStopped { task: &'static str, reason: String },
/// Routes were to be installed in the kernel, and cannot be.
#[error(transparent)]
Kernel(#[from] bgp_fib::FibError),
}
/// Why a configuration was rejected.
@@ -199,41 +130,14 @@ pub enum ServerError {
pub enum ConfigError {
#[error("invalid router_id {0:?}: must be a dotted-quad IPv4 address other than 0.0.0.0")]
RouterId(String),
#[error("invalid cluster_id {0:?}: must be a dotted-quad IPv4 address other than 0.0.0.0")]
ClusterId(String),
#[error("local_asn must be non-zero")]
LocalAsn,
#[error("invalid IP address {0:?}")]
Address(String),
#[error(transparent)]
Endpoint(#[from] EndpointProblem),
#[error("port {0} is out of range")]
Port(u32),
#[error("http_endpoint.port is required")]
HttpPort,
#[error("rpki: no caches are listed")]
RpkiNoCaches,
#[error("export: {0}")]
Export(#[from] ExportProblem),
#[error("export_limits: {0}")]
ExportLimits(ImportProblem),
#[error("community_names {community:?}: {problem}")]
CommunityName {
community: String,
problem: crate::marks::MarksProblem,
},
#[error("community_names {community:?}: the name is empty")]
CommunityNameEmpty { community: String },
#[error(
"accept_unconfigured_peers is set and export.learned_routes passes learned routes on: \
a peer that is not configured has no role, so the routes it sends would go to every \
peer, and every route to it"
)]
UnconfiguredPeersWithoutRole,
#[error("rpki: the port of cache {0:?} is required")]
RpkiCachePort(String),
#[error("graceful_restart: {0}")]
GracefulRestart(#[from] GracefulRestartProblem),
#[error("peer {peer:?}: {problem}")]
Peer { peer: String, problem: PeerProblem },
#[error("originate {prefix:?}: {problem}")]
@@ -266,82 +170,6 @@ pub enum PeerProblem {
AddressFamily,
#[error("add_path has a value that is not one of its settings")]
AddPath,
#[error("role is set for a peer in our own AS; roles are between networks")]
RoleInOwnAs,
#[error("require_role is set without a role")]
RequireRoleWithoutRole,
#[error("role has a value that is not one of its settings")]
Role,
#[error("groups has a 0 in it; groups are numbered from 1")]
Group,
#[error("route_reflector_client is set for a peer in another AS; it is for routers of our own")]
ReflectorClientOutsideAs,
#[error(
"role is not set, and learned routes are passed on (export.learned_routes): without a \
role, the routes the peer sends would go to every other peer, and it would be told of \
every route. Set role to PROVIDER, CUSTOMER, PEER, ROUTE_SERVER or ROUTE_SERVER_CLIENT"
)]
NoRole,
#[error("export_limits: {0}")]
ExportLimits(ImportProblem),
#[error("import: {0}")]
Import(#[from] ImportProblem),
#[error("export: {0}")]
Export(#[from] ExportProblem),
#[error("graceful_restart: {0}")]
GracefulRestart(#[from] GracefulRestartProblem),
}
/// What is wrong with a `graceful_restart`.
#[derive(Debug, Clone, PartialEq, Eq, Error)]
pub enum GracefulRestartProblem {
#[error("restart_time_seconds is {0}; it is at most {max}", max = crate::restart::MAX_RESTART_TIME)]
RestartTime(u32),
}
/// What is wrong with a peer's `import` policy.
#[derive(Debug, Clone, PartialEq, Eq, Error)]
pub enum ImportProblem {
#[error("prefix {prefix:?}: {problem}")]
Prefix {
prefix: String,
problem: bgp_primitives::ip_prefix::PrefixError,
},
#[error(
"prefix {prefix:?}: max_length {max_length} is not between its own length and {longest}"
)]
MaxLength {
prefix: String,
max_length: u32,
longest: u8,
},
#[error("{field} of {value} is longer than an address of {longest} bits")]
PrefixLength {
field: &'static str,
value: u32,
longest: u8,
},
#[error("max_as_path_length of {0} is more than a path can hold")]
PathLength(u32),
#[error(transparent)]
Marks(#[from] crate::marks::MarksProblem),
#[error("rule {number}: {problem}")]
Rule {
/// From 1, as an operator counts.
number: usize,
problem: Box<ImportProblem>,
},
}
/// What is wrong with an `export` policy.
#[derive(Debug, Clone, PartialEq, Eq, Error)]
pub enum ExportProblem {
#[error(transparent)]
Marks(#[from] crate::marks::MarksProblem),
#[error("as_path_prepend of {value} is more than {max}")]
Prepend { value: u32, max: u32 },
#[error("all_paths needs learned_routes")]
AllPathsWithoutLearnedRoutes,
}
/// What is wrong with one `originate` entry.
@@ -351,8 +179,8 @@ pub enum OriginateProblem {
Prefix(#[from] bgp_primitives::ip_prefix::PrefixError),
#[error("origin {0:?} must be IGP, EGP or INCOMPLETE")]
Origin(String),
#[error("as_path_prepend of {value} is more than {max}")]
Prepend { value: u32, max: u32 },
#[error("as_path_prepend must be at most {max}")]
Prepend { max: u32 },
#[error("{0}")]
Community(#[from] bgp_packet::community::CommunityError),
#[error("listed more than once")]
@@ -416,31 +244,5 @@ mod tests {
notification(SessionError::Io(io::ErrorKind::BrokenPipe.into())),
None
);
// Going over an export limit is ours to explain: an Administrative
// Shutdown, with the reason for the peer's operators (RFC 9003).
let (error, data) = notification(SessionError::ExportLimitExceeded {
limit: 50,
routes: 900_000,
})
.unwrap();
assert_eq!(error, BgpError::Cease(CeaseSubcode::AdminShutdown));
let text = std::str::from_utf8(&data[1..]).unwrap();
assert_eq!(usize::from(data[0]), text.len());
assert!(
text.starts_with("export limit of 50 routes reached"),
"{text}"
);
}
/// A Shutdown Communication is at most 255 octets, and whole
/// characters (RFC 9003 §2).
#[test]
fn a_shutdown_communication_is_cut_to_fit() {
let long = "é".repeat(200);
let data = shutdown_communication(&long);
assert_eq!(usize::from(data[0]), data.len() - 1);
assert_eq!(data[0], 254);
assert!(std::str::from_utf8(&data[1..]).is_ok());
assert_eq!(shutdown_communication(""), [0]);
}
}
crates/server/src/export.rs
-1426
View File
File diff suppressed because it is too large. Load diff
crates/server/src/fsm.rs
+25 -616
View File
@@ -14,18 +14,13 @@
use std::net::Ipv4Addr;
use std::time::Duration;
use bgp_packet::errors::{BgpError, CeaseSubcode, DecodeError, OpenMsgSubcode};
use bgp_packet::message::{AS_TRANS, BGP4_VERSION, Message, NotificationMessage, UpdateMessage};
use bgp_packet::open::{
AddPathEntry, Capability, ExtendedNextHopEntry, OpenMessage, OpenOption, SendReceive,
};
use bgp_primitives::constants::{Afi, AfiSafi, Safi};
use bgp_packet::errors::{BgpError, DecodeError, OpenMsgSubcode};
use bgp_packet::message::{AS_TRANS, BGP4_VERSION, Message, UpdateMessage};
use bgp_packet::open::{AddPathEntry, AddPathSendRecv, Capability, OpenMessage, OpenOption};
use crate::error::SessionError;
use crate::family::FamilySet;
use crate::metrics::message_type_name;
use crate::policy::PeerRole;
use crate::restart::{self, Agreed, Leave, Offer};
/// How long to wait for the peer's OPEN, and for its first KEEPALIVE when
/// the negotiated hold time is zero. RFC 4271 §8.2.2 suggests four minutes.
@@ -69,19 +64,7 @@ pub struct FsmConfig {
pub families: FamilySet,
/// The directions to offer ADD-PATH (RFC 7911) for, on each of
/// `families`; `None` for neither.
pub add_path: Option<SendReceive>,
/// What the peer is to us, if that is configured. Our OPEN then says
/// what we are to it, and the peer's must not say otherwise (RFC 9234).
/// Not for a peer in our own AS.
pub role: Option<PeerRole>,
/// Refuse a peer that does not say what it is (RFC 9234 §4.2, "strict
/// mode"). Means nothing without `role`.
pub require_role: bool,
/// Offer to exchange IPv4 routes with IPv6 next hops (RFC 8950): for a
/// session over a link that has only IPv6 addresses.
pub extended_next_hop: bool,
/// What graceful restart (RFC 4724) is offered as, if it is.
pub graceful_restart: Option<Offer>,
pub add_path: Option<AddPathSendRecv>,
}
/// What the two OPENs settled.
@@ -95,8 +78,6 @@ pub struct Negotiated {
pub hold_time_secs: u16,
/// Whether AS numbers are four octets wide on this session (RFC 6793).
pub four_octet_asn: bool,
/// Whether the peer can be asked to send its routes again (RFC 2918).
pub route_refresh: bool,
/// The address families both sides offered: the ones routes may be
/// exchanged for. Never empty.
pub families: FamilySet,
@@ -107,14 +88,6 @@ pub struct Negotiated {
/// The families for which we send the peer path identifiers: we
/// offered to send and the peer to receive.
pub add_path_send: FamilySet,
/// Whether the peer said what it is to us, and it is what we have it
/// configured as (RFC 9234).
pub role_confirmed: bool,
/// Whether IPv4 routes go with IPv6 next hops on this session
/// (RFC 8950): both sides offered it, and the session carries IPv4.
pub extended_next_hop: bool,
/// Graceful restart (RFC 4724), if both sides offered it.
pub graceful_restart: Option<Agreed>,
}
/// A message from the peer, with what the decoder found out besides.
@@ -125,9 +98,6 @@ pub struct Received {
/// decoder kept them (`DecodedUpdate::attributes_wire`). They go with
/// the UPDATE to whoever stores its routes.
pub attributes_wire: Option<Vec<u8>>,
/// For an UPDATE: whether the decoder changed it because of errors in
/// it (RFC 7606). An UPDATE emptied that way is no End-of-RIB marker.
pub repaired: bool,
}
impl From<Message> for Received {
@@ -135,7 +105,6 @@ impl From<Message> for Received {
Self {
message,
attributes_wire: None,
repaired: false,
}
}
}
@@ -158,8 +127,6 @@ pub enum Event {
/// table has no room for the peer's routes, say. The peer is told as
/// the error has it.
LocalFailure(SessionError),
/// The daemon is going, and the session with it.
Leave(Leave),
}
/// Something the driver must do, in the order given.
@@ -181,8 +148,6 @@ pub enum Action {
/// The peer sent this many routes of address families that were not
/// negotiated. They have been left out of the update that follows.
RoutesIgnored(usize),
/// The peer has sent all its routes of this family once (RFC 4724 §2).
EndOfRib(AfiSafi),
/// Routes arrived; put them in the routing table.
DeliverUpdate {
update: UpdateMessage,
@@ -265,7 +230,6 @@ impl Fsm {
_ => vec![],
},
Event::Outbound(msg) => self.on_outbound(msg),
Event::Leave(leave) => self.leave(leave),
}
}
@@ -273,7 +237,6 @@ impl Fsm {
let Received {
message,
attributes_wire,
repaired,
} = received;
match (&self.phase, message) {
// A NOTIFICATION ends the session in any state, and is not
@@ -299,14 +262,6 @@ impl Fsm {
(Phase::Established(negotiated), Message::KeepAlive) => heard_from_peer(negotiated),
(Phase::Established(negotiated), Message::Update(mut update)) => {
let mut actions = heard_from_peer(negotiated);
// Before the families are restricted, which can leave an
// UPDATE that looks like one.
if let Some(family) = update.end_of_rib_family().filter(|_| !repaired) {
if negotiated.families.contains(family) {
actions.push(Action::EndOfRib(family));
}
return actions;
}
// A route of a family the session does not carry is ignored
// rather than treated as an error: it harms nothing, and
// resetting the session over it would.
@@ -352,48 +307,42 @@ impl Fsm {
});
}
let role_confirmed = match self.negotiate_role(&open) {
Ok(confirmed) => confirmed,
Err(e) => return self.fail(e),
};
let ours = self.config.families.clone();
let theirs = FamilySet::offered_by(&open);
let families = ours.intersection(&theirs);
if families.is_empty() {
return self.fail(SessionError::NoCommonAddressFamily { ours, theirs });
}
let (add_path_receive, add_path_send) = self.negotiate_add_path(&open, &families);
// RFC 4724 §3: in use if both offered it.
let graceful_restart = self
.config
.graceful_restart
.and_then(|_| Agreed::of(&open, &families));
// RFC 8950 §4: in use only if both said so.
let extended_next_hop = self.config.extended_next_hop
&& families.contains(AfiSafi::IPV4_UNICAST)
&& open.capabilities().any(|capability| {
matches!(capability, Capability::ExtendedNextHop { entries }
if entries.contains(&IPV4_WITH_IPV6_NEXT_HOP))
// ADD-PATH is in use in a direction, for a family the session
// carries, when one side offered to send and the other to receive
// (RFC 7911 §4).
let (mut add_path_receive, mut add_path_send) =
(FamilySet::default(), FamilySet::default());
if let Some(ours) = self.config.add_path {
let theirs = open.capabilities().flat_map(|capability| match capability {
Capability::AddPath { entries } => entries.as_slice(),
_ => &[],
});
for entry in theirs.filter(|entry| families.contains(entry.family())) {
if ours.receives() && entry.send_recv.sends() {
add_path_receive.insert(entry.family());
}
if ours.sends() && entry.send_recv.receives() {
add_path_send.insert(entry.family());
}
}
}
let negotiated = Negotiated {
extended_next_hop,
peer_asn,
ibgp: peer_asn == self.config.local_asn,
peer_identifier: open.identifier,
hold_time_secs: self.config.hold_time_secs.min(open.hold_time),
four_octet_asn: capability_asn.is_some(),
route_refresh: open
.capabilities()
.any(|capability| matches!(capability, Capability::RouteRefresh)),
families,
add_path_receive,
add_path_send,
role_confirmed,
graceful_restart,
};
// With a hold time of zero the timer will not run once Established,
// but a peer that never confirms must still be given up on.
@@ -422,21 +371,9 @@ impl Fsm {
/// Ends the session because of `error`, telling the peer with the
/// NOTIFICATION that goes with it if there is one.
///
/// With graceful restart agreed for NOTIFICATIONs too (RFC 8538), one
/// that goes for an error is a Hard Reset: the peer is to drop our
/// routes, and not keep them for a restart. Not for an expired hold
/// timer, which RFC 8538 §4 has the peer treat as a restart; nor for a
/// collision, where another connection takes over at once.
fn fail(&mut self, error: SessionError) -> Vec<Action> {
let for_good = !matches!(
error,
SessionError::HoldTimerExpired | SessionError::Collision
);
let notification = error
let mut actions: Vec<Action> = error
.notification()
.map(|notification| self.as_sent(notification, for_good));
let mut actions: Vec<Action> = notification
.map(|notification| Action::Send(Message::Notification(notification)))
.into_iter()
.collect();
@@ -444,123 +381,12 @@ impl Fsm {
actions
}
/// Whether graceful restart applies to sessions that end with a
/// NOTIFICATION, other than a Hard Reset (RFC 8538).
fn notifications_restart(&self) -> bool {
match &self.phase {
Phase::OpenConfirm(negotiated) | Phase::Established(negotiated) => negotiated
.graceful_restart
.as_ref()
.is_some_and(|agreed| agreed.notification),
Phase::OpenSent | Phase::Closed => false,
}
}
/// Ends the session because the daemon is going: for good, with a
/// NOTIFICATION that has the peer drop our routes now; or to come back
/// at once, by closing the connection without one where graceful
/// restart is agreed, which has the peer keep them (RFC 4724 §4.2).
fn leave(&mut self, leave: Leave) -> Vec<Action> {
let agreed = match &self.phase {
Phase::Established(negotiated) => negotiated.graceful_restart.is_some(),
_ => false,
};
if leave == Leave::Restart && agreed {
return self.close(Ok(()));
}
let shutdown = NotificationMessage {
error: BgpError::Cease(CeaseSubcode::AdminShutdown),
data: vec![],
};
let notification = self.as_sent(shutdown, true);
let mut actions = vec![Action::Send(Message::Notification(notification))];
actions.extend(self.close(Ok(())));
actions
}
/// What the peer says it is to us, held against what is configured
/// (RFC 9234 §4.2): whether the two agree, or an error if they do not.
/// A peer that does not say is taken at our word for it, unless it is
/// required to; what a peer says of itself means nothing where nothing
/// is configured to hold it against.
fn negotiate_role(&self, open: &OpenMessage) -> Result<bool, SessionError> {
let mut their_roles = open
.capabilities()
.filter_map(|capability| match capability {
Capability::BgpRole { role } => Some(*role),
_ => None,
});
let their_role = their_roles.next();
let mismatch = |theirs| SessionError::RoleMismatch {
configured: self.config.role,
theirs,
};
// Saying it twice is saying it once; saying two things is saying
// nothing that can be relied on.
if their_roles.any(|role| Some(role) != their_role) {
return Err(mismatch(None));
}
match (self.config.role, their_role) {
(Some(configured), Some(theirs)) if theirs == configured.theirs() => Ok(true),
(Some(_), Some(theirs)) => Err(mismatch(Some(theirs))),
(Some(_), None) if self.config.require_role => Err(mismatch(None)),
(Some(_), None) | (None, _) => Ok(false),
}
}
/// The families the peer sends us several routes for a prefix in, and
/// those we send it several in: ADD-PATH is in use in a direction, for
/// a family the session carries, when one side offered to send and the
/// other to receive (RFC 7911 §4).
fn negotiate_add_path(
&self,
open: &OpenMessage,
families: &FamilySet,
) -> (FamilySet, FamilySet) {
let (mut receive, mut send) = (FamilySet::default(), FamilySet::default());
let Some(ours) = self.config.add_path else {
return (receive, send);
};
let theirs = open.capabilities().flat_map(|capability| match capability {
Capability::AddPath { entries } => entries.as_slice(),
_ => &[],
});
for entry in theirs.filter(|entry| families.contains(entry.family())) {
if ours.receives() && entry.send_recv.sends() {
receive.insert(entry.family());
}
if ours.sends() && entry.send_recv.receives() {
send.insert(entry.family());
}
}
(receive, send)
}
/// `notification` as it goes to the peer: where NOTIFICATIONs count as
/// restarts (RFC 8538) and this one is `for_good`, as a Hard Reset, so
/// that the peer drops our routes rather than keeping them.
fn as_sent(&self, notification: NotificationMessage, for_good: bool) -> NotificationMessage {
if for_good && self.notifications_restart() {
restart::hard_reset(notification)
} else {
notification
}
}
fn close(&mut self, result: Result<(), SessionError>) -> Vec<Action> {
self.phase = Phase::Closed;
vec![Action::Close(result)]
}
}
/// IPv4 unicast routes with next hops that are IPv6 addresses, as the
/// Extended Next Hop capability names it (RFC 8950 §3).
const IPV4_WITH_IPV6_NEXT_HOP: ExtendedNextHopEntry = ExtendedNextHopEntry {
afi: Afi::IPV4,
safi: Safi::UNICAST,
nh_afi: Afi::IPV6,
};
/// Any message from the peer shows it is alive (RFC 4271 §4.4).
fn heard_from_peer(negotiated: &Negotiated) -> Vec<Action> {
hold_time(negotiated)
@@ -588,17 +414,6 @@ fn build_open(config: &FsmConfig) -> Message {
// not accept routes of a family that was not offered (RFC 4760 §8).
capabilities.extend(config.families.iter().map(Capability::multiprotocol));
capabilities.push(Capability::RouteRefresh);
if let Some(role) = config.role {
capabilities.push(Capability::BgpRole { role: role.ours() });
}
if config.extended_next_hop && config.families.contains(AfiSafi::IPV4_UNICAST) {
capabilities.push(Capability::ExtendedNextHop {
entries: vec![IPV4_WITH_IPV6_NEXT_HOP],
});
}
if let Some(offer) = &config.graceful_restart {
capabilities.push(restart::capability(offer, &config.families));
}
if let Some(send_recv) = config.add_path {
capabilities.push(Capability::AddPath {
entries: config
@@ -648,8 +463,6 @@ mod tests {
use bgp_rib::table::TooManyPeers;
use bgp_packet::open::BgpRole;
use super::*;
use crate::family::SUPPORTED;
@@ -664,10 +477,6 @@ mod tests {
expected_asn: None,
families: FamilySet::supported(),
add_path: None,
role: None,
require_role: false,
extended_next_hop: false,
graceful_restart: None,
}
}
@@ -763,14 +572,10 @@ mod tests {
peer_identifier: PEER_ID,
hold_time_secs: 30,
four_octet_asn: false,
route_refresh: false,
// The peer listed no families, so it speaks IPv4 unicast.
families: only(AfiSafi::IPV4_UNICAST),
add_path_receive: FamilySet::default(),
add_path_send: FamilySet::default(),
role_confirmed: false,
extended_next_hop: false,
graceful_restart: None,
};
assert!(matches!(
&actions[..],
@@ -900,7 +705,8 @@ mod tests {
#[test]
fn add_path_is_in_use_where_one_side_sends_and_the_other_receives() {
use bgp_packet::open::SendReceive::{Both, Receive, Send};
let negotiated = |ours: Option<SendReceive>, theirs: Vec<(AfiSafi, SendReceive)>| {
let negotiated = |ours: Option<AddPathSendRecv>,
theirs: Vec<(AfiSafi, AddPathSendRecv)>| {
let (mut fsm, first) = Fsm::new(FsmConfig {
add_path: ours,
..config()
@@ -1000,197 +806,6 @@ mod tests {
));
}
/// What becomes of a session with a peer configured as `role`, whose
/// OPEN carries the Role capabilities `theirs`: whether the peer
/// confirmed its role, or the error the session ended with.
fn roles_meet(
role: Option<PeerRole>,
require_role: bool,
theirs: &[BgpRole],
) -> Result<bool, BgpError> {
let (mut fsm, _) = Fsm::new(FsmConfig {
role,
require_role,
..config()
});
let open = OpenMessage {
options: vec![OpenOption::Capabilities(
theirs
.iter()
.map(|role| Capability::BgpRole { role: *role })
.collect(),
)],
..peer_open(90)
};
let actions = fsm.handle(received(Message::Open(open)));
match &actions[..] {
[Action::Admit(negotiated), ..] => Ok(negotiated.role_confirmed),
_ => Err(ending(&fsm, actions).0.expect("a NOTIFICATION")),
}
}
/// RFC 8950: IPv4 routes with IPv6 next hops are in use only if both
/// sides offer it, and we offer it only where asked to.
#[test]
fn ipv4_with_ipv6_next_hops_is_in_use_only_if_both_offer_it() {
let offer = Capability::ExtendedNextHop {
entries: vec![IPV4_WITH_IPV6_NEXT_HOP],
};
let outcome = |ours: bool, theirs: Vec<Capability>| {
let (mut fsm, first) = Fsm::new(FsmConfig {
extended_next_hop: ours,
..config()
});
let Action::Send(open) = &first[0] else {
panic!("expected an OPEN first, got {first:?}");
};
assert_eq!(capabilities(open).contains(&&offer), ours);
let mut offered = vec![
Capability::multiprotocol(AfiSafi::IPV4_UNICAST),
Capability::multiprotocol(AfiSafi::IPV6_UNICAST),
];
offered.extend(theirs);
let open = OpenMessage {
options: vec![OpenOption::Capabilities(offered)],
..peer_open(90)
};
match &fsm.handle(received(Message::Open(open)))[..] {
[Action::Admit(negotiated), ..] => negotiated.extended_next_hop,
other => panic!("expected the session to go on, got {other:?}"),
}
};
assert!(outcome(true, vec![offer.clone()]));
assert!(!outcome(true, vec![]));
assert!(!outcome(false, vec![offer.clone()]));
// Offered for something else: IPv6 routes with IPv4 next hops,
// say.
let other = Capability::ExtendedNextHop {
entries: vec![ExtendedNextHopEntry {
afi: Afi::IPV6,
safi: Safi::UNICAST,
nh_afi: Afi::IPV4,
}],
};
assert!(!outcome(true, vec![other]));
}
/// RFC 9234 §4.1: our OPEN says what we are to the peer, which is the
/// other half of what the peer is to us.
#[test]
fn our_open_says_what_we_are_to_the_peer() {
for (peer_is, we_are) in [
(PeerRole::Provider, BgpRole::Customer),
(PeerRole::Customer, BgpRole::Provider),
(PeerRole::Peer, BgpRole::Peer),
(PeerRole::RouteServer, BgpRole::RouteServerClient),
(PeerRole::RouteServerClient, BgpRole::RouteServer),
] {
let (_, actions) = Fsm::new(FsmConfig {
role: Some(peer_is),
..config()
});
let Action::Send(open) = &actions[0] else {
panic!("expected an OPEN first, got {actions:?}");
};
let roles: Vec<_> = capabilities(open)
.into_iter()
.filter(|capability| matches!(capability, Capability::BgpRole { .. }))
.collect();
assert_eq!(
roles,
[&Capability::BgpRole { role: we_are }],
"{peer_is:?}"
);
}
// Nothing is said where nothing is configured.
let (_, actions) = Fsm::new(config());
let Action::Send(open) = &actions[0] else {
panic!("expected an OPEN first, got {actions:?}");
};
assert!(
!capabilities(open)
.iter()
.any(|capability| matches!(capability, Capability::BgpRole { .. }))
);
}
/// RFC 9234 §4.2.
#[test]
fn a_peer_that_says_it_is_something_else_than_configured_is_refused() {
let mismatch = Err(BgpError::OpenMsg(OpenMsgSubcode::RoleMismatch));
let all = [
BgpRole::Provider,
BgpRole::RouteServer,
BgpRole::RouteServerClient,
BgpRole::Customer,
BgpRole::Peer,
];
for configured in [
PeerRole::Provider,
PeerRole::Customer,
PeerRole::Peer,
PeerRole::RouteServer,
PeerRole::RouteServerClient,
] {
for theirs in all {
let expected = if theirs == configured.theirs() {
Ok(true)
} else {
mismatch.clone()
};
assert_eq!(
roles_meet(Some(configured), false, &[theirs]),
expected,
"configured as {configured:?}, says {theirs:?}"
);
}
// A role nobody has defined is not the one configured.
assert_eq!(
roles_meet(Some(configured), false, &[BgpRole::Unknown(9)]),
mismatch
);
}
}
#[test]
fn a_peer_that_does_not_say_what_it_is_is_refused_only_if_it_must() {
let mismatch = Err(BgpError::OpenMsg(OpenMsgSubcode::RoleMismatch));
// A peer that knows nothing of roles is taken at our word for it,
assert_eq!(roles_meet(Some(PeerRole::Customer), false, &[]), Ok(false));
// unless it is required to say.
assert_eq!(roles_meet(Some(PeerRole::Customer), true, &[]), mismatch);
assert_eq!(
roles_meet(Some(PeerRole::Customer), true, &[BgpRole::Customer]),
Ok(true)
);
// What a peer says where nothing is configured confirms nothing.
assert_eq!(roles_meet(None, false, &[BgpRole::Provider]), Ok(false));
}
#[test]
fn a_peer_that_says_two_things_is_refused() {
let mismatch = Err(BgpError::OpenMsg(OpenMsgSubcode::RoleMismatch));
// The same thing twice is the same thing.
assert_eq!(
roles_meet(Some(PeerRole::Peer), false, &[BgpRole::Peer, BgpRole::Peer]),
Ok(true)
);
// Two things, one of them right, is not.
assert_eq!(
roles_meet(
Some(PeerRole::Peer),
false,
&[BgpRole::Peer, BgpRole::Customer]
),
mismatch
);
// Not even where we have nothing configured to compare with.
assert_eq!(
roles_meet(None, false, &[BgpRole::Peer, BgpRole::Customer]),
mismatch
);
}
#[test]
fn an_unacceptable_open_is_refused_with_the_matching_subcode() {
let refused = |open: OpenMessage, config: FsmConfig| {
@@ -1342,29 +957,7 @@ mod tests {
let actions = fsm.handle(received(Message::KeepAlive));
assert!(matches!(&actions[..], [Action::ResetHoldTimer(d)] if *d == hold));
let withdrawal = UpdateMessage {
withdrawn_routes: vec![IpPrefix::try_from("192.0.2.0/24").unwrap().into()],
..empty_update()
};
let actions = fsm.handle(received(Message::Update(withdrawal)));
assert!(matches!(
&actions[..],
[Action::ResetHoldTimer(d), Action::DeliverUpdate { .. }] if *d == hold
));
// An UPDATE with nothing in it is the End-of-RIB marker for IPv4
// (RFC 4724 §2), and has no routes to deliver.
let actions = fsm.handle(received(Message::Update(empty_update())));
assert!(matches!(
&actions[..],
[Action::ResetHoldTimer(d), Action::EndOfRib(AfiSafi::IPV4_UNICAST)] if *d == hold
));
// Unless the decoder emptied it for errors in it (RFC 7606).
let actions = fsm.handle(Event::Message(Received {
message: Message::Update(empty_update()),
attributes_wire: None,
repaired: true,
}));
assert!(matches!(
&actions[..],
[Action::ResetHoldTimer(d), Action::DeliverUpdate { .. }] if *d == hold
@@ -1438,188 +1031,4 @@ mod tests {
assert!(fsm.handle(event).is_empty());
}
}
// ---- graceful restart (RFC 4724, RFC 8538) ----------------------------
const OFFER: Offer = Offer {
restart_time: 120,
restarted: true,
forwarding_preserved: false,
};
/// A peer's capability: its restart time, N bit and the families with
/// their F bits.
fn peer_restart(notification: bool, entries: &[(AfiSafi, bool)]) -> Capability {
Capability::GracefulRestart {
restart_state: false,
restart_time: 90,
notification,
entries: entries
.iter()
.map(
|(family, forwarding)| bgp_packet::open::GracefulRestartEntry {
afi: family.afi,
safi: family.safi,
preserve_forwarding: *forwarding,
},
)
.collect(),
}
}
/// A machine offering graceful restart, Established with a peer that
/// speaks both families and offers it as `restart` says, if at all.
fn restarting_fsm(restart: Option<Capability>) -> Fsm {
let (mut fsm, _) = Fsm::new(FsmConfig {
graceful_restart: Some(OFFER),
..config()
});
let mut capabilities: Vec<Capability> = SUPPORTED
.iter()
.map(|family| Capability::multiprotocol(*family))
.collect();
capabilities.extend(restart);
fsm.handle(received(Message::Open(OpenMessage {
options: vec![OpenOption::Capabilities(capabilities)],
..peer_open(90)
})));
fsm.handle(received(Message::KeepAlive));
assert_eq!(fsm.state(), State::Established);
fsm
}
#[test]
fn graceful_restart_is_offered_with_notifications_for_every_family() {
let (_, actions) = Fsm::new(FsmConfig {
graceful_restart: Some(OFFER),
..config()
});
let Action::Send(open) = &actions[0] else {
panic!("the OPEN comes first: {actions:?}");
};
let offered = capabilities(open)
.into_iter()
.find(|c| matches!(c, Capability::GracefulRestart { .. }))
.cloned();
assert_eq!(
offered,
Some(Capability::GracefulRestart {
restart_state: true,
restart_time: 120,
notification: true,
entries: SUPPORTED
.iter()
.map(|family| bgp_packet::open::GracefulRestartEntry {
afi: family.afi,
safi: family.safi,
preserve_forwarding: false,
})
.collect(),
})
);
// Not offered where it is not configured.
let (_, actions) = Fsm::new(config());
let Action::Send(open) = &actions[0] else {
panic!("the OPEN comes first: {actions:?}");
};
assert!(
!capabilities(open)
.iter()
.any(|c| matches!(c, Capability::GracefulRestart { .. }))
);
}
#[test]
fn graceful_restart_is_agreed_for_the_families_the_peer_lists() {
let fsm = restarting_fsm(Some(peer_restart(
true,
&[
(AfiSafi::IPV6_UNICAST, true),
(AfiSafi::new(Afi::L2VPN, Safi::EVPN), true),
],
)));
let agreed = fsm.negotiated().unwrap().graceful_restart.clone().unwrap();
assert_eq!(agreed.restart_time, 90);
assert!(agreed.notification);
// A family the session does not carry is left out.
assert_eq!(agreed.families, only(AfiSafi::IPV6_UNICAST));
assert_eq!(agreed.forwarding, only(AfiSafi::IPV6_UNICAST));
// Not agreed where only one side offers it.
let fsm = restarting_fsm(None);
assert_eq!(fsm.negotiated().unwrap().graceful_restart, None);
let mut fsm = fsm_in(State::OpenSent, 90);
fsm.handle(received(Message::Open(OpenMessage {
options: vec![OpenOption::Capabilities(vec![peer_restart(true, &[])])],
..peer_open(90)
})));
assert_eq!(fsm.negotiated().unwrap().graceful_restart, None);
}
/// Leaving to restart is closing the connection without a word, which
/// a peer that agreed takes as a restart; without the agreement, and
/// when leaving for good, the peer is told, with a Hard Reset where it
/// would otherwise keep our routes.
#[test]
fn leaving_says_goodbye_as_graceful_restart_has_it() {
let shutdown = BgpError::Cease(CeaseSubcode::AdminShutdown);
let hard_reset = BgpError::Cease(CeaseSubcode::HardReset);
let both = &[(AfiSafi::IPV4_UNICAST, true), (AfiSafi::IPV6_UNICAST, true)];
for (restart, leave, expected) in [
(Some(peer_restart(true, both)), Leave::Restart, None),
(Some(peer_restart(false, both)), Leave::Restart, None),
(None, Leave::Restart, Some(shutdown.clone())),
(
Some(peer_restart(true, both)),
Leave::Stop,
Some(hard_reset.clone()),
),
(
Some(peer_restart(false, both)),
Leave::Stop,
Some(shutdown.clone()),
),
(None, Leave::Stop, Some(shutdown.clone())),
] {
let mut fsm = restarting_fsm(restart.clone());
let actions = fsm.handle(Event::Leave(leave));
let (sent, result) = ending(&fsm, actions);
assert_eq!(sent, expected, "{leave:?} with {restart:?}");
assert!(result.is_ok());
}
// Before Established there is nothing to keep.
let mut fsm = fsm_in(State::OpenConfirm, 90);
let actions = fsm.handle(Event::Leave(Leave::Restart));
assert_eq!(ending(&fsm, actions).0, Some(shutdown));
}
/// With NOTIFICATIONs covered (RFC 8538), one for an error of ours is a
/// Hard Reset, with the error in its data; an expired hold timer is
/// not.
#[test]
fn errors_end_the_session_with_a_hard_reset_where_notifications_restart() {
let both = &[(AfiSafi::IPV4_UNICAST, true), (AfiSafi::IPV6_UNICAST, true)];
let mut fsm = restarting_fsm(Some(peer_restart(true, both)));
let actions = fsm.handle(Event::LocalFailure(SessionError::PrefixLimitExceeded {
limit: 10,
}));
let Action::Send(Message::Notification(sent)) = &actions[0] else {
panic!("a NOTIFICATION first: {actions:?}");
};
assert_eq!(sent.error, BgpError::Cease(CeaseSubcode::HardReset));
assert_eq!(sent.data, [6, 1]);
let mut fsm = restarting_fsm(Some(peer_restart(true, both)));
let actions = fsm.handle(Event::HoldTimerExpired);
assert_eq!(ending(&fsm, actions).0, Some(BgpError::HoldTimer));
let mut fsm = restarting_fsm(Some(peer_restart(false, both)));
let actions = fsm.handle(Event::LocalFailure(SessionError::PrefixLimitExceeded {
limit: 10,
}));
assert_eq!(
ending(&fsm, actions).0,
Some(BgpError::Cease(CeaseSubcode::MaxPrefixes))
);
}
}
crates/server/src/holddown.rs
-174
View File
@@ -1,174 +0,0 @@
//! Peers that are being kept down for a while: those that went over a
//! limit, ours or theirs, and those whose sessions keep failing.
use std::collections::HashMap;
use std::sync::Mutex;
use std::time::{Duration, Instant};
use crate::address::PeerAddress;
use crate::sync::IgnorePoison;
/// Why a peer is being kept down.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum HeldDown {
/// It went over its limit of routes.
OverLimit,
/// We were to announce it more routes than its export limit allows.
OverExportLimit,
/// Its sessions keep coming up and failing.
Flapping,
}
impl HeldDown {
pub fn as_str(self) -> &'static str {
match self {
Self::OverLimit => "held down after going over its limit of routes",
Self::OverExportLimit => {
"held down after more routes were to be advertised to it than its export limit allows"
}
Self::Flapping => "held down because its sessions keep failing",
}
}
}
/// A session that fails sooner than this after coming up counts as a
/// flap.
const FLAP_WITHIN: Duration = Duration::from_secs(60);
/// This many flaps in a row are let pass: a router being restarted, a
/// cable put back.
const FLAPS_ALLOWED: u32 = 2;
/// How long a peer is kept down after the first flap too many; twice as
/// long after each one more, up to `FLAP_HOLD_MAX`.
const FLAP_HOLD: Duration = Duration::from_secs(1);
const FLAP_HOLD_MAX: Duration = Duration::from_secs(300);
/// Peers that are being kept down, and until when: those that went over
/// their limit, and those whose sessions keep failing. Shared by the
/// sessions, which put peers here, and by the dialer and the listeners,
/// which leave them alone meanwhile.
///
/// Every session that comes up and goes costs a look at every route in
/// the table, and tells every other peer of the changes. A peer that
/// does so over and over, by fault or by design, is slowed down.
#[derive(Debug, Default)]
pub struct HoldDowns {
held: Mutex<HashMap<PeerAddress, (Instant, HeldDown)>>,
/// How many times in a row each peer's session has failed soon after
/// coming up.
flaps: Mutex<HashMap<PeerAddress, u32>>,
}
impl HoldDowns {
/// Keeps the peer at `address` down for `duration` from now.
pub fn hold(&self, address: PeerAddress, duration: Duration, why: HeldDown) {
let until = Instant::now() + duration;
let mut held = self.held.locked();
// Not for less long than it already is.
if held
.get(&address)
.is_none_or(|(already, _)| *already < until)
{
held.insert(address, (until, why));
}
}
/// How much longer the peer at `address` is kept down, if it is, and
/// why.
pub fn remaining(&self, address: PeerAddress) -> Option<(Duration, HeldDown)> {
let mut held = self.held.locked();
let (until, why) = *held.get(&address)?;
let remaining = until.saturating_duration_since(Instant::now());
if remaining.is_zero() {
held.remove(&address);
return None;
}
Some((remaining, why))
}
/// A session with the peer at `address` that had come up has ended,
/// `lasted` after it did; `failed` if not at our own wish. Returns for
/// how long the peer is kept down for it, if it is.
pub fn session_ended(
&self,
address: PeerAddress,
lasted: Duration,
failed: bool,
) -> Option<Duration> {
let mut flaps = self.flaps.locked();
if !failed || lasted >= FLAP_WITHIN {
flaps.remove(&address);
return None;
}
let count = flaps.entry(address).or_insert(0);
*count += 1;
let over = count.checked_sub(FLAPS_ALLOWED + 1)?;
let hold = FLAP_HOLD
.checked_mul(1u32.checked_shl(over).unwrap_or(u32::MAX))
.map_or(FLAP_HOLD_MAX, |hold| hold.min(FLAP_HOLD_MAX));
drop(flaps);
self.hold(address, hold, HeldDown::Flapping);
Some(hold)
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn a_peer_is_held_down_for_as_long_as_was_said() {
let held = HoldDowns::default();
let address = PeerAddress::from("192.0.2.1".parse::<std::net::IpAddr>().unwrap());
let other = PeerAddress::from("192.0.2.2".parse::<std::net::IpAddr>().unwrap());
// Two failures soon after coming up are let pass; from the third
// the peer is kept down, twice as long each time, up to a limit.
let flaps = HoldDowns::default();
let soon = Duration::from_secs(5);
assert_eq!(flaps.session_ended(address, soon, true), None);
assert_eq!(flaps.session_ended(address, soon, true), None);
assert_eq!(flaps.session_ended(address, soon, true), Some(FLAP_HOLD));
assert_eq!(
flaps.session_ended(address, soon, true),
Some(2 * FLAP_HOLD)
);
assert_eq!(
flaps.remaining(address).map(|(_, why)| why),
Some(HeldDown::Flapping)
);
assert_eq!(flaps.remaining(other), None, "each peer for itself");
for _ in 0..40 {
flaps.session_ended(address, soon, true);
}
assert_eq!(
flaps.session_ended(address, soon, true),
Some(FLAP_HOLD_MAX)
);
// A session that lasted, or that we ended ourselves, starts the
// count over.
assert_eq!(flaps.session_ended(address, FLAP_WITHIN, true), None);
assert_eq!(flaps.session_ended(address, soon, true), None);
assert_eq!(flaps.session_ended(address, soon, false), None);
assert_eq!(flaps.session_ended(address, soon, true), None);
assert_eq!(held.remaining(address), None);
held.hold(address, Duration::from_secs(600), HeldDown::OverLimit);
let (remaining, why) = held.remaining(address).expect("held");
assert!(remaining > Duration::from_secs(590), "{remaining:?}");
assert_eq!(why, HeldDown::OverLimit);
assert_eq!(held.remaining(other), None);
// Not made shorter by a lesser reason.
held.hold(address, Duration::from_secs(1), HeldDown::Flapping);
let (remaining, why) = held.remaining(address).expect("held");
assert!(remaining > Duration::from_secs(590), "{remaining:?}");
assert_eq!(why, HeldDown::OverLimit);
held.hold(other, Duration::ZERO, HeldDown::OverLimit);
assert_eq!(held.remaining(other), None);
assert_eq!(
held.held.locked().len(),
1,
"a hold that is over is forgotten"
);
}
}
crates/server/src/lib.rs
+4 -13
View File
@@ -1,27 +1,18 @@
pub mod address;
pub mod advertise;
pub mod collision;
pub mod config;
mod connection;
mod dialer;
pub mod connection;
pub mod dialer;
pub mod error;
pub mod export;
pub mod family;
mod fsm;
pub(crate) mod holddown;
pub mod marks;
pub mod fsm;
pub mod metrics;
mod panic;
pub mod policy;
mod reach;
pub mod rejections;
pub mod restart;
mod routes;
pub mod rpki;
pub mod server;
pub mod session;
mod sync;
mod throttle;
pub mod tracing;
pub mod proto {
include!(concat!(env!("OUT_DIR"), "/protos/mod.rs"));
Loaded 100 of 202 files, more files were not shown because too many files have changed in this diff. Show more