rayhaan/ferrix
0
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add a documentation website, built with Zola
ci / check (push) Canceled after 0s
Details
ci / docs (push) Canceled after 0s
Details
ci / interop (push) Canceled after 0s
Details
ci / fuzz (push) Canceled after 0s
Details

Browse Source 
The documentation moves from READMEs into a site in docs/: getting
started, a user guide, operations, a reference, and development pages,
written in the style of network vendors' documentation (docs/STYLE.md).
bin/bgp/README.md and docs/RFCs.md are folded into it.

- A theme of its own in the International Typographic Style, in blues,
  with self-hosted Inter and JetBrains Mono, search over Zola's index,
  dark mode, and no resources loaded from other hosts.
- Example configurations in docs/examples/, shown on the pages and
  checked with `bgp --check` in CI.
- A test that fails when a field of config.proto is missing from the
  configuration reference.
- A CI job that builds the site; Zola fails on broken links and anchors.
- A hardened systemd unit, tried under systemd with kernel routes on.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Rayhaan JaufeerallyandClaude Opus 5.5 committed 2026-10-03 15:12:43 +02:00
1 parent 9820a5d3bc
commit dc275f99d7
61 files changed
+5632 -623
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

bin/bgp/README.md
+11 -554
View File
@@ -1,560 +1,17 @@
# BGP
# bgp
This crate provides the `bgp` binary, which runs the Border Gateway Protocol
daemon together with its read-only HTTP status interface.
The daemon: the BGP speaker, the RTR client, the kernel route installer and
the status page, in one binary.
```
cargo run -- path/to/config.textpb
cargo run -p bgp -- --check docs/examples/quick-start.textpb
cargo run -p bgp -- docs/examples/quick-start.textpb
```
`--check` reads the configuration, says whether it is in order, and stops
without listening on anything or touching a route: for trying a change
before the daemon is restarted with it.
How to configure, run and operate it is in the documentation, which is
built from `docs/` (`zola --root docs serve`):
The configuration format is defined in `crates/server/proto/config.proto`;
`testdata/server.textpb` is a worked example. The daemon is configured
entirely through that file: peers, and the routes it originates
(`originate { prefix: ... }`). There is no way to change either at runtime.
When `http_endpoint` is set the binary also serves:
| Path | Purpose |
|---|---|
| `GET /statusz` | Status page: configuration, peers and session state, originated and received routes, live event log |
| `GET /api/status` | Router ID, ASN, uptime and table sizes |
| `GET /api/config` | The configuration file as read at startup |
| `GET /api/peers` | Configured peers and established sessions |
| `GET /api/routes/local` | Originated routes |
| `GET /api/routes/received` | Routes learned from peers; see "Finding a route" |
| `GET /api/routes/rejected` | Routes peers announced that were refused, by peer |
| `GET /api/rpki` | Validation with the RPKI: the cache, the data and its age |
| `GET /api/kernel` | The routes in the kernel: how many, and why the others are not; the machine's interfaces |
| `GET /api/events` | Server-sent events for session and route changes |
| `GET /metricz` | Prometheus metrics |
Everything under it is read-only and unauthenticated. Bind it to loopback or
put it behind a reverse proxy.
## What it does today
It establishes sessions (both dialling configured peers and accepting
connections, with RFC 4271 collision detection), announces its originated
routes to every peer with itself as next hop, judges what peers send by each
peer's import policy, keeps what passes, chooses a best route for each
prefix, and on Linux can put those best routes in the kernel so that the
machine forwards by them. Peers whose `export` says so are told the best of
what the others announced, as the roles of RFC 9234 allow.
## Who may connect
Connections are taken only from the addresses of configured peers, and from
each of those only a few at a time. Anything else is closed unread and
counted in `bgp_connections_refused_total`. `accept_unconfigured_peers: true`
lets any host open a session as any AS; it exists for tests on a closed
network.
## What is taken from a peer
Every route a peer announces is judged before it goes into the routing
table (`import` in a peer's configuration; see `config.proto`):
- Refused for any peer unless turned off: prefixes that are not routed on
the Internet (private, documentation, multicast, reserved), prefixes longer
than /24 or /48, a default route, paths with a private or reserved AS
number, paths that do not begin with the peer's own AS, and paths with an
AS_SET in them (RFC 9774).
- The next hop has to be the peer itself (`accept_third_party_next_hop`
turns that off; a peer whose role is `ROUTE_SERVER` is exempt). A next
hop is where our traffic will be sent: a peer may draw it to itself, not
point it at another member of an exchange or at a machine on some other
network of ours.
- The prefixes this router originates, and anything inside them, are not
taken from another network, unless the peer's `prefixes` list names them
(a customer with part of our space).
- Always refused: a path with our own AS number in it, and a next hop that
is no address to send to (unspecified, loopback, multicast, our own).
- If configured: a list of the prefixes the peer may announce and of the AS
numbers its routes may originate from.
- `max_prefixes`: one route more ends the session with Cease / Maximum
Number of Prefixes Reached and keeps the peer down for ten minutes.
A configured peer is one router: a second connection from its address
under another BGP Identifier is refused, so the limits above are not
multiplied by opening more sessions. And a peer whose sessions keep coming
up and failing within a minute is kept down, from the third time for a
second and for twice as long each time after, up to five minutes: every
such session costs a look through the whole table.
A refused route is dropped, along with the route it would have replaced.
`bgp_routes_rejected_total` counts them by reason, and the status page shows
how many routes each peer has and how many were refused.
## Passing routes on
A peer whose `export` has `learned_routes: true` is told the best route of
every prefix the others announced, not only what we originate. The rules
are the usual ones and cannot be turned off: not back to the peer a route
came from, not to a peer whose AS is on its path already, not from one
peer in our own AS to another, not past NO_EXPORT or NO_ADVERTISE. A
route goes out with our AS added to its path and ourselves as next hop,
without the MED it came with; to a peer in our own AS the path is as it
was and LOCAL_PREF goes along; to a `ROUTE_SERVER_CLIENT` path, MED and
(where the client can use it) next hop are left as they came, as a route
server sends them (RFC 7947).
**What goes where is decided by roles (RFC 9234).** A route learned from a
provider, a peer or a route server is marked "only to customer" on the way
in and goes on to customers only; a customer's route goes to everyone. So
with roles on every peer, `learned_routes` makes us transit for our
customers and nothing else. A peer without a role has no such rule, and is
told every route; the daemon warns about that at startup. The top-level
`export` is for every peer that has none of its own:
```
export { learned_routes: true }
peers { name: "customer-a" role: CUSTOMER ... }
peers { name: "transit-a" role: PROVIDER ... }
```
**Marks.** `import` puts communities on every route taken from a peer, and
takes some off first; `export` does the same for every route announced to
one. Where a route was learned, which router it went through:
```
peers {
name: "ixp-a-rs"
import {
remove_communities: "64601:*" # nobody else's word for our marks
add_communities: "64601:1001" # learned at IXP A
add_large_communities: "64601:1:1001"
local_pref: 200
}
}
export { add_large_communities: "64601:2:1" } # went through router 1
```
Patterns for `remove_communities` are `64601:100`, `64601:*`, `*:*`, or
with three parts for large communities. `import.local_pref` is what route
selection goes by: of two routes for a prefix the one with the higher is
chosen, whatever else they have.
**Control communities.** With `control_communities: true`, a route's
communities say which peers it is passed on to, in the form Internet
exchanges' route servers use (`L` our AS, `P` a peer's):
| Standard | Large | |
|---|---|---|
| `0:P` | `L:0:P` | not to the peer with AS `P` |
| `L:P` | `L:1:P` | to the peer with AS `P`, and then only to peers named so |
| `0:L` | `L:0:0` | to nobody |
"Not" wins over "to"; a route with none of them goes to everyone it
otherwise would. Standard communities only have room for AS numbers below
65536; the large forms work for any. They are taken off a route as it is
passed on, and they hold for routes we originate too. With this on,
standard communities `L:n` mean "to the peer with AS n", so mark routes
with large communities (`L:2:n` and up) instead.
**Cost.** A decision is a few comparisons on the route's communities and
path. Routes that go out alike go in one UPDATE: changes are gathered for
50 ms before they go, and a peer that is given the whole table (when it
comes up, or asks for it again) has it gathered first, so that the
prefixes that share attributes, which are spread over the table's shards,
go together. At most four sessions gather a table at once; the others wait
their turn. `cargo bench -p bgp-bench --bench transit`, with one peer
sending 1.23 million routes over a session and another being told of them,
on a laptop: in our table after 0.47 s, all of it heard by the other peer
after 2.1 s, in UPDATEs of four routes each on average.
**Not there yet.** Per-client best path selection for a route server (every
client is told the one best route); ADD-PATH for learned routes (only the
best is sent); route reflection; a way to send pre-built messages to
several peers at once.
## Route origin validation (RPKI)
With `rpki { caches { address: "..." port: 3323 } }` in the configuration,
the daemon keeps the validated ROA payloads of a validator (Routinator,
rpki-client with StayRTR, FORT) over the RPKI-to-Router protocol, and
validates every route from peers outside its own AS (RFC 6811):
- **invalid** (the holder of the addresses authorised another origin, or a
shorter prefix): refused, like any route the import policy refuses, unless
the peer has `accept_rpki_invalid: true`;
- **valid** and **not found**: taken. Validity does not otherwise change
which route is chosen.
The data changes all the time, and the routes follow without the peer having
to send them again: a route in the table that becomes invalid is taken out,
and a route refused as invalid is kept aside ("held") and taken when it no
longer is. If no cache confirms the data for the expire interval (two hours
unless the cache or the configuration says otherwise) it is dropped, and
nothing is invalid any more.
The status page shows the cache, how fresh the data is, each route's
validity, and how many of each peer's routes are held. `/api/rpki` has the
same as JSON; `bgp_rpki_vrps{family}`, `bgp_rpki_aspas` and
`bgp_rpki_updates_total` are on `/metricz`. Refused routes are counted under
the reasons `RPKI invalid` and `ASPA invalid`.
The connection to the cache is plain TCP, without authentication: run the
validator on the same host or on a network you trust.
## Roles and route leaks (RFC 9234)
`role` on a peer says what the peer is to us: `PROVIDER`, `CUSTOMER`, `PEER`,
`ROUTE_SERVER` or `ROUTE_SERVER_CLIENT`. It is not for peers in our own AS.
With it:
- **The two ends have to agree.** Our OPEN says what we are to the peer (its
provider, if it is our customer). A peer whose OPEN says it is something
else than configured is refused with OPEN Message Error / Role Mismatch:
one side thinking "customer" and the other "peer" is how leaks happen. A
peer that says nothing (it does not implement the RFC) is accepted, unless
`require_role: true`. The status page shows the role, with a tick when the
peer confirmed it.
- **Routes are marked "only to customer"** (the OTC attribute) when they come
in from a provider, a peer or a route server, and when ours go out to a
customer, a peer or a route server client. The mark stays on the route.
- **A marked route coming the wrong way is a leak and is refused**: from a
customer or a route server client always, and from a peer unless the peer
put the mark there itself. The reason shown is `route leak (Only to
Customer)`. No setting turns this off, as the RFC requires.
This catches leaks even where nobody on the path has an ASPA, as long as the
networks on either side of the leaker mark their routes.
## AS path verification (ASPA)
Origin validation says who may originate a prefix. It says nothing about the
rest of the path, so it does not catch a route leak (a network passing its
provider's or peer's routes on to another provider or peer), nor a hijack
that keeps the right origin at the end of a made-up path. ASPAs do: in one,
a network lists its providers, and a path that goes from that network to
anyone else, where only a provider could come next, is invalid
(draft-ietf-sidrops-aspa-verification).
Two things are needed for paths to be verified:
- a validator that hands out ASPAs, which takes version 2 of the
RPKI-to-Router protocol (draft-ietf-sidrops-8210bis). The daemon asks for
version 2 and falls back to 1 or 0 if that is what the validator speaks;
the status page says which is in use, and warns when it is one without
ASPAs;
- `role` on the peer, saying what the peer is to us, because what a path may
look like depends on who hands it over:
| `role` | The peer | Its paths |
|--------|----------|-----------|
| `CUSTOMER` | buys transit from us | must lead up through providers all the way |
| `PEER` | exchanges its own and its customers' routes with us | the same |
| `ROUTE_SERVER` | is an exchange's route server | the same; the path need not begin with the route server's AS |
| `ROUTE_SERVER_CLIENT` | is a client of ours, where we are the route server | the same |
| `PROVIDER` | sells us transit | may lead up, across one peering, and down |
A peer without a role has its origins validated and its paths not.
A path is **invalid** (refused, unless the peer has
`accept_aspa_invalid: true`), **valid**, or **unknown**, which is what most
paths are while few networks have an ASPA; unknown ones are taken. A path
with an AS_SET in it is invalid, and so is one that does not begin with the
peer's AS (other than from a route server). As with origins, routes follow
the data: held while invalid, taken when they no longer are.
```
rpki { caches { address: "127.0.0.1" port: 3323 } }
peers {
name: "transit"
endpoint { address: "192.0.2.1" }
remote_asn: 64496
role: PROVIDER
}
```
## IPv4 over a link that has only IPv6 addresses (RFC 8950)
A session over IPv6 carries IPv4 routes too, with IPv6 next hops, if the
peer agrees: the daemon offers the Extended Next Hop capability on every
session over IPv6 whose address families include IPv4 unicast, and nothing
has to be configured for it on this side. (On FRR it is `neighbor ...
capability extended-nexthop`, on BIRD `extended next hop on` in the IPv4
channel.) The peers table on the status page says "IPv4 with IPv6 next
hops" for a session that agreed on it.
Our IPv4 routes then go out with the session's own IPv6 address as next
hop, and the peer's arrive with its IPv6 address, or with a global and a
link-local one. In the kernel such a route is an IPv4 route by way of an
IPv6 neighbor (`10.2.0.0/24 via inet6 2001:db8::2 dev eth0`, Linux 5.2 and
later).
## Peers at link-local addresses
A link with no addresses but the link-local ones every interface has
(`fe80::/10`) is enough for a session. Such an address means something
only on its link, so the peer is the address and the interface together:
```
peers {
name: "upstream"
endpoint { address: "fe80::1%eno3" }
remote_asn: 64602
}
```
- The interface is looked up by its name each time it is needed: one that
is not there yet, or is made anew with another number (a tunnel that is
restarted), is found when it is. Until then the peer's state says there
is no such interface.
- The same address on two interfaces is two peers. A link-local address
without an interface is refused, as is an interface on any other
address.
- To take the peer's connections the daemon must listen on `::` (as it
does by default), not on one global address.
- Both address families go over the session (IPv4 as above, if the peer
agrees). Our routes go with our link-local address as next hop, in both
places MP_REACH_NLRI has for one, as FRR sends its own. The peer's next
hop is taken in any of the forms in use: the link-local address alone,
twice, or after `::`. In the kernel the routes go by way of that address
on the session's interface (`10.2.0.0/24 via inet6 fe80::1 dev eno3`).
- `local_address`, if set to a link-local address of ours, is taken to be
on the peer's interface.
With FRR, configure the peer by interface (`neighbor eno3 interface
remote-as ...`, "BGP unnumbered"). FRR then learns who is on the link from
IPv6 router advertisements, which this daemon does not send: run `radvd`
(or the like) on the interface beside it. FRR's older `neighbor fe80::1
interface eno3` brings the session up but FRR 10.5 then takes every route
from us for unreachable. BIRD needs only `neighbor fe80::1%eno3 as ...`.
## Forwarding: routes in the kernel (Linux)
With `kernel {}` in the configuration the daemon puts the best route of every
prefix in the kernel's routing table, over netlink, and keeps it there as the
best routes change. Without it the daemon only speaks BGP and the machine's
forwarding is not touched.
```
kernel {
table: 254 # the main table; the default
metric: 20 # the default
}
```
- **What is installed.** A best route whose next hop is a neighbor: an
address in the network of one of the machine's interfaces that is up. For
an IPv6 route with a link-local next hop, the interface is the one the
peer is a neighbor on. A route whose next hop is further away (a route
from a peer in our own AS that did not set itself as next hop, say)
cannot be installed: looking up the way to such a next hop through other
routes is not done yet.
- **Routes that cannot be installed are not chosen.** With `kernel {}`, a
route whose next hop cannot be reached is not taken into the table, so
that it cannot be the best route of its prefix while another peer has
one that works (RFC 4271 §9.1.2.1). It is held aside, counted as refused
for "next hop unreachable" and shown as held on the status page, and
taken as soon as the machine's interfaces change so that it can be
reached; a route in the table whose next hop goes away is held in the
same way. Without `kernel {}` the daemon does not know what can be
reached, and takes every route.
- **Nexthop objects.** Where the kernel has them (Linux 5.3 and later) a
gateway is put there once, as a nexthop object, and routes name it; `ip
route` shows such routes with `nhid`, and `ip nexthop` the objects. The
kernel then checks the gateway once and not for every route, which makes
a full table about a third faster to install, and lets all of them be
removed at once when the daemon stops. The objects the daemon makes are
its own: it does not use anyone else's, puts its own back if one is
removed, and removes them when it stops. `without_nexthop_objects: true`
has every route carry its own gateway instead, as on older kernels.
- **What is the machine's own.** A route for a network the machine is
itself on, or for part of one, is never installed, whatever the policy
let into the table: the kernel reaches those directly. A peer on a shared
LAN that announces part of the LAN would otherwise take the traffic to
its neighbors, and our sessions with them. The status page counts such
routes as "inside a network of this machine".
- **What is ours.** Routes are marked `proto bgp`. Every route with that
mark in the configured table with the configured metric is taken to be
this daemon's, and no other route is ever changed or removed. So do not
run two daemons that install BGP routes into the same table with the same
metric.
- **Starting and stopping.** At startup, routes of ours left by an earlier
run are kept, so that the machine goes on forwarding while the peers send
their tables again; those that no peer has announced again after
`stale_seconds` (120 unless set) are removed. On SIGINT or
SIGTERM the daemon closes its sessions and then removes its routes. After
a crash they stay until the next start.
- **Interfaces.** The daemon learns the machine's interfaces and addresses
from the kernel and is told when they change; a second after the last
change, and five seconds after the first at the latest, it works all
routes out again, since any next hop may resolve differently.
- **When the kernel does not answer.** The kernel says which requests it
refused. If it had more to say than there was room for, or says nothing
for ten seconds, the daemon asks it what it has of our routes and puts
right what is not as it should be.
- **Privileges.** Changing routes needs the `CAP_NET_ADMIN` capability, and
listening on port 179 needs `CAP_NET_BIND_SERVICE`; neither needs root.
Without the first the daemon stops as soon as the kernel refuses it.
```
setcap cap_net_admin,cap_net_bind_service+ep /usr/local/bin/bgp
```
`ip route show proto bgp` shows what is there. `/api/kernel` and the status
page show how many routes are installed, how many are not and why, the last
refusal by the kernel, and the interfaces; `/metricz` has `bgp_kernel_routes`,
`bgp_kernel_routes_unresolved`, `bgp_kernel_routes_refused`,
`bgp_kernel_route_changes_total{change}` and `bgp_kernel_errors_total`.
To forward packets for others the machine also needs
`net.ipv4.ip_forward=1` and `net.ipv6.conf.all.forwarding=1`; the daemon does
not set them.
### How long it takes
A million IPv4 routes, about the size of the Internet's table, go into the
kernel in a little over a second and come out in half a second (release
build, in a virtual machine on an Apple M-series laptop; the test
`many_routes_are_installed_and_removed` with `FERRIX_ROUTES=1000000`).
About three quarters of that is the kernel's own work on each route.
What keeps the rest small: requests go to the kernel hundreds to a write,
with one acknowledgement for each write and not for each route; a task of
its own does the writing, so the next batch is worked out while the kernel
is busy with the last; and what a next hop resolves to is worked out once
for each next hop, not once for each route.
### Testing against a real kernel
The tests that change a kernel's routes are not run by `cargo test`.
`scripts/kernel-tests.sh` runs them on Linux, each in a network namespace
made for it (`scripts/in-namespace.sh` runs any one command that way): the route installer against `ip route`, the daemon with a peer,
and two daemons that route for each other with pings across. From macOS,
`scripts/linux-vm.sh` runs it (or any command, such as `cargo test`) in a
Lima virtual machine that it creates on first use.
## Interoperability
`scripts/interop/lab.sh` runs the daemon against FRR, BIRD and Routinator,
each in a network namespace of its own on one exchange LAN, and checks what
every side ends up with. It needs Linux (`scripts/linux-vm.sh
scripts/interop/lab.sh` from macOS; `scripts/interop/install.sh` installs
the others on Ubuntu). Last run against FRR 10.5.1, BIRD 2.18 and
Routinator 0.15.2. What it covers:
| Phase | What is checked |
|-------|-----------------|
| `basics` | Sessions over IPv4 and IPv6; routes both ways with path, next hop, communities and large communities intact; four-octet AS numbers; ADD-PATH from BIRD; pings across by the installed routes |
| `roles` | RFC 9234 roles agree with FRR (peer) and BIRD (provider); Only to Customer is set where it should be, in both directions |
| `rpki` | RTR version 2 with Routinator; an invalid route from FRR is refused and held, and taken after Routinator restarts with data that allows it |
| `refresh` | Routes added and withdrawn by FRR; FRR's ROUTE-REFRESH answered; BIRD re-sending |
| `flaps` | Five session resets; FRR and BIRD killed and restarted; a cut wire noticed by the hold timer |
| `restart` | The daemon stopped in good order (kernel emptied, peers told) and killed (routes kept, stale one cleared by the next start) |
| `scale` | A table from BIRD (`SCALE_ROUTES`, a million for the Internet's size) into the table and the kernel; the daemon killed and restarted under it; the table withdrawn |
| `limits` | A peer whose role does not fit gets no session; a peer over its prefix limit gets Cease / Maximum Number of Prefixes and is held down |
| `hijacks` | FRR announces our own prefix, a route whose next hop is BIRD, and the exchange LAN's own prefix: none is taken |
| `nexthops` | A better route from FRR whose next hop cannot be reached is held and BIRD's is chosen and installed; they swap when the next hop becomes a neighbor, and swap back when it stops being one |
| `transit` | FRR as our customer and BIRD as our provider: FRR's routes reach BIRD with our AS on the path, BIRD's reach FRR marked only to customer, one of FRR's is kept from BIRD by a community, and packets go between their networks through us |
| `v6only` | The exchange LAN without IPv4 addresses: one IPv6 session with each router for both families, IPv4 routes with IPv6 next hops (RFC 8950) in BGP and in the kernel, IPv4 pings across |
| `linklocal` | The exchange LAN with link-local addresses only, and the same ones on a second interface: sessions with FRR (by interface, with radvd beside the daemon) and BIRD at `fe80::…%ix0`, both families in BGP and in the kernel, pings across, the interface taken down and up |
`scripts/interop/real-rpki.sh` checks the RTR client against a Routinator
that has the real RPKI, ASPAs included: the daemon has to end up with as
many VRPs and ASPAs as the validator says it has.
Not covered: routers other than FRR and BIRD; IPv6 routes over an IPv4
session (the daemon does not send them); an interface that is removed and
made anew under a session; long runs.
## Finding a route
The status page has a look-up box over the received routes, and
`/api/routes/received` takes the same questions:
| Query | Answer |
| --- | --- |
| `?prefix=193.0.0.0/21` | Every route for that prefix, the best first |
| `?address=193.0.0.1` | Every prefix that covers the address, most specific first |
| `?within=193.0.0.0/16` | That prefix and everything inside it |
| `&peer=NAME` | Only routes from that peer (its name or its address) |
| `&best=true` | Only the route chosen for each prefix |
| `&limit=N` | At most N routes (1000 unless given, 100000 at most) |
`within`, `peer` and `best` on their own look through the whole table, which
on a full table takes a moment.
## Seeing and alerting on refused routes
A refused route is not in the table. What is kept instead, for each peer, is
how many were refused for each reason and the latest 200 of them with their
AS paths. The status page lists them under "Refused routes";
`/api/routes/rejected[?peer=NAME]` has them as JSON. The record is kept when
the session ends, so a peer that was cut off can still be looked into. The
log has a line for a peer's first refused route and then at most one a
minute.
For alerting, `/metricz` has:
| Metric | Meaning |
| --- | --- |
| `bgp_routes_rejected_total{peer,reason}` | Routes refused, by peer and why |
| `bgp_peer_routes{peer}` | Routes an established peer has in the table |
| `bgp_peer_max_prefixes{peer}` | The peer's limit, where it has one |
| `bgp_prefix_limit_exceeded_total{peer}` | Sessions ended for going over the limit |
| `bgp_connections_refused_total{reason}` | Connections closed without a session |
| `bgp_routes_exported_total{change}` | Learned routes announced to, and withdrawn from, other peers |
| `bgp_peer_up{peer}` | 1 while the session with a configured peer is Established, else 0 |
| `bgp_build_info{version}` | Always 1; says which version is running |
Rules to start from:
```yaml
groups:
- name: bgp
rules:
# A peer is announcing what its policy does not allow: a leak or a
# hijack on its side, or a list on ours that is out of date.
- alert: BgpRoutesRefused
expr: sum by (peer, reason) (increase(bgp_routes_rejected_total[10m])) > 0
labels: { severity: warning }
# A route that has been through our own AS came back: a loop, or
# someone announcing our AS number.
- alert: BgpOwnAsInPath
expr: increase(bgp_routes_rejected_total{reason="own AS in path"}[10m]) > 0
labels: { severity: page }
# A peer is close to its limit; the limit may need raising before the
# session is cut.
- alert: BgpPeerNearPrefixLimit
expr: bgp_peer_routes / on (peer) bgp_peer_max_prefixes > 0.8
for: 10m
labels: { severity: warning }
- alert: BgpPrefixLimitExceeded
expr: increase(bgp_prefix_limit_exceeded_total[1h]) > 0
labels: { severity: page }
# A configured peer has no session.
- alert: BgpPeerDown
expr: bgp_peer_up == 0
for: 5m
labels: { severity: page }
# Someone is trying to have our own prefixes, or our traffic, go
# where they should not.
- alert: BgpHijackAttempt
expr: increase(bgp_routes_rejected_total{reason=~"prefix is our own|next hop is not the peer"}[10m]) > 0
labels: { severity: page }
# The kernel is refusing routes, or its answers are being lost.
- alert: BgpKernelErrors
expr: increase(bgp_kernel_errors_total[10m]) > 0
labels: { severity: warning }
```
## Trying it without a router
`fake_peer` is a minimal BGP speaker that connects to the daemon, announces a
prefix, prints every UPDATE it receives and, with `--echo`, re-announces each
received prefix back with its own ASN prepended (dropping NO_EXPORT routes).
The prefix it announces shows up in the received table. The ones it echoes
have been through our AS already, so they are refused as loops and counted
in the peer's refused routes:
```
cargo run -- testdata/server.textpb
cargo run -p bgp-integration-tests --example fake_peer -- \
--server 127.0.0.1:179 --asn 65001 --announce 203.0.113.0/24 --echo
```
- [Quick start](../../docs/content/docs/getting-started/quick-start.md)
- [Configuration reference](../../docs/content/docs/reference/configuration.md)
- [Deployment](../../docs/content/docs/operations/deployment.md) and
[monitoring](../../docs/content/docs/operations/monitoring.md)