179 lines
6.7 KiB
Markdown
179 lines
6.7 KiB
Markdown
# Running ferrix_bgp on the production router
|
|
|
|
ferrix and BIRD run side by side on pr01 and peer over `::1`. Students reach the status page through a reverse proxy that adds
|
|
TLS and a password. Nothing here has been run on the router; treat each step
|
|
as something to check.
|
|
|
|
## 1. Build a static Linux binary (on the Mac)
|
|
|
|
The `x86_64-unknown-linux-musl` target and Homebrew's musl cross-linker are
|
|
enough; the router needs no Rust toolchain and no `protoc`.
|
|
|
|
CARGO_TARGET_X86_64_UNKNOWN_LINUX_MUSL_LINKER=x86_64-linux-musl-gcc \
|
|
cargo build --release -p bgp --target x86_64-unknown-linux-musl
|
|
|
|
scp target/x86_64-unknown-linux-musl/release/bgp pr01:/tmp/ferrix-bgp
|
|
|
|
Check the router really is x86-64 first (`uname -m`).
|
|
|
|
## 2. Install on the router
|
|
|
|
install -m 0755 /tmp/ferrix-bgp /usr/local/bin/ferrix-bgp
|
|
install -d /etc/ferrix-bgp
|
|
install -m 0644 ferrix.textpb /etc/ferrix-bgp/ferrix.textpb
|
|
install -m 0644 ferrix-bgp.service /etc/systemd/system/ferrix-bgp.service
|
|
|
|
## 3. Loopback
|
|
|
|
Peering over `::1` needs no address setup. `05-lab-lo.network` only carries
|
|
the AnyIP route for the lab space; merge it into the networkd file that
|
|
matches `lo` if it is not there already, then `networkctl reload`.
|
|
|
|
## 4. BIRD
|
|
|
|
Replace the `protocol bgp ferrix` stanza with `bird-ferrix-local.conf`, then
|
|
`birdc configure check` and `birdc configure timeout 300` / `confirm`.
|
|
|
|
## 5. Start
|
|
|
|
systemctl daemon-reload
|
|
systemctl enable --now ferrix-bgp
|
|
journalctl -u ferrix-bgp -f
|
|
birdc show protocols all ferrix
|
|
|
|
The journal should say `Ping monitor listening for ICMPv6 echo requests`. If
|
|
it says the monitor is disabled, the unit is missing `CAP_NET_RAW`; everything
|
|
else works without it and the status page says why the table is off.
|
|
|
|
To check the monitor before the class, announce a prefix on the page, turn on
|
|
auto-refresh and ping an address inside it from somewhere else:
|
|
|
|
ping -c 3 2a0d:d741:1::1
|
|
|
|
The ping shows up under "Pings to announced addresses" with its source, the
|
|
interface it arrived on and the hop limit it arrived with. Check that
|
|
"arrived on" names the real ingress port (the exchange or transit interface)
|
|
and not `lo`.
|
|
|
|
## 6. Let students in
|
|
|
|
The API has no authentication and announces into a real AS, so do not expose
|
|
port 8080. Put a reverse proxy in front with TLS and a class password. The
|
|
event stream is long-lived, so the proxy must not buffer it.
|
|
|
|
Apache (tested against a local instance: 401 without the password, pages,
|
|
API and POSTs through the proxy, and events arriving within 50 ms):
|
|
|
|
<VirtualHost *:80>
|
|
ServerName lab.example.net
|
|
Redirect permanent "/" "https://lab.example.net/"
|
|
</VirtualHost>
|
|
|
|
<VirtualHost *:443>
|
|
ServerName lab.example.net
|
|
|
|
SSLEngine on
|
|
SSLCertificateFile /etc/letsencrypt/live/lab.example.net/fullchain.pem
|
|
SSLCertificateKeyFile /etc/letsencrypt/live/lab.example.net/privkey.pem
|
|
|
|
# Everything behind one class password.
|
|
<Location "/">
|
|
AuthType Basic
|
|
AuthName "BGP lab"
|
|
AuthUserFile /etc/apache2/ferrix.htpasswd
|
|
Require valid-user
|
|
</Location>
|
|
|
|
# Use Apache's built-in error messages in this vhost. Servers with the
|
|
# multi-language error documents enabled fetch them from /error/,
|
|
# which this vhost would proxy to ferrix; a 401 then comes back as
|
|
# ferrix's empty 404: a blank page and no password prompt. With the
|
|
# built-in ones nothing is fetched, so nothing needs exempting from
|
|
# the login.
|
|
ErrorDocument 400 default
|
|
ErrorDocument 401 default
|
|
ErrorDocument 403 default
|
|
ErrorDocument 404 default
|
|
ErrorDocument 500 default
|
|
ErrorDocument 502 default
|
|
ErrorDocument 503 default
|
|
ErrorDocument 504 default
|
|
|
|
ProxyPreserveHost On
|
|
ProxyPass "/" "http://127.0.0.1:8080/"
|
|
ProxyPassReverse "/" "http://127.0.0.1:8080/"
|
|
|
|
# ferrix has no use for the password; do not pass it on.
|
|
RequestHeader unset Authorization
|
|
|
|
# The status page follows a long-lived event stream. If mod_deflate
|
|
# compresses it, events sit in the compressor and never arrive.
|
|
SetEnvIf Request_URI "^/api/events" no-gzip=1
|
|
</VirtualHost>
|
|
|
|
Modules: `proxy proxy_http ssl headers setenvif auth_basic authn_file`.
|
|
Create the password file with `htpasswd -c -B /etc/apache2/ferrix.htpasswd student`.
|
|
To also limit it to known networks, replace `Require valid-user` with
|
|
|
|
<RequireAll>
|
|
Require valid-user
|
|
Require ip 192.0.2.0/24 2001:db8::/32
|
|
</RequireAll>
|
|
|
|
Caddy:
|
|
|
|
lab.example.net {
|
|
basic_auth {
|
|
student <hash from `caddy hash-password`>
|
|
}
|
|
reverse_proxy 127.0.0.1:8080 {
|
|
flush_interval -1
|
|
}
|
|
}
|
|
|
|
nginx, inside a TLS `server` block:
|
|
|
|
location / {
|
|
auth_basic "BGP lab";
|
|
auth_basic_user_file /etc/nginx/lab.htpasswd;
|
|
proxy_pass http://127.0.0.1:8080;
|
|
proxy_http_version 1.1;
|
|
proxy_buffering off;
|
|
proxy_read_timeout 1h;
|
|
}
|
|
|
|
Restricting the proxy to the university's address ranges as well is cheap
|
|
and worth doing. Only the Apache example has been tested.
|
|
|
|
## 7. Load test before the class
|
|
|
|
`tools/loadtest.py` makes N simulated students announce their /48 at the same
|
|
instant, then verifies the table, ownership under load, the event stream and
|
|
the number of BGP UPDATEs actually sent, and withdraws everything again.
|
|
Python 3 standard library only.
|
|
|
|
tools/loadtest.py 128 --url https://lab.example.net --user student
|
|
|
|
Every test route carries `210036:0:0`, so BIRD accepts it but exports it to no
|
|
neighbour (apart from `home_v6`, which is outside the export control). While
|
|
it runs, `birdc show route protocol ferrix count` should rise to the number of
|
|
students and fall back to what it was. BIRD's `import limit 130` means more
|
|
than 130 lab routes at once are blocked there, by design.
|
|
|
|
## Things to know
|
|
|
|
- The ping monitor only listens. The kernel answers the pings, with or
|
|
without ferrix running, because of the AnyIP route. It sees echo requests
|
|
to the lab space from anyone on the Internet, so source addresses of
|
|
strangers will appear on the page next to the students' own. It remembers
|
|
at most 2048 source and destination pairs and publishes at most 50 events a
|
|
second, so a ping flood costs little; `ping_monitor_echo_requests_total` and
|
|
`ping_monitor_events_suppressed_total` on `/metricz` show what it saw.
|
|
|
|
- Announced routes live in memory. A restart of the service forgets them and
|
|
students have to announce again.
|
|
- Stopping the service sends BIRD a Cease first, so restarts reconnect at
|
|
once.
|
|
- ferrix's own safety net is `local_route_policy`; BIRD's import filter and
|
|
import limit are the second one. Keep both.
|