ferrix/docs/lab/prod/on-router/README.md

Running ferrix_bgp on the production router

ferrix and BIRD run side by side on pr01 and peer over ::1. Students reach the status page through a reverse proxy that adds TLS and a password. Nothing here has been run on the router; treat each step as something to check.

1. Build a static Linux binary (on the Mac)

The x86_64-unknown-linux-musl target and Homebrew's musl cross-linker are enough; the router needs no Rust toolchain and no protoc.

CARGO_TARGET_X86_64_UNKNOWN_LINUX_MUSL_LINKER=x86_64-linux-musl-gcc \
  cargo build --release -p bgp --target x86_64-unknown-linux-musl

scp target/x86_64-unknown-linux-musl/release/bgp pr01:/tmp/ferrix-bgp

Check the router really is x86-64 first (uname -m).

2. Install on the router

install -m 0755 /tmp/ferrix-bgp /usr/local/bin/ferrix-bgp
install -d /etc/ferrix-bgp
install -m 0644 ferrix.textpb /etc/ferrix-bgp/ferrix.textpb
install -m 0644 ferrix-bgp.service /etc/systemd/system/ferrix-bgp.service

3. Loopback

Peering over ::1 needs no address setup. 05-lab-lo.network only carries the AnyIP route for the lab space; merge it into the networkd file that matches lo if it is not there already, then networkctl reload.

4. BIRD

Replace the protocol bgp ferrix stanza with bird-ferrix-local.conf, then birdc configure check and birdc configure timeout 300 / confirm.

5. Start

systemctl daemon-reload
systemctl enable --now ferrix-bgp
journalctl -u ferrix-bgp -f
birdc show protocols all ferrix

6. Let students in

The API has no authentication and announces into a real AS, so do not expose port 8080. Put a reverse proxy in front with TLS and a class password. The event stream is long-lived, so the proxy must not buffer it.

Apache (tested against a local instance: 401 without the password, pages, API and POSTs through the proxy, and events arriving within 50 ms):

<VirtualHost *:80>
    ServerName lab.example.net
    Redirect permanent "/" "https://lab.example.net/"
</VirtualHost>

<VirtualHost *:443>
    ServerName lab.example.net

    SSLEngine on
    SSLCertificateFile    /etc/letsencrypt/live/lab.example.net/fullchain.pem
    SSLCertificateKeyFile /etc/letsencrypt/live/lab.example.net/privkey.pem

    # Everything behind one class password.
    <Location "/">
        AuthType Basic
        AuthName "BGP lab"
        AuthUserFile /etc/apache2/ferrix.htpasswd
        Require valid-user
    </Location>

    # Use Apache's built-in error messages in this vhost. Servers with the
    # multi-language error documents enabled fetch them from /error/,
    # which this vhost would proxy to ferrix; a 401 then comes back as
    # ferrix's empty 404: a blank page and no password prompt. With the
    # built-in ones nothing is fetched, so nothing needs exempting from
    # the login.
    ErrorDocument 400 default
    ErrorDocument 401 default
    ErrorDocument 403 default
    ErrorDocument 404 default
    ErrorDocument 500 default
    ErrorDocument 502 default
    ErrorDocument 503 default
    ErrorDocument 504 default

    ProxyPreserveHost On
    ProxyPass        "/" "http://127.0.0.1:8080/"
    ProxyPassReverse "/" "http://127.0.0.1:8080/"

    # ferrix has no use for the password; do not pass it on.
    RequestHeader unset Authorization

    # The status page follows a long-lived event stream. If mod_deflate
    # compresses it, events sit in the compressor and never arrive.
    SetEnvIf Request_URI "^/api/events" no-gzip=1
</VirtualHost>

Modules: proxy proxy_http ssl headers setenvif auth_basic authn_file. Create the password file with htpasswd -c -B /etc/apache2/ferrix.htpasswd student. To also limit it to known networks, replace Require valid-user with

<RequireAll>
    Require valid-user
    Require ip 192.0.2.0/24 2001:db8::/32
</RequireAll>

Caddy:

lab.example.net {
    basic_auth {
        student <hash from `caddy hash-password`>
    }
    reverse_proxy 127.0.0.1:8080 {
        flush_interval -1
    }
}

nginx, inside a TLS server block:

location / {
    auth_basic "BGP lab";
    auth_basic_user_file /etc/nginx/lab.htpasswd;
    proxy_pass http://127.0.0.1:8080;
    proxy_http_version 1.1;
    proxy_buffering off;
    proxy_read_timeout 1h;
}

Restricting the proxy to the university's address ranges as well is cheap and worth doing. Only the Apache example has been tested.

7. Load test before the class

tools/loadtest.py makes N simulated students announce their /48 at the same instant, then verifies the table, ownership under load, the event stream and the number of BGP UPDATEs actually sent, and withdraws everything again. Python 3 standard library only.

tools/loadtest.py 128 --url https://lab.example.net --user student

Every test route carries 210036:0:0, so BIRD accepts it but exports it to no neighbour (apart from home_v6, which is outside the export control). While it runs, birdc show route protocol ferrix count should rise to the number of students and fall back to what it was. BIRD's import limit 130 means more than 130 lab routes at once are blocked there, by design.

Things to know

• Announced routes live in memory. A restart of the service forgets them and students have to announce again.
• Stopping the service sends BIRD a Cease first, so restarts reconnect at once.
• ferrix's own safety net is local_route_policy; BIRD's import filter and import limit are the second one. Keep both.