ferrix/docs/content/docs/operations/monitoring.md

+++ title = "Monitoring Ferrix" description = "Describes the status page, route lookup, rejected route records, and the metrics and alerting rules for monitoring Ferrix." weight = 2

[extra] nav = "Monitoring" +++

Ferrix provides a status page, a JSON API, an event stream, and Prometheus metrics over HTTP. To enable the HTTP interface, configure http_endpoint:

http_endpoint { address: "127.0.0.1" port: 8080 }

Table 1 lists the HTTP endpoints. All endpoints are read-only.

Path	Description
/statusz	Status page
/api/...	JSON API. See HTTP API Reference.
/api/events	Server-sent events for session and route changes
/metricz	Prometheus metrics. See Metrics Reference.

Status Page

The status page, /statusz, displays the following information:

• The configuration that was read at startup.
• Each neighbor, with its session state, address families, role and role confirmation, number of accepted routes, rejected announcements, and held routes, and last error.
• The originated routes, and a lookup of received routes.
• RPKI validation: the cache, the RTR version, and the age of the data.
• Kernel route installation: the number of installed routes, the number of routes not installed and the reason, the last error reported by the kernel, and the interfaces.
• Rejected routes, by neighbor and by reason.
• A live event log.

To display the name of the holder of an AS number, point to the number. The browser requests the name from RIPEstat; Ferrix does not send any requests itself. If the browser cannot reach RIPEstat, the page displays "name not available".

Route Lookup

The status page provides a lookup of received routes. The HTTP endpoint /api/routes/received accepts the same queries. Table 2 lists the query parameters.

Parameter	Result
prefix=193.0.0.0/21	All routes for the prefix, best path first
address=193.0.0.1	All prefixes that contain the address, most specific first
within=193.0.0.0/16	The prefix and all more specific prefixes
peer=NAME	Only routes from the neighbor, identified by name or address
best=true	Only the best path for each prefix
limit=N	At most N routes. Default: 1000. Maximum: 100000.

The following example displays the best path for the destination 193.0.0.1:

curl -s '127.0.0.1:8080/api/routes/received?address=193.0.0.1&best=true' | jq

Note

The within, peer, and best parameters, when used without prefix or address, scan the entire routing table. With a full routing table, such a query takes noticeably longer.

Rejected Routes

Ferrix does not store rejected routes in the routing table. For each neighbor, it records the number of rejected routes for each reason, and the 200 most recent rejected routes with all the path attributes they were received with. The status page displays this information under "Rejected routes"; to display the attributes of a rejected route, select show in its row. The HTTP endpoint /api/routes/rejected returns the same information as JSON. To display the records for one neighbor, add ?peer=NAME.

The numbers count rejected announcements, not prefixes. When a neighbor announces a route again, for example because an attribute changed, and Ferrix rejects it again, the count increases again. Two neighbors that advertise the same routes can therefore have different counts. The number of routes currently held, because they are RPKI or ASPA invalid or their next hop is unreachable, is displayed in the neighbor table as held now.

[{"peer":"transit-a","total":3,"by_reason":{"RPKI invalid":2,"bogon prefix":1},
  "recent":[{"prefix":"203.0.113.0/24","reason":"RPKI invalid","as_path":[64496,64511]}]}]

The records are retained after the session ends. Ferrix logs the first rejected route from each neighbor, and at most one rejected route per minute thereafter.

For a description of each rejection reason, see Import Policy.

Alerting Rules

The following Prometheus alerting rules cover the conditions that require operator action: route hijack and route leak attempts, prefix limits, session failures, loss of RPKI data, and kernel errors.

groups:
  - name: bgp
    rules:
      # A neighbor advertises routes that its import policy rejects. The
      # cause is a route leak or hijack at the neighbor, or an outdated
      # prefix list.
      - alert: BgpRoutesRejected
        expr: sum by (peer, reason) (increase(bgp_routes_rejected_total[10m])) > 0
        labels: { severity: warning }

      # A route that contains the local AS number was received: a routing
      # loop, or another network using the local AS number.
      - alert: BgpOwnAsInPath
        expr: increase(bgp_routes_rejected_total{reason="own AS in path"}[10m]) > 0
        labels: { severity: page }

      # A neighbor advertised a prefix of the local network, or a route
      # with a third-party next hop.
      - alert: BgpHijackAttempt
        expr: increase(bgp_routes_rejected_total{reason=~"prefix is our own|next hop is not the peer"}[10m]) > 0
        labels: { severity: page }

      # A neighbor is approaching its prefix limit.
      - alert: BgpPeerNearPrefixLimit
        expr: bgp_peer_routes / on (peer) bgp_peer_max_prefixes > 0.8
        for: 10m
        labels: { severity: warning }

      # A session was torn down because the neighbor exceeded its prefix
      # limit.
      - alert: BgpPrefixLimitExceeded
        expr: increase(bgp_prefix_limit_exceeded_total[1h]) > 0
        labels: { severity: page }

      # A configured neighbor has no established session.
      - alert: BgpPeerDown
        expr: bgp_peer_up == 0
        for: 5m
        labels: { severity: page }

      # No RPKI data is available. Routes are not validated.
      - alert: BgpRpkiNoData
        expr: sum(bgp_rpki_vrps) == 0
        for: 15m
        labels: { severity: page }

      # The kernel rejected route installation requests.
      - alert: BgpKernelErrors
        expr: increase(bgp_kernel_errors_total[10m]) > 0
        labels: { severity: warning }

Dashboard Queries

Table 3 lists queries for a monitoring dashboard.

Panel	Query
Established sessions	sum(bgp_peer_up), of count(bgp_peer_up)
Routes by neighbor	bgp_peer_routes
Prefixes with a best path	bgp_rib_prefixes
Rejected routes by reason	sum by (reason) (rate(bgp_routes_rejected_total[5m]))
UPDATE messages received and sent	rate(bgp_messages_received_total{type="update"}[5m]), rate(bgp_messages_sent_total{type="update"}[5m])
Kernel routes	bgp_kernel_routes, bgp_kernel_routes_unresolved
RPKI data	bgp_rpki_vrps, bgp_rpki_aspas

The metric bgp_build_info{version} reports the running version.