A router feeding a collector every path (all_paths) was 4.8 GB resident
with a table that should take a third of that. A heap profile of the
daemon in the Linux VM, with four full tables and such a collector,
showed where it went: the exporter kept, for every route it had
advertised, the route's outgoing form, the parsed attribute list it was
dressed with, through an `Arc<Dressed>` in the per-route record. One
such form lived for every distinct attribute set advertised, about a
kilobyte each, for the life of the session, on top of the exporter's
cache of 262,144 of them. The record needs only to tell whether a route
changed, and the table's own attribute set, which is in memory anyway,
tells it that: the outgoing form is a function of the set, the peer the
route came from and the policy.
This change:
* `Sent` holds an `Arc<AttrSet>` instead of an `Arc<Dressed>`, and a
route is sent again when its set differs. `Verdict::All` carries both
while a flush is on.
* `DRESSED_KEPT` goes from 262,144 to 65,536: a kilobyte a set, so 64 MB
rather than 256 MB a session at most; what a flush is sending holds
its own.
* The status page's header shows the daemon's memory as jemalloc
accounts for it (allocated and resident), `/api/status` has the six
figures (`memory`: allocated, active, metadata, resident, mapped,
retained), and `bgp_memory_bytes{kind}` exports them. jemalloc is built
with statistics for this, and `bgp-web` reads them through
`tikv-jemalloc-ctl`.
* `scripts/profile/full-table.sh` gains `COLLECTOR=1`, a collector
session over IPv6 that takes both families with ADD-PATH, and `HEAP=1`,
which builds the daemon with jemalloc's heap profiler (the new
`heap-profiling` feature of the daemon crate, not for production) and
turns the profile into a flame graph with jeprof; and it reports
memory with the table in and again 30 s later. The feeder
(`crates/bench/examples/feed.rs`) plays the collector with
`FEED_COLLECTOR=1`.
Measured in the VM, four peers' full tables and a collector sent every
path (4.9 million routes advertised): 1,723 MiB resident before, 1,286
MiB after, with jemalloc's allocated figure from 1,500 MiB to 946 MiB.
Without the collector the same tables take 630 MiB, so the collector's
bookkeeping is now about 50 bytes a route advertised. Of the resident
figure, some 400 MiB was pages jemalloc had freed and not yet returned:
it returns them on allocator activity only, and a router that has its
tables in allocates little. jemalloc is now built with its background
thread on (the `background_threads` feature), which returns them on
time: 1,020 MiB resident 30 s after the load, against 1,286 MiB without
it.
Documentation: `memory` in the HTTP API reference, `bgp_memory_bytes`
in the metrics reference and the monitoring guide, the script's new
settings in the testing guide.
Testing: the ADD-PATH integration tests pass, including the collector
ones; the heap profile and the figures above are from
`PEERS=4 COLLECTOR=1 HEAP=1 scripts/profile/full-table.sh` in the VM.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
13 KiB
+++ title = "HTTP API Reference" description = "Describes the endpoints of the Ferrix HTTP interface: the JSON API, the event stream, the status page, and the metrics endpoint." weight = 3
[extra] nav = "HTTP API" +++
Ferrix serves its HTTP interface on the address configured with
http_endpoint. All endpoints are read-only and use the GET method.
An error response has a text/plain body that describes the error. The
server returns the following error statuses:
| Status | Meaning |
|---|---|
| 400 (Bad Request) | A query parameter is invalid. |
| 500 (Internal Server Error) | The server could not produce the response, for example because the metrics could not be encoded. |
| 503 (Service Unavailable) | A table is unavailable because the daemon is shutting down. |
Caution
The HTTP interface does not require authentication. Bind it to a loopback address, or place it behind a reverse proxy that requires authentication.
Endpoints
| Endpoint | Description |
|---|---|
/ |
Redirects to /statusz. |
/statusz |
Status page. |
/api/status |
Router ID, AS number, uptime, table sizes, memory use (memory, as jemalloc accounts for it: allocated, active, metadata, resident, mapped, retained octets), and whether control communities are enabled (control_communities). |
/api/config |
The configuration that was read at startup. |
/api/peers |
Configured neighbors and their sessions. |
/api/routes/local |
Originated routes. |
/api/routes/received |
Received routes. |
/api/routes/rejected |
Rejected routes, by neighbor. |
/api/rpki |
RPKI cache, data, and data age. |
/api/kernel |
Kernel route installation state and interfaces. |
/api/tcp |
The TCP connection of one session, as the kernel reports it. |
/api/events |
Session and route changes, as server-sent events. |
/metricz |
Prometheus metrics. See Metrics Reference. |
Peers
/api/peers returns an array with one object for each configured
neighbor:
{
"name": "test-peer",
"peer_addr": "127.0.0.1:64815",
"peer_asn": 65001,
"state": "Established",
"direction": "inbound",
"local_addr": "127.0.0.1:1790",
"uptime_secs": 2,
"four_octet_asn": true,
"extended_next_hop": false,
"address_families": ["IPv4 unicast", "IPv6 unicast"],
"add_path_receive": [],
"add_path_send": [],
"routes": 1,
"rejected_routes": 1,
"advertised_routes": 2,
"held_routes": 0,
"expected_routes": null,
"role": null,
"role_confirmed": null,
"last_error": null,
"retry_at_unix": null,
"graceful_restart": {
"restart_time_secs": 120,
"notification": true,
"address_families": ["IPv4 unicast", "IPv6 unicast"],
"forwarding_preserved": ["IPv4 unicast", "IPv6 unicast"]
},
"stale_routes": null,
"stale_until_unix": null
}
| Field | Description |
|---|---|
state |
Session state: Established, Connecting, Backoff, Idle, or Passive (a passive neighbor that has no session). |
direction |
inbound if the neighbor initiated the connection, outbound otherwise. |
routes |
Number of the neighbor's routes in the routing table. |
rejected_routes |
Number of the neighbor's routes rejected by its import policy. |
advertised_routes |
Number of routes advertised to the neighbor and not withdrawn since. With ADD-PATH and all_paths, every route advertised counts. |
held_routes |
Number of held routes: routes rejected for a reason that can change without a new UPDATE, such as RPKI validity or next hop resolution. |
expected_routes |
The expected_routes value from the configuration, or null if it is not set. |
role_confirmed |
Whether the neighbor advertised a BGP role that matches the configured role. |
retry_at_unix |
For a neighbor without a session, the time of the next connection attempt. |
graceful_restart |
Graceful restart, if both neighbors advertised it: the restart time of the neighbor, whether the N bit is in use (RFC 8538), the address families whose routes are retained, and those for which the neighbor preserved its forwarding state. |
stale_routes |
Number of the neighbor's routes retained from its previous session and not yet advertised again, or null if there are none. |
stale_until_unix |
For a restarting neighbor without a session, the time at which its retained routes are removed. |
Received Routes
/api/routes/received returns received routes, one page at a time. Table 1
lists the query parameters.
| Parameter | Description |
|---|---|
prefix |
Returns all routes for the prefix, in order of path selection. Example: prefix=193.0.0.0/21. |
address |
Returns all routes for the prefixes that contain the address, most specific first. Example: address=193.0.0.1. |
within |
Returns the routes for the prefix and all more specific prefixes. Example: within=193.0.0.0/16. |
peer |
Returns only routes from the neighbor, identified by name or address. |
origin |
Returns only routes originated by the AS: the last AS of the AS path. The value is a number, with or without the prefix AS. Routes with an empty AS path are originated by the local AS. Example: origin=AS64496. |
best |
If true, returns only the best path for each prefix. |
limit |
Specifies the approximate number of routes in a page. Default: 1000. Maximum: 100000. |
after |
Returns the page after the specified prefix. Use the next_after value of the previous page. |
before |
Returns the page before the specified prefix. Use the prev_before value of the next page. |
A page contains only complete prefixes. It ends before the prefix whose
routes would exceed limit, but always contains at least one prefix.
Prefixes are sorted by address, with IPv4 prefixes first.
{
"routes": [{
"peer_name": "test-peer",
"peer_addr": "127.0.0.1:64815",
"peer_asn": 65001,
"prefix": "203.0.113.0/24",
"path_id": 0,
"best": true,
"rank": 1,
"stale": false,
"rpki": null,
"aspa": null,
"summary": {
"origin": "IGP",
"as_path": [65001],
"next_hop": "10.0.0.2",
"med": null,
"local_pref": null,
"communities": ["65001:1"],
"large_communities": [],
"other": [{ "name": "aggregator", "value": "AS65001 10.0.0.2" }]
},
"attributes": [ ... ]
}],
"next_after": null,
"prev_before": null
}
| Field | Description |
|---|---|
path_id |
ADD-PATH path identifier, or 0. |
best |
Whether the route is the best path for the prefix. |
rank |
The position of the route in path selection for the prefix: 1 for the best path, 2 for the path that would be selected if the best path were withdrawn, and so on. |
stale |
Whether the route is retained from a previous session of the neighbor and has not been advertised again (graceful restart). |
rpki, aspa |
Route origin validation and ASPA verification state, if RPKI validation is configured. |
summary |
The path attributes as text. other lists, in received order, the attributes that have no field of their own: atomic aggregate, aggregator, extended communities, AS4_PATH, only to customer, originator ID, cluster list, and unrecognized attributes. |
attributes |
All path attributes, as received. |
Rejected Routes
/api/routes/rejected returns, for each neighbor with rejected routes, the
number of rejected routes for each reason and the 200 most recent rejected
routes. To return the records for one neighbor, add ?peer=NAME.
[{
"peer": "test-peer",
"peer_addr": "127.0.0.1",
"total": 1,
"by_reason": { "own AS in path": 1 },
"recent": [{
"prefix": "198.51.100.0/24",
"path_id": 0,
"reason": "own AS in path",
"as_path": [65001, 64500],
"at_unix": 1791020702,
"summary": {
"origin": "IGP",
"as_path": [65001, 64500],
"next_hop": "10.0.0.2",
"med": null,
"local_pref": null,
"communities": [],
"large_communities": [],
"other": []
},
"attributes": [
{ "Origin": 0 },
{ "AsPath": { "segments": [{ "ordered": true, "path": [65001, 64500] }] } },
{ "NextHop": "10.0.0.2" }
]
}]
}]
| Field | Description |
|---|---|
summary, attributes |
The path attributes of the rejected route, in the same format as for received routes. For a route rejected when RPKI data changed, the attributes as stored, including those added by the import policy. |
Kernel
/api/kernel returns the state of kernel route installation:
{
"enabled": true,
"table": 254,
"metric": 20,
"synced": true,
"installed": 0,
"kept": 0,
"blackholes": 0,
"unresolved": {},
"refused": 0,
"errors": 0,
"last_error": null,
"interfaces": [
{ "name": "eth0", "index": 2, "up": true,
"addresses": ["192.168.5.15/24", "fe80::5055:55ff:fed4:2526/64"] }
]
}
| Field | Description |
|---|---|
synced |
Whether the kernel routing table has been synchronized with the routing table since the daemon started. |
installed |
Number of routes installed. |
kept |
Number of routes from a previous run that are retained during startup. |
blackholes |
Number of installed routes that are blackhole routes, which discard traffic for their prefix (RFC 7999). |
unresolved |
Number of prefixes whose active route is not installed, by reason. |
refused |
Number of prefixes whose active route the kernel rejected. |
errors |
Number of requests that the kernel rejected. |
TCP Connection
/api/tcp?peer=ADDRESS returns what the kernel reports about the TCP
connection of the session with the neighbor at ADDRESS, given as
peer_addr in /api/peers shows it: the round-trip time, the congestion
and receive windows, the queues, the retransmissions, and how long the
connection has been held up by the neighbor's receive window or by the
local send buffer. Linux only; on other systems, and for a session the
kernel has no connection for, available is false and reason says
why.
{
"available": true,
"reason": null,
"connection": {
"state": "established",
"congestion": "cubic",
"rtt_ms": 0.62,
"rtt_variance_ms": 0.11,
"min_rtt_ms": 0.4,
"rto_ms": 201,
"mss": 1428,
"pmtu": 1500,
"cwnd_segments": 10,
"ssthresh_segments": null,
"send_window": 1048576,
"receive_window": 3145728,
"receive_space": 65536,
"send_window_scale": 7,
"receive_window_scale": 10,
"send_queue": 0,
"not_sent": 0,
"recv_queue": 0,
"unacked_segments": 0,
"retrans_segments": 0,
"lost_segments": 0,
"total_retrans": 0,
"bytes_retrans": 0,
"pacing_rate": 2309920000,
"delivery_rate": 184860000,
"bytes_sent": 312993,
"bytes_acked": 312994,
"bytes_received": 190,
"segments_out": 230,
"segments_in": 118,
"busy_ms": 41,
"limited_by_receive_window_ms": 0,
"limited_by_send_buffer_ms": 0,
"limited_by_receive_window_share": 0,
"limited_by_send_buffer_share": 0,
"last_data_sent_ms": 1234,
"last_data_received_ms": 1300,
"last_ack_received_ms": 1234
}
}
Times are in milliseconds, sizes in octets, rates in octets per second.
send_window is the neighbor's receive window, which bounds what Ferrix
can send; receive_window is what Ferrix lets the neighbor send. The two
_share values are the fractions of the busy time, 0 to 1, in which the
connection was held up by the neighbor's window and by the local send
buffer. A session that advertises a table to a neighbor that reads it
slowly shows a high limited_by_receive_window_share. The status page
shows the same information under the tcp link of each established
session.
Events
/api/events is a stream of
server-sent events.
Each event has the name bgp and contains a JSON object. The type field
identifies the event, as listed in Table 2.
type |
Description |
|---|---|
session_up |
A session was established. The peer field has the format of /api/peers. |
session_down |
A session ended. Fields: peer_addr, peer_asn. |
route_refresh_requested |
A neighbor sent a ROUTE-REFRESH message. |
local_announced, local_withdrawn |
An originated route was advertised or withdrawn. |
best_changed |
The best path for a prefix changed. The route field contains the new best path. |
best_withdrawn |
A prefix no longer has a route. |
lagged |
The client did not read events quickly enough and missed events on stream. Reload the corresponding table. The missed field gives the number of missed events for the sessions and local_routes streams; the best_routes stream omits it because the number is not known. |
The following command displays the event stream:
curl -sN 127.0.0.1:8080/api/events