session: write messages in batches, with TCP_NODELAY
A session wrote every message to its socket as it came: one write and, with Nagle's algorithm, mostly one segment for each UPDATE, a hundred or two octets. On the IPv6 cross-connect between the two routers, whose round trip is 50 microseconds, a million routes took 42 seconds to go out: the status page's new TCP row showed the connection busy for that long with neither window nor buffer limiting it, a congestion window stuck at ten segments, and a smoothed round-trip time of 18 ms. That is the receiver's delayed acknowledgements, 40 ms at a time, on a stream of small segments, holding the sender to a few segments a round trip. Now a session gathers what is waiting for it, up to the outbound queue's length, in one wake, encodes the messages into one buffer, and writes the buffer when it reaches 64 KiB or when the actions of the wake are done; the socket has TCP_NODELAY, so a batch's last, partial segment is not held back for the acknowledgement of the one before. The NOTIFICATION that ends a session, including one sent after a panic, is flushed on its own. Measured in the Linux VM with four peers' tables going to a collector over a veth pair, 151 MiB: 121,850 segments instead of 313,773, the connection busy 0.8 s instead of 2.0 s, and the congestion window grown to 47 segments instead of 13. Over loopback the transit benchmark is unchanged at 2.5 s for the second neighbor to have the table: there the receiver's parsing sets the pace, not the segments. Testing: the session tests, including the one that sends a NOTIFICATION after a panic; the integration tests; the kernel tests and every lab phase. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
1 parent
b0dd357986
commit
9276bc9f7a
2 files changed
+74
-9
No files matched your search
@@ -195,6 +195,9 @@ if [ "$collector" = 1 ]; then
|
||||
last=$now_held
|
||||
done
|
||||
echo "collector: $(advertised) routes advertised after $(since "$start") s"
|
||||
collector_addr=$(status peers '.[] | select(.name == "collector") | .peer_addr')
|
||||
run curl -sf "http://127.0.0.1:8080/api/tcp?peer=$collector_addr" \
|
||||
| jq -r '.connection | "collector tcp: busy \(.busy_ms / 1000 | . * 10 | round / 10) s, rwnd-limited \(.limited_by_receive_window_share * 100 | round)%, rtt \(.rtt_ms) ms, cwnd \(.cwnd_segments), sent \(.bytes_sent / 1048576 | round) MiB in \(.segments_out) segments"'
|
||||
fi
|
||||
memory_report() {
|
||||
rss=$(awk '/VmRSS/ { printf "%.0f", $2 / 1024 }' "/proc/$daemon_pid/status")
|
||||
|
||||
Reference in new issue
Block a user