Fetching the Tcp Rtt in Linux

Extract TCP round trip time (RTT) estimations on linux

This can be done using tcpprobe, which is a module that inserts a hook into the tcp_recv processing path using kprobe records the state of a TCP connection in response to incoming packets.

Let's say you want to probe tcp connection on port 443, you need to do the following:

sudo modprobe tcp_probe port=443 full=1
sudo chmod 444 /proc/net/tcpprobe
cat /proc/net/tcpprobe > /tmp/output.out &
pid=$!

full=1: log on every ack packet received

full=0: log on only condo changes (if you use this your output might be empty)

Now pid is the process which is logging the probe. To stop, simply kill this process:

kill $pid

The format of output.out (according to the source at line 198):

[time][src][dst][length][snd_nxt][snd_una][snd_cwnd][ssthresh][snd_wnd][srtt][rcv_wnd]

Are RTT (round trip time) statistics kept for TCP connections in Linux?

No, because the round-trip-time isn't fixed. That is, you can't know how long a response will take before the requesting packet is actually sent. You can have an idea, and you can accumulate statistics, but you can't ask "What's the round trip time to such-and-such server".

How do you get the current RTT estimation for a tcp socket?

You can use the linux-specific socket option TCP_INFO for that (defined in linux/tcp.h)

struct tcp_info ti;
socklen_t tisize = sizeof(ti);
getsockopt(fd, IPPROTO_TCP, TCP_INFO, &ti, &tisize);

Now the rtt-estimation is in ti.tcpi_rtt (unit is milliseconds). There are several more interesting values, just look into the structure tcp_info.

TCP send queue getting stuck

Is there any way that I can check current TCP send buffer status, so that I can stop outgoing traffic if buffer is getting full?

It will automatically stop sending if the send buffer is full: If the send buffer is full a send/write will either block (blocking socket) or fail (non-blocking socket). If you want to stop earlier just decrease the buffer with setsockopt(..., SO_SNDBUF,...)
– Steffen Ullrich

Getting disconnection notification using TCP Keep-Alive on write blocked socket

If you pull the network connection before all the data is transmitted, then the connection is not idle and thus in some implementations the keepalive timer does not start. (Keep in mind that keepalive is NOT part of the TCP specification and as a result it is implemented inconsistently if at all.) In general, because of the combination of exponential backoff and large number of retries (tcp_retries2 defaults to 15) it can take up to 30 minutes for transmission retries to time out before the keepalive timer starts.

The workaround, if there is one, depends on the particular TCP implementation you are using. Some newer versions of Linux (kernel version 2.6.37 released 4 January, 2011) implement TCP_USER_TIMEOUT. More info here.

The usual recommendation is to implement communication timeouts at the application level rather than use TCP-based keepalive anyway. See, for example, HTTP Keep-Alive.

---

Related Topics