How Does a Linux Socket Buffer Overflow

How to find the socket buffer size of linux

If you want see your buffer size in terminal, you can take a look at:

• /proc/sys/net/ipv4/tcp_rmem (for read)
• /proc/sys/net/ipv4/tcp_wmem (for write)

They contain three numbers, which are minimum, default and maximum memory size values (in byte), respectively.

How the buffering work in socket on linux

For UDP socket client will never know - the server side will just start dropping packets after the receive buffer is filled.

TCP, on the other hand, implements flow control. The server's kernel will gradually reduce the window, so the client will be able to send less and less data. At some point the window will go down to zero. At this point the client fills up its send buffer and receives an error from the send(2).

What will happen if the socket buffer size is not enough

If there is not enough space in the buffer data will be discarded. In the case of TCP this will result in the peer sending the data again since they were not ack'ed. In case of UDP the data are simply lost. TCP will also advertise its receive window so that the peer will hopefully not send the data faster than they can be processed (i.e. taken out of the buffer).

Are there any side effects to increasing the socket buffer size in Linux?

The only side-effect is memory usage. Increase them gradually and monitor the system. As long as you leave enough memory for existing processes you should be golden.

What happens on buffer overflow?

Short answer is this. "send" calls on a TCP socket will just block until the TCP sliding window (or internal queue buffers) opens up as a result of the remote endpoint receiving and consuming data. It's not much different than trying to write bytes to a file faster than the disk can save it.

If your socket is configured for non-blocking mode, send will return EWOULDBLOCK or EAGAIN, until data can be sent. Standard poll, select, and epoll calls will work as expected so you know when to "send" again.

What are the differences between Kernel Buffer, TCP Socket Buffer and Sliding Window

Linux does not handle TCP's sliding window as a separate buffer, rather as several indices indicating how much has already been received / read. The Linux kernel packet handling process can be described in many ways and can be divided to small parts as yo go deeper, but the general flow is as follows:

1. The kernel prepares to receive data over a network interface, it prepares SKB (Socket Buffer) data structures and map them to the interface Rx DMA buffer ring.
2. When packets arrive, they fill these preconfigured buffers and notify the kernel in an interrupt context of the packets arrival. In this context, the buffers are moved to a recv queue for the network stack to handle them out of an interrupt context.
3. The network stack retrieves these packets and handles them accordingly, eventually arriving to the TCP layer (if they are indeed TCP packets) which in turn handles the window.
4. See struct tcp_sock member u32 rcv_wnd which is then used in tp->rcvq_space.space as the per-connection space left in window.
5. The buffer is added to socket receive queue and is read accordingly as stream data in tcp_recvmsg()

The important thing to remember here is that copies is the worst thing regarding performance. Therefore, the kernel will always (unless absolutely necessary) will avoid copies and use pointers instead.

What happens in TCP when the internal buffer fills up

Can the internal network buffer fill up?

There are two internal buffers: the send buffer and the receive buffer. Both can fill up.

I assume the answer is yes...

Yes.

If so, then does the TCP protocol specify what will happen in this scenario?

Yes.

Are these discarded packets

There are no discarded packets. TCP does not discard packets in this circumstance. What happens when the send buffer fills depends on whether you are in blocking or non-blocking mode, or whether you are using an asynchronous API:

• blocking mode: the sender blocks
• non-blocking mode: the sender gets an error EAGAIN/EWOULDBLOCK
• asynchronous: the operation continues to be deferred.

treated as if they were lost in transit and just re-transmitted like any other lost packages?

No. See above.

Or are these packets truly lost

No. See above.

and this something I have to consider when desiging my own communication protocol on top of the TCP?

No. See above. But the various conditions are certainly something you have to consider in your implementation of the protocol.

Does the answer vary between operating systems?

No.

For a record, the system above should have some kind of congestion control mechanism

TCP already has a congestion control mechanism.

which server could use to tell the client that it's under heavy load

How? if the client isn't reading, how can the server tell it anything?

or it's free again. However, I'm curious to know how the TCP by the default would behave in this scenario.

See above.

---

Related Topics