Difference Between Tcp_Max_Syn_Backlog and Somaxconn

What is the difference between tcp_max_syn_backlog and somaxconn?

sysctl is an API. So you can just read the Linux kernel documentation for appropriate version:

tcp_max_syn_backlog - INTEGER
    Maximal number of remembered connection requests, which have not
    received an acknowledgment from connecting client.
    The minimal value is 128 for low memory machines, and it will
    increase in proportion to the memory of machine.
    If server suffers from overload, try increasing this number.

somaxconn - INTEGER
    Limit of socket listen() backlog, known in userspace as SOMAXCONN.
    Defaults to 128.  See also tcp_max_syn_backlog for additional tuning
    for TCP sockets.

Let's consider a TCP-handshake.. tcp_max_syn_backlog represents the maximal number of connections in SYN_RECV queue. I.e. when your server received SYN, sent SYN-ACK and haven't received ACK yet. This is a separate queue of so-called "request sockets" - reqsk in code (i.e. not fully-fledged sockets, "request sockets" occupy less memory. In this state we can save some memory and not yet allocate a full socket because the full connection may not be at all in the future if ACK will not arrive). The value of this queue is affected (see this post) by listen()'s backlog argument and limited by tcp_max_syn_backlog in kernel.

somaxconn represents the maximal size of ESTABLISHED queue. This is another queue.

Recall the previously mentioned SYN_RECV queue - your server is waiting for ACK from client. When the ACK arrives the kernel roughly speaking makes the big full-fledged socket from "request socket" and moves it to ESTABLISHED queue. Then you can do accept() on this socket. This queue is also affected by listen()'s backlog argument and limited by somaxconn in kernel.

Useful links: 1, 2.

What's tcp-backlog in redis.conf

Is tcp-backlog the size of "complete connection queue" (three-way handshake complete, what is described here) or "incomplete connection queue"?

tcp-backlog is the size of complete connection queue. In fact, Redis passes this configuration as the second parameter of the listen(int s, int backlog) call.

@GuangshengZuo already had a good answer for this question. So I'll focus on the other one.

If it means "complete connection queue" then why should I raise tcp_max_syn_backlog which limits the size of an incomplete connection queue?

Quote from the doc you mentioned:

The implementation uses two queues, a SYN queue (or incomplete connection queue) and an accept queue (or complete connection queue). Connections in state SYN RECEIVED are added to the SYN queue and later moved to the accept queue when their state changes to ESTABLISHED, i.e. when the ACK packet in the 3-way handshake is received. As the name implies, the accept call is then implemented simply to consume connections from the accept queue. In this case, the backlog argument of the listen syscall determines the size of the accept queue.

We can see that items in complete connection queue are moved from the incomplete connection queue.

If you have a large somaxconn with a small tcp_max_syn_backlog, then you might NOT have enough items to be moved to the complete connection queue, and the complete connection queue might never be full. Many requests might have already been dropped from the first queue before they have the chance to be moved to the second.

So only raise the value of somaxconn might NOT work. You have to raise both of them.

listen() backlog upper limits

somaxconn is the number of complete connections waiting.

tcp_max_syn_backlog is the number of incomplete connections waiting.

They aren't the same thing. It's all described in the man page.

Invalid argument setting key net.core.somaxconn

Same issue I got into when I tried to fine tune my nginx. This is the problem with the patch that been made to Ubuntu kernel.
The

sk_max_ack_backlog field of the sock structure is defined as unsigned short.

Therefore, the backlog argument in inet_listen()
shouldn't exceed USHRT_MAX. The backlog argument in the listen() syscall is truncated to the somaxconn value. So, the somaxconn value shouldn't exceed 65535 (USHRT_MAX).

So in short to make your net.core.somaxconn work you should not give value greater then 65535

net.core.somaxconn = 65535

This is sad but we have to live with it until unless you are ok to repatch your kernel:
https://lists.ubuntu.com/archives/kernel-team/2013-October/033041.html

Does listen's backlog number include SYN-received connections count in case of TCP in Linux?

In case of 4.3 kernel you specified it's something like:

tcp_v4_do_rcv()->tcp_rcv_state_process()->tcp_v4_conn_request()->tcp_conn_request()->inet_csk_reqsk_queue_is_full()

Here we can see the most important details about queues:

/* TW buckets are converted to open requests without
 * limitations, they conserve resources and peer is
 * evidently real one.
 */
if ((sysctl_tcp_syncookies == 2 ||
     inet_csk_reqsk_queue_is_full(sk)) && !isn) {
    want_cookie = tcp_syn_flood_action(sk, skb, rsk_ops->slab_name);
    if (!want_cookie)
        goto drop;
}

/* Accept backlog is full. If we have already queued enough
 * of warm entries in syn queue, drop request. It is better than
 * clogging syn queue with openreqs with exponentially increasing
 * timeout.
 */
if (sk_acceptq_is_full(sk) && inet_csk_reqsk_queue_young(sk) > 1) {
    NET_INC_STATS_BH(sock_net(sk), LINUX_MIB_LISTENOVERFLOWS);
    goto drop;
}

Pay your attention to inet_csk_reqsk_queue_is_full():

static inline int inet_csk_reqsk_queue_is_full(const struct sock *sk)
{
    return inet_csk_reqsk_queue_len(sk) >= sk->sk_max_ack_backlog;
}

Finally it compares current queue icsk_accept_queue with sk_max_ack_backlog size which was previously set by inet_csk_listen_start(). So yep, backlog affects incoming queue in current case.

You can see that both sk_acceptq_is_full() and inet_csk_reqsk_queue_is_full() make comparison with the same socket's sk_max_ack_backlog which is set through the listen():

static inline bool sk_acceptq_is_full(const struct sock *sk)
{
    return sk->sk_ack_backlog > sk->sk_max_ack_backlog;
}

Useful links: 1, 2

In a Node.js HTTP server's listen() method, what is the backlog parameter?

This parameter will help you throttle connections to your server and, hence, protect it from "too many connections" issues.

Without this parameter, your server will (theoretically speaking and without OS limitations) accepts any number of connections. And since each connection consumes memory from your underlying machine, you will get to an "out of memory" situation when there are too many connections to your server and your server will stop running or, at least, will behaving in unexpected manner.

Receiving too many connections can be either normal (too many valid users trying to use your server) or abnormal (a result of a DOS/DDOS attack). In all cases, it is a good practice to put a limit to the number of connections your server can handle to ensure a high quality of service (connections above the limit will be "polity" declined).

Hopefully, Linux put some system-wide limitations that are tcp_max_syn_backlog and somaxconn. Even if you set the backlog parameter to a high value, Linux will review it to align it with the value set for these two system parameters. You can read more about this feature in this post.

---

Related Topics