Do HTML Websockets Maintain an Open Connection for Each Client? Does This Scale

Do HTML WebSockets maintain an open connection for each client? Does this scale?

In most ways WebSockets will probably scale better than AJAX/HTML requests. However, that doesn't mean WebSockets is a replacement for all uses of AJAX/HTML.

Each TCP connection in itself consumes very little in terms server resources. Often setting up the connection can be expensive but maintaining an idle connection it is almost free. The first limitation that is usually encountered is the maximum number of file descriptors (sockets consume file descriptors) that can be open simultaneously. This often defaults to 1024 but can easily be configured higher.

Ever tried configuring a web server to support tens of thousands of simultaneous AJAX clients? Change those clients into WebSockets clients and it just might be feasible.

HTTP connections, while they don't create open files or consume port numbers for a long period, are more expensive in just about every other way:

• Each HTTP connection carries a lot of baggage that isn't used most of the time: cookies, content type, conetent length, user-agent, server id, date, last-modified, etc. Once a WebSockets connection is established, only the data required by the application needs to be sent back and forth.

• Typically, HTTP servers are configured to log the start and completion of every HTTP request taking up disk and CPU time. It will become standard to log the start and completion of WebSockets data, but while the WebSockets connection doing duplex transfer there won't be any additional logging overhead (except by the application/service if it is designed to do so).

• Typically, interactive applications that use AJAX either continuously poll or use some sort of long-poll mechanism. WebSockets is a much cleaner (and lower resource) way of doing a more event'd model where the server and client notify each other when they have something to report over the existing connection.

• Most of the popular web servers in production have a pool of processes (or threads) for handling HTTP requests. As pressure increases the size of the pool will be increased because each process/thread handles one HTTP request at a time. Each additional process/thread uses more memory and creating new processes/threads is quite a bit more expensive than creating new socket connections (which those process/threads still have to do). Most of the popular WebSockets server frameworks are going the event'd route which tends to scale and perform better.

The primary benefit of WebSockets will be lower latency connections for interactive web applications. It will scale better and consume less server resources than HTTP AJAX/long-poll (assuming the application/server is designed properly), but IMO lower latency is the primary benefit of WebSockets because it will enable new classes of web applications that are not possible with the current overhead and latency of AJAX/long-poll.

Once the WebSockets standard becomes more finalized and has broader support, it will make sense to use it for most new interactive web applications that need to communicate frequently with the server. For existing interactive web applications it will really depend on how well the current AJAX/long-poll model is working. The effort to convert will be non-trivial so in many cases the cost just won't be worth the benefit.

Update:

Useful link: 600k concurrent websocket connections on AWS using Node.js

Websockets and scalability

1) Single server instance and single client ==> How many websockets will be created and how many connections to websockets?

If your client creates one webSocket connection, then that's what there will be one webSocket connection on the client and one on the server. It's the client that creates webSocket connections to the server so it is the client that determines how many there will be. If it creates 3, then there will be 3. If it creates 1, then there will be 1. Usually, the client would just create 1.

2) Single server instance and 10 clients ==> How many websockets will be created and how many connections to websockets?

As described above, it depends upon what the client does. If each client creates 1 webSocket connection and there are 10 clients connected to the server, then the server will see a total of 10 webSocket connections.

3) Single server instance and 1000 clients ==> How many websockets will be created and how many connections to websockets?

Same as point #2.

How do you scale with webscokets when your application has a 1000’s of user base?

A single server, configured appropriately can handle hundreds of thousands of simultaneous webSocket connections that are mostly idle since an idle webSocket uses pretty much no server CPU. For even larger scale deployments, one can cluster the server (run multiple server processes) and use sticky load balancing to spread the load.

There are many other articles like these on Google worth reading if you're pursuing large scale webSocket or socket.io deployments:

The Road to 2 Million Websocket Connections in Phoenix

600k concurrent websocket connections on AWS using Node.js

10 million concurrent webSockets

Ultimately, the achievable scale per a properly configured server will likely have more to do with how much activity there is per connection and how much computation is needed to deliver that.

What happens when many clients access the same websocket server simultaneously?

what will the 200th user see? will he just wait 200 seconds and see the usual reply?

Surely not. Even in HTTP example, the server can usually handle some of the requests simultaneously. The number of requests that can be replied at the same time actually depends on many thing, specially your web-server (or application-server) configuration. Since each request usually takes a thread, they don't wait for each other to end. Unless there are too many connected threads in the thread-pool so the webserver has to wait to answer and close one, to begin processing the next request.

Only there is a slight difference in websockets. Unlike HTTP which is stateless and the connection closes right after the response is sent, Websockets are kept alive to interact with server. So if each socket takes a thread, still application server can handle their request and response simultaneously. But the big problem is about the size of the thread-pool.

If there are too many active connections, usually the new connection cant be made because the pool is full and unlike HTTP, there might not be an open space for new connection any time soon. Application server on this situation usually handles it by not accepting new connections.

To work around this problem you must have enough resources and you should find suitable configurations to handle as many as concurrent connections you need. I'm not a python developer, but in Java and Tomcat application server, this is usually done by a right configuration which increases the application server thread-pool size. You must find same kind of configuration in what ever application server you use.

update
To answer the question in comment, I add this update.

So, suppose there are 10 threads in the pool and 11 users, what will the 11th user see when trying to connect?

If maximum session pool size is 10, the 11th connection usually can not be made. Depending on application server one of these scenarios might happen.

• Connection keeps waiting to connect till it reaches it's connect timeout.
• Server immediately drops the new connection with some status code. Sometimes the status code might be 503 service unavailable.
• Its rare but server might even drop some in-active or idle connections from pool and accept the new connection.

How many system resources will be held for keeping 1,000,000 websocket open?

Updated Answer

Short answer: yes, but it's expensive.

Long answer:

This question is not unique to WebSockets since WebSockets are fundamentally long-lived TCP sockets with a HTTP-like handshake and minimal framing for messages.

The real question is: could a single server handle 1,000,000 simultaneous socket connections and what server resources would this consume? The answer is complicated by several factors, but 1,000,000 simultaneous active socket connections is possible for a properly sized system (lots of CPU, RAM and fast networking) and with a tuned server system and optimized server software.

The number of connections is not the primary problem (that's mostly just a question of kernel tuning and enough memory), it is the processing and sending/receiving data to/from each of those connections. If the incoming connections are spread out over a long period, and they are mostly idle or infrequently sending small chunks of static data then you could probably get much higher than even 1,000,000 simultaneous connections. However, even under those conditions (slow connections that are mostly idle) you will still run into problems with networks, server systems and server libraries that aren't configured and designed to handle large numbers of connections.

See Alessandro Alinone's answer about approximate resource usage for 500,000 connections.

Here are some older but still applicable resources to read on how you would configure your server and write your server software to support large numbers of connections:

• What is the theoretical maximum number of open TCP connections that a modern Linux box can have
• http://www.kegel.com/c10k.html

How many SSE connections can a web server maintain?

Tomcat 8 (web server to give an example) and above that uses the NIO connector for handling incoming requst. It can service max 10,000 concurrent connections(docs). It does not say anything about max connections pers se. They also provide another parameter called acceptCount which is the fall back if connections exceed 10,000.

socket connections are treated as files. Every incoming connection to tomcat is like opening a socket and depending on the OS e.g in linux depends on the file-descriptor policy. You will find a common error when too many connections are open or max connections have been reached as the following

java.net.SocketException: Too many files open

You can change the number of open files by editing

/etc/security/limits.conf

It is not clear what is max limit that is allowed. Some say default for tomcat is 1096 but the (default) one for linux is 30,000 which can be changed.

On the article I have shared the linkedIn team were able to go 250K connections on one host.

So that should give you a pretty good idea about max sse connections possible. depends on your web server max connection configuration, OS capacity etc.

Websockets. Tyrus. Time for which the client is connected to a socket

There is no such feature.

I'd suggest creating @OnOpen method, which will save the time to current instance (Endpoints are by default WebSocket Session scoped) and work with that value.

Feel free to file an enhancement against Tyrus.

WebSocket - how to keep connection?

The Websocket protocol implements so called PING/PONG messages to keep Websockets alive, even behind proxies, firewalls and load-balancers. The server sends a PING message to the client through the Websocket, which then replies with PONG. If the client does not reply, the server closes the connection.

Check Your server config probably a wrong config have this side effect.

---

Related Topics