Server Sent Events and Rails Streaming

Server Sent Events and Rails Streaming

The way that SSEs are built is by the client opening a connection to the server, which is then left open until the server has some data to send. This is part of the SSE spec, and not a thing specific to ActionController::Live. It's effectively the same as long-polling, but with the connection not being closed after the first bit of data is returned, and with the mechanism built into the browser.

As such, the only way it can be implemented is by having multiple open client connections to the webserver which sit there indefinitely. As to what resources are required to deal with them, I'm not sure, as I've not yet tried to benchmark this, but it'll need enough servers for Puma to keep open thousands of connections if you have that many users with a page open.

The default limit for puma is 16 concurrent connections. Several blogs posts about setting up SSEs for Rails mention upping this to a larger value, but none that I've found suggest what this higher value should be. They do mention that the number of DB connections will need to be the same, as each Rails thread keeps one running. Sort of sounds like an expensive way to run things.

"Run a benchmark" is the only answer really.

I can't comment as to reverse proxying as I've not tried it, but as SSEs are done over standard HTTP, I shouldn't think it'll need any special setup.

Server-sent events in Rails not delivered asynchronously

I ran into a similar issue with a basic 'out the book' Rails 5 SSE app. The issue turned out to be a Rack update that lead to buffering of the stream. More info here https://github.com/rack/rack/issues/1619 and fixed by including

config.middleware.delete Rack::ETag

in config/application.rb

Implement Server Send Events with Rails 4

Found the solution here:
http://api.rubyonrails.org/classes/ActionController/Live.html

Just include the Action Controller live, and write a simple action:

class MyController < ActionController::Base
  include ActionController::Live

  def stream
    response.headers['Content-Type'] = 'text/event-stream'
    100.times {
      response.stream.write "hello world\n"
      sleep 1
    }
  ensure
    response.stream.close
  end
end

Why SSE in rails hang on the connection?

Problem is not in closing the connection, it's about when you are doing it - this happens way later(dozens of seconds, may be even into hours) than for a regular request, which is usually handled below couple of seconds even for the heaviest ones.

For example, let's say (very simplified and approximated):

1. you have 10 threads/workers, no SSE
2. you serve each html page in under 0.1 seconds
3. a user will wait up to a 1 second for page to load before busting in tears and going away from your site
4. the user will then read the page for 9 seconds before requesting next one

this way each thread can serve 10 pages a second, all threads - 100 pages a second, since every user requests at most one per 10 seconds - you can handle about 1000 users using your app simultaneously.

Now adding SSE updates to these pages, step by step:

1. first user connects, gets html in 0.1 sec, then one thread is occupied with SSE request for 9 seconds before page reload, and after that reload it will again be locked by the same users' request
2. you have only 9 idle threads
3. second user connects, the same repeats

This way only maximum of 10 users can be handled by the same system, that's 100x less. And you cannot just increase threads to 1000, because they are not free (memory, scheduler overhead etc.).

The catch is that most of the time most such connections are not doing anything, just waiting for events, so they actually do not need a thread reserved for them. So it's logical to free the thread for other requests without closing the connection, that's what hijack does.

PS. This approach can be taken even further - client live updates connection can be kept open by a process other than rails server (non-ruby and more efficient) while still doing all the events logic in rails. For example with anycable backend for ActionCable you can keep thousands of concurrent connections easily

Must a server sent event always be firing regardless of what page a user is on?

In your controller when handling SSE you're expected to do updates in a loop,
then ActionController::Live::ClientDisconnected is raised by response.stream.write once client is gone:

def regular_update
  response.headers['Content-Type'] = 'text/event-stream'
  sse = SSE.new(response.stream, event: 'notice')
  loop do
    sse.write(NoticeTask.perform) # custom code returning the JSOn 
    sleep 60
  end
rescue ClientDisconnected
  logger.info "Client is gone"
ensure
  sse.close
end

your code disconnects the client after first update and delay, but everything appears to be working because EventSource automatically reconnects (thus you're getting long-polling updates).

On client EventSource should be close()d once it is not needed. Usually it is done automatically upon navigation away from page containing it, so:

1. make sure that eventsource javascript is only on the dashboard page, not in javascript bundle (or is in bundle, but only enabled on specific page)
2. if you're using turbolinks - you have to close() the connection manually, as a quick solution - try adding <meta name="turbolinks-visit-control" content="reload"> to page header or disabling turbolinks temporarily.

Also think again whether you actually need SSE for this specific task, because for plain periodic updates you can just poll a json action from client side code, that will render the same data. This will make controller simpler, will not keep connection busy for each client, has wider server compatibility etc.

For SSE to be reasoned - at least check if something has really changed and skip message if there's nothing. Better way is to use some kind of pub-sub (like Redis' SUBSCRIBE/PUBLISH, or Postgres' LISTEN/NOTIFY) - emit events to a topic every time something that affects the dashboard changes, subscribe on SSE connect and so on (may be also throttle updates, depends on your application). Similar can be implemented with ActionCable (is a bit overkill, but can be handy, since already has pub-sub integrated)

How scalable are server-sent events in Rails?

If Puma creates a new thread for each connection, don't use it. Not only do you plan for hundreds of thousands users at once but provided it's going to be a web application, users can have multiple instances open in multiple browser tabs. Even the SSE specification warns against the "multiple tabs" problem because browsers may have their own limits of the number of simultaneous connections to one host:

Clients that support HTTP's per-server connection limitation might run
into trouble when opening multiple pages from a site if each page has
an EventSource to the same domain. Authors can avoid this using the
relatively complex mechanism of using unique domain names per
connection, or by allowing the user to enable or disable the
EventSource functionality on a per-page basis, or by sharing a single
EventSource object using a shared worker.

Use an evented server where connections do not block. The above-mentioned gevent, something built on Node JS or something else in Ruby (which I don't know and thus can't recommend anything).

For other readers who land on this page and might get confused, Rich Peck's answer is wrong. Server-Sent Events don't rely on long polling, nor do they send requests every couple of seconds. They are long-lived HTTP connections without the need to reopen the connection after every event. There are no "constant requests to the server".

---

Related Topics