How to Use Redis in a Multi-Threaded Rails Environment? (Puma/Sidekiq)

What is the best way to use Redis in a Multi-threaded Rails environment? (Puma / Sidekiq)

You use a separate global connection pool for your application code. Put something like this in your redis.rb initializer:

require 'connection_pool'
REDIS = ConnectionPool.new(size: 10) { Redis.new }

Now in your application code anywhere, you can do this:

REDIS.with do |conn|
  # some redis operations
end

You'll have up to 10 connections to share amongst your puma/sidekiq workers. This will lead to better performance since, as you correctly note, you won't have all the threads fighting over a single Redis connection.

All of this is documented here: https://github.com/mperham/sidekiq/wiki/Advanced-Options#connection-pooling

How can I place Sidekiq alongside redis on a different server than the app server?

Yes, you'll make things more scalable, although things will be slightly slower this way since you'll increase network IO when communicating between server A and server B (where in the first case they were on the same server). You'll also need to pay for and manage that extra box. If you're close to the point where you can't get enough throughput in scenario A, switching makes sense. If you're not, this seems like premature optimization.

Which Redis commands does Sidekiq use?

There is no canonical list but the current Sidekiq supports Redis 2.8 so none of the newer commands in 3.0+.

What is the best way to use the same redis connection as sideqik?

Sidekiq takes a block, and the equivalent method is:

Sidekiq.redis { |x| x.redis }

---

Related Topics