$Redis Global Variable with Ruby on Rails

$redis global variable with ruby on rails

expanding further on mestachs suggestion, namespacing a module in your initializer as below

config/initializers/redis.rb

module ReadCache
  class << self
    def redis
      @redis ||= Redis.new(:url => (ENV["REDIS_URL"] || 'redis://127.0.0.1:6379'))
    end
  end
end

then in unicorn.rb

 before_fork do |server, worker|
    ...
   if defined?(ReadCache.redis)
    ReadCache.redis.quit
   end
    ...
 end

 after_fork do |server, worker|
    ...
   if defined?(ReadCache.redis)
    ReadCache.redis.client.reconnect
   end
    ...
 end

Is a Ruby global variable equivalent to a class variable on Class Object?

1) JS: My understanding is that variables set outside the "outer" function's scope are global in context

This depends on what exactly you mean by "variables". Variables declared with const or let are lexically scoped.

Variables declared with var in the top-level context aren't actually variables at all, they become properties of the top-level object (e.g. window, document, global, … depending on your environment).

and store a location to the assigned value.

That is true of all variables in both Ruby and ECMAScript, as well as properties in ECMAScript.

3) "Everything" in Ruby is an object (except blocks and individual index items within an array), including classes, primitives, etc.

This really depends on what exactly you mean by "everything". There are lots of "objects" (in the sense of "things we can talk about") in Ruby that aren't "objects" (in the sense of "things we can assign to variables, pass around, manipulate in Ruby code"). For example, variables aren't objects in Ruby.

A) Is the Rails.application object actually an instance of the Application Class?

I have no idea what this has to do with global variables.

Is there any difference between a Ruby global variable:

$var = "A Ruby global variable"

and

Class Object
  @@var = "A class variable set on the Class Object"
end

Yes, there is the pretty obvious difference that objects and classes which don't have Object in their ancestors chain won't have access to Object's class variables:

class BasicObject
  @@var
end
# NameError: uninitialized class variable @@var in BasicObject

Is Class Object, as the Class from which all other Classes/Objects ultimately inherit, therefore, Ruby's "global" context (or not, and I missed something huge)?

The Object class acts as a sort-of global context for some stuff, but that is not because it is global, but rather because it is the parent of most classes (basically anything that doesn't extend directly from BasicObject).

Anything that uses inheritance (class variables, methods, and constants) and is defined in Object will also be available in everything that descends from Object. But that has nothing to do with "global". That's just how inheritance works. Note that Object itself inherits from Kernel and BasicObject, so this is true for anything defined in those two as well.

C) (added as an edit) Because a global variable needs to be initiated and is available in a global scope, it is a class variable and not an instance variable. Is that reasoning sound?

No, it is not. A global variable is neither an instance variable nor a class variable. It is a global variable.

If a global variable is an object, it has to inherit from Class Object, right? This would mean the "global" context is still wrapped within Class Object.

Variables aren't objects in Ruby.

Configure Redis connection on initialize

This is how Redis is initialized in general.

Firstly, a good practice would be adding this to your config/environments/[environment_name].rb. So you can maintain different locations for Redis when you change environments.

config.redis_host   = "localhost"

Then in your application's config/initializers path create redis.rb and place the code below to initialize Redis.

require 'redis'

## Added rescue condition if Redis connection is failed
begin
  $redis = Redis.new(:host => Rails.configuration.redis_host, :port => 6379) 
rescue Exception => e
  puts e
end

Then you'll be able to use the global variable $redis within your application for Redis-related commands.

$redis.hset "my_hash", item.id, business.id

Here is a helpful article with more details.

---

Now in your case as this documentation suggests, here is what you should do:

In config/initializers/predictor.rb,

Predictor.redis = Redis.new(:url => ENV["PREDICTOR_REDIS"])

Or, to improve performance, add hiredis as your driver (you'll need to install the hiredis gem first)

Predictor.redis = Redis.new(:url => ENV["PREDICTOR_REDIS"], :driver => :hiredis)

Then, be sure to include include Predictor::Base in all models you want to use it,

class CourseRecommender
  include Predictor::Base
  ...
end

Here is the code responsible for the error you getting.

Ruby on Rails always pointing local redis Server

As per Question:

// LOCAL REDIS SERVER
Redis.current = Redis.new(:host => '127.0.0.1', :port => 6379)
// REMOTE REDIS SERVER
#Redis.current = Redis.new(:host => '<VM IP>', :port => <VM PORT>, :password => '<PASSWORD>')

Model.rb

 DataMapper.setup(:default, {:adapter  => "redis"})

In above code, I am overloading Redis configuration in User model.
In that case, I need to write remote configuration in model as well.

DataMapper.setup(:default, {:adapter  => "redis", :host => '<VM IP>', :port => <VM PORT>, :password => '<PASSWORD>'})

Benefits of connection pooling with Redis and Unicorn

Connection pool is only used when ActiveRecord talks to the SQL databases defined in your databases.yml config file. It is not related to Redis at all and the SO answer that you cite is actually not relevant for Redis.

So, unless you wanted to use some custom connection pool solution for Redis, you don't have to deal with it at all, as there is no pool for Redis in Rails by default. I guess the custom pool might be suitable if you had multiple threads in your applications which is not your case.

Update: Does building a connection pool make sense in your scenario? I doubt it. Connection pool is a way to reuse open connections (typically among multiple threads / requests). But you say that you:

1. use unicorn, the workers of which are separate, independent processes, not threads,
2. open a stable connection (or two) during after_fork, a connection which is then open all the time the unicorn worker lives
3. do not use threads in your application anywhere (I'd check if this is true again - it's not only Sidekiq but it might be any gem that tends to do things in the background).

In such scenario, pooling connection to Redis makes no sense to me as there seems to be no code that would benefit from reusing the connection - it is open all the time anyway.

Accessing a variable within a rails thread

UPDATED EDIT AT END: Shows working code. Main module unmodified except for debugging code. Note: I did experience the issue I already noted regarding the need to unsubscribe prior to termination.

The code looks correct. I'd like to see how you are instantiating it.

In config/application.rb, you probably have at least something like:

require 'ws_communication'
config.middleware.use WsCommunication

Then, in your JavaScript client, you should have something like this:

var ws = new WebSocket(uri);

Do you instantiate another instance of WsCommunication? That would set @clients to an empty array and could exhibit your symptoms. Something like this would be incorrect:

var ws = new WsCommunication;

It would help us if you would show the client and, perhaps, config/application.rb if this post does not help.

By the way, I agree with the comment that @clients should be protected by a mutex on any update, if not reads as well. It's a dynamic structure that could change at any time in an event-driven system. redis-mutex is a good option. (Hope that link is correct as Github seems to be throwing 500 errors on everything at the moment.)

You might also note that $redis.publish returns an integer value of the number of clients that received the message.

Finally, you might find that you need to ensure that your channel is unsubscribed before termination. I've had situations where I've ended up sending each message multiple, even many, times because of earlier subscriptions to the same channel that weren't cleaned up. Since you are subscribing to the channel within a thread, you will need to unsubscribe within that same thread or the process will just "hang" waiting for the right thread to magically appear. I handle that situation by setting an "unsubscribe" flag and then sending a message. Then, within the on.message block, I test for the unsubscribe flag and issue the unsubscribe there.

The module you provided, with only minor debugging modifications:

require 'faye/websocket'
require 'redis'

class WsCommunication
  KEEPALIVE_TIME = 15 #seconds
  CHANNEL = 'vip-deck'

  def initialize(app)
    @app = app
    @clients = []
    uri = URI.parse(ENV['REDISCLOUD_URL'])
    $redis = Redis.new(host: uri.host, port: uri.port, password: uri.password)
    Thread.new do
      redis_sub = Redis.new(host: uri.host, port: uri.port, password: uri.password)
      redis_sub.subscribe(CHANNEL) do |on|
        on.message do |channel, msg|
          puts "Message event. Clients receiving:#{@clients.count};"
          @clients.each { |ws| ws.send(msg) }
        end
      end
    end
  end

  def call(env)
    if Faye::WebSocket.websocket?(env)
      ws = Faye::WebSocket.new(env, nil, {ping: KEEPALIVE_TIME})

      ws.on :open do |event|
        @clients << ws
        puts "Open event. Clients open:#{@clients.count};"
      end

      ws.on :message do |event|
        receivers = $redis.publish(CHANNEL, event.data)
        puts "Message published:#{event.data}; Receivers:#{receivers};"
      end

      ws.on :close do |event|
        @clients.delete(ws)
        puts "Close event. Clients open:#{@clients.count};"
        ws = nil
      end

      ws.rack_response
    else
      @app.call(env)
    end
  end
end

The test subscriber code I provided:

# encoding: UTF-8
puts "Starting client-subscriber.rb"
$:.unshift File.expand_path '../lib', File.dirname(__FILE__)
require 'rubygems'
require 'eventmachine'
require 'websocket-client-simple'

puts "websocket-client-simple v#{WebSocket::Client::Simple::VERSION}"

url = ARGV.shift || 'ws://localhost:3000'

EM.run do

  ws = WebSocket::Client::Simple.connect url

  ws.on :message do |msg|
    puts msg
  end

  ws.on :open do
    puts "-- Subscriber open (#{ws.url})"
  end

  ws.on :close do |e|
    puts "-- Subscriber close (#{e.inspect})"
    exit 1
  end

  ws.on :error do |e|
    puts "-- Subscriber error (#{e.inspect})"
  end

end

The test publisher code I provided. Publisher and Subscriber could easily be combined, as these are just tests:

# encoding: UTF-8
puts "Starting client-publisher.rb"
$:.unshift File.expand_path '../lib', File.dirname(__FILE__)
require 'rubygems'
require 'eventmachine'
require 'json'
require 'websocket-client-simple'

puts "websocket-client-simple v#{WebSocket::Client::Simple::VERSION}"

url = ARGV.shift || 'ws://localhost:3000'

EM.run do
  count ||= 0
  timer = EventMachine.add_periodic_timer(5+rand(5)) do
    count += 1
    send({"MESSAGE": "COUNT:#{count};"})
  end

  @ws = WebSocket::Client::Simple.connect url

  @ws.on :message do |msg|
    puts msg
  end

  @ws.on :open do
    puts "-- Publisher open"
  end

  @ws.on :close do |e|
    puts "-- Publisher close (#{e.inspect})"
    exit 1
  end

  @ws.on :error do |e|
    puts "-- Publisher error (#{e.inspect})"
    @ws.close
  end

  def self.send message
    payload = message.is_a?(Hash) ? message : {payload: message}
    @ws.send(payload.to_json)
  end
end

A sample config.ru which runs all this at the rack middleware layer:

require './controllers/main'
require './middlewares/ws_communication'
use WsCommunication
run Main.new

This is Main. I stripped it down out of my running version so it might need tweaked if you use it:

%w(rubygems bundler sinatra/base json erb).each { |m| require m }
ENV['RACK_ENV'] ||= 'development'
Bundler.require
$: << File.expand_path('../', __FILE__)
$: << File.expand_path('../lib', __FILE__)

Dir["./lib/*.rb", "./lib/**/*.rb"].each { |file| require file }
env = ENV['OS'] == 'Windows_NT' ? 'development' : ENV['RACK_ENV']

  class Main < Sinatra::Base

    env = ENV['OS'] == 'Windows_NT' ? 'development' : ENV['RACK_ENV']
    get "/" do
      erb :"index.html"
    end

    get "/assets/js/application.js" do
      content_type :js
      @scheme = env == "production" ? "wss://" : "ws://"
      erb :"application.js"
    end
  end

---

Related Topics