Rails.Cache Error in Rails 3.1 - Typeerror: Can't Dump Hash With Default Proc

Rails.cache error in Rails 3.1 - TypeError: can't dump hash with default proc

This might be a little verbose but I had to spend some time with the Rails source code to learn how the caching internals work. Writing things down aids my understanding and I figure that sharing some notes on how things work can't hurt. Skip to the end if you're in a hurry.

---

Why It Happens

This is the offending method inside ActiveSupport:

def should_compress?(value, options)
  if options[:compress] && value
    unless value.is_a?(Numeric)
      compress_threshold = options[:compress_threshold] || DEFAULT_COMPRESS_LIMIT
      serialized_value = value.is_a?(String) ? value : Marshal.dump(value)
      return true if serialized_value.size >= compress_threshold   
    end
  end
  false  
end

Note the assignment to serialized_value. If you poke around inside cache.rb, you'll see that it uses Marshal to serialize objects to byte strings before they go into the cache and then Marshal again to deserialize objects. The compression issue isn't important here, the important thing is the use of Marshal.

The problem is that:

Some objects cannot be dumped: if the objects to be dumped include bindings, procedure or method objects, instances of class IO, or singleton objects, a TypeError will be raised.

Some things have state (such as OS file descriptors or blocks) that can't be serialized by Marshal. The error you're noting is this:

can't dump hash with default proc

So someone in your model has an instance variable that is a Hash and that Hash uses a block to supply default values. The column_methods_hash method uses such a Hash and even caches the Hash inside @dynamic_methods_hash; column_methods_hash will be called (indirectly) by public methods such as respond_to? and method_missing.

One of respond_to? or method_missing will probably get called on every AR model instance sooner or later and calling either method makes your object unserializable. So, AR model instances are essentially unserializable in Rails 3.

Interestingly enough, the respond_to? and method_missing implementations in 2.3.8 are also backed by a Hash that uses a block for default values. The 2.3.8 cache is "[...]is meant for caching strings." so you were getting lucky with a backend that could handle whole objects or it used Marshal before your objects had hash-with-procs in them; or perhaps you were using the MemoryStore cache backend and that's little more than a big Hash.

Using multiple scope-with-lambdas might end up storing Procs in your AR objects; I'd expect the lambdas to be stored with the class (or singleton class) rather than the objects but I didn't bother with an analysis as the problem with respond_to? and method_missing makes the scope issue irrelevant.

What You Can Do About It

I think you've been storing the wrong things in your cache and getting lucky. You can either start using the Rails cache properly (i.e. store simple generated data rather than whole models) or you can implement the marshal_dump/marshal_load or _dump/_load methods as outlined in Marshal. Alternatively, you can use one of the MemoryStore backends and limit yourself to one distinct cache per server process.

---

Executive Summary

You can't depend on storing ActiveRecord model objects in the Rails cache unless you're prepared to handle the marshalling yourself or you want to limit yourself to the MemoryStore cache backends.

---

The exact source of the problem has changed in more recent versions of Rails but there are still many instances of default_procs associated with Hashes.

Rails - 'can't dump hash with default proc' during custom validation

Without a stack trace (does it lead anywhere, or does it just not appear?) it is difficult to know what exactly is happening, but here's how you can reproduce this error in a clean environment:

# initialize a new hash using a block, so it has a default proc
h = Hash.new {|h,k| h[k] = k } 

# attempt to serialize it:
Marshal.dump(h)
#=> TypeError: can't dump hash with default proc

Ruby can't serialize procs, so it wouldn't be able to properly reconstitute that serialized hash, hence the error.

If you're reasonably sure that line is the source of your trouble, try refactoring it to see if that solves the problem.

def already_wants? want_name
  wants.any? {|want| want_name == want.name }
end

or

def already_wants? want_name
  wants.where(name: want_name).count > 0
end

Marshal can't dump hash with default proc (TypeError)

Ruby doesn't have a Marshal format for code, only for data. You cannot marshal Procs or lambdas.

Your apps hash has a default_proc, because

hsh = Hash.new { some_block }

is more or less the same as

hsh = {}
hsh.default_proc = ->{ some_block }

IOW: your apps hash contains code, and code cannot marshalled.

Problems with Rails.cache: Exception while duping an array on Rails: TypeError (can't modify frozen object)

The solution is to upgrade to Rails 3.1 or higher since dup is introduced then.

can't dump File during Rails.cache.fetch and Rails.cache.write

The problem is that

Website.includes(:account)

does not load the data, it just provides you with a proxy object that will load the objects on demand when you do something like calling #each or #to_s, if you force the fetching by adding a #to_aat the end it should work, e.g. try

Marshal.dump(Website.includes(:account))
Marshal.dump(Website.includes(:account).to_a)

#all(x=>y) does not do the same thing as #x(y), that is why you can do X.includes.joins.where but not X.all.where.

Not sure where the reference to a File comes from though.

---

Related Topics