Streaming CSV Download from Rails 3.2 App

Streaming CSV Download from Rails 3.2 app

OK, after a bit more research I hacked together the following in my controller. It'll stream if response_body is given something enumeratable (is that a word?). Also, the server needs to be able to stream (I am using Unicorn on Heroku). I'd like very much to not have all this stuff in the controller, so my next step is to extract it out somehow.

  format.csv {
    @entries = Entry.all
    @columns = ["First Name", "Last Name"].to_csv
    @filename = "entries-#{Date.today.to_s(:db)}"

    self.response.headers["Content-Type"] ||= 'text/csv'
    self.response.headers["Content-Disposition"] = "attachment; filename=#{@filename}"
    self.response.headers["Content-Transfer-Encoding"] = "binary"

    self.response_body = Enumerator.new do |y|
      @entries.each_with_index do |entry, i|
        if i == 0
          y << @columns
        end
        y << [entry.first_name, entry.last_name].to_csv
      end
    end
  }

Streaming large gzipped data to browser with minimal resources

Here's an older blog post that shows an example of streaming: http://patshaughnessy.net/2010/10/11/activerecord-with-large-result-sets-part-2-streaming-data

You might also have luck with the new Streaming API and Batches. If I'm reading the documentation correctly, you'd need to do your queries and output formatting in a view template rather than your controller in order to take advantage of the streaming.

As for gzipping, it looks like the most common way to do that in Rails is Rack::Deflator. In older versions of Rails, the Streaming API didn't play well Rack::Deflator. That might be fixed now, but if not that SO question has a monkey patch that might help.

Update

Here's some test code that's working for me with JRuby on Torquebox:

# /app/controllers/test_controller.rb
def index
  respond_to do |format|
    format.csv do
      render stream: true, layout: false
    end
  end
end

# /app/views/test/index.csv.erb
<% 100.times do -%>
<%= (1..1000).to_a.shuffle.join(",") %>
<% end -%>

# /config/application.rb
module StreamTest
  class Application < Rails::Application
    config.middleware.use Rack::Deflater
  end
end

Using that as an example, you should be able to replace your view code with something like this to render your CSV

Name,Created At
<% Model.scope.find_each do |model| -%>
"<%= model.name %>","<%= model.created_at %>"
<% end -%>

As far as I can tell, Rails will continue to generate the response if the user hits stop half-way through. I think this is a limitation with HTTP, but I could be wrong. This should meet the rest of your requirements, though.

Export to CSV File

Controller:

respond_to do |format|
        format.csv { send_data @report.to_csv, filename: @report.name + ".csv"} #to export csv
end

In the view

<%= link_to 'CSV', report_path(report, format: :csv)%>

Ruby on Rails 3: Streaming data through Rails to client

It looks like this isn't available in Rails 3

https://rails.lighthouseapp.com/projects/8994/tickets/2546-render-text-proc

This appeared to work for me in my controller:

self.response_body =  proc{ |response, output|
  output.write "Hello world"
}

---

Related Topics