Rails - X-Sendfile + Temporary Files

rails - x-sendfile + temporary files

Given that Rails3 uses x-sendfile when it is available, and there is no way to deactivate it, you just can't use send_file with a library such as TempFile. The best option is the one I mentioned in the question: use a regular File, and have a cron task that removes old temp files periodically.

EDIT: The removal of unused files has now been easier to deal with with the maid gem:

https://github.com/benjaminoakes/maid

Rails - Creating temp files in a portable way

tmp/ is definitively the right place to put the files.

The best way I've found of creating files on that folder is using ruby's tempfile library.

The code looks like this:

require 'tempfile'

def foo()
  # creates a temporary file in tmp/
  Tempfile.open('prefix', Rails.root.join('tmp') ) do |f|
    f.print('a temp message')
    f.flush
    #... do more stuff with f
  end
end

I like this solution because:

It generates random file names automatically (you can provide a prefix)
It automatically deletes the files when they are no longer used. For example, if invoked on a rake task, the files are removed when the rake task ends.

Delete folder after send_file in Rails

Are you sure the send_file is not still sending the file when you are removing the dir, it may be asynchronous if it uses X-SendFile? That would cause an error when trying to remove the dir.

So you should probably be queuing this delete action, or doing it with a sweeper later, rather than trying to do it straight after sending the file to streaming.

I'm not completely clear on which file you are sending, so it would be useful to include an actual example of file path, and file type, and how it is created in your question.

Possible help with debugging:

Write out a very large file (> 60MB say), and check there is no invisible file created during your file creation process - I'm not clear on which file you are actually sending
Set up a large file transfer on a slow connection, and watch for the creation and possibly growing of this file (it might be related to compressing the file served on the fly for example).

Given that sendfile may still be sending (for large files) via the web server (x-send-file is now default) when you try to delete, I'd try looking into delayed solutions.

Possible solutions:

Use send_data rather than send_file (if files are small)
Schedule the deletion of the folder for later with something like delayed_job
Set up a sweeper which removes the folders at the end of each day

Rails sends 0 byte files using send_file

send_file has :x_sendfile param which defaults to true in Rails 3.
This feature offloads streaming download to front server - Apache (with mod_xsendfile) or lighttpd, by returning empty response with X-Sendfile header with path.

Nginx uses X-Accel-Redirect header for same functionality but you have to
configure Rails properly in proper environment file:

config.action_dispatch.x_sendfile_header = 'X-Accel-Redirect'

Rails 3 update: this line already exists in production.rb, just uncomment it.

Add sendfile on; to your nginx config to utilize header sent by Rails.
Remember the absolute path must be used and nginx must have read access to file.

Another way for aliased files:

For better security I use aliases in nginx instead of absolute paths,
however send_file method checks existence of file which fails with alias.
Thus I changed my action to:

  head(
        'X-Accel-Redirect'=> file_item.location,
        'Content-Type' => file_item.content_type,
        'Content-Disposition' => "attachment; filename=\"#{file_item.name}\"");
  render :nothing => true;

Where to store ftp files in rails app

There's no common place for this in a rails app file structure, so it's really up to you. As long as it's a known place on the file system then it should be fine. However, a couple of pointers:

Avoid storing them in the OS's temporary directory (or rails' tmp directory), as these are cleared in certain cases.
If you're going to use capistrano to deploy your application then it's probably best to keep the files in a directory that's outside of the rails app altogether, as a deployment will swap the app directory with a fresh copy. If this is a problem, and you're determined to keep the directory within the rails app, then you will have to put it in the shared directory that capistrano creates and create a symbolic link/shortcut.

Rails - X-Sendfile + Temporary Files