Migrating from Local Paperclip Storage to S3

Migrating from local paperclip storage to S3

Looks like I needed to go beyond ruby on this one, s3cmd seemed to be the most appropriate tool for this sort of job. In my case, the sync command did the trick:

s3cmd sync my-app/public/system/ s3://mybucket 

migrating old paperclip images from filesystem to S3

Solved it... I changed my rake task into this, I am saving my every ToyCar object image again and it does the job....

namespace :migrate_to_s3 do

  desc "Upload images to S3"
  task :toy_cars => :environment do
    toy_cars = ToyCar.all
    toy_cars.each do |car|
      image = Dir["#{Rails.root}/public/system/#{car.logo.path}"].first
      if image.present?
        car.update(:logo => File.open(image))
        puts "uploading"
      end
    end
  end

end

Updating Paperclip path file names from on server to s3

Say you have a model User with an attachment named profile_pic;

Go into the rails console eg. rails c and then get an object for the model you have the attachment on, eg. u = User.find(100).

Now type u.profile_pic.url to get the url or u.profile_pic_file_name to get the filename.

To see the effect of other options (for example your old options) you can do;

p = u.profile_pic # gets the paperclip attachment for profile_pic
puts p.url # gets the current url
p.options.merge!(url: '/blah/:class/:attachment/:id_partition/:style/:filename')
puts p.url # now shows url with the new options

Similarly p.path will show the local file path with whatever options you pick.

Long story short, something like;

User.where('created_at < some_date').map do |x| 
  "#{x.id} #{x.profile_pic_file_name} #{x.profile_pic.path}" 
end

should give you what you want :)

Paperclip multiple storage

Maybe you'd benefit from this:

http://airbladesoftware.com/notes/asynchronous-s3/

What you'll have to do is firstly upload to your local storage, and then "asynchronously" upload to S3

This is typically done through the likes of Resque or DelayedJob (as the tutorial demonstrates), and will require you to run some sort of third-party processing engine on your server (typically Redis or similar)

From the tutorial:

### Models ###

class Person < ActiveRecord::Base
  has_attached_file :local_image,
                    path: ":rails_root/public/system/:attachment/:id/:style/:basename.:extension",
                    url:  "/system/:attachment/:id/:style/:basename.:extension"

  has_attached_file :image,
                    styles: {large: '500x500#', medium: '200x200#', small: '70x70#'},
                    convert_options: {all: '-strip'},
                    storage:         :s3,
                    s3_credentials:  "#{Rails.root}/config/s3.yml",
                    s3_permissions:  :private,
                    s3_host_name:    's3-eu-west-1.amazonaws.com',
                    s3_headers:      {'Expires'             => 1.year.from_now.httpdate,
                                      'Content-Disposition' => 'attachment'},
                    path:            "images/:id/:style/:filename"

  after_save :queue_upload_to_s3

  def queue_upload_to_s3
    Delayed::Job.enqueue ImageJob.new(id) if local_image? && local_image_updated_at_changed?
  end

  def upload_to_s3
    self.image = local_image.to_file
    save!
  end
end

class ImageJob < Struct.new(:image_id)
  def perform
    image = Image.find image_id
    image.upload_to_s3
    image.local_image.destroy
  end
end

### Views ###

# app/views/people/edit.html.haml
# ...
= f.file_field :local_image

# app/views/people/show.html.haml
- if @person.image?
  = image_tag @person.image.expiring_url(20, :small)
- else
  = image_tag @person.local_image.url, size: '70x70'

---

Related Topics