How to List All Files in an S3 Folder Using Aws-Sdk Gem in Ruby on Rails

list all the files from s3 using aws-sdk gem

You can simple iterate over bucket objects and use the with_prefix method

s3.buckets[YOUR BUCKET NAME].objects.with_prefix('videos/my_videos/college').each.collect(&:key)
#=> ["videos/my_videos/college/myfirst_day.mp4"]

OR use the as_tree method

s3.buckets[YOUR BUCKET NAME].as_tree(prefix:'videos/my_videos/college').select(&:leaf?).collect(&:key)
 #=> ['videos/my_videos/college/myfirst_day.mp4']

Obviously these are fictional since I have no access to your bucket but take a look at ObjectCollection and Tree for more methods in the AWSSDK.

There are quite a few methods for bucket traversal available such as Tree responds to children which will list both LeafNodes (File) and BranchNodes (Directory). BranchNodes will then also respond to children so you can make this recursive if needed.

To get the suffix (e.g. just the filename) you could possibly patch these in.

class LeafNode
  def suffix
    @member.key.split(delimiter).pop
  end
end
class S3Object
  def suffix
    @key.split("/").pop
  end
end

I have not fully tested these in any way but they should work for returning just the file name itself if it is nested inside a branch.

Retrieving list of files in certain folder in AWS S3(v 2.4.4) and deleting them

You have syntax error, options need to be a Hash. Your command should be:

bucket.objects({prefix: '/tmp-files/'})

For more options, refer docs: http://docs.aws.amazon.com/sdkforruby/api/Aws/S3/Bucket.html#objects-instance_method

list all items under a key of my bucket of AWS S3

Q1. With AWS Ruby SDK, how can I list all the items under
my-bucket/customers/products/?

bucket.objects({prefix: "customers/products/"})

Q2. How can I check e.g. my-bucket/customers/products/data3.txt
exists?

Use the S3 Object #exists? method like:

bucket.objects["customers/products/data3.txt"].exists?

aws-sdk for Ruby access folder within bucket

probably the S3 bucket are trying to use is located outside the US-EAST (default location), so this should help you:

s3 = AWS::S3.new(
    :access_key_id => "KEY",
    :secret_access_key => "SECRET"
    :s3_endpoint => 's3-eu-west-1.amazonaws.com'
)

Choose your S3 enpdpoint from the following list:

US Standard *                           s3.amazonaws.com(default)
US West (Oregon) Region                 s3-us-west-2.amazonaws.com
US West (Northern California) Region    s3-us-west-1.amazonaws.com
EU (Ireland) Region                     s3-eu-west-1.amazonaws.com
Asia Pacific (Singapore) Region         s3-ap-southeast-1.amazonaws.com
Asia Pacific (Tokyo) Region             s3-ap-northeast-1.amazonaws.com
South America (Sao Paulo) Region        s3-sa-east-1.amazonaws.com

In terms of object access, the bucket name is my_bucket, but my_folder should be a part of object.

Best way to return an array of files from Amazon S3 and sort by date?

If you are using the AWS SDK is fairly straightforward.

Once you have the array of files:

bucket.objects.sort_by &:last_modified

---

Related Topics