Using Send_File to Download a File from Amazon S3

Using send_file to download a file from Amazon S3?

In order to send a file from your web server,

• you need to download it from S3 (see @nzajt's answer) or

• you can redirect_to @attachment.file.expiring_url(10)

NodeJS Amazon AWS S3 getObject how to send file in API response to download

Server Side

const aws = require('aws-sdk');

router.get('/getfilefroms3', async (req, res, next) => {

  aws.config.update({
    secretAccessKey: config.secret_access_key,
    accessKeyId: config.access_key_id,
    signatureVersion: config.signature_version,
    region: config.region
  })

  const s3 = new aws.S3({ });

  var params = { Bucket: config.sample_bucket_name, Key: req.query.filename };

  s3.getObject(params, function (err, data) {
    if (err) {
      res.status(200);
      res.end('Error Fetching File');
    }
    else {
      res.attachment(params.Key); // Set Filename
      res.type(data.ContentType); // Set FileType
      res.send(data.Body);        // Send File Buffer
    }
  });

})

Client Side

If you are using Web Application you can use any HTTP REST API Client like Axios or Fetch, The Download Manager will capture the file.

curl --location --request GET 'http://localhost:5001/getfilefroms3?filename=sample.pdf'

If you are using NodeJS Application

var http = require('http');
var fs = require('fs');

var download = function (url, destination, callback) {
    var file = fs.createWriteStream(destination);
    http.get(url, function (response) {
        response.pipe(file);
        file.on('finish', function () {
            file.close(callback);
        });
    });
}

var fileToDownload = "sample.pdf"

download("http://localhost:5001/getfilefroms3?filename=" + fileToDownload, "./" + fileToDownload, () => { console.log("File Downloaded") })

NodeJS How do I Download a file to disk from an aws s3 bucket?

This is the entire code using streaming on the latest version of aws-sdk

var express = require('express');
var app = express();
var fs = require('fs');

app.get('/', function(req, res, next){
    res.send('You did not say the magic word');
});


app.get('/s3Proxy', function(req, res, next){
    // download the file via aws s3 here
    var fileKey = req.query['fileKey'];

    console.log('Trying to download file', fileKey);
    var AWS = require('aws-sdk');
    AWS.config.update(
      {
        accessKeyId: "....",
        secretAccessKey: "...",
        region: 'ap-southeast-1'
      }
    );
    var s3 = new AWS.S3();
    var options = {
        Bucket    : '/bucket-url',
        Key    : fileKey,
    };

    res.attachment(fileKey);
    var fileStream = s3.getObject(options).createReadStream();
    fileStream.pipe(res);
});

var server = app.listen(3000, function () {
    var host = server.address().address;
    var port = server.address().port;
    console.log('S3 Proxy app listening at http://%s:%s', host, port);
});

Rails: How to send file from S3 to remote server

Take a look at:
https://github.com/rest-client/rest-client/blob/master/lib/restclient/payload.rb

RestClient definitely supports streamed uploads. The condition is that in payload you pass something that is not a string or a hash, and that something you pass in responds to read and size. (so basically a stream).

On the S3 side, you basically need to grab a stream, not read the whole object before sending it. You use http://docs.aws.amazon.com/sdkforruby/api/Aws/S3/Client.html#get_object-instance_method and you say you want to get an IO object in the response target (not a string). For this purpose you may use an IO.pipe

reader, writer = IO.pipe

fork do 
    reader.close
    s3.get_object(bucket: 'bucket-name', key: 'object-key') do |chunk|
      writer.write(chunk)
    end
end

writer.close

you pass in the reader to the RestClient::Payload.generate and use that as your payload. If the reading part is slower than the writing part you may still read a lot in memory. you want, when writing to only do accept the amount you are willing to buffer in memory. You can read the size of the stream with writer.stat.size (inside the fork) and spin on it once it gets past a certain size.

Rails: allow download of files stored on S3 without showing the actual S3 URL to user

Yes, this is possible - just fetch the remote file with Rails and either store it temporarily on your server or send it directly from the buffer. The problem with this is of course the fact that you need to fetch the file first before you can serve it to the user. See this thread for a discussion, their solution is something like this:

#environment.rb
require 'open-uri'

#controller
def index
  data = open(params[:file])
  send_data data, :filename => params[:name], ...
end

This issue is also somewhat related.

Download a file from S3 to Local machine

You'll need to send it to the user. So, I think you have an expressJS and the user can get the element using your API endpoint.

After all you have done in your question, you will need to send it to the user.

res.sendFile('/path/to/downloaded/s3/object')

---

Related Topics