Node.Js Can't Create Blobs

Node.js can't create Blobs?

The Solution to this problem is to create a function which can convert between Array Buffers and Node Buffers. :)

Convert a binary NodeJS Buffer to JavaScript ArrayBuffer

In recent node versions it's just:

let buffer = Buffer.from(arraybuffer);
let arraybuffer = Uint8Array.from(buffer).buffer;

How do I create a Blob in Node.js?

Node.js has experimental Blob support now, since v14.18 / v15.7.0, part of the ongoing effort to bring web APIs to Node.js so code can be reused without being rewritten.

Node.js ~~doesn't~~ didn't used to have Blob, it uses Buffer (not to be confused with ArrayBuffer) and typed arrays.

The blob npm module you tried to use isn't for use in Node.js, it's for use in a browser, to smooth over historical differences in how you create Blobs in different browsers. From its description:

A cross-browser Blob that falls back to BlobBuilder when appropriate. If neither is available, it exports undefined.

(my emphasis)

Somewhat confusingly, browser-targeted packages have been appearing (in droves) on npm the last few years. Modules using require have been a feature of bundlers like Webpack, Rollup, etc., for a while, and so people started using npm for common modules for browsers just like using it for common modules for Node.js. In fact, some modules are written to work in either environment. (But blob doesn't appear to be one of them.)

In comments, you've said you want to upload a file from your Node.js process. You don't need a Blob for that, the way you do this in Node.js is different from the way you do it in a browser. So you probably want to research how to upload files from Node.js, without worrying about Blobs. That would be a different question, though. (One which may be answered here or, if you're willing to use Express, here.)

Why JSZip can't create blob file?

JSZip throws this error at jszip/lib/utils.js:352:15 because of the value of support.blob, defined by at jszip/lib/support.js (lines 11 to 32). I don't know for your machine, but while trying to run this script in NodeJS, I've debugged it to the conclusion that JSZip detects blobs to be not supported because self is not defined, so line 23 throws an error and support.blob is set to false.

JSZip seems to support the Node Buffer type on Node though - the following doesn't throw any errors on my machine:

// JSZip v3.2.1
// NodeJS v8.10.0
// Ubuntu 18.04
const JSZip = require('jszip');

const zip = new JSZip();
zip.file('hello.txt', 'Hello world\n');
zip
    .generateAsync({type: 'nodebuffer'}) // blob -> nodebuffer
    .then(console.log);

How to create Blob from local file with require in NodeJS?

Use the Fetch API to request the file from the server. Then read it as an ArrayBuffer and decode it to an AudioBuffer by using the BaseAudioContext.decodeAudioData() method.

/**
 * Get a file, read it as an ArrayBuffer and decode it an AudioBuffer.
 * @param {string} file
 * @returns {Promise<AudioBuffer>}
 */
const fetchAudioBuffer = async file => {
  const audioContext = new AudioContext();

  try {
    const response = await fetch(file);
    const arrayBuffer = await response.arrayBuffer();
    return audioContext.decodeAudioData(arrayBuffer);
  } catch (error) {
    console.error(error);
  }
};

// Fetch the file and decode it as an AudioBuffer.
fetchAudioBuffer('path/to/assets/filename.mp3').then(audioBuffer => {
  // Use your audioBuffer here.
});

Can't upload blob to node js backend

I am able to write a workable code. Hope that it works for you also

HTML(EJS):

myview.ejs

<html>

<head>
  <title>File upload Node.</title>
</head>

<body>
  <button id='new-item'>BUTTON</button>

</body>
<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.7.1/jquery.min.js"></script>
<script src="http://cdnjs.cloudflare.com/ajax/libs/jquery.form/3.51/jquery.form.min.js"></script>

<script>
  function postItem() {
    let obj =
      'o' +
      '\nv -0.500000 -0.500000 0.500000' +
      '\nv 0.500000 -0.500000 0.500000' +
      '\nv -0.500000 0.500000 0.500000' +
      '\nvt 0.000000 0.000000' +
      '\nvt 1.000000 0.000000' +
      '\nvt 0.000000 1.000000' +
      '\nvn 0.000000 0.000000 1.000000' +
      '\nf 1/1/1 2/2/1 3/3/1';
    blob = new Blob([obj]);
    console.log(blob);
    let formData = new FormData();
    formData.append("files[]", blob, "test.wav");
    formData.append("test", "test");
    $.ajax({
      url: '/api/askjnj',
      type: 'POST',
      data: formData,
      success: function(data) {
        console.log(data);
      },
      error: function(err) {
        console.log(err);
      },
      cache: false,
      contentType: false,
      processData: false
    });
  } //postItem()

  $('#new-item').on('click', function() {
    postItem();
  });
</script>

</html>

Router:

myroutes.js

var express = require('express');
var app=express();

//var router = express.Router();
const multer = require('multer');
const upload = multer();
app.set('view engine', 'ejs');

app.get('/', function (req, res) {
    console.log('marhaba');
    res.render('myview', {});
});



app.post('/api/askjnj', upload.any(), (req, res) => {
    console.log('Here');
    console.log('Files: ', req.files);
    res.send('ok');
});
app.listen(3000, function () {
    console.log("Working on port 3000");
});

command : node myview.js

createBlockBlobFromStream() is overriding existing content instead of appending

The issue is that you're using createBlockBlobFromStream which will overwrite the contents of a blob. This method is used to create a blob in a single request (i.e. complete blob data is passed as input). From the documentation here:

Uploads a block blob from a stream. If the blob already exists on the
service, it will be overwritten. To avoid overwriting and instead
throw an error if the blob exists, please pass in an accessConditions
parameter in the options object.

In your case, you're uploading chunks of the data. What you would need to do is use createBlockFromStream method for each chunk that you're uploading.

Once all chunks are uploaded, you would need to call commitBlocks method to create the blob.

UPDATE

How can we generate the blockId?

A block id is simply a string. Key thing to remember is that when you're calling createBlockFromStream, the length of block id of each block you're sending must be the same. You can't use 1,2,3,4,5,6,7,8,9,10,11... for example. You will have to use something like 01,02,03,04,05,06,07,08,09,10,11... so that they're of same length. You can use GUID for that purpose. Also, the maximum length of the block id is 50 characters.

should it be unique for all?

Yes. For a blob, the block ids must be unique otherwise it will overwrite the content of a previous block uploaded with the same id.

can u show the example code so that i can try similar way to
implement?

Please take a look here.

Basically the idea is very simple: On your client side, you're chunking the file and sending each chunk separately. What you will do is apart from sending that data, you will also send a block id string. You will also need to keep that block id on your client side. You will repeat the same process for all the chunks of your file.

Once all chunks are uploaded successfully, you will make one more request to your server and send the list of all the block ids. Your server at that time will call commitBlocks to create the blob.

Please note that the order of block ids in your last request is important. Azure Storage Service will use this information to stitch the blob together.

Module 'buffer' has no exported member 'Blob'

We had the same problem after upgrading nest 7.5.x to 8.0.0. The dependency "supertest" for "nestjs/testing" has a dependency on "@types/supertest" which wildcards "@types/superagent": "*", and that dependency has another wildcard dependency "@types/node": "*", but the types within @types/supertest actually require @types/node >=16.X.X.

So nestjs/testing -> supertest -> @types/supertest -> @types/superagent -> @types/node >= 16.X.X is your problem and error.

The comments mentioned are accurate because these package managers wildcard their dependencies to get the latest version of dependencies. They should but do not add peerDependencies with dependencies requirements such as "@types/node": "">=12.0.0 <16.0.0". Instead they say anything, "@types/node": "*" so the error is post package install, no npm warnings/errors. "It worked yesterday but not today" is your big red flag because when you ran npm install, with these wildcard dependencies even though you did not know it installed the latest version. Since it installed everything wildcard today, but not yesterday, it worked yesterday.

In addition, but also important is that you are have pinned @types/node <16.0.0 thus your error in combination with the other package changes.

One option: revert your package-lock.json changes and run npm ci

Another option: set your package.json dependency for @types/node to -> "@types/node": "^16.0.0",.

Another option: accept that wildcards are wrong and you don't trust what is going on there so pin the @types/superagent dependency to the one prior.

As for me and my family, we use nestjs with AWS lambda which runtime does not include nodejs 16, and not everyone on my team runs npm ci we more typically run npm install so the solution was

package.json

    ...
    "devDependencies": {
        ...
        "@types/node": "14.18.2",
        "@types/superagent": "4.1.10",
        "@types/supertest": "^2.0.11",
        ...

Node.Js Can't Create Blobs