How to Create Streams from String in Node.Js

How to create streams from string in Node.Js?

From node 10.17, stream.Readable have a from method to easily create streams from any iterable (which includes array literals):

const { Readable } = require("stream")

const readable = Readable.from(["input string"])

readable.on("data", (chunk) => {
  console.log(chunk) // will be called once with `"input string"`
})

Note that at least between 10.17 and 12.3, a string is itself a iterable, so Readable.from("input string") will work, but emit one event per character. Readable.from(["input string"]) will emit one event per item in the array (in this case, one item).

Also note that in later nodes (probably 12.3, since the documentation says the function was changed then), it is no longer necessary to wrap the string in an array.

https://nodejs.org/api/stream.html#stream_stream_readable_from_iterable_options

How do I turn a String into a Readable Stream?

What you need as you say yourself is a readable stream not a transformation stream. Additionally you have a bug because string.split('') always return the same Array and then .shift() will always return the same letter. Your code once rewritten is as follows:

'use strict'

Readable = require('stream').Readable

stringer = (string) ->
  array = string.split('')
  new Readable
    read: (size) ->
      @push array.shift()
      return

readable = stringer('hello')

readable.on 'readable', ->
  console.log 'readable'
  return

readable.pipe process.stdout

Convert String to Readable in TypeScript

The problem in the first place was that i had an old @types/node module installed i updated it and now it Works.

npm install @types/node

Here is the workaround that worked with the older Version:

import { Readable } from 'stream'
/* tslint:disable-next-line: no-string-literal */
const dataStream = Readable['from'](["My String"])

how to create a readable stream from a remote url in nodejs?

fs.createReadStream() does not work with http URLs only file:// URLs or filename paths. Unfortunately, this is not described in the fs doc, but if you look at the source code for fs.createReadStream() and follow what it calls you can find that it ends up calling fileURULtoPath(url) which will throw if it's not a file: URL.

function fileURLToPath(path) {
  if (typeof path === 'string')
    path = new URL(path);
  else if (!isURLInstance(path))
    throw new ERR_INVALID_ARG_TYPE('path', ['string', 'URL'], path);
  if (path.protocol !== 'file:')
    throw new ERR_INVALID_URL_SCHEME('file');
  return isWindows ? getPathFromURLWin32(path) : getPathFromURLPosix(path);
}

It would suggest using the got() library to get yourself a readstream from a URL:

const got = require('got');
const mp4Url = 'https://www.example.com/path/to/mp4Video.mp4';

app.get('/video', (req, res) => {
    got.stream(mp4Url).pipe(res);
});

More examples described in this article: How to stream file downloads in Nodejs with Got.

---

You can also use the plain http/https modules to get the readstream, but I find got() to be generally useful at a higher level for lots of http request things so that's what I use. But, here's code with the https module.

const https = require('https');
const mp4Url = 'https://www.example.com/path/to/mp4Video.mp4';

app.get("/", (req, res) => {
    https.get(mp4Url, (stream) => {
        stream.pipe(res);
    });
});

More advanced error handling could be added to both cases.

How do I read the contents of a Node.js stream into a string variable?

(This answer is from years ago, when it was the best answer. There is now a better answer below this. I haven't kept up with node.js, and I cannot delete this answer because it is marked "correct on this question". If you are thinking of down clicking, what do you want me to do?)

The key is to use the data and end events of a Readable Stream. Listen to these events:

stream.on('data', (chunk) => { ... });
stream.on('end', () => { ... });

When you receive the data event, add the new chunk of data to a Buffer created to collect the data.

When you receive the end event, convert the completed Buffer into a string, if necessary. Then do what you need to do with it.

how to send data with Readable stream from NodeJS to C++

Implementing this with all the bells and whistles is not trivial. As always, the most complex part is handling all the errors.

I suggest you go check in Node.js sources src/js_stream.cc which contains something similar and you will understand why a good/fast implementation is difficult.

If you want to keep it simple, you can start by creating a JS reference to your C++ on_data function.

Then you can register this function as an event receiver.

Napi::Value ondata(const Napi::CallbackInfo &info) {
  if (!info[0].IsObject()) printf("Something strange received");
  else printf("received %s\n", info[0].ToString().Utf8Value().c_str());
  return info.Env().Undefined();
}

Napi::Value stream(const Napi::CallbackInfo &info) {
  Napi::Object stream = info[0].ToObject();
  Napi::Function on_data_ref = Napi::Function::New(info.Env(), ondata, "on_data");
  Napi::Value onValue = stream.Get("on");
  if (!onValue.IsFunction()) throw Napi::Error::New(info.Env(), "This is not an event emitter");
  Napi::Function on = onValue.As<Napi::Function>();
  on.Call(stream, {Napi::String::New(info.Env(), "data"), on_data_ref});
  return info.Env().Undefined();
}

If you do this, data will start flowing. But you still have to do correctly the error handling and everything else.

Node.js - How to get stream into string

Instead of using the second nested function, try the toString() method on the stream. This can then be piped wherever you want it to go, and you can also write a through() method that can assign it to a variable or use it directly in the one function.

var http = require('http');

var string = '';
var request = http.get('http://www.google.cz', function (err, data){
  if (err) console.log(err);
  string = data.toString();
  //any other code you may want
});
//anything else

One last note--the http.get() method takes two parameters: the url, and a callback. This requires two parameters, and you may have been getting nothing because it was an empty error message.

---

Related Topics