Piping Stdin to R

Piping stdin to R

This does not happen if you explicitly open the stdin connection.

#!/usr/bin/env Rscript
f <- file("stdin")
open(f)
while(length(line <- readLines(f,n=1)) > 0) {
  write(line, stderr())
  # process line
}

R, STDIN to Rscript

I still like r over Rscript here (but then I am not unbiased in this ...)

edd@rob:~$ (echo "Hello,World";echo "Bye,Bye") | r -e 'X <- readLines(stdin());print(X)' -
Hello,World
Bye,Bye
[1] "Hello,World" "Bye,Bye"
edd@rob:~$ 

r can also read.csv() directly:

edd@rob:~$ (echo "X,Y"; echo "Hello,World"; echo "Bye,Bye") | r -d -e 'print(X)' -
      X     Y
1 Hello World
2   Bye   Bye
edd@rob:~$

The -d is essentially a predefined 'read stdin into X via read.csv' which I think I borrowed as an idea from rio or another package.

Edit: Your example works with small changes:

1. Make it executable: chmod 0755 ex.R
2. Pipe output in correctly, ie use echo not cat
3. Use the ./ex.R notation for a file in the current dir
4. I changed it to use print(x)

Then:

edd@rob:~$ echo AAA | ./ex.R
[1] "AAA"
edd@rob:~$

Have an Rscript read or take input from stdin

You need to read text from the connection created by file("stdin") in order to pass anything useful to the text argument of writeLines(). This should work

#!/usr/bin/Rscript 
writeLines(readLines(file("stdin")))

How to make R script takes input from pipe and user given parameter

For very basic usage, have a look at ?commandArgs.

For more complex usage, two popular packages for command-line arguments and options parsing are getopt and optparse. I use them all the time, they get the job done. I also see argparse, argparser, and GetoptLong but have never used them before. One I missed: Dirk recommended that you look at docopt which does seem very nice and easy to use.

Finally, since you seem to be passing arguments via pipes you might find this OpenRead() function useful for generalizing your code and allowing your arguments to be pipes or files.

---

I wanted to test docopt so putting it all together, your script could look like this:

#!/usr/bin/env Rscript

## Command-line parsing ##

'usage: my_prog.R [-v -m <msg>] <param> <file_arg> 

options:
 -v        verbose
 -m <msg>  Message' -> doc

library(docopt)
opts <- docopt(doc)

if (opts$v) print(str(opts)) 
if (!is.null(opts$message)) cat("MESSAGE: ", opts$m)

## File Read ##

OpenRead <- function(arg) {
   if (arg %in% c("-", "/dev/stdin")) {
      file("stdin", open = "r")
   } else if (grepl("^/dev/fd/", arg)) {
      fifo(arg, open = "r")
   } else {
      file(arg, open = "r")
   }
}

dat.con <- OpenRead(opts$file_arg)
dat <- read.table(dat.con, sep = " ", header = FALSE)

# do something with dat and opts$param

And you can test running:

echo "1 2 3" | ./test.R -v -m HI param_1 -

or

./test.R -v -m HI param_1 some_file.txt

how do I pipe output of a python script into an Rscript?

Yes, a little change in frequency.R (reading from stdin) allows to duplicate a .csv:

l@np350v5c:~$ cat foo.sh 
cat gbr_Country_en_csv_v2.csv | Rscript frequency.R

l@np350v5c:~$ cat frequency.R 
f <- file("stdin")
dat <- read.csv(f, header=TRUE)
write.csv(dat, file = "out2.csv")

Is it possible to pipe code to R or Rscript in commandline?

Rscript will not listen to stdin:

$ echo "2 + 2" | Rscript

Usage: /path/to/Rscript [--options] [-e expr [-e expr2 ...] | file] [args]

--options accepted are
  --help              Print usage and exit
  --version           Print version and exit
  --verbose           Print information on progress
  --default-packages=list
                      Where 'list' is a comma-separated set
                        of package names, or 'NULL'
or options to R, in addition to --slave --no-restore, such as
  --save              Do save workspace at the end of the session
  --no-environ        Don't read the site and user environment files
  --no-site-file      Don't read the site-wide Rprofile
  --no-init-file      Don't read the user R profile
  --restore           Do restore previously saved objects at startup
  --vanilla           Combine --no-save, --no-restore, --no-site-file
                        --no-init-file and --no-environ

'file' may contain spaces but not shell metacharacters
Expressions (one or more '-e <expr>') may be used *instead* of 'file'
See also  ?Rscript  from within R
$

But littler has been doing this just fine as it was built for this (and more):

$ echo "2 + 2" | r -p                # -p switch needed for print
[1] 4
$ echo "print(2 + 2)" | r 
[1] 4
$ 

Note that operations are by default "silent" either explicit print() statements or the -p flag are your friend.

For completeness, R can now do it too but I forget when it was added:

$ echo "2 + 2" | R --slave
[1] 4
$ 

I have an older blog post comparing the start-up speeds, so my money is still on littler for these things -- and I have numerous scripts and cron jobs using it as it "just works".

R check if `stdin()` is empty

You can use blocking = FALSE. The following script will check input from stdin every half second and terminate when a full line is received.

#!/usr/bin/Rscript
inp <- file("stdin", "r", blocking = FALSE)
i <- 0
while (TRUE) {
    line <- readLines(inp, n = 1)
    if (length(line) > 0) {
        cat(sprintf("read a line: %s\n", line))
        break;
    }
    Sys.sleep(0.5)
    cat(sprintf("%d - nothing\n", i))
    i <- i + 1
}

---

Related Topics