R: Serialize Objects to Text File and Back Again

R: serialize objects to text file and back again

JD, we do that in the digest package via serialize() to/from raw. That is nice as you can store serialized objects in SQL and other places. I would actually store this as RData as well which is way quicker to load() (no parsing!) and save().

Or, if it has to be RawToChar() and ascii then use something like this (taken straight from help(digest) where we compare serialization of the file COPYING:

 # test 'length' parameter and file input
 fname <- file.path(R.home(),"COPYING")
 x <- readChar(fname, file.info(fname)$size) # read file
 for (alg in c("sha1", "md5", "crc32")) {
   # partial file
   h1 <- digest(x    , length=18000, algo=alg, serialize=FALSE)
   h2 <- digest(fname, length=18000, algo=alg, serialize=FALSE, file=TRUE)
   h3 <- digest( substr(x,1,18000) , algo=alg, serialize=FALSE)
   stopifnot( identical(h1,h2), identical(h1,h3) )
   # whole file
   h1 <- digest(x    , algo=alg, serialize=FALSE)
   h2 <- digest(fname, algo=alg, serialize=FALSE, file=TRUE)
   stopifnot( identical(h1,h2) )
 }

so with that your example becomes this:

R> outCon <- file("/tmp/jd.txt", "w")
R> mychars <- rawToChar(serialize(1:10, NULL, ascii=T))
R> cat(mychars, file=outCon); close(outCon)
R> fname <- "/tmp/jd.txt"
R> readChar(fname, file.info(fname)$size)
[1] "A\n2\n133633\n131840\n13\n10\n1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n"
R> unserialize(charToRaw(readChar(fname, file.info(fname)$size)))
[1]  1  2  3  4  5  6  7  8  9 10
R>

R: Creating a CSV out of serialized objects

The package caTools has a Base64 encoder-decoder that you can use:

> library(caTools)
> s<-base64encode(serialize("blah",NULL))
> s
[1] "WAoAAAACAAIKAQACAwAAAAAQAAAAAQAAAAkAAAAEYmxhaA=="
> unserialize(base64decode(s,"raw"))
[1] "blah"

Unserialize objects from in-memory ASCII instead of from a file connection

Does this fit your needs?

It follows the general strategy in your Approach 2. The only difference is that it uses as.character() to convert the serialized object to a character vector before passing it to toJSON(), and then uses as.raw(as.hexmode()) to convert it back to a raw vector "on the other side". (I've marked the two edited lines with comments reading ## <<- Edited.)

library(forecast)

## SERVER: estimates initial model and writes JSON to socket
model <- auto.arima(AirPassengers, trace = TRUE)
fc <- as.data.frame(forecast(model))
serialized <- as.character(serialize(model, NULL)) ## <<- Edited
class(serialized)

json_out <- list(data = AirPassengers, model = serialized, fc = fc)
json_out <- jsonlite::toJSON(json_out)

## CLIENT: keeps estimated model, updates data, writes to socket
json_in <- jsonlite::fromJSON(json_out)
json_in$data <- window(AirPassengers, end = 1949 + (1/12 * 14))

## SERVER: reads new JSON and applies model to new data
data <- json_in$data
model_0 <- as.raw(as.hexmode(json_in$model))       ## <<- Edited

unserialize(model_0)
## Series: AirPassengers 
## ARIMA(0,1,1)(0,1,0)[12]                    
## 
## Coefficients:
##           ma1
##       -0.3184
## s.e.   0.0877
## 
## sigma^2 estimated as 137.3:  log likelihood=-508.32
## AIC=1020.64   AICc=1020.73   BIC=1026.39

Import text file to R and change it's format

Solution 1

That looks a lot like javascript code. Execute the javascript (using a web browser) and save the result to JSON, then open the file with R with jsonlite.

With your example, create this file and save it as my_page.html:

<html>
<header>
<script>

// Initialize locations to be able to push more values in it
// probably not required with your full code
var locations = [];

vLatitude ='23.8145833';       
vLongitude ='90.4043056';
vcontents ='LRP: LRPS</br>Start of Road From the End of Banani Rail Crossing Over Pass</br>Division:Gazipur</br>Sub-Division:Tongi';
                
vLocations = new Array(vcontents, vLatitude, vLongitude);   
locations.push(vLocations);

// convert locations to json
var jsonData = JSON.stringify(locations);

// actually write the json to file
function download(content, fileName, contentType) {
    var a = document.createElement("a");
    var file = new Blob([content], {type: contentType});
    a.href = URL.createObjectURL(file);
    a.download = fileName;
    a.click();
}
download(jsonData, 'export_json.txt', 'text/plain');

</script>
</header>
<body>
Download should start automatically. You can look at the web console for errors.
</body>
</html>

When you open it with your web browser it should "download" a file, that you can open with R:

jsonlite::read_json("export_json.txt",simplifyVector = TRUE)

One problem is that the javascript code is created an array without names. So the names are not exported. I don't see how you could make javascript export it.

Solution 2

Instead of relying on a browser to execute the javascript code, you could do it directly in R with a javascript engine. It should give you the same result, but makes communication between the two easier.

Solution 3

If the file really looks like that all along, you might be able to remove the javascript lines that organize the arrays, and only keep the lines that define variables. In R, the symbols = and ; are technically valid, it's not too hard to rewrite the javascript into R code. Note this solution could be very fragile depending on what else is in your javascript code!

js_script <- "var locations = [];

vLatitude ='23.8145833';       
vLongitude ='90.4043056';
vcontents ='LRP: LRPS</br>Start of Road From the End of Banani Rail Crossing Over Pass</br>Division:Gazipur</br>Sub-Division:Tongi';
                
vLocations = new Array(vcontents, vLatitude, vLongitude);   
locations.push(vLocations);

// convert locations to json
var jsonData = JSON.stringify(locations);" %>%
  str_split(pattern = "\n", simplify=TRUE) %>%
  as.character() %>%
  str_trim()

# Find the lines that look like defining variables
js_script <- js_script[str_detect(js_script, pattern = "^\\w+ ?= ?'.*' ?;$")]

# make it into an R expression
r_code <- str_remove(js_script, ";$") %>%
  paste(collapse = ",")

r_code <- paste0("c(", r_code, ")")

# Execute
eval(str2expression(r_code))

cat appends CR to output in R

The solution here was similar to the answer found here: R: Getting Unix-like linebreak LF writing files with cat()

    f<-file("out.txt", open="wb");
    cat(txtFile,file=f,sep="");
    close(f);

Resulting text file output in notepad++:
Sample Image

Reliably convert any object to String and then back again

Yes, it is called serialization!

 String serializedObject = "";

 // serialize the object
 try {
     ByteArrayOutputStream bo = new ByteArrayOutputStream();
     ObjectOutputStream so = new ObjectOutputStream(bo);
     so.writeObject(myObject);
     so.flush();
     serializedObject = bo.toString();
 } catch (Exception e) {
     System.out.println(e);
 }

 // deserialize the object
 try {
     byte b[] = serializedObject.getBytes(); 
     ByteArrayInputStream bi = new ByteArrayInputStream(b);
     ObjectInputStream si = new ObjectInputStream(bi);
     MyObject obj = (MyObject) si.readObject();
 } catch (Exception e) {
     System.out.println(e);
 }

Write xml-object to disk

xml2 objects have external pointers that become invalid when you serialize them naively. The package provides xml_serialize() and xml_unserialize() objects to handle this for you. Unfortunately the API is slightly cumbersome because base::serialize() and base::unserialize() assume an open connection.

library(xml2)

x <- read_xml("<foo>
              <bar>text <baz id = 'a' /></bar>
              <bar>2</bar>
              <baz id = 'b' />
              </foo>")

# function to save and read object
roundtrip <- function(obj) {
  tf <- tempfile()
  con <- file(tf, "wb")
  on.exit(unlink(tf))

  xml_serialize(obj, con)
  close(con)
  con <- file(tf, "rb")
  on.exit(close(con), add = TRUE)
  xml_unserialize(con)
}
x
#> {xml_document}
#> <foo>
#> [1] <bar>text <baz id="a"/></bar>
#> [2] <bar>2</bar>
#> [3] <baz id="b"/>
(y <- roundtrip(x))
#> {xml_document}
#> <foo>
#> [1] <bar>text <baz id="a"/></bar>
#> [2] <bar>2</bar>
#> [3] <baz id="b"/>

identical(x, y)
#> [1] FALSE
all.equal(x, y)
#> [1] TRUE
xml_children(y)
#> {xml_nodeset (3)}
#> [1] <bar>text <baz id="a"/></bar>
#> [2] <bar>2</bar>
#> [3] <baz id="b"/>
as_list(y)
#> $bar
#> $bar[[1]]
#> [1] "text "
#> 
#> $bar$baz
#> list()
#> attr(,"id")
#> [1] "a"
#> 
#> 
#> $bar
#> $bar[[1]]
#> [1] "2"
#> 
#> 
#> $baz
#> list()
#> attr(,"id")
#> [1] "b"

Also in regards to the second part of your question, I would seriously consider using XPATH expressions to extract the desired data, even if you have to rewrite code.

How to serialise request.POST to a database and back again

Django's serializer interface works with django model objects. It will not work with other objects.

You may try to use json

if request.POST:
     # save checkpoint
     obj.checkpoint = json.dumps(request.POST)
     post_data = request.POST
else:
     # load last version from database.
     post_data = json.loads(obj.checkpoint)

formset = MyFormSet(post_data)

R: Serialize Objects to Text File and Back Again