Converting nested list (unequal length) to data frame
We get the length of list element ('indx') by looping with sapply. In the recent version of R, we can use lengths to replace the sapply(.., length) step. We change the length of each element to the max length from the 'indx' (length<-) and thereby pad NA values at the end of the list elements with length less than the max length. We can rbind the list elements, convert to data.frame and change the column names.
indx <- sapply(lst, length)
#indx <- lengths(lst)
res <- as.data.frame(do.call(rbind,lapply(lst, `length<-`,
max(indx))))
colnames(res) <- names(lst[[which.max(indx)]])
res
# sk ques pval diff imp
#1 10 sfsf 0.05 <NA> <NA>
#2 24 wwww 0.11 0.3 <NA>
#3 24 wwww 0.11 0.3 2
data
lst <- list(structure(c("10", "sfsf", "0.05"), .Names = c("sk", "ques",
"pval")), structure(c("24", "wwww", "0.11", "0.3"), .Names = c("sk",
"ques", "pval", "diff")), structure(c("24", "wwww", "0.11", "0.3",
"2"), .Names = c("sk", "ques", "pval", "diff", "imp")))
Convert list with nested lists of different lengths and empty elements into dataframe in R
library(tidyverse)
test %>%
map_df(~.x %>%
map(~if(length(.)) . else NA) %>%
do.call(what = cbind) %>%
as_tibble)
Gives
V1 V2 V3
<chr> <chr> <chr>
1 a b NA
2 a c NA
3 a d NA
4 j r NA
5 j s NA
6 h NA i
Create a pandas dataframe from a nested lists of unequal lengths
The zip_longest function from itertools does this:
>>> import itertools, pandas
>>> pandas.DataFrame((_ for _ in itertools.zip_longest(*nest)), columns=['aa', 'bb', 'cc'])
aa bb cc
0 aa1 bb1 cc1
1 aa2 bb2 cc2
2 aa3 bb3 cc3
3 aa4 bb4 None
4 aa5 None None
If you have an older version of pandas you may need to wrap zip_longest in a list constructor. On older Python you may need to call izip_longest instead of zip_longest.
Convert nested list with different names to data.frame filling NA and adding column
A shorter solution in base R would be
make_df <- function(a = NA, b = NA, z = NA) {
data.frame(a = unlist(a), b = unlist(b), z = unlist(z))
}
do.call(rbind, lapply(mylist, function(x) do.call(make_df, x)))
#> a b z
#> 1 1 2 <NA>
#> 2 3 NA <NA>
#> 3 NA 5 <NA>
#> 4 9 NA k
Update
A more general solution using the same method, but which does not require specific names would be:
build_data_frame <- function(obj) {
nms <- unique(unlist(lapply(obj, names)))
frmls <- as.list(setNames(rep(NA, length(nms)), nms))
dflst <- setNames(lapply(nms, function(x) call("unlist", as.symbol(x))), nms)
make_df <- as.function(c(frmls, call("do.call", "data.frame", dflst)))
do.call(rbind, lapply(mylist, function(x) do.call(make_df, x)))
}
This allows
build_data_frame(mylist)
#> a b z
#> 1 1 2 <NA>
#> 2 3 NA <NA>
#> 3 NA 5 <NA>
#> 4 9 NA k
Put nested lists of different lengths into a dataframe one bellow another
Like this?
do.call(rbind, lapply(NestedList, function(x)as.data.frame(x, col.names=c("First", "Second", "Third"))))
gives,
First Second Third
1 1 1 1
2 2 2 2
3 3 3 3
4 4 4 4
5 5 5 5
6 6 6 6
7 7 7 7
8 8 8 8
9 9 9 9
10 10 10 10
11 11 11 11
12 12 12 12
13 13 13 13
14 14 14 14
15 15 15 15
16 16 16 16
17 17 17 17
18 18 18 18
19 19 19 19
20 20 20 20
21 21 21 21
22 22 22 22
23 23 23 23
24 24 24 24
25 25 25 25
26 26 26 26
27 27 27 27
28 28 28 28
29 29 29 29
30 30 30 30
31 31 31 31
32 32 32 32
33 33 33 33
34 34 34 34
35 35 35 35
36 36 36 36
37 37 37 37
38 38 38 38
39 39 39 39
40 40 40 40
41 41 41 41
42 42 42 42
43 43 43 43
44 44 44 44
45 45 45 45
46 46 46 46
47 47 47 47
48 48 48 48
49 49 49 49
50 50 50 50
51 1 1 1
52 2 2 2
53 3 3 3
54 4 4 4
55 5 5 5
56 6 6 6
57 7 7 7
58 8 8 8
59 9 9 9
60 10 10 10
61 11 11 11
62 12 12 12
63 13 13 13
64 14 14 14
65 15 15 15
66 16 16 16
67 17 17 17
68 18 18 18
69 19 19 19
70 20 20 20
71 21 21 21
72 22 22 22
73 23 23 23
74 24 24 24
75 25 25 25
76 26 26 26
77 27 27 27
78 28 28 28
79 29 29 29
80 30 30 30
81 1 1 1
82 2 2 2
83 3 3 3
84 4 4 4
85 5 5 5
86 6 6 6
87 7 7 7
88 8 8 8
89 9 9 9
90 10 10 10
91 11 11 11
92 12 12 12
93 13 13 13
94 14 14 14
95 15 15 15
96 16 16 16
97 17 17 17
98 18 18 18
99 19 19 19
100 20 20 20
So, lapply goes through the list converting each element (i.e., each nested list) to a data frame and giving the columns the same names. do.call then binds these data frames together using rbind.
uneven nested list to dataframe
You could unlist and store it in a matrix and then change to a data frame.
# Helper function
conv=function(x) {
hr=floor(x/3600)
mins1=x%%3600
mins=floor(mins1/60)
secs=mins1%%60
return(paste0(hr,":",mins,":",secs))
}
# Mutation
library(dplyr)
library(lubridate)
n=names(unlist(res))[1:10]
f=matrix(unlist(res), ncol=10, byrow=TRUE)
f=data.frame(f, stringsAsFactors = FALSE)
colnames(f)=n
g=rename(f, b1=beta.b1, b2=beta.b2) %>%
mutate(across(time_iter:b2, as.numeric),
time_iter=time_iter*10000,
time_iter=conv(time_iter),
time_iter=as.difftime(time_iter, "%H:%M:%S", "secs"),
time_iter=time_iter/10000)
output
modeltype time_iter seed nobs hyperpar1 hyperpar2 metric1 metric2 b1 b2
1 tree 0.7099 secs 1 75 0.5 0.5 0.4847 0.2576 0.575 0.745
2 tree 0.0580 secs 2 75 0.5 0.5 0.4013 0.2569 0.535 0.775
3 tree 0.0460 secs 1 75 0.8 0.5 0.4755 0.2988 0.541 0.702
4 tree 0.0474 secs 2 75 0.8 0.5 0.2413 0.2147 0.545 0.793
5 tree 0.0502 secs 1 75 0.5 1.0 0.7131 0.5024 0.500 0.722
6 tree 2.9419 secs 2 75 0.5 1.0 0.4254 0.2824 0.555 0.712
7 tree 0.0410 secs 1 75 0.8 1.0 0.6709 0.4092 0.578 0.701
8 tree 0.0396 secs 2 75 0.8 1.0 0.4585 0.4115 0.501 0.777
convert nested list to data frame
This is a bit awkward because of the inconsisteny nesting levels, but we could write a recursive function to extract the lists that have "x" in their name. For example
find_x <- function(x) {
if (is.list(x) && !"x" %in% names(x)) {
return(do.call("rbind", lapply(x, find_x)))
} else if ( !is.list(x)) {
return(NULL)
} else {
return(x)
}
}
find_x(l)
# x y
# [1,] 1 2 3 4
# [2,] 3 4 5 6
# [3,] 1 2 3 4
# [4,] 2 3 4 5
You can change the "x" part to whatever marker you have for your own data of interest
Turning lists of unequal length lists into data frames
Here is an option using tidyverse
library(tidyverse)
out <- mylist %>%
transpose %>%
map(~ if(all(lengths(.x) == 1)) unlist(.x) else
map(.x, as_tibble) %>%
reduce(full_join, by = 'V1') %>%
mutate_all(funs(.[order(!is.na(.))])))
Now, we can extract the list elements
out$price %>%
set_names(c("Day", paste0("Price", LETTERS[1:3])))
# A tibble: 7 x 4
# Day PriceA PriceB PriceC
# <dbl> <dbl> <int> <int>
#1 1.00 NA NA 21
#2 2.00 NA NA 22
#3 3.00 NA 12 23
#4 4.00 NA 13 24
#5 5.00 7.00 14 25
#6 6.00 8.00 15 26
#7 7.00 9.00 16 27
Related Topics
Range Standardization (0 to 1) in R
What Does the Function Invisible() Do
Get Date Difference in Years (Floating Point)
Dealing with Very Small Numbers in R
Split a String Column into Several Dummy Variables
Compute the Minimum of a Pair of Vectors
Why Has Data.Table Defined := Rather Than Overloading <-
Saving and Loading a Model in R
Output in R, Avoid Writing "[1]"
In R, What Does a Negative Index Do
Ggplot2: Drop Unused Factors in a Faceted Bar Plot But Not Have Differing Bar Widths Between Facets