Extract nested list elements using bracketed numbers and names
The problem seems to be you not knowing how to extract list elements. As you said, there are two Sphericity Corrections data frames, so I will how to get the p[GG] value for both.
using bracketed number
For the first one, we do RM_test[[1]][[3]][[3]]. You can do it step by step to understand it:
x1 <- RM_test[[1]]; str(x1)
x2 <- x1[[3]]; str(x2)
x3 <- x2[[3]]; str(x3)
For the second one, do RM_test[[3]][[3]].
using bracketed name
Instead of using numbers for indexing, we can use names. For the first, do
RM_test[["ANOVA"]][["Sphericity Corrections"]][["p[GG]"]]
For the second, do
RM_test[["Sphericity Corrections"]][["p[GG]"]]
using $
For the first one, do
RM_test$ANOVA$"Sphericity Corrections"$"p[GG]"
For the second one, do
RM_test$"Sphericity Corrections"$"p[GG]"
Note the use of quote "" when necessary.
Nested lists multiple levels get elements by name
There isn't very much data in these files. I'm not sure what you're expecting. There are data frames called years and quarters in both lists. They both contain dates and values. Most of the content is NA. Each is a time-series data set starting in 1962. You don't need httr, either. fromJSON accepts a URL as a source without content or GET. m <- map(1:2, ~fromJSON(d[.x], flatten = T) where d is a vector with the two URLs. I think that's what @John Nielsen was stating. I used map, but lapply works fine, too.
You're using brackets, but you don't have to.
library(tidyverse) # for map and ggplot
library(jsonlite)
# get the JSON data and flatten it
mm <- map(1:2, ~fromJSON(d[.x], flatten = T))
# name each list, based on the URL
names(mm) <- c("Domestic", "International")
attributes(mm) # validate change
You can access the content with the environment pane, to get an idea of what you have.
You can use $ to access the content or [[]]. You can use numbers or names with [[]].
all.equal(mm[[1]][[1]], mm$Domestic$years)
# [1] TRUE
all.equal(mm[["Domestic"]][["years"]], mm$Domestic$years)
# [1] TRUE
To extract the data or to make a data frame-only object, you just need to assign it to an object name.
newDF <- mm$Domestic$years
You don't have to create a new object, though. If you wanted to plot the Domestic years data frame, you can use it as it is.
mm$Domestic$years %>%
mutate(year = as.integer(year),
value = as.numeric(value)) %>%
ggplot(aes(year, value)) + geom_path() + theme_bw()
The Domestic quarters data
mm$Domestic$quarters %>%
mutate(value = as.numeric(value)) %>%
ggplot(aes(seq_along(value), value)) + geom_path() +
scale_x_continuous(labels = mm$Domestic$quarters[seq(1, 220, 50), ]$date,
breaks = seq(1, 220, 50)) +
xlab("") + theme_bw()
You can do the same thing with the International data.
mm$International$years %>%
mutate(year = as.integer(year),
value = as.numeric(value)) %>%
ggplot(aes(year, value)) + geom_path() + theme_bw()
Having trouble accessing content of a nested list item
As it is a nested list, we can do
v1 <- docs[[200]][1]$content
v1
#[1] "I like cats"
Extract strings between brackets and nested brackets
You can do this with splits. If you separate the string using '_(' instead of only '_', the second part onward will be an enclosed keyword. you can strip the closing parentheses and split those parts on the '(' to get either one component (if there was no nested parentesis) or two components. You then form either a one-element list or dictionary depending on the number of components.
line = ";star/stellar_(class(ification))_(chart)"
if line.startswith(";"):
parts = [ part.rstrip(")") for part in line.split("_(")[1:]]
parts = [ part.split("(",1) for part in parts ]
parts = [ part if len(part)==1 else dict([part]) for part in parts ]
print(parts)
[{'class': 'ification'}, ['chart']]
Note that I assumed that the first part of the string is never included in the process and that there can only be one nested group at the end of the parts. If that is not the case, please update your question with relevant examples and expected output.
extracting unique elements from nested list in dataframe
within would be good for this. It allows for reassignment of the variables within the expression. Also, you could adjust your regular expression in strsplit so that you can remove those spaces and the commas in one go.
> within(b[-3],{
unique.participants <- sapply(strsplit(participants, "(,)|(, )"), unique)
rm(participants)
})
# id unique.participants
# 1 1-191 Takfir wa'l Hijra, AIS, GIA, AQIM, MUJAO
# 2 1-191 Takfir wa'l Hijra, AIS, GIA, AQIM, MUJAO
# 24 1-192 FLEC-R, FLEC-FAC
Since I'm seeing
I would like to perform these steps only for each ID (session number), which can be also repetitive.
in your question, I'm sticking with the duplicate row.
Extract data from a nested list and return a data.frame
Edited answer based on OP's new sample with nested lists, each representing one user (modified version of dataset reproduced below because there were more names than elements in each category, which didn't really make sense).
Data:
z2 <- structure(list(code = list("1"),
note = list("success"),
category = structure(list(name = list("Mary"),
department = structure(list(name = list("Math")), .Names = "name", id = "300"),
department = structure(list(name = list("English")), .Names = "name", id = "301"),
department = structure(list(name = list("Chinese")), .Names = "name", id = "302f"),
department = structure(list(name = list("Music")), .Names = "name", id = "317")),
.Names = c("name", "department", "department", "department", "department")),
category = structure(list(name = list("Kevin"),
department = structure(list(name = list("Physics")), .Names = "name", id = "12G0"),
department = structure(list(name = list("German")), .Names = "name", id = "321"),
department = structure(list(name = list("French")), .Names = "name", id = "325"),
department = structure(list(name = list("Spanish")), .Names = "name", id = "427")),
.Names = c("name", "department", "department", "department", "department")),
category = structure(list(name = list("Andy"),
department = structure(list(name = list("Swedish")), .Names = "name", id = "330"),
department = structure(list(name = list("Danish")), .Names = "name", id = "331"),
department = structure(list(name = list("Russian")), .Names = "name", id = "332"),
department = structure(list(name = list("Japanese")), .Names = "name", id = "341")),
.Names = c("name", "department", "department", "department", "department")),
category = structure(list(name = list("Nana"),
department = structure(list(name = list("Arabic")), .Names = "name", id = "200"),
department = structure(list(name = list("African")), .Names = "name", id = "201"),
department = structure(list(name = list("Sport")), .Names = "name", id = "202"),
department = structure(list(name = list("Korean")), .Names = "name", id = "211")),
.Names = c("name", "department", "department", "department", "department")),
category = structure(list(name = list("Sandy"),
department = structure(list(name = list("Vocals")), .Names = "name", id = "100"),
department = structure(list(name = list("Language")), .Names = "name", id = "515")),
.Names = c("name", "department", "department"))),
.Names = c("code", "note", "category", "category", "category", "category", "category"))
Drop unneeded elements from the list:
# keep only category elements (i.e. drop code, note, etc. from the list)
z2 <- z2[which(names(z2)=="category")]
Convert each nested list into a data frame & bind them together:
do.call(rbind,
lapply(unname(z2),
function(y){
data.frame(
name = y[[which(names(y)=="name")]][[1]],
department = sapply(y[which(names(y)=="department")], function(x){x[[1]][[1]]}),
id = sapply(y[which(names(y)=="department")], function(x){attr(x, "id")})
)
}))
name department id
1 Mary Math 300
2 Mary English 301
3 Mary Chinese 302f
4 Mary Music 317
5 Kevin Physics 12G0
6 Kevin German 321
7 Kevin French 325
8 Kevin Spanish 427
9 Andy Swedish 330
10 Andy Danish 331
11 Andy Russian 332
12 Andy Japanese 341
13 Nana Arabic 200
14 Nana African 201
15 Nana Sport 202
16 Nana Korean 211
17 Sandy Vocals 100
18 Sandy Language 515
Extract the first item of the list from nested lists
Try :
dt$col2 <-sapply(dt$col1 , "[", 1)
(The elements of your lists are not lists, just vectors, so one bracket is fine)
EDIT :
For a column inside a data.table :
dt$col2 <- sapply(dt$col1,function(x){sapply(x,'[',1)})
Finding common items between nested list and another list by casting them to set
Use set intersection:
s1 = set(l1)
i = s1.intersection( e[0] for e in l2 )
print(i) # set(['a', 'and', 'that', 'of', 'to', 'in', 'the'])
Set intersection (the method) can take any iterable to find the intersection with the set you call it on.
Your error stems from incorrectly using the lambda:
map(lambda x: set(l2[x][x]), l2[0:6]))
each x is one element of l2 (you only take the first six elements of l2. map takes each element of the input iterable and applies the function you provide. For the first element of l2 this would be:
set(l2[('the', 637)][('the', 637)])
wich is clearly wrong.
Extract name hierarchy for each leaf of a nested list
wdots <- names(rapply(l,length))
This works for the example given, but I only found it thanks to @flodel's comment. If you want the dollar symbols, there's
wdols <- gsub('\\.','\\$',wdots)
though, as @thelatemail pointed out, this will not give you what you want if any of the names in your hierarchy contain "."
Extract a delimited list from a string inside brackets RegEx Python
My 2 cents:
list(map(lambda y: [x.split('~') for x in re.findall(r'\[([^\].*\[]*)\]', y)], all_strings))
where all_strings is the list of string in the question plus '["if ([Business Layer~Scenario~Scenario Name] = ''Budget 2013'' and [Business Layer~GL Period~GL Year Number] = 2013) Then ([Business Layer~GL Balances~Period Net DR Amt]-[Business Layer~GL Balances~Period Net CR Amt]) else (0)", ''if'']'.
Here the result for each string in all_strings:
[Business Layer~Project Owning Org~Proj Owning Dept ID] --> [['Business Layer', 'Project Owning Org', 'Proj Owning Dept ID']]
[Business Layer~Project Owning Org~Proj Owning Org Name] --> [['Business Layer', 'Project Owning Org', 'Proj Owning Org Name']]
[Business Layer~Project~Proj No] --> [['Business Layer', 'Project', 'Proj No']]
[Business Layer~Project~Proj Name] --> [['Business Layer', 'Project', 'Proj Name']]
([Business Layer~Project~Proj No]) || COALESCE((' - ' || ([Business Layer~Project~Proj Name])), ' - ') --> [['Business Layer', 'Project', 'Proj No'], ['Business Layer', 'Project', 'Proj Name']]
[Project Assignment Fact~Task~Task No] --> [['Project Assignment Fact', 'Task', 'Task No']]
[Project Assignment Fact~Task~Task Name] --> [['Project Assignment Fact', 'Task', 'Task Name']]
([Project Assignment Fact~Task~Task No]) || COALESCE((' - ' || ([Project Assignment Fact~Task~Task Name])), ' - ') --> [['Project Assignment Fact', 'Task', 'Task No'], ['Project Assignment Fact', 'Task', 'Task Name']]
([Business Layer~Project~Proj No]) || COALESCE((' - ' || ([Project Assignment Fact~Task~Task No])), ' - ') || COALESCE((' - ' || ([Project Assignment Fact~Task~Task Name])), ' - ') --> [['Business Layer', 'Project', 'Proj No'], ['Project Assignment Fact', 'Task', 'Task No'], ['Project Assignment Fact', 'Task', 'Task Name']]
[Business Layer~Project Cost~Short Code Alias] --> [['Business Layer', 'Project Cost', 'Short Code Alias']]
[Business Layer~Expenditure Type~Expenditure Category Name] --> [['Business Layer', 'Expenditure Type', 'Expenditure Category Name']]
[Business Layer~Expenditure Type~Expenditure Type Parent Code] --> [['Business Layer', 'Expenditure Type', 'Expenditure Type Parent Code']]
[Business Layer~Expenditure Type~Expend Type Desc] --> [['Business Layer', 'Expenditure Type', 'Expend Type Desc']]
[Business Layer~Expenditure Owning Org~Exp Owning Org Name] --> [['Business Layer', 'Expenditure Owning Org', 'Exp Owning Org Name']]
[Business Layer~Transaction Source~Trans Source] --> [['Business Layer', 'Transaction Source', 'Trans Source']]
[Business Layer~Employee~Employee Name] --> [['Business Layer', 'Employee', 'Employee Name']]
[Business Layer~Project Cost~Expend Comment] --> [['Business Layer', 'Project Cost', 'Expend Comment']]
[Business Layer~Project Cost~PO No] --> [['Business Layer', 'Project Cost', 'PO No']]
[Business Layer~Project Cost~PV Invoice No] --> [['Business Layer', 'Project Cost', 'PV Invoice No']]
[Business Layer~Vendor~Vendor Name] --> [['Business Layer', 'Vendor', 'Vendor Name']]
[Business Layer~Scenario~Scenario Name] --> [['Business Layer', 'Scenario', 'Scenario Name']]
[Business Layer~ERS Employee~ERS Employee Name] --> [['Business Layer', 'ERS Employee', 'ERS Employee Name']]
[Business Layer~ERS Employee~ERS Employee Number] --> [['Business Layer', 'ERS Employee', 'ERS Employee Number']]
[Business Layer~Project Cost~Vehicle Tag No] --> [['Business Layer', 'Project Cost', 'Vehicle Tag No']]
[Business Layer~Project Cost~Vehicle Make] --> [['Business Layer', 'Project Cost', 'Vehicle Make']]
[Business Layer~Project Cost~Vehicle Model] --> [['Business Layer', 'Project Cost', 'Vehicle Model']]
[Business Layer~Project Cost~Vehicle Mileage] --> [['Business Layer', 'Project Cost', 'Vehicle Mileage']]
[Business Layer~Project Type~Proj Type Code] --> [['Business Layer', 'Project Type', 'Proj Type Code']]
[Business Layer~GL Period~GL Period Start Date] --> [['Business Layer', 'GL Period', 'GL Period Start Date']]
[Business Layer~Project Cost~Burdened Cost Amt] --> [['Business Layer', 'Project Cost', 'Burdened Cost Amt']]
["if ([Business Layer~Scenario~Scenario Name] = Budget 2013 and [Business Layer~GL Period~GL Year Number] = 2013) Then ([Business Layer~GL Balances~Period Net DR Amt]-[Business Layer~GL Balances~Period Net CR Amt]) else (0)", if] --> [['Business Layer', 'Scenario', 'Scenario Name'], ['Business Layer', 'GL Period', 'GL Year Number'], ['Business Layer', 'GL Balances', 'Period Net DR Amt'], ['Business Layer', 'GL Balances', 'Period Net CR Amt']]
Related Topics
Shiny Doesn't Show Me the Entire Selectinput When I Have Choices > 1000
How to Convert Ensembl Id to Gene Symbol in R
How to Hide Code in Rmarkdown, with Option to See It
Change Background Color of R Plot
Convert Ggplot Object to Plotly in Shiny Application
Writing Functions in R, Keeping Scoping in Mind
How to Optimize Read and Write to Subsections of a Matrix in R (Possibly Using Data.Table)
How to Calculate Wind Direction from U and V Wind Components in R
Reshape Wide Format, to Multi-Column Long Format
How Many Non-Na Values in Each Row for a Matrix
How to Save a Plot Made with Ggplot2 as Svg
Predicting Lda Topics for New Data
Ggplot2: Overlay Density Plots R
How to Split the Main Title of a Plot in 2 or More Lines
Real Time, Auto Updating, Incremental Plot in R