How to extract multiple JSON objects from one file?
Use a json array, in the format:
[
{"ID":"12345","Timestamp":"20140101", "Usefulness":"Yes",
"Code":[{"event1":"A","result":"1"},…]},
{"ID":"1A35B","Timestamp":"20140102", "Usefulness":"No",
"Code":[{"event1":"B","result":"1"},…]},
{"ID":"AA356","Timestamp":"20140103", "Usefulness":"No",
"Code":[{"event1":"B","result":"0"},…]},
...
]
Then import it into your python code
import json
with open('file.json') as json_file:
data = json.load(json_file)
Now the content of data is an array with dictionaries representing each of the elements.
You can access it easily, i.e:
data[0]["ID"]
Parse a json file with multiple json objects
You can read the file to a list of dictionaries:
import json
my_list = json.load(open("test.json"))
>>> my_list
[{'uuid': '4c85cb89-88bb-4019-81ee-1215363c9e9a',
'ulid': '01FX3473AG0WJHEQ29A62HGQA9'},
{'uuid': '1d5af2a4-9ed4-4f96-996c-6d1580a691b4',
'ulid': '01FX34BBRC5P3C1ECMDG9Q5VJQ'}]
Loading and parsing a JSON file with multiple JSON objects
You have a JSON Lines format text file. You need to parse your file line by line:
import json
data = []
with open('file') as f:
for line in f:
data.append(json.loads(line))
Each line contains valid JSON, but as a whole, it is not a valid JSON value as there is no top-level list or object definition.
Note that because the file contains JSON per line, you are saved the headaches of trying to parse it all in one go or to figure out a streaming JSON parser. You can now opt to process each line separately before moving on to the next, saving memory in the process. You probably don't want to append each result to one list and then process everything if your file is really big.
If you have a file containing individual JSON objects with delimiters in-between, use How do I use the 'json' module to read in one JSON object at a time? to parse out individual objects using a buffered method.
How to read multiple JSON objects in a single file?
for line in f:
data = json.load(f)
This makes no sense. You are trying to parse the file over and over again, as many times as the number of lines in the file. This is more problematic than it sounds since f
is exhausted after the first call to json.load(f)
.
You don't need the loop, just pass f
to json.load
:
with open('/home/data.json') as f:
data = json.load(f)
print(data)
outputs
[{'uuid': '6f476e26', 'created': '2018-09-26T06:57:04.142232', 'creator': 'admin'},
{'uuid': '11d1e78a', 'created': '2019-09-21T11:19:39.845876', 'creator': 'admin'}]
Now you can loop over data
or directly access a specific index, ie data[0]
or data[1]
.
Parsing Json containing multiple json objects wrapped in string from s3
You'll have to do it line by line. This will produce a list of objects.
import boto3
import json
s3 = boto3.resource('s3')
content_object = s3.Object(bucket,key)
file_content = content_object.get()['Body'].read().decode('utf-8')
json_content = [json.loads(line) for line in file_content.splitlines()]
print(json_content)
Parse file with multiple JSON objects in Python
If every json object is on its own line, you should be able to do something like
with open('/path/to/file') as data:
objects = [json.loads(line) for line in data]
how to load multiple json objects in python
The content of file you described is not a valid JSON object this is why bot approaches are not working.
To transform in something you can load with json.load(fd)
you have to:
- add a
[
at the beginning of the file - add a
,
between each object - add a
]
at the very end of the file
then you can use the Method 2.
For instance:
[ { "a": 1,
"b" : 2,
"c" : {
"d":3
}
}, { "e" : 4,
"f" : 5,
"g" : {
"h":6
}
}
]
is a valid JSON array
If the file format is exactly as you've described you could do
with open(filename, 'r') as infile:
data = infile.read()
new_data = data.replace('}{', '},{')
json_data = json.loads(f'[{new_data}]')
JavaScript JSON parsing failing with multiple objects from a json file
Only valid JSON will be parsed correctly and as you are having more than one object, you should wrap those objects inside []
like this:
student.json
[{
"name": "Sara",
"age": 23,
"gender": "Female",
"department": "History",
"car": "Honda"
},
{
"name": "Sara",
"age": 23,
"gender": "Female",
"department": "History",
"car": "Honda"
}]
JS Code:
const fs = require('fs');
let rawdata = fs.readFileSync('student.json');
let students = JSON.parse(rawdata);
students.forEach((student) => {
console.log(student);
});
Related Topics
Best Way to Get the Max Value in a Spark Dataframe Column
Clicking Links With Python Beautifulsoup
How to Restart a Program Based on User Input
How to Extract List from List of Lists When Any One Element Match With Another List'S Element
Python Pandas .Isnull() Does Not Work on Nat in Object Dtype
How to Import a File in Python With Spaces in the Name
Move Files Between Two Aws S3 Buckets Using Boto3
Valueerror: Cannot Reshape Array of Size 30470400 into Shape (50,1104,104)
How to Clear All Variables in the Middle of a Python Script
Using Python, How to Access a Shared Folder on Windows Network
How to Verify If a Button Is Enabled and Disabled in Webdriver Python
How to Get Max Output from a While Loop
How to Check Whether a Number Is Divisible by Another Number
How to Get One Key and Value from a Json in Python
A Way to Quick Preview .Ipynb Files
How to Extract Hours and Minutes from a Datetime.Datetime Object