Python Read JSON File and Modify

Python read JSON file and modify

Set item using data['id'] = ....

import json

with open('data.json', 'r+') as f:
    data = json.load(f)
    data['id'] = 134 # <--- add `id` value.
    f.seek(0)        # <--- should reset file position to the beginning.
    json.dump(data, f, indent=4)
    f.truncate()     # remove remaining part

How to update a JSON file by using Python?

You did not save the changed data at all. You have to first load, then modify, and only then save. It is not possible to modify JSON files in-place.

with open('my_file.json', 'r') as f:
    json_data = json.load(f)
    json_data['b'] = "9"

with open('my_file.json', 'w') as f:
    f.write(json.dumps(json_data))

You may also do this:

with open('my_file.json', 'r+') as f:
    json_data = json.load(f)
    json_data['b'] = "9"
    f.seek(0)
    f.write(json.dumps(json_data))
    f.truncate()

If you want to make it safe, you first write the new data into a temporary file in the same folder, and then rename the temporary file onto the original file. That way you will not lose any data even if something happens in between.

If you come to think of that, JSON data is very difficult to change in-place, as the data length is not fixed, and the changes may be quite significant.

Working on .json files (read/write/change) on-the-fly with Python

I would recommend you to just use the json library. Read the file once and do occasionally a save, let's say every five minutes.
Maybe you want to create a new class in a singleton pattern. This will ensure that you have one object that holds the current up-to-date information and only this object can update the file.

Here is a quick sketch on how this class may look like:

# SaveFileHandler.py

import json

class _SaveFileHandler:
    def __init__(self):
        self._data = {}
        self._path = ""

    def read(self, path):
        self._path = path
        with open(path) as file:
            self._data = json.load(file)

    def write(self):
        with open(self._path) as file:
            json.dump(self._data, file)

    # now for every value you want data to have do the following
    # it allows you to check the value for errors and makes the usage more readable
    @property
    def key(self):
        return self._data["key"]

    @key.setter
    def key(self, value):
        self._data["key"] = value

# the is the only object you will ever use
save_file = _SaveFileHandler()

You can also read the save file directly in the init, this will make sure your data is always available.

How to modify a JSON file in a script?

import json
with open(path_to_json_file) as f:
    data = f.read()
d = json.loads(data)
d["browser"]["enabled_labs_experiments"] = ["ssl-version-max@2"]
with open(path_to_json_file, 'w') as f:
    f.write(json.dumps(d))

How to modify the multiple values in the JSON file using python

The code works as expected if you have resolved the variable name difference.

Further, I would also suggest to open the file using context manager with so that it gets closed automagically when the with block is exited.

import json, os
with open(".\example.json", 'r') as f:
    data = json.load(f)

for feature in data['images']:
    feature['file_name'] = os.path.basename(feature["file_name"])
    
print(data)

Output:

Data in json file:

{'images': [{'id': 1, 'file_name': 'Folder1/Folder2/Folder3/Folder4/1110.jpg', 'height': 3024, 'width': 4032}, {'id': 2, 'file_name': 'Folder1/Folder2/Folder3/Folder4/1111.jpg', 'height': 3024, 'width': 4032}, {'id': 3, 'file_name': 'Folder1/Folder2/Folder3/Folder4/1112.jpg', 'height': 3024, 'width': 4032}]}

Formatted data:

{'images': [{'id': 1, 'file_name': '1110.jpg', 'height': 3024, 'width': 4032}, {'id': 2, 'file_name': '1111.jpg', 'height': 3024, 'width': 4032}, {'id': 3, 'file_name': '1112.jpg', 'height': 3024, 'width': 4032}]}

How to open and edit an incorrect json file?

The file isn't JSON (the filename is incorrect); instead, it's composed of valid Python literals. There's no reason to try to transform it to JSON. Don't do that; instead, just tell Python to parse it as-is.

#!/usr/bin/env python3
import ast, json

results = [ ast.literal_eval(line) for line in open('qa_Appliances.json') ]
print(json.dumps(results))

...properly gives you a list named results with all your lines in it, and (for demonstration purposes) dumps it to stdout as JSON.

Update a JSON file with Python and keep the original format

Try json.dumps(json_obj, indent=4)

Example:

import json

status_wifi  = "ok" 

with open("config_wifi.json", "r") as jsonFile:
    data = json.load(jsonFile)
    data['wifi_session']['status'] = status_wifi 

with open("config_wifi.json", "w") as jsonFile:
    json.dump(json.dumps(data, indent=4), jsonFile)

The indent is the number of spaces for a tab.
If you set this parameter, the JSON will be formatted.

You can read more about it here.

How to modify JSON in python

Some errors in your code:

You don't need to re-iterate over the annotations again and again for every item in images, remove it out of the loop
You don't need to update all the keys of element for every item in annotations. Remove it out of the loop.
The usage of all other auxiliary containers such as listElements, listAnnotations, and kk are not needed. You can update the target elements directly if you refactored your code.

With the points above, consider this approach.

First, get all the annotations grouped by image_id
Then, iterate each image. For each image, just get the target annotation from the previous step. Construct the target dictionary and append to the result.

from collections import defaultdict

elements = []
annotation = defaultdict(list)

dataset_dicts = {"categories": [{"supercategory": "", "id": 0, "name": "Negative", "tiles": 2489, "bboxes": 5527, "className": "Negative"}, {"supercategory": "", "id": 1, "name": "Positive", "tiles": 5227, "bboxes": 15362, "className": "Positive"}], "images": [{"id": 224, "file_name": "img1.jpg", "height": 512, "width": 512}, {"id": 225, "file_name": "img2.jpg", "height": 512, "width": 512}], "annotations": [{"id": 716, "image_id": 224, "category_id": 0, "iscrowd": 0, "area": 2856.0, "bbox": [298, 18, 56, 51]}, {"id": 715, "image_id": 224, "category_id": 0, "iscrowd": 0, "area": 4096.0, "bbox": [185, 68, 64, 64]}, {"id": 714, "image_id": 224, "category_id": 0, "iscrowd": 0, "area": 2744.0, "bbox": [354, 10, 56, 49]}, {"id": 717, "image_id": 225, "category_id": 0, "iscrowd": 0, "area": 4096.0, "bbox": [374, 397, 64, 64]}]}

for item in dataset_dicts['annotations']:
    annotation[item["image_id"]].append(
        {
            "bbox": list(map(float, item["bbox"])),  # Or just item["bbox"].copy() if you don't intend it to be float
            "bbox_mode": 1,
            "category_id": str(item["category_id"]),  # Or just item["category_id"] if you don't intend it to be string
        }
    )

for item in dataset_dicts["images"]:
    elements.append(
        {
            "image_id": item["id"],
            "file_name": item["file_name"],
            "height": item["height"],
            "width": item["width"],
            "annotations": annotation[item["id"]],
        }
    )

print(elements)

Output

[
    {
        "image_id": 224,
        "file_name": "img1.jpg",
        "height": 512,
        "width": 512,
        "annotations": [
            {
                "bbox": [
                    298.0,
                    18.0,
                    56.0,
                    51.0
                ],
                "bbox_mode": 1,
                "category_id": "0"
            },
            {
                "bbox": [
                    185.0,
                    68.0,
                    64.0,
                    64.0
                ],
                "bbox_mode": 1,
                "category_id": "0"
            },
            {
                "bbox": [
                    354.0,
                    10.0,
                    56.0,
                    49.0
                ],
                "bbox_mode": 1,
                "category_id": "0"
            }
        ]
    },
    {
        "image_id": 225,
        "file_name": "img2.jpg",
        "height": 512,
        "width": 512,
        "annotations": [
            {
                "bbox": [
                    374.0,
                    397.0,
                    64.0,
                    64.0
                ],
                "bbox_mode": 1,
                "category_id": "0"
            }
        ]
    }
]

Python Read JSON File and Modify