Http Requests and JSON Parsing in Python

HTTP requests and JSON parsing in Python

I recommend using the awesome requests library:

import requests

url = 'http://maps.googleapis.com/maps/api/directions/json'

params = dict(
    origin='Chicago,IL',
    destination='Los+Angeles,CA',
    waypoints='Joplin,MO|Oklahoma+City,OK',
    sensor='false'
)

resp = requests.get(url=url, params=params)
data = resp.json() # Check the JSON Response Content documentation below

JSON Response Content: https://requests.readthedocs.io/en/master/user/quickstart/#json-response-content

What's the best way to parse a JSON response from the requests library?

You can use json.loads:

import json
import requests

response = requests.get(...)
json_data = json.loads(response.text)

This converts a given string into a dictionary which allows you to access your JSON data easily within your code.

Or you can use @Martijn's helpful suggestion, and the higher voted answer, response.json().

Parsing HTTP Response in Python

When I printed response.read() I noticed that b was preprended to the string (e.g. b'{"a":1,..). The "b" stands for bytes and serves as a declaration for the type of the object you're handling. Since, I knew that a string could be converted to a dict by using json.loads('string'), I just had to convert the byte type to a string type. I did this by decoding the response to utf-8 decode('utf-8'). Once it was in a string type my problem was solved and I was easily able to iterate over the dict.

I don't know if this is the fastest or most 'pythonic' way of writing this but it works and theres always time later of optimization and improvement! Full code for my solution:

from urllib.request import urlopen
import json

# Get the dataset
url = 'http://www.quandl.com/api/v1/datasets/FRED/GDP.json'
response = urlopen(url)

# Convert bytes to string type and string type to dict
string = response.read().decode('utf-8')
json_obj = json.loads(string)

print(json_obj['source_name']) # prints the string with 'source_name' key

How to parse data from JSON request?

Your weather_data is a dictionary. One of the keys of the dictionary, description points to a list of dictionaries. You need to look at each item in that list and pull out the description. The easiest way is with a list comprehension:

descriptions = [item['description'] for item in weather_data['weather']]
# ['snow', 'mist']

That will give you a new list with just the descriptions.

Parsing and printing JSON result after GET request using Python

You can use Response.json(). Then iterate over data key:

import requests

resp = requests.get("https://smspva.com/api/rent.php?method=getcountries")
data = resp.json()
for country in data.get('data', []):
    print(f"Name:{country.get('name')} Code:{country.get('code')}")

Using Python to send HTTP request and parse JSON

With the information you provide, you can do something like this:

import requests
import csv

url = 'http://test_url'
with open('user.csv', 'rU') as data_file:
     data = csv.DictReader(data_file)
     for row in data:
         current_user = row['mdn']
         r = requests.get(url) # probably you also need to send current_user somehow. "r = requests.get(url+'?user='+current_user)" maybe?
         string_json = r.text.encode('utf-8')
         json = eval(string_json)   # only use this command if you are sure that string_json only contains a valid json and no malicious code instead
         with open('outputfile.csv','a') as outfile:
             outfile.write(current_user+';'+json["msg"]["mes"]+';'+json["msg"]["high"]+';'+json["msg"]["low"]+'\n') 

Please provide more information about the HTTP request usage and about the format of your csv for a more precise answer

How to parse JSON data from API response in Python?

Like @danil-kondratiev said, you can use response.json() and you don't need to encode/decode. Will this work for you?

import requests

response = requests.get('url with keys')

json_data = response.json() if response and response.status_code == 200 else None

if json_data and 'hoststatuslist' in json_data:
    if 'hoststatus' in json_data['hoststatuslist']:
        for hoststatus in json_data['hoststatuslist']['hoststatus']:
            host_name = hoststatus.get('name')
            status_text = hoststatus.get('status_text')

How do I parse a JSON response from Python Requests?

The manual suggests: if self.response.status_code == requests.codes.ok:

If that doesn't work:

if json.loads(self.response.text)['result'] == 'success':
   whatever()

---

Related Topics