Parsing JSON from Url

How to get JSON from URL in JavaScript?

You can use jQuery .getJSON() function:

$.getJSON('http://query.yahooapis.com/v1/public/yql?q=select%20%2a%20from%20yahoo.finance.quotes%20WHERE%20symbol%3D%27WRC%27&format=json&diagnostics=true&env=store://datatables.org/alltableswithkeys&callback', function(data) {
    // JSON result in `data` variable
});

If you don't want to use jQuery you should look at this answer for pure JS solution.

Parsing JSON from URL

1. First you need to download the URL (as text):

   private static String readUrl(String urlString) throws Exception {
       BufferedReader reader = null;
       try {
           URL url = new URL(urlString);
           reader = new BufferedReader(new InputStreamReader(url.openStream()));
           StringBuffer buffer = new StringBuffer();
           int read;
           char[] chars = new char[1024];
           while ((read = reader.read(chars)) != -1)
               buffer.append(chars, 0, read); 
   
           return buffer.toString();
       } finally {
           if (reader != null)
               reader.close();
       }
   }
   

2. Then you need to parse it (and here you have some options).

   • GSON (full example):

     static class Item {
         String title;
         String link;
         String description;
     }
     
     static class Page {
         String title;
         String link;
         String description;
         String language;
         List<Item> items;
     }
     
     public static void main(String[] args) throws Exception {
     
         String json = readUrl("http://www.javascriptkit.com/"
                               + "dhtmltutors/javascriptkit.json");
     
         Gson gson = new Gson();        
         Page page = gson.fromJson(json, Page.class);
     
         System.out.println(page.title);
         for (Item item : page.items)
             System.out.println("    " + item.title);
     }
     

     Outputs:

     javascriptkit.com
         Document Text Resizer
         JavaScript Reference- Keyboard/ Mouse Buttons Events
         Dynamically loading an external JavaScript or CSS file
     

   • Try the java API from json.org:

     try {
         JSONObject json = new JSONObject(readUrl("..."));
     
         String title = (String) json.get("title");
         ...
     
     } catch (JSONException e) {
         e.printStackTrace();
     }
     

Parsing JSON data from a URL

Your return response; won't be of any use. You can pass a function as an argument to get_json, and have it receive the result. Then in place of return response;, invoke the function. So if the parameter is named callback, you'd do callback(response);.

// ----receive function----v
function get_json(url, callback) {
    http.get(url, function(res) {
        var body = '';
        res.on('data', function(chunk) {
            body += chunk;
        });

        res.on('end', function() {
            var response = JSON.parse(body);
// call function ----v
            callback(response);
        });
    });
}

         // -----------the url---v         ------------the callback---v
var mydata = get_json("http://webapp.armadealo.com/home.json", function (resp) {
    console.log(resp);
});

Passing functions around as callbacks is essential to understand when using NodeJS.

How would I parse json from an external url?

First of all you forgot to pass callback function to getJSON as second parameter, which is supposed to be called
when your xhr returns with the data.
Second, you do not need to parse data to json when you are asking for JSON file from server and setting responseType to JSON, this would be automatically done for you.

var getJSON = function(url, callback) {
var xhr = new XMLHttpRequest();
xhr.open('GET', url, true);
xhr.responseType = 'json';
xhr.onload = function() {
  var status = xhr.status;
  if (status === 200) {
    callback(null, xhr.response);
  } else {
    callback(status, xhr.response);
  }
};
xhr.send();
 };

function yourCallBackFunction(err, data){
    if(err){
        //Do something with the error 
    }else{
        //data  is the json response that you recieved from the server
    }

}

 function statsget() {
 var uname = document.getElementById("nameget").value;
 var data = getJSON(`https://www.reddit.com/user/${uname}/circle.json`, yourCallBackFunction);

 }

Let me know if you need more details on this.

How to parse JSON response from an API url using https request in java?

look what happens when you call the provided url from a shell curl:

> $ curl 'https://www.nseindia.com/api/quote-derivative?symbol=RELIANCE'                                                 [±master ●●▴]
<HTML><HEAD>
<TITLE>Access Denied</TITLE>
</HEAD><BODY>
<H1>Access Denied</H1>

You don't have permission to access "http://www.nseindia.com/api/quote-derivative?" on this server.<P>
Reference #18.15b5655f.1595338785.264b3809
</BODY>
</HTML>

So, I guess nseindia is blocking the request because is not coming from a browser.
But, if you try this:

curl 'https://www.nseindia.com/api/quote-derivative?symbol=RELIANCE' -H 'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Firefox/78.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' -H 'Accept-Language: en-US,en;q=0.5' --compressed -H 'Connection: keep-alive' -H 'Cookie: ak_bmsc=D709927FAB6A44F1C243E20E9199ABA35F65B5158B040000E1EF165F44B95756~plIifDJa9J1EHznQiR/zQpjmGEFWOE88N83Yuqa0oOcgvvYh7eLlgD5MSFVhCZZOObVMZ7UDwEzmOQ2V2lJ6W9pPf6zEbQT1Je27i6h3hrAIyBYFMplV1EDo7rSLxXmCk+HGL3ynHmuPJsePDPD7WTljjTMnM1qpvixsCMEtM1BlmVmuXQijd8cKjWxLeLaf/cNAEJB6VC7SXOq+j6uj6oi9xF/Z2NYB905XnUg9YppXM=' -H 'Upgrade-Insecure-Requests: 1' -H 'Pragma: no-cache' -H 'Cache-Control: no-cache'

you get a json in the response.

{"info":{"symbol":"RELIANCE","companyName":"Reliance Industries Limited",.......}

So you need to add a couple of HTTP Headers to your request, something like:

con.setRequestProperty("User-Agent", "Mozilla/5.0");
con.setRequestProperty("Accept-Language", "en-US,en;q=0.5");
con.setRequestProperty("Accept", "text/html");

How to parse JSON from URL

The actual JSON object is quite, unusual - and it does not lend itself to a simple way of turning it into a struct.

For demonstration, I will use JSONGen, a great utility for turning JSON into Go structs.

For the problem at hand, I would probably use a two step approach.

First, parse the whole document (assuming msft.json contains an API response):

$ cat msft.json | JSONGen
type _ struct {
    MetaData struct {
        Information   string `json:"1. Information"`
        Interval      string `json:"4. Interval"`
        LastRefreshed string `json:"3. Last Refreshed"`
        OutputSize    string `json:"5. Output Size"`
        Symbol        string `json:"2. Symbol"`
        TimeZone      string `json:"6. Time Zone"`
    } `json:"Meta Data"`
    TimeSeries1min struct {
        _ struct {
            Close  string `json:"4. close"`
            High   string `json:"2. high"`
            ...

The problem is the repeated elements, keyed by datetime, which would probably be better modelled as a list. Anyway, with jq we can parse out a relevant piece and generate another struct:

$ cat msft.json | jq '.["Time Series (1min)"]["2018-05-24 15:47:00"]' | JSONGen
type _ struct {
    Close  string `json:"4. close"`
    High   string `json:"2. high"`
    Low    string `json:"3. low"`
    Open   string `json:"1. open"`
    Volume string `json:"5. volume"`
}

Now we can combine the two structs into one. Here is a complete program to parse the JSON
input.

package main

import (
    "encoding/json"
    "fmt"
    "log"
    "os"
)

type Item struct {
    Close  string `json:"4. close"`
    High   string `json:"2. high"`
    Low    string `json:"3. low"`
    Open   string `json:"1. open"`
    Volume string `json:"5. volume"`
}

type Response struct {
    MetaData struct {
        Information   string `json:"1. Information"`
        Interval      string `json:"4. Interval"`
        LastRefreshed string `json:"3. Last Refreshed"`
        OutputSize    string `json:"5. Output Size"`
        Symbol        string `json:"2. Symbol"`
        TimeZone      string `json:"6. Time Zone"`
    } `json:"Meta Data"`
    TimeSeries1min map[string]Item `json:"Time Series (1min)"`
}

We can model the time series as a map of OHLC items. The parsing now gets really simple:

func main() {
    resp := Response{}
    if err := json.NewDecoder(os.Stdin).Decode(&resp); err != nil {
        log.Fatal(err)
    }
    for k, v := range resp.TimeSeries1min {
        fmt.Printf("%s\t%s\t%s\t%s\t%s\t%s\t%s\n",
            resp.MetaData.Symbol, resp.MetaData.LastRefreshed, k,
            v.Open, v.High, v.Low, v.Close)
    }
}

While will output something like:

$ go run main.go < msft.json
MSFT    2018-05-25 10:53:00 2018-05-25 10:49:00 98.6292 98.6292 98.5700 98.5750
MSFT    2018-05-25 10:53:00 2018-05-25 10:40:00 98.8700 98.8701 98.7900 98.8300
MSFT    2018-05-25 10:53:00 2018-05-25 10:22:00 98.6460 98.6500 98.6000 98.6300

...

---

Related Topics