F# Asynchronous Http Request - Parse JSON Response

F# asynchronous HTTP request - parse JSON response

Check the following code:

open System
open System.Net
open FSharp.Data

type Response = JsonProvider<""" { "name":"John", "age":94 } """>

[<EntryPoint>]
let main argv =
  let request () =
    async {
      let url = "https://www.googleapis.com/identitytoolkit/v3/relyingparty/signupNewUser?key=XXXXXXXXXXXX"
      return! Http.AsyncRequestString
        ( url,
          headers = [ "Content-Type", HttpContentTypes.Json ],
          body = TextRequest """ {"returnSecureToken": true} """ )
    } |>  Async.Catch
  let result = request ()
               |> Async.RunSynchronously
  match result with
  | Choice1Of2 text -> let json = Response.Parse text
                       printfn "name: %A" json.Name   
  | Choice2Of2 e -> printfn "request failed: %A" e
  0 // return an integer exit code

If type provides are not working with your setup, this may help: https://github.com/Microsoft/visualfsharp/issues/3303

Replace sample JSON with what is expected from the service.

Get Asynchronous HttpResponse through Silverlight (F#)

The whole point of async is that you don't have to deal with state and IAsyncResult and Callbacks and whatnot. Below is a somewhat cleaned-up version of your code...

open System 
open System.IO  
open System.Net  
open System.Text  
open System.Web  
open System.Security.Authentication  
open System.Runtime.Serialization  

[<DataContract>]  
type Result<'TResult> = {  
    [<field: DataMember(Name="code") >]  
    Code:string  
    [<field: DataMember(Name="result") >]  
    Result:'TResult array 
    [<field: DataMember(Name="message") >]  
    Message:string  
    }  

// The elements in the list 
[<DataContract>]  
type ChemicalElement = {  
    [<field: DataMember(Name="name") >]  
    Name:string  
    [<field: DataMember(Name="boiling_point") >]  
    BoilingPoint:string  
    [<field: DataMember(Name="atomic_mass") >]  
    AtomicMass:string  
}  

//http://blogs.msdn.com/b/dsyme/archive/2007/10/11/introducing-f-asynchronous-workflows.aspx 
//http://lorgonblog.spaces.live.com/blog/cns!701679AD17B6D310!194.entry 
type System.Net.HttpWebRequest with 
    member x.GetResponseAsync() = 
        Async.FromBeginEnd(x.BeginGetResponse, x.EndGetResponse) 

let getHttpWebRequest (query:string) =  
    let query = query.Replace("'","\"")  
    let queryUrl = sprintf "http://api.freebase.com/api/service/mqlread?query=%s" "{\"query\":"+query+"}"  

    let request : HttpWebRequest = downcast WebRequest.Create(queryUrl)  
    request.Method <- "GET"  
    request.ContentType <- "application/x-www-form-urlencoded"  
    request 

let printResults (stream: Stream)=  
    let result =  
        try  
            use reader = new StreamReader(stream)  
            reader.ReadToEnd();  
        finally  
            () 

    let data = Encoding.Unicode.GetBytes(result);  
    let stream = new MemoryStream()  
    stream.Write(data, 0, data.Length);  
    stream.Position <- 0L  

    let JsonSerializer = Json.DataContractJsonSerializer(typeof<Result<ChemicalElement>>)  
    let result = JsonSerializer.ReadObject(stream) :?> Result<ChemicalElement>  

    if result.Code<>"/api/status/ok" then  
        raise (InvalidOperationException(result.Message))  
    else  
        result.Result |> Array.iter(fun element->printfn "%A" element)  

let test = async {
    // Call Query (w/ generics telling it you wand an array of ChemicalElement back, the query string is wackyJSON too –I didn’t build it don’t ask me! 
    let request = getHttpWebRequest "[{'type':'/chemistry/chemical_element','name':null,'boiling_point':null,'atomic_mass':null}]" 
    try 
        use! response = request.AsyncGetResponse()
        use responseStream = response.GetResponseStream() 
        printResults responseStream 
    with  
    | :? WebException as e 
        ->  printfn "WebException raised!" 
            printfn "\n%s" e.Message 
            printfn "\n%s" (e.Status.ToString()) 
    | _ as e 
        ->  printfn "Exception raised!" 
            printfn "Source : %s" e.Source 
            printfn "Message : %s" e.Message 
}

test |> Async.RunSynchronously 
System.Console.ReadLine() |> ignore 

Fsharp.Data HttpRequestBody getting 'The mandatory 'grant_type' parameter is missing'

I think the problem here is that you are sending the body as JSON, when it should be form values:

body =
    FormValues [
        "grant_type", "password"
        "password", "xxx"
        "username", "y@..."
    ]

I haven't tested this, though.

F# - How to create an async function dynamically based on return type

If you can leverage generics as Aaron suggests, then doing that will be a better idea. However, if you need to choose a type at runtime, then you can make your code work by changing functionThatReturnsAsync to look as follows:

let functionThatReturnsAsync (irrelevantArgs: obj list) (returnType: Type) = 
    if returnType.GUID = typeof<string>.GUID
    then box (async { return "some text" })
    elif returnType.GUID = typeof<int>.GUID
    then box (async { return 42 })
    elif returnType.GUID = typeof<bool>.GUID
    then box (async { return true })
    else box (async { return (null:obj) })

This is almost the same as what you had - but rather than boxing the values inside async computations, it is boxing the whole async computation (which then returns the value of the right type) - so the casting works!

JsonProvider failed to parse JSON from API written myself

I checked the FSharp.Data source code (its the function asyncRead) to see how they download the JSON. It basically boils down to this:

let readString =
  async {
    let contentTypes  = [ HttpContentTypes.Json ]
    let headers       = [ 
                          HttpRequestHeaders.UserAgent ("F# Data JSON Type Provider") 
                          HttpRequestHeaders.Accept (String.concat ", " contentTypes) 
                        ]
    let! text     = Http.AsyncRequestString("http://www.kujiale.com/api/askinvitesearch?query=cc", headers = headers) 

    return text
  }

If one runs this code against the url http://www.kujiale.com/api/askinvitesearch?query=cc we see something interesting about what's returned:

"[{\"linkToIdeaBook\":\"/u/3FO4K4UR89F1/huabao\",\"linkToDesi

Note that the content starts with " and that the "strings" are "escaped" with \. So it seems the JSON document is returned as an escaped string. According to json.org the root object must be either an object or array so the parser fails at character 0.

If one switches to contentType HttpContentTypes.Text it starts like this:

[{"linkToIdeaBook":"/u/3FO4K4UR89F1/huabao","linkToDesignCollect":"/u

Which actually turns out to be a valid JSON object.

To me it seems somewhat odd that if you ask for content with content type JSON you get an escaped string but that seems to be the root cause of the failure.

How to resolve it is more difficult to say. One way forward would be a PR to FSharp.Data to allow users to specify the content type used to download content.

Composing async computations in F#

When writing asynchronous code in F# , I find it easier to just use the built-in computation asynchronous workflow syntax rather than trying to build more sophisticated combinators.

In your example, you would not really duplicate any code if you just write a straightforward function, so I think the following does not break the DRY principle and it is considerably simpler (and it is also easy to extend the code to handle exceptions, which would be otherwise difficult):

let getJson<'T> (request:Net.WebRequest) = async { 
  let! response = request.AsyncGetResponse()
  use reader = new StreamReader(response.GetResponseStream())
  let! data = reader.AsyncReadToEnd()
  return Json.JsonConvert.DeserializeObject<'T> data }

Of course, you could split the code into downloadData and getJson if you need to download data for other purposes elsewhere in the code.

In general, when writing functional code, you have two options how to compose computations:

• Using the language syntax (such as loops, let and try .. with, use in both plain F# and in asynchronous workflows). This approach generally works well if you're writing some computation, because the language is designed to describe computations and can do that well.

• Using custom combinators (such as map, >> and |> or library-specific operators). This is needed if you're modeling something that is not just a computation, for example an interactive animation, a stock option, a user interface component or a parser. However, I would only follow this path if the basic features of the language are not sufficient to express the problem.

Aside, you might be interested in the F# Data library, which implements helpers for various tasks including HTTP requests, JSON parsing and JSON type provider, which might make your life even easier.

C# async/await to F# using Azure ML example

I was not able to compile and run the code, but you probably need something like this:

let invokeRequestResponseService() = async {
    use client = new HttpClient()
    let scoreData = (...)
    let apiKey = "abc123"
    client.DefaultRequestHeaders.Authorization <- 
        new AuthenticationHeaderValue("Bearer", apiKey)
    client.BaseAddress <- Uri("https://ussouthcentral..../score");
    let! response = client.PostAsJsonAsync("", scoreRequest) |> Async.AwaitTask
    if response.IsSuccessStatusCode then
        let! result = response.Content.ReadAsStringAsync() |> Async.AwaitTask
        Console.WriteLine("Result: {0}", result);
    else
        Console.WriteLine("Failed with status code: {0}", response.StatusCode) }

• Wrapping the code in the async { .. } block makes it asynchronous and lets you use let! inside the block to perform asynchronous waiting (i.e. in places where you'd use await in C#)

• F# uses type Async<T> instead of .NET Task, so when you're awaiting a task, you need to insert Async.AwaitTask (or you can write wrappers for the most frequently used operations)

• The invokeRequestResponseService() function returns F# async, so if you need to pass it to some other library function (or if it needs to return a task), you can use Async.StartAsTask

F# syntax for async controller methods in ASP.NET Core

UPDATED 2016-09-28

Thanks to Ruben Bartelink this is what my controller looks like now correctly implemented as async and handling the nuances that differ between C# and F# async patterns:

namespace FSharp.WebLib

open System
open Microsoft.AspNetCore.Mvc
open Microsoft.AspNetCore.Routing
open Microsoft.AspNetCore.JsonPatch
open FSharp.Models

module ActionResult =
    let ofAsync (res: Async<IActionResult>) =
        res |> Async.StartAsTask

[<Route("api/[controller]")>]
type FSToDoController(commands: IToDoCommands, queries: IToDoQueries) =
    inherit Controller()

    [<HttpGet>]
    member this.Get() = 
        ActionResult.ofAsync <| async {
            let! data = queries.GetAll()
            return JsonResult(data) :> _ } 

    [<HttpGet("{id}", Name = "GetFsToDo")>]
    member this.Get(id) =
        ActionResult.ofAsync <| async {
            let! res = queries.Find id
            match res with 
            | None -> return this.NotFound() :> _
            | Some data -> return ObjectResult(data) :> _ } 

    // create
    [<HttpPost>]
    member this.Post([<FromBody>] item:ToDoItem) =
        ActionResult.ofAsync <| async {
            if not this.ModelState.IsValid then
                return this.BadRequest() :> _
            else  
                let item = { item with Id = Guid.NewGuid() |> string }
                do! commands.Add item
                let rv = RouteValueDictionary()
                rv.Add("id",item.Id)
                return this.CreatedAtRoute("GetFsToDo", rv, item) :> _ } 

    // update
    [<HttpPut("{id}")>]
    member this.Put(id:String, [<FromBody>] item:ToDoItem) =
        ActionResult.ofAsync <| async {
            if (not this.ModelState.IsValid) || String.IsNullOrEmpty item.Id then
                return this.BadRequest() :> _
            else
                let! res = queries.Find id
                match res with
                | None -> return this.NotFound() :> _
                | Some toDo ->
                    do! commands.Update item
                    return NoContentResult() :> _ } 

for anyone else interested in learning F# particularly for use in ASP.NET Core, this is part of a proof of concept project on github that has both C# and F# implementations of a ToDo list back end web api both of which are consumed from a front end implemented with polymer web components. Models and data access are also implemented in both languages to provide a good comparison for C# devs like myself to learn F#

---

Related Topics