How to Make a Post Request with Header and Data Options in R Using Httr::Post

How to make a POST request with header and data options in R using httr::POST?

You could try this; with content type and headers added:

link <- "http://www.my-api.com"
df <- list(name="Fred", age="5")

httr::POST(url = link,
           body =  jsonlite::toJSON(df, pretty = T, auto_unbox = T),
           httr::add_headers(`accept` = 'application/json'), 
           httr::content_type('application/json'))

R httr post with headers

The trick here seemed to be to construct the data as a list of a named list so that it is encoded to json as the API expects.

Note, most of the supplied headers proved unnecessary (or are automatically generated by httr) in order to retrieve the desired response.

library(httr)

# data as list of named list
data <- list(
  list(
    lat = 3.003090,
    lng = 101.678300,
    classification = "LOW_RISK_NS"
  )
)

params <- list(type = "search")

response <- POST("https://mysejahtera.malaysia.gov.my/register/api/nearby/hotspots",
                 query = params,
                 body = data, 
                 add_headers(Authorization = "Basic N0ZFRkRCMTMtN0Q2MC00NEQxLUE5MTctM0"),
                 encode = "json"
)
content(response)
#> $hotSpots
#> list()
#> 
#> $zoneType
#> [1] "RED"
#> 
#> $messages
#> $messages$ms_MY
#> [1] "Hai {name}, terdapat 21 kes COVID-19 dalam lingkungan radius 1km dari lokasi ini yang dilaporkan dalam masa 14 hari yang lepas."
#> 
#> $messages$en_US
#> [1] "Hi {name}, there have been 21 reported case(s) of COVID-19 within a 1km radius from your searched location in the last 14 days."
#> 
#> 
#> $note
#> NULL

^{Created on 2021-06-21 by the reprex package (v1.0.0)}

Some Explanation

JSON from List vs List of List

See here the difference (i.e., the addition of square brackets) in the resulting strings of json between a named list and a list of a named list:

data <- list(lat = 3, lng = 101, class = "LOW")

# list
jsonlite::toJSON(data)
#> {"lat":[3],"lng":[101],"class":["LOW"]}

# list of a named list
jsonlite::toJSON(list(data))
#> [{"lat":[3],"lng":[101],"class":["LOW"]}]

^{Created on 2021-06-23 by the reprex package (v1.0.0)}

Even Simpler Approach

Here I do more of the encoding work myself rather than letting httr take care of it as in the answer above.

Note that the square brackets surrounding the values are now missing as compared to the second (list of list) json string above. This is representative of the "unboxing" httr does.

library(httr)

response <- POST("https://mysejahtera.malaysia.gov.my/register/api/nearby/hotspots?type=search",
                 body = '[{"lat":3.003090,"lng":101.678300,"classification":"LOW_RISK_NS"}]', 
                 add_headers(Authorization = "Basic N0ZFRkRCMTMtN0Q2MC00NEQxLUE5MTctM0",
                             "Content-Type" = "application/json")
)
content(response)
#> $hotSpots
#> list()
#> 
#> $zoneType
#> [1] "RED"
#> 
#> $messages
#> $messages$ms_MY
#> [1] "Hai {name}, terdapat 22 kes COVID-19 dalam lingkungan radius 1km dari lokasi ini yang dilaporkan dalam masa 14 hari yang lepas."
#> 
#> $messages$en_US
#> [1] "Hi {name}, there have been 22 reported case(s) of COVID-19 within a 1km radius from your searched location in the last 14 days."
#> 
#> 
#> $note
#> NULL

^{Created on 2021-06-23 by the reprex package (v1.0.0)}

R httr package - Make POST request to an API with body

Try to make sure the body is a json character vector instead of an r list.

httr::POST("http://localhost:5000/api/login", body = jsonlite::toJSON(login), verbose(), content_type("application/json"))

How to get httr POST request to work in R?

You are missing order and you need to explicitly convert your request to json. Give it a try:

library("RCurl")
library("rjson")

# Accept SSL certificates issued by public Certificate Authorities
options(RCurlOptions = list(cainfo = system.file("CurlSSL", "cacert.pem", package = "RCurl")))

h = basicTextGatherer()
hdr = basicHeaderGatherer()

req = list(
        order = list(instrument="EUR_USD",
                        units="1", 
                        side="buy", 
                        type="market")
)

body = enc2utf8(toJSON(req))
Token = "abc123" # Replace this with the API key for the web service
authz_hdr = paste('Bearer', Token, sep=' ')

h$reset()
curlPerform(url = "",
            httpheader=c('Content-Type' = "application/json", 'Authorization' = authz_hdr),
            postfields=body,
            writefunction = h$update,
            headerfunction = hdr$update,
            verbose = TRUE
            )

headers = hdr$value()
httpStatus = headers["status"]
if (httpStatus >= 400)
{
    print(paste("The request failed with status code:", httpStatus, sep=" "))

    # Print the headers - they include the requert ID and the timestamp, which are useful for debugging the failure
    print(headers)
}

print("Result:")
result = h$value()
print(fromJSON(result))

My guess is that it is automatically conveted into a text, and there is no matching handler method for text

making specific request with httr

this works for me, does it work for you?

httr::POST(
  "https://api.dropboxapi.com/2/files/list_folder",
  add_headers(Authorization = "Bearer <token>"),
  content_type_json(),
  body = "{\"path\": \"/folder\",\"recursive\": false,\"include_media_info\": false,\"include_deleted\": false}",
  encode = "json"
)

If you want to generalize a bit for many folders:

library("httr")
foobar <- function(x) {
  content(POST(
    "https://api.dropboxapi.com/2/files/list_folder",
    add_headers(Authorization = "Bearer <token>"),
    content_type_json(),
    body = list(path = paste0("/", x), recursive = FALSE, 
                include_media_info = FALSE, include_deleted = FALSE),
    encode = "json"
  ))
}

lapply(c('a', 'b', "c"), foobar)

R - httr POST request to website investing.com to get a JSON response

If you are not too tied to the syntax used, you can switch as follows, noting I have added a cookie header to allow for onward redirect within httr:

library(httr)
library(jsonlite)

headers = c(
  'user-agent' = 'Safari/537.36',
  'x-requested-with' = 'XMLHttpRequest',
  'cookie' = 'adBlockerNewUserDomains=on')

data = list(
  'search_text' = 'FR0000120404'
)

r <- httr::POST(url = 'https://fr.investing.com/search/service/searchTopBar', httr::add_headers(.headers=headers), 
                  body =data, encode = 'form') |> 
  content() |> 
  html_element('p') |> 
  html_text() |> 
  jsonlite::parse_json()

r

How to structure httr POST request to return site data?

What you are encountering here is actually a pretty common problem. httr uses RCurl for the heavy lifting. The default user_agent header sent in a GET or POST request by RCurl is NULL, which frequently confuses scripts. This is why you get different results from your browser and httr(...). If you spoof a meaningful user agent, you get the results you want.

base_url <- "http://web1.ncaa.org/stats/exec/records"
ua       <- "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:33.0) Gecko/20100101 Firefox/33.0"
library(httr)
library(XML)
doc <- POST(base_url, 
            query = list(academicYear = "2014", sportCode = "MFB",orgId = "721"),
            user_agent(ua))

html <- content(doc, useInternalNodes=T)
df.list <- readHTMLTable(html)
df      <- df.list[[4]]
head(df)
#    Opponent  Game Date Air ForceScore OppScore  Loc Neutral SiteLocation GameLength Attend
# 1   Colgate 08/31/2013             38       13 Home                               - 32,095
# 2  Utah St. 09/07/2013             20       52 Home                               - 32,716
# 3 Boise St. 09/13/2013             20       42 Away                               - 36,069
# 4   Wyoming 09/21/2013             23       56 Home                               - 35,389
# 5    Nevada 09/28/2013             42       45 Away                               - 24,545
# 6      Navy 10/05/2013             10       28 Away                               - 38,225

Note also that this website uses tables for just about everything, so readHTMLTable(...) actually returns a list of 4 data frames. The 4th is the one you want.

You don't need rvest.

---

Related Topics