Ruby Mechanize Post with Header

How to set the body of a POST request using Ruby Mechanize?

I don't really like the answer you linked to in your comment because it employs to_json() which is a rails method, and the tags for your question do not indicate that your question pertains to rails. In any case, I think the answer needs some discussion.

Here is the mechanize method:

Mechanize#post(url, query, headers)

...and your stated goal is:

I want to set the body of the POST request

Mechanize#post() allows you to set the body of the request to anything you want, but you also have to consider the question:

What is the server side expecting?  

You gave an example of a jquery ajax() request for what you want to do. jquery uses the following default Content-Type header when sending an ajax() request:

application/x-www-form-urlencoded; charset=UTF-8

That tells the server that the body of the post request is going to be written in a specific secret code. Well, it's not much of a secret; it looks like this:

name1=val1&name2=val2

That secret code's name is x-www-form-urlencoded. Because the server is given the name of the secret code in the Content-Type header, the server knows how to read the body of the post request.

In the Mechanize#post() method, the second parameter is 'query', and the mechanize docs say this about the query argument:

The query is specified by either a string, or

a list of key-value pairs represented by a hash, or

an array of arrays.

http://rubydoc.info/gems/mechanize/Mechanize#post-instance_method

If you want to use the secret code named x-www-form-urlencoded in the body of your Mechanize#post() request, then you can provide a Hash with name/value pairs, e.g.

my_hash = {
  'data' => '{"key1":"value1","key2":"value2"}'
}

Then you call Mechanize#post() like this:

my_agent.post(
  'http://target_site.com', 
  my_hash, 
  {'Content-Type' => 'application/x-www-form-urlencoded; charset=UTF-8'},
)

Then Mechanize will convert the 'query' Hash into a String using the secret code named x-www-form-urlencoded and insert the string into the body of the post request. On the server side, the application that receives the post request can retrieve the json string doing something like this:

json_str = post_variables['data']   

You should be aware that there are other secret codes that can be used for the body of a post request. One of them is called json, which is a string formatted using javascript syntax, for example:

  '{
    "id": 1,
    "name": "A green door",
    "price": 12.50,
    "tags": ["home", "green"]
  }'

Note how there are no '=' signs or '&' symbols in the json format--as there are with the x-www-form-urlencoded format, so the json secret code is much different from the x-www-form-urlencoded secret code.

If you want to use the json secret code in the body of your post request, you need to change two things when you call Mechanize#post(url, query, headers):

1. Provide a String for the 'query' argument.
2. Tell the server that the body of the post request uses the json secret code.

Like this:

json_str = '{"key1":"value1","key2":"value2"}'

my_agent.post(
  'http://target_site.com', 
  json_str, 
  {'Content-Type' => 'application/json'},
)

When you pass a String argument for the query parameter, Mechanize doesn't do any processing of the String before inserting the String into the body of the post request. On the server side, the application that receives the post request can retrieve the json string by doing something like this:

json_str = request.body.read
#Then probably:
hash = JSON.parse(json_str)

The one hitch is that the server can ignore the Content-Type header and try to read the body of the post request using a secret code that it has already decided upon. If the body of your post request is not written in the secret code that the server expects, then you will get an error.

Note that the 'data' string you posted isn't valid json because it uses single quotes around the properties and values.

How to set the Referer header before loading a page with Ruby mechanize?

The docs say:

get(uri, parameters = [], referer = nil, headers = {}) { |page| ... }

so for example:

agent.get 'http://www.google.com/', [], agent.page.uri, {'foo' => 'bar'}

alternatively you might like:

agent.request_headers = {'foo' => 'bar'}
agent.get url

How to get request headers using Mechanize?

you need to run a pre_connect_hooks to get the request header

 agent.pre_connect_hooks << lambda do |agent, request|
   request['X-Requested-With'] = 'XMLHttpRequest'
 end

Check this thread so-1

Read Response Header

puts page.header['location']
puts page.header['server']

loop through the header array

trying to POST with ruby mechanize

post() doesn't take a block. Try this:

page = browser.post('http://www.mysite.com/login', {
  "email" => "myemail%40gmail.com",
  "password" => "something",
  "remember" => "1",
  "loginSubmit" => "Login",
  "url" => ""
})

edit: changed for accuracy

Mechanize send a POST to populate data on a page

That Content-Type header should be: application/x-www-form-urlencoded

So try:

bot.post url, vars, ({'Content-Type' => 'application/x-www-form-urlencoded'})

How can I perform a Head request using mechanize in Ruby

Just like get but it's head instead:

page = agent.head 'http://www.google.com/'
page.body.length
#=> 0
page.header.keys
#=> ["date", "expires", "cache-control", "content-type", "set-cookie", "p3p", "server", "x-xss-protection", "x-frame-options", "transfer-encoding"]

POST request capture with Mechanize

The post method expects the parameters as a hash. Try:

agent.post("http://www.etr.gov.ar/getSmsResponse.php", {"parada" => "4152", "linea"=>"112"})

---

Related Topics