How to Test a Ruby Application Which Uses Mechanize

How to test a ruby application which uses mechanize

Use Fakeweb to stub out the internet responses.

For the Google example, first go to the website and save the html of the page you desire. In this case, let's assume you saved www.google.com via your browser or curl. Add the Fakeweb gem to your test.rb file

Then your code is

stream = File.read("saved_google_page.html")
FakeWeb.register_uri(:get, 
    "http://www.google.com", 
    :body => stream, 
    :content_type => "text/html")

When you do your standard Mechanize call of

agent = Mechanize.New
page = agent.get("http://www.google.com/")

Fakeweb will return the page you saved with the content_type headers set in a way that Mechanize will think it accessed the internet successfully. Make sure that content_type header is set since otherwise Mechanize treats the response as Mechanize::File instead of Mechanize::Page. You can test that it's fully working by running tests on your machine with the network connection unplugged.

p.s. I'm answering this 6 months after the question was asked since this is the top result in Google but it's unanswered. I just spent 30 minutes figuring this out myself and thought I'd share the solution.

Ruby: Problems using Mechanize to access my form!

How about

target_form.field_with(:name => "post[title]").value = "test"
target_form.field_with(:name => "post[description]").value = "test"

Testing links in a web app

What you are asking for is beyond most of the test tools once you throw in the ability to spider the site.

That final requirement pushes you into the realm of hand-coding something. Using the Mechanize gem you could do all those things, but, you get to code a lot of the navigation of the site.

Mechanize uses Nokogiri internally, so it's easy to grab all links in a page, which you could store in a database to be checked by a different thread, or some subsequent code. That said, writing a spider is not hard if you're the owner of the pages you're hitting, because you can be pretty brutal about accessing the server and let the code run at full speed without worrying about being banned for excessive bandwidth use.

How to use Mechanize to parse local file

Mechanize uses URI strings to point to what it's supposed to parse. Normally we'd use a "http" or "https" scheme to point to a web-server, and that's where Mechanize's strengths are, but other schemes are available, including "file", which can be used to load a local file.

I have a little HTML file on my Desktop called "test.rb":

<!DOCTYPE html>
<html>
<head></head>
<body>
<p>
Hello World!
</p>
</body>
</html>

Running this code:

require 'mechanize'

agent = Mechanize.new
page = agent.get('file:/Users/ttm/Desktop/test.html')
puts page.body

Outputs:

<!DOCTYPE html>
<html>
<head></head>
<body>
<p>
Hello World!
</p>
</body>
</html>

Which tells me Mechanize loaded the file, parsed it, then accessed the body.

However, unless you need to actually manipulate forms and/or navigate pages, then Mechanize is probably NOT what you want to use. Instead Nokogiri, which is under Mechanize, is a better choice for parsing, extracting data or manipulating the markup and it's agnostic as to what scheme was used or where the file is actually located:

require 'nokogiri'

doc = Nokogiri::HTML(File.read('/Users/ttm/Desktop/test.html'))
puts doc.to_html

which then output the same file after parsing it.

Back to your question, how to find the node only using Nokogiri:

Changing test.html to:

<!DOCTYPE html>
<html>
<head></head>
<body>
<div class="product_name">Hello World!</div>
</body>
</html>

and running:

require 'nokogiri'

doc = Nokogiri::HTML(File.read('/Users/ttm/Desktop/test.html'))
doc.search('div.product_name').map(&:text)
# => ["Hello World!"]

shows that Nokogiri found the node and returned the text.

This code in your sample could be better:

text = node.text  
puts "product name: " + text.to_s

node.text returns a string:

doc = Nokogiri::HTML('<p>hello world!</p>')
doc.at('p').text # => "hello world!"
doc.at('p').text.class # => String

So text.to_s is redundant. Simply use text.

How to confirm a JavaScript popup using Nokgiri or Mechanize

Nokogiri, and Mechanize because it is built on top of Nokogiri, can parse the HTML and return the <script> tag's contents. The tag's content is text so at that point it's necessary to look inside the text to find what you want:

require 'nokogiri'

doc = Nokogiri::HTML(<<EOT)
<html>
<head>
<script>alert("TEST");</script>
</head>
</html>
EOT

script_content = doc.at('script').content # => "alert(\"TEST\");"

It's easy to check to see if a sub-string exists at that point:

script_content['alert("TEST");'] # => "alert(\"TEST\");"

or:

!!script_content['alert("TEST");'] # => true

Note: It's not possible with Nokogiri, or Mechanize, to tell if a pop-up occurred as that'd happen inside a browser as it runs the JavaScript. Neither Nokogiri or Mechanize understand or interpret JavaScript. Only a tool like Watir or that interprets JavaScript could do that.

Remote testing with capybara-mechanize: Form submission results in incorrect URL (404 = Net::HTTPNotFound)

This appears to be acting correctly since your action path is relative. For the behavior you expect the forms action attribute would need to be “create” (relative), “/something/create” (absolute), or “../something/create” (relative) depending on whether or not you’re using the same form at different URLs and how it should behave at those too.

how to get google search results links and store them in array using mechanize

Something like this should work:

@searchword = params[:q]
@sitesurl = Array.new
agent = Mechanize.new
page = agent.get("http://www.google.com")
search_form = page.form_with(:name => "f")
search_form.field_with(:name => "q").value = @searchword.to_s     
search_results = agent.submit(search_form)

(search_results/"li.g").each do |result|
  @sitesurl << (result/"a").first.attribute('href') if result.attribute('class').to_s == 'g knavi'
end

How to use Cucumber to test non-Ruby, non-Rack API's

I wouldn't use Capybara to test a remote API because Capybara is made for testing applications is used for testing applications with a HTML UI (as Aslak points out in the comments).

Instead, I would use Cucumber* in conjunction with something like HTTParty which would be the tool used to make the HTTP requests and parse them neatly. Here's an idea:

When /^I visit the API path '(.*?)'/ do |path|
  @result = HTTParty.get("http://theapi.com/#{path}")
end

Then /^I should see the following result:$/ do |result|
  @result.should == result
end

The final step here you would use like this:

Then I should see the following result:
   """
     { success: true }
   """

* I would actually use RSpec personally, I find the syntax less clumsy.

How to Test a Ruby Application Which Uses Mechanize