Why Does Accessing a Ssl Site with Mechanize on Windows Fail, But on MAC Work

Why does accessing a SSL site with Mechanize on Windows fail, but on Mac work?

The version of OpenSSL (the library used to establish secure connections with Net::HTTPS) is not able to properly find the certificate chain in your computer.

To our bad, OpenSSL was never able to use the Windows installed cert storage to validate remote servers so is failing because of that.

From your example, you can do:

a.agent.http.verify_mode = OpenSSL::SSL::VERIFY_NONE

To avoid the verification, however that is far from ideal (due clear security issues)

I recommend you download some cert bundles (like the ones from curl):

http://curl.haxx.se/ca

And modify your code to something like this:

require "rbconfig"
require "mechanize"

a = Mechanize.new

# conditionally set certificate under Windows
# http://blog.emptyway.com/2009/11/03/proper-way-to-detect-windows-platform-in-ruby/
if RbConfig::CONFIG["host_os"] =~ /mingw|mswin/
  # http://curl.haxx.se/ca
  ca_path = File.expand_path "~/Tools/bin/curl-ca-bundle.crt"

  a.agent.http.ca_file = ca_path
end

page = a.get "https://github.com/"

That seems to work, Ruby 1.9.3-p0 (i386-mingw32), Windows 7 x64 and mechanize 2.1.pre.1

Hope that helps.

What can I do about Mechanize waiting on an unresponsive web site?

There's a couple ways to deal with it.

Open-Uri, and Net::HTTP have ways of passing in timeout values, which then tell the underlying networking stack how long you are willing to wait. For instance, Mechanize lets you get at its settings when you initialize an instance, something like:

mech = Mechanize.new { |agent|
  agent.open_timeout   = 5
  agent.read_timeout   = 5
}

It's all in the docs for new but you'll have to view the source to see what instance variables you can get at.

Or you can use Ruby's timeout module:

require 'timeout'
status = Timeout::timeout(5) {
  # Something that should be interrupted if it takes too much time...
}

Why am I getting an OpenSSL::SSL::SSLError only on Windows?

appears it happens because ruby doesn't ship with certificates? http://betterlogic.com/roger/2011/08/github-jruby-ssl-woe/

Ruby Mechanize 404 = Net::HTTPNotFound

Ok,

The problème was that the website disable the Mechanize user agent.

I just changed it to : mechanize.user_agent_alias = 'Windows Chrome'

installing mechanize on a mac with an error

I just had the same issue. Consider installing rbenv https://github.com/rbenv/rbenv which makes it easy to switch between installed versions of Ruby. With rbenv you can download/install almost any version of Ruby and then install the mechanize Gem.

—OR—

Run this command for CL tools:

$ xcode-select --install

You will be prompted to install tools or receive this error:

xcode-select: error: command line tools are already installed, use "Software Update" to install updates

The run these commands to install the Gem (use sudo at your own discretion):

gem update --system
gem uninstall nokogiri
echo "gem: -n/usr/local/bin" >> ~/.gemrc
sudo chown -R $(whoami):admin /usr/local
sudo gem install nokogiri -n/usr/local/bin -v '1.6.3.1' -- --with-opt-include=$(xcrun --show-sdk-path)/usr/include
sudo gem install mechanize

Mechanize requires minimum version 1.6, so Nokogiri 1.6.3.1 will work: https://github.com/sparklemotion/mechanize/blob/master/mechanize.gemspec#L56

---

For further issues see:

• http://www.nokogiri.org/tutorials/installing_nokogiri.html
• https://github.com/sparklemotion/nokogiri/issues
• https://github.com/sparklemotion/mechanize/issues

Scraping: SSL: CERTIFICATE_VERIFY_FAILED error for http://en.wikipedia.org

Once upon a time I stumbled with this issue. If you're using macOS go to Macintosh HD > Applications > Python3.6 folder (or whatever version of python you're using) > double click on "Install Certificates.command" file. :D

---

Related Topics