using mechanize with ssl gives "OpenSSL::SSL::SSLError: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed" error
26 Mar 2012
gem, openssl, mechanize
I was given a task to scrape a site that uses ssl. So the first thing I thought of was using Mechanize. When going to the site that uses ssl, I was presented with the following error
OpenSSL::SSL::SSLError: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed
At first it was a real pain having to find the right tutorial to set the certificates that you'd use and how to tell mechanize to use them. To help those who might need this in the future, here it is.
First, download the list of certificates from http://curl.haxx.se/docs/caextract.html and save it to a filename. We'll use
cacert.pem for this blog. Move
cacert.pem inside the
Next part is setting up the scraper agent.
cert_store = OpenSSL::X509::Store.new cert_store.add_file = 'lib/cacert.pem' agent = Mechanize.new agent.cert_store = cert_store agent.get 'https://github.com'
And that's it! Enjoy!