OpenUri causing 401 Unauthorized error with HTTPS URL
The given URL will be redirected to /v1/KJV/passages.xml?q[]=john+3%3A1-5
with HTTP status code 302 Found
. OpenURI understands the redirection, but automatically deletes authentication header (maybe) for security reason. (*)
If you access "http://biblesearch.americanbible.org/v1/KJV/passages.xml?q[]=john+3%3A1-5"
directly, you will get the expected result. :-)
(*) You can find in open-uri.rb
:
if redirect
### snip ###
if options.include? :http_basic_authentication
# send authentication only for the URI directly specified.
options = options.dup
options.delete :http_basic_authentication
end
OpenURI::HTTPError: 401 Unauthorized with open-uri
the URL you're trying to access requires authentication.
There are couple of example / answers to authentication questions using OpenURI in stackoverflow. Here are some of them:
OpenUri causing 401 Unauthorized error with HTTPS URL
and if you are using a proxy:
Ruby open-uri proxy authentication fails
OpenURI::HTTPError: 401 Unauthorized with Rails, HTTParty, and Paperclip
I've noticed in newer versions of Paperclip, I've needed to wrap any URL strings in URI.parse
when asking Paperclip to import the asset via URL.
So for your example:
@event.image = URI.parse("https://img.evbuc.com/http%3A%2F%2Fcdn.evbuc.com%2Fimages%2F18699100%2F162201087004%2F1%2Foriginal.jpg?h=200&w=450&rect=0%2C194%2C2000%2C1000&s=67649c6c8e7ef66e409189bf08474203")
Access a SQL Server 2005 Express Edition from a network computer
See this KB Article. How to configure SQL Server 2005 to allow remote connections.
Oh, and remember that the SQLServer name will probably be MyMachineName\SQLExpress
OpenURI::HTTPError (500 Internal Server Error) with open-uri
Try the following with HTTPS URL
require 'open-uri'
url = "https://graph.facebook.com/v4.0/10224315028492218/picture?access_token=EAAPhFIKqb2IBAO6ekZAGnQyoqkxrmoBRAN8Qfc115SJkOzu5wOmCZB6C7IKtql3ZCIEw8NU66RXrmoFZAszU6G3VeXaz7KHZAmS2NVzF5uRKTEonQ43r1ZB8q1otb5MUmEW7YySXcINREet1xG8i1II5apMif97TAxpMoqDXwz9AZDZD"
begin
File.open('test.jpeg', 'wb') do |file|
file << open(url).read
end
rescue Exception => e
p e.message, e.backtrace.inspect
end
Opening a non-HTTP proxy URI on https domain using OpenURI
Okay, so I've found out how to get the page, but I had to switch open-uri for net/https, also, I set OpenSSL to VERIFY_NONE, since it's a self signed certificate (company server):
require 'rubygems'
require 'nokogiri'
require 'net/https'
require 'openssl'
class JenkinsTest
# Request the Jenkins webpage
def request_jenkins_webpage
uri = URI.parse("https://https://yadayad.yada.yada.com:8443")
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
request = Net::HTTP::Get.new(uri.request_uri)
response = http.request(request)
@@page = Nokogiri::HTML(response.body)
end
def print_jenkins_webpage
puts @@page
end
end
It looks ugly, if anybody finds out a better way to put this, please edit this post, but as of now, it's working fine.
JAX-WS WebService Client - Response: '401: Unauthorized' for url
It was a Weblogic 10 problem due to its URLStreamHandler.
I solved it by doing the following:
Instantiated the WSDL Url like this:
URLStreamHandler handler = new sun.net.www.protocol.http.Handler(); //standard http handler
URL url = null;
try {
url = new URL(null, wsdlUrl, handler); //forced this http handler here
ws = ServiceUtilityWS.create(url, qName);
} catch (MalformedURLException e) {
}After solving that, I had a similar exception on the service operation call. To solve that, I added a custom Handler to the SOAP Message chain:
Binding aBinding = bindingProvider.getBinding();
List<Handler> handlerChain = aBinding.getHandlerChain();
handlerChain.add(new SOAPHandler<SOAPMessageContext>() {
public Set<QName> getHeaders() {
return new TreeSet<QName>();
}
public boolean handleMessage(SOAPMessageContext context) {
final Boolean outInd = (Boolean) context.get(MessageContext.MESSAGE_OUTBOUND_PROPERTY);
if (outInd.booleanValue()) {
try {
context.put(BindingProvider.USERNAME_PROPERTY, user);
context.put(BindingProvider.PASSWORD_PROPERTY, password);
} catch (final Exception e) {
return false;
}
}
return true;
}
Ruby's Mechanize Error 401 while sending a POST request (Steam trade offer send)
I found the issue by debugging the python POST request.
What was happening: when I log in, I get a sessionid indeed, however that sessionid is valid for 'store.steampowered.com' and 'help.steampowered.com' precisely '.storesteapowered.com'.
in my code I was blindly identifying my session cookie (without paying attention to which website it belongs), as a result a the sessionid variable that was being sent in the POST request params was not equal to the cookie the POST request was sending the in header so I got 401 Unauthorized.
so we need to set/get a session id for steamcommunity.com.
fixes :
1)set a random CSRF sessionid cookie for steamcommunity.com or, like I did, set steampowered.com's session id cookie to steamcommunity.com (marked in the code)
2)in params => 'json_tradeoffer' => "new_version"
should be "newversion"
to avoid error 400 BAD REQUEST
3)the headers of the post request should be:
{'Referer' =>'https://steamcommunity.com/tradeoffer/new', 'Origin' =>'https://steamcommunity.com' }
4)convert params => json_tradeoffer
& params => 'trade_offer_create_params'
values to string using to_json
IMPORTANT: this code is for 1 offer send, if you are going to send more than 1 you MUST always update your sessionid variable cause the cookie value will change every time you communicate with steamcommunity.com
here is the code fixed:
require 'mechanize'
require 'json'
require 'open-uri'
require 'openssl'
require 'base64'
require 'time'
def fa(shared_secret)
timestamp = Time.new.to_i
math = timestamp / 30
math = math.to_i
time_buffer =[math].pack('Q>')
hmac = OpenSSL::HMAC.digest('sha1', Base64.decode64(shared_secret), time_buffer)
start = hmac[19].ord & 0xf
last = start + 4
pre = hmac[start..last]
fullcode = pre.unpack('I>')[0] & 0x7fffffff
chars = '23456789BCDFGHJKMNPQRTVWXY'
code= ''
for looper in 0..4 do
copy = fullcode #divmod
i = copy % chars.length #divmod
fullcode = copy / chars.length #divmod
code = code + chars[i]
end
puts code
return code
end
def pass_stamp(username,password,mech)
response = mech.post('https://store.steampowered.com/login/getrsakey/', {'username' => username})
data = JSON::parse(response.body)
mod = data["publickey_mod"].hex
exp = data["publickey_exp"].hex
timestamp = data["timestamp"]
key = OpenSSL::PKey::RSA.new
key.e = OpenSSL::BN.new(exp)
key.n = OpenSSL::BN.new(mod)
ep = Base64.encode64(key.public_encrypt(password.force_encoding("utf-8"))).gsub("\n", '')
return {'password' => ep, 'timestamp' => timestamp }
end
user = 'user'
password = 'password'
session = Mechanize.new { |agent|
agent.user_agent_alias = 'Windows Mozilla'
agent.follow_meta_refresh = true
agent.add_auth('https://steamcommunity.com/tradeoffer/new/send/', user, password)
agent.log = Logger.new("mech.log")
}
data = pass_stamp(user,password, session)
ep = data["password"]
timestamp = data["timestamp"]
session.add_auth('https://steamcommunity.com/tradeoffer/new/send/', user, ep)
send = {
'password' => ep,
'username' => user,
'twofactorcode' =>fa('twofactorcode'), #update
'emailauth' => '',
'loginfriendlyname' => '',
'captchagid' => '-1',
'captcha_text' => '',
'emailsteamid' => '',
'rsatimestamp' => timestamp,
'remember_login' => 'false'
}
login = session.post('https://store.steampowered.com/login/dologin', send )
responsejson = JSON::parse(login.body)
if responsejson["success"] != true
puts "didn't sucded"
puts "probably 2fa code time diffrence, retry "
exit
end
responsejson["transfer_urls"].each { |url|
getcookies = session.post(url, responsejson["transfer_parameters"])
}
## SET COOKIE FOR STEAM COMMUNITY.COM
steampowered_sessionid = ''
session.cookies.each { |c|
if c.name == "sessionid"
steampowered_sessionid = c.value
puts c.domain
end
}
cookie = Mechanize::Cookie.new :domain => 'steamcommunity.com', :name =>'sessionid', :value =>steampowered_sessionid, :path => '/'
session.cookie_jar << cookie
sessionid = steampowered_sessionid
### END SET COOKIE
offer_link = 'https://steamcommunity.com/tradeoffer/new/?partner=410155236&token=H-yK-GFt'
token = offer_link.split('token=', 2)[1]
theirs = [{"appid" => 753,"contextid"=> "6","assetid" => "6705710171","amount" => 1 }]
mine = []
params = {
'sessionid' => sessionid,
'serverid' => 1,
'partner' => '76561198370420964',
'tradeoffermessage' => '',
'json_tradeoffer' => {
"newversion" => true, ## FIXED newversion to avoid 400 BAD REQUEST
"version" => 4,
"me" => {
"assets" => mine, #create this array
"currency" => [],
"ready" => false
},
"them" => {
"assets" => theirs, #create this array
"currency" => [],
"ready" => false
}
}.to_json, # ADDED TO JSON TO AVOID 400 BAD REQUEST
'captcha' => '',
'trade_offer_create_params' => {'trade_offer_access_token' => token}.to_json ## ADDED TO JSON FIX TO AVOID ERROR 400 BAD REQUEST
}
begin
send_offer = session.post(
'https://steamcommunity.com/tradeoffer/new/send',
params,
{'Referer' => 'https://steamcommunity.com/tradeoffer/new', 'Origin' => 'https://steamcommunity.com' } ##FIXED THIS
)
puts send_offer.body
rescue Mechanize::UnauthorizedError => e
puts e
puts e.page.content
end
Related Topics
Quote All Fields in CSV Output
Split Float into Integer and Decimals in Ruby
How to Pass Multi Value Query Params in Swagger
Accessing a Ruby Hash with a Variable as the Key
Gmaps4Rails:Setting Map Width and Height
How to Cancel Evaluating a Required Ruby File? (A.K.A. Top-Level Return)
What Is an Eoferror in Ruby File I/O
Defined' and 'Unless' Not Working as Expected
Why Can't We Override '||' and '&&'
How to Override the System Timezone in Ruby
How to Fix Ruby Script Which Fails with Encoding Error: "\Xd8" on Us-Ascii
Intelligently Generating Combinations of Combinations
What's a Semantically-Correct Way to Parse CSV from SQL Server 2008
Rails 3.2 Force_Ssl Except on Landing Page
How to Sort So That "Vitamin B12" Is Not in Front of "Vitamin B6"
Algorithm to Shuffle an Array Randomly Based on Different Weights