Requests: How to Disable/Bypass Proxy

requests: how to disable / bypass proxy

The only way I'm currently aware of for disabling proxies entirely is the following:

• Create a session
• Set session.trust_env to False
• Create your request using that session

import requests

session = requests.Session()
session.trust_env = False

response = session.get('http://www.stackoverflow.com')

This is based on this comment by Lukasa and the (limited) documentation for requests.Session.trust_env.

Note: Setting trust_env to False also ignores the following:

• Authentication information from .netrc (code)
• CA bundles defined in REQUESTS_CA_BUNDLE or CURL_CA_BUNDLE (code)

---

If however you only want to disable proxies for a particular domain (like localhost), you can use the NO_PROXY environment variable:

import os
import requests

os.environ['NO_PROXY'] = 'stackoverflow.com'

response = requests.get('http://www.stackoverflow.com')

switch off proxy in requests library

Set the trust_env variable on the session to False. If not true (True is the default), proxy information from the environment is ignored altogether:

session = requests.Session()
session.trust_env = False
response = session.get(url)

This does also disable .netrc authentication support. If you need that still, then you have two more options that I can see:

• add a NO_PROXY environment variable; set to * means no proxies should be used at all. You could do this by directly setting the key in the os.environ dictionary.

• simply delete the proxy keys from os.environ.

  Take into account that on OSX and Windows Python will look for proxies in the system configuration too (so the registry on Windows, and SysConf on Mac OS X).

Altering os.environ is safe. It is a regular dictionary, adding or deleting keys in your program is fine, the parent shell environment won't be altered.

Requests.get not finishing, doesn't raise any error

It seems that requests on MacOS is a bit buggy. Try to disable the proxies by setting trust_env to False:

session = requests.Session()
session.trust_env = False  # No proxy settings from the OS
r = session.get(url)

curl --noproxy * equivalent in Python's request module

Setting proxies to None is like omitting it. You want to specify that the http and https proxies are None by passing the value:

proxies={'http': None, 'https': None}

to the requests.get function.

Regarding NO_PROXY (or no_proxy), its value is a domain name, so you shouldn't add http:// or https:// in front of its value, just use the domain name hostname in your case.

How do I make curl ignore the proxy?

I assume curl is reading the proxy address from the environment variable http_proxy and that the variable should keep its value. Then in a shell like bash, export http_proxy=''; before a command (or in a shell script) would temporarily change its value.

(See curl's manual for all the variables it looks at, under the ENVIRONMENT heading.)

How do I disable the security certificate check in Python requests

From the documentation:

requests can also ignore verifying the SSL certificate if you set
verify to False.

>>> requests.get('https://kennethreitz.com', verify=False)
<Response [200]>

If you're using a third-party module and want to disable the checks, here's a context manager that monkey patches requests and changes it so that verify=False is the default and suppresses the warning.

import warnings
import contextlib

import requests
from urllib3.exceptions import InsecureRequestWarning

old_merge_environment_settings = requests.Session.merge_environment_settings

@contextlib.contextmanager
def no_ssl_verification():
    opened_adapters = set()

    def merge_environment_settings(self, url, proxies, stream, verify, cert):
        # Verification happens only once per connection so we need to close
        # all the opened adapters once we're done. Otherwise, the effects of
        # verify=False persist beyond the end of this context manager.
        opened_adapters.add(self.get_adapter(url))

        settings = old_merge_environment_settings(self, url, proxies, stream, verify, cert)
        settings['verify'] = False

        return settings

    requests.Session.merge_environment_settings = merge_environment_settings

    try:
        with warnings.catch_warnings():
            warnings.simplefilter('ignore', InsecureRequestWarning)
            yield
    finally:
        requests.Session.merge_environment_settings = old_merge_environment_settings

        for adapter in opened_adapters:
            try:
                adapter.close()
            except:
                pass

Here's how you use it:

with no_ssl_verification():
    requests.get('https://wrong.host.badssl.example/')
    print('It works')

    requests.get('https://wrong.host.badssl.example/', verify=True)
    print('Even if you try to force it to')

requests.get('https://wrong.host.badssl.example/', verify=False)
print('It resets back')

session = requests.Session()
session.verify = True

with no_ssl_verification():
    session.get('https://wrong.host.badssl.example/', verify=True)
    print('Works even here')

try:
    requests.get('https://wrong.host.badssl.example/')
except requests.exceptions.SSLError:
    print('It breaks')

try:
    session.get('https://wrong.host.badssl.example/')
except requests.exceptions.SSLError:
    print('It breaks here again')

Note that this code closes all open adapters that handled a patched request once you leave the context manager. This is because requests maintains a per-session connection pool and certificate validation happens only once per connection so unexpected things like this will happen:

>>> import requests
>>> session = requests.Session()
>>> session.get('https://wrong.host.badssl.example/', verify=False)
/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py:857: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
<Response [200]>
>>> session.get('https://wrong.host.badssl.example/', verify=True)
/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py:857: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
<Response [200]>

Bypass proxy when HTTPGET https://localhost/

You can look to use something like this to disable use of the system proxy, if your HttpClient is picking it up from the system settings:

var handler = new HttpClientHandler() { UseProxy = false }; 
httpClient = new HttpClient(handler);

---

Related Topics