Requests (Caused by Sslerror("Can't Connect to Https Url Because the Ssl Module Is Not Available.") Error in Pycharm Requesting Website

SSLError(Can't connect to HTTPS URL because the SSL module is not available.) in pip command

TL;DR you're probably missing some system dependencies. sudo apt install build-essential zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libreadline-dev libffi-dev wget

Read below for the full details on how we got there.

The error states that the SSL python module is not available; meaning you either don't have an appropriate ssl lib installed (probably not since you state the system python can pip install fine), or the python you built from source or otherwise installed doesn't include the ssl module.

If you built it from source, you need to be sure to set the configuration option --with-openssl.

Also, I'd really caution against sudo pip installing anything. Use something like virtualenv to have separate python environments from the system python or other python versions you install.

EDIT:

Upon closer examination, the python configure script appears to enable ssl support by default, assuming the dev headers are present. Make sure all the dev headers are present with sudo apt install build-essential zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libreadline-dev libffi-dev wget, then retry configuring and building python.

(Caused by SSLError(Can't connect to HTTPS URL because the SSL module is not available.)) ) pyscript

The requests packages uses the operating system's TCP Socket API. That API is not available inside the web browser virtual sandbox. This is a security restriction on all browser applications.

Rewrite your code to use either pyfetch() or fetch().

There might be a potential problem that developers run into. That is CORS. The browser enforces CORS when accessing HTTP endpoints. The endpoint that you are calling must support CORS headers on responses. Otherwise, you will get a CORS error in the browser. This is another security restriction. The only solution if there is a CORS error is to ask the owner of the endpoint to fix that problem (add CORS response headers).

Example using pyfetch():

from pyodide.http import pyfetch, FetchResponse

response = await pyfetch(url, method="GET", headers=headers)

data_dict = await response.json()

If you need access to the response headers, you must use fetch() because pyfetch() does not return them. pyfetch is a wrapper for JavaScript fetch().

Example using fetch():

from js import fetch, Object
from pyodide.http import to_js

response = await fetch(
    url,
    method="GET",
    headers=Object.fromEntries(to_js(headers))

data_dict = (await response.json()).to_py()

(Caused by SSLError(Can't connect to HTTPS URL because the SSL module is not available.))

Your problem is caused by importing the requests package.

The Requests package uses the operating system TCP Socket API. That API is not available in web browsers. That limitation is a security restriction imposed on all browser-based applications.

Python packages that import the requests package are not supported in PyScript.

Modify your code using one of the following methods.

from pyodide.http import open_url

page = open_url('https://github.com/Nitink2001/EnergyUpgraded.git')

Note: There is a usage restriction for open_url(). That API can only return text based data and does not support binary.

PyScript has the function pyfetch():

import asyncio
from pyodide.http import pyfetch

# this code must be in an "async" function
    
async def myfunc():
    response = await pyfetch('https://github.com/Nitink2001/EnergyUpgraded.git')
    data = await response.text()

PyScript also supports the browser Fetch API, which is the API I recommend using:

import asyncio
from js import fetch

# this code must be in an "async" function

async def myfunc():
    response = await fetch('https://github.com/Nitink2001/EnergyUpgraded.git')
    data = await response.text()

Python sendgrid library error when run from command line

I was able to find the answer here: Requests (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.") Error in PyCharm requesting website

This issue has to do with some DLLs in anaconda. I am not sure what the best fix is for this. I copy pasted the DLLS as shown in one answer here. I am going to copy the text of this answer though, because it is not the top answer on this thread, and this search result does not come up when using sendgrid, so I think my question actually does add to the community. From the other SO post:

"""

I have faced it on 04/2020. These are the options that I tried and the last solution get me through.

Problem:

Requests module works fine when I use in Spyder IDE but when I try to execute the script in windows it fails with SSL error. It works fine for HTTP requests but for HTTPS requests i got SSL error.

I tired with Veify=True, False , also with Certs. Same error.
Removed Certifi - conda remove certifi - Did not work Updated
openssl , certifi - Still same error ( Refer : https://github.com/ContinuumIO/anaconda-issues/issues/494) Added
the path variables - Same error
Created new environment in Anaconda - same error

Solution that fixed it:

Solution from: https://github.com/conda/conda/issues/8273

I have copied the following files from Anaconda3\Library\bin to \Anaconda3\DLL

libcrypto-1_1-x64.* libssl-1_1-x64.*

"""

urllib and SSL: CERTIFICATE_VERIFY_FAILED Error

If you just want to bypass verification, you can create a new SSLContext. By default newly created contexts use CERT_NONE.

Be careful with this as stated in section 17.3.7.2.1

When calling the SSLContext constructor directly, CERT_NONE is the default. Since it does not authenticate the other peer, it can be insecure, especially in client mode where most of time you would like to ensure the authenticity of the server you’re talking to. Therefore, when in client mode, it is highly recommended to use CERT_REQUIRED.

But if you just want it to work now for some other reason you can do the following, you'll have to import ssl as well:

input = input.replace("!web ", "")      
url = "https://domainsearch.p.mashape.com/index.php?name=" + input
req = urllib2.Request(url, headers={ 'X-Mashape-Key': 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' })
gcontext = ssl.SSLContext()  # Only for gangstars
info = urllib2.urlopen(req, context=gcontext).read()
Message.Chat.SendMessage ("" + info)

This should get round your problem but you're not really solving any of the issues, but you won't see the [SSL: CERTIFICATE_VERIFY_FAILED] because you now aren't verifying the cert!

To add to the above, if you want to know more about why you are seeing these issues you will want to have a look at PEP 476.

This PEP proposes to enable verification of X509 certificate signatures, as well as hostname verification for Python's HTTP clients by default, subject to opt-out on a per-call basis. This change would be applied to Python 2.7, Python 3.4, and Python 3.5.

There is an advised opt out which isn't dissimilar to my advice above:

import ssl

# This restores the same behavior as before.
context = ssl._create_unverified_context()
urllib.urlopen("https://no-valid-cert", context=context)

It also features a highly discouraged option via monkeypatching which you don't often see in python:

import ssl

ssl._create_default_https_context = ssl._create_unverified_context

Which overrides the default function for context creation with the function to create an unverified context.

Please note with this as stated in the PEP:

This guidance is aimed primarily at system administrators that wish to adopt newer versions of Python that implement this PEP in legacy environments that do not yet support certificate verification on HTTPS connections. For example, an administrator may opt out by adding the monkeypatch above to sitecustomize.py in their Standard Operating Environment for Python. Applications and libraries SHOULD NOT be making this change process wide (except perhaps in response to a system administrator controlled configuration setting).

If you want to read a paper on why not validating certs is bad in software you can find it here!

How to get around python requests SSL and proxy error?

The problem is very likely not the authentication. Unfortunately, you don't provide details of the proxy configuration and the URL you use for the proxy. The only thing you provide is:

proxies = { 'https' : eampleIpWithAuth }

Based on the reference to _connect_tls_proxy in the stacktrace the eampleIpWithAuth is very likely something like https://..., i.e. you try to access the proxy itself over HTTPS. Note that accessing a proxy over HTTPS is different from using a HTTP proxy for HTTPS. When accessing a HTTPS URL over a HTTPS proxy one essentially does double encryption to the proxy:

client --- [HTTPS wrapped inside HTTPS] --- proxy --- [HTTPS] --- server

Whereas with a HTTPS URL over a "normal" HTTP proxy there is only single encryption, i.e. it looks (simplified) like this:

client --- [HTTPS wrapped inside HTTP]  --- proxy --- [HTTPS] --- server

Very likely the proxy you want to use is a plain HTTP proxy, and not a HTTPS proxy. This is actually the most common case.

The error happens since the proxy is not able to speak TLS but gets accessed by TLS. The fix is to use http://proxy and not https://proxy as the proxy address. Note that the latter worked in older versions of Python since proxy over HTTPS was not supported and a value of https:// for the protocol was treated the same as http://.