Python and Openssl Version Reference Issue on Os X

Python and OpenSSL version reference issue on OS X

You can install a version of python that uses a newer version of openssl.

First you can brew install a new version of open SSL

brew update
brew install openssl
brew link --force openssl

You should see a newer version with

openssl version -a

Then you can install a different version of python which uses that newer version of openssl. If you use pyenv, that would be:

CFLAGS="-I$(brew --prefix openssl)/include" \
LDFLAGS="-L$(brew --prefix openssl)/lib" \
pyenv install -v 3.4.3

You can find more information about installing a version of python with a brew installed version of openssl here: https://github.com/yyuu/pyenv/wiki/Common-build-problems

SSL module in Python is not available (on OSX)

The ssl module as well as its underlying C extension appears to be a part of the python formula:

Mac-Admin:~ admin$ python3
Python 3.7.4 (default, Sep  7 2019, 18:27:02) 
[Clang 10.0.1 (clang-1001.0.46.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
>>> ssl
<module 'ssl' from '/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ssl.py'>
>>> import _ssl
>>> _ssl
<module '_ssl' from '/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload/_ssl.cpython-37m-darwin.so'>

so it being missing most probably means package installation corruption which brew reinstall python should fix.

---

Also note that while Homebrew allows multiple versions to coexist, its installation logic isn't quite designed to keep the alternative versions operational unless they are installed via a versioned formula (and e.g. routinely removes old versions in the regular brew cleanup).

So consider using pyenv (also available via brew) if you need to routinely switch between Python versions -- or some 3rd-party tap that offers versioned formulae for it.

How can I compile Python 3.6.2 on macOS with openSSL from homebrew?

None of the previous answers I found earlier worked for me, but I did eventually figure this out with the help of another answer not mentioned earlier. Here was the actual fix:
https://stackoverflow.com/a/20740964/2934226

Basically, CPPFLAGS and LDFLAGS can't be set in the environment; you need to set them alongside the configure command, like this:

./configure CPPFLAGS="-I[openSSL install location]/include" LDFLAGS="-L[openSSL install location]/lib" [other flags here]

And then after compiling and installing, it worked!

$ python3 -c "import ssl; print(ssl.OPENSSL_VERSION)"
OpenSSL 1.0.2l  25 May 2017

Here are the things that didn't work, and why:

How do I compile Python 3.4 with custom OpenSSL? doesn't help because you can't set LDFLAGS, CFLAGS, or CPPFLAGS in the environment; setup.py doesn't pass them along to the actual compilation steps. And even if setting LD_LIBRARY_PATH might work, you don't want to do that because it's dangerous (see http://xahlee.info/UnixResource_dir/_/ldpath.html). Finally, --with-ssl isn't a valid configure argument, and the patch listed there to add it doesn't apply cleanly.

Homebrew refusing to link OpenSSL doesn't apply when you're trying to build something from source, rather than trying to get an an already-compiled dylib to find a relocated library. Furthermore, making symlinks in /usr/local is dangerous, and can cause programs to compile against the newer headers but use the older system binaries.

how to include ssl with python build on MacOS doesn't work properly. Editing setup.py to add the lib and include directories for where you've installed my openSSL partially works, and lets you compile in SSL support. Alas, they aren't importable because the old version is still getting used:

Following modules built successfully but were removed because they could not be imported:
_hashlib              _ssl                                    

[...]

building '_ssl' extension
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -I/usr/local/opt/openssl/include/ -I./Include -I/oebuild/python/python-3.6.1/include -I. -I/usr/local/include -I/oebuild/python/src/Python-3.6.1/Include -I/oebuild/python/src/Python-3.6.1 -c /oebuild/python/src/Python-3.6.1/Modules/_ssl.c -o build/temp.macosx-10.11-x86_64-3.6/oebuild/python/src/Python-3.6.1/Modules/_ssl.o
gcc -bundle -undefined dynamic_lookup build/temp.macosx-10.11-x86_64-3.6/oebuild/python/src/Python-3.6.1/Modules/_ssl.o -L/oebuild/python/python-3.6.1/lib -L/usr/local/lib -lssl -lcrypto -o build/lib.macosx-10.11-x86_64-3.6/_ssl.cpython-36m-darwin.so
building '_hashlib' extension
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -I/usr/local/opt/openssl/include/ -I./Include -I/oebuild/python/python-3.6.1/include -I. -I/usr/local/include -I/oebuild/python/src/Python-3.6.1/Include -I/oebuild/python/src/Python-3.6.1 -c /oebuild/python/src/Python-3.6.1/Modules/_hashopenssl.c -o build/temp.macosx-10.11-x86_64-3.6/oebuild/python/src/Python-3.6.1/Modules/_hashopenssl.o
gcc -bundle -undefined dynamic_lookup build/temp.macosx-10.11-x86_64-3.6/oebuild/python/src/Python-3.6.1/Modules/_hashopenssl.o -L/oebuild/python/python-3.6.1/lib -L/usr/local/lib -lssl -lcrypto -o build/lib.macosx-10.11-x86_64-3.6/_hashlib.cpython-36m-darwin.so
*** WARNING: renaming "_ssl" since importing it failed: dlopen(build/lib.macosx-10.11-x86_64-3.6/_ssl.cpython-36m-darwin.so, 2): Symbol not found: _CRYPTO_THREADID_set_callback
  Referenced from: build/lib.macosx-10.11-x86_64-3.6/_ssl.cpython-36m-darwin.so
  Expected in: flat namespace
 in build/lib.macosx-10.11-x86_64-3.6/_ssl.cpython-36m-darwin.so
*** WARNING: renaming "_hashlib" since importing it failed: dlopen(build/lib.macosx-10.11-x86_64-3.6/_hashlib.cpython-36m-darwin.so, 2): Symbol not found: _HMAC_CTX_copy
  Referenced from: build/lib.macosx-10.11-x86_64-3.6/_hashlib.cpython-36m-darwin.so
  Expected in: flat namespace
 in build/lib.macosx-10.11-x86_64-3.6/_hashlib.cpython-36m-darwin.so

otool -L shows the problem:

$ otool -L build/lib.macosx-10.11-x86_64-3.6/_ssl.cpython-36m-darwin_failed.so 
build/lib.macosx-10.11-x86_64-3.6/_ssl.cpython-36m-darwin_failed.so:
    /usr/lib/libssl.0.9.8.dylib (compatibility version 0.9.8, current version 0.9.8)
    /usr/lib/libcrypto.0.9.8.dylib (compatibility version 0.9.8, current version 0.9.8)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)

(CRYPTO_THREADID was introduced in version 1.0.0, according to https://wiki.openssl.org/index.php/Manual:Threads(3)#HISTORY)

---

Related Topics