Docker Container Cmake Gives Crypto/Fips/Fips.C:153: Openssl Internal Error: Fatal Fips Selftest Failure - Dracut-Fips Crypto.Fips_Enabled = 1

Docker container CMAKE gives crypto/fips/fips.c:153: OpenSSL internal error: FATAL FIPS SELFTEST FAILURE - dracut-fips crypto.fips_enabled = 1

Found the solution

WHY this issue was coming:
Docker build (to create the image) was run on a machine (the other machine, where cmake was working inside the docker container)... i.e. where FIPS was not installed.
During the build step, Docker file was running
zypper install cmake (or yum install cmake).

As FIPS was disabled on the machine where docker image was built, cmake was installed in docker image, without understanding FIPS being enabled / dracut-fips being installed.

Then, when you copy this image and consume it on a machine where FIPS is actually ENABLED, then cmake was failing with that error mesg: crypto/fips/fips.c:153: OpenSSL internal error: FATAL FIPS SELFTEST FAILURE

Actually there are 2 solutions.

Solution #1: Tag your docker image appropriately.

Summary:

PS: If you do install cmake using the above, then it'll only work on target machine where you run container, if that machine's FIPS is disabled or enabled at build time. i.e. if FIPS was enabled, you install cmake and run it on a machine where it's NOT same FIPS setting as the host, where image was built, then you'll be reading this post for help.

Better way in case you want to install cmake using the above package managers, would be, tag your docker image appropriately during image creation time i.e.:

docker build -t <image-name>-fips-enabled ... if FIPS is enabled

and

docker build -t <image-name>-fips-disabled ... if FIPS on that that machine is disabled.

That way, you can pick the correct docker image imagename-fips-enabled vs imagename-fips-disabled acc. to what your target machine FIPS setting is (where you'll actually perform docker run ... using this image).

Solution #2: Don't use zypper (OpenSuse) or yum if you have RedHat container.
and this solution is flexible in the sense, that it's independent of FIPS setting = 0 / 1 on the host, where image was built.

I didn't use zypper/yum to install cmake inside Dockerfile, but just grabbed cmake-3.18.2-Linux-x86_64.tar.gz bundle file.

Within Dockerfile, I simply extracted this .tar.gz file inside some directory.
Also I set export PATH:/path/where/I/installed/cmake-3.18.2../bin:/..some_other_paths:/...:/.... in Dockerfile inside RUN statement.

i.e.
RUN export PATH=/path/where/I/installed/cmake-3.18.2../bin:/...... && <more cmds here> && <some other cmds here> && ... etc, so it can find the extracted cmake 3.18.2 for any buil-time (cmake operations) and also SET the same PATH=/... variable as ENV PATH=/.... same value used during RUN for PATH so at runtime, when the container runs, $PATH is all set for finding cmake (3.18.2 version) rather than using any existing /usr/bin/cmake or some other shit).

Dockerfile snapshot:

#    curl -k -sSf -H "X-JFrog-Art-Api:dslfhjlieurqwihlj233lk2l4j6p9usdkajdfasddl809842iijhlkhflhafOHIHFLyeaGoodLuck" \
#    -o /tmp/cmake.tar.gz https://artifactory.company.com/artifactory/some-Local/cmake/cmake-3.18.2-Linux-x86_64.tar.gz && \

and

as my umask settings were set to 022, I didn't have to do any chicken chmod operation post untar:

#    echo -e "\n-- Installing CMake ...\n" && \
#    tar -xvzpf /tmp/cmake.tar.gz -C /home/docker_nonroot_user/tools/ && \

Inside the docker container, as ENV PATH=/... was set in Dockerfile for this target path as well, the correct cmake 3.18.2 was used at docker container runtime action.

cmake installed location in my case was:

/home/docker_nonroot_user/tools/cmake-3.18.2-Linux-x86_64/bin/cmake

PATH inside docker container was:

/home/docker_nonroot_user/tools/cov-analysis/bin:/home/docker_nonroot_user/tools/cmake-3.18.2-Linux-x86_64/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

RESULT:

Using a Docker image built using SOLUTION #2 above, i.e. docker image built on a machine where FIPS = 0 aka DISABLED and then later, using this same image to create a container on a totally different target host machine where FIPS = 1 aka ENABLED, I see:

87d8104d8c41:/home/docker_nonroot_user # sysctl -a|grep fips_en
crypto.fips_enabled = 1
87d8104d8c41:/home/docker_nonroot_user #
87d8104d8c41:/home/docker_nonroot_user # which cmake
/usr/bin/cmake
87d8104d8c41:/home/docker_nonroot_user #
87d8104d8c41:/home/docker_nonroot_user # cmake --version
crypto/fips/fips.c:153: OpenSSL internal error: FATAL FIPS SELFTEST FAILURE
Aborted (core dumped)
87d8104d8c41:/home/docker_nonroot_user #

87d8104d8c41:/home/docker_nonroot_user #
87d8104d8c41:/home/docker_nonroot_user # ls -l /home/docker_nonroot_user/tools/cmake-3.18.2-Linux-x86_64/bin
total 75504
-rwxr-xr-x 1 root root 11908568 Aug 20  2020 ccmake
-rwxr-xr-x 1 root root 12096216 Aug 20  2020 cmake
-rwxr-xr-x 1 root root 27476480 Aug 20  2020 cmake-gui
-rwxr-xr-x 1 root root 12398808 Aug 20  2020 cpack
-rwxr-xr-x 1 root root 13318712 Aug 20  2020 ctest
87d8104d8c41:/home/docker_nonroot_user #
87d8104d8c41:/home/docker_nonroot_user # ls -l /home/docker_nonroot_user/tools/cmake-3.18.2-Linux-x86_64/bin/cmake
-rwxr-xr-x 1 root root 12096216 Aug 20  2020 /home/docker_nonroot_user/tools/cmake-3.18.2-Linux-x86_64/bin/cmake
87d8104d8c41:/home/docker_nonroot_user #
87d8104d8c41:/home/docker_nonroot_user # /home/docker_nonroot_user/tools/cmake-3.18.2-Linux-x86_64/bin/cmake --version
cmake version 3.18.2

CMake suite maintained and supported by Kitware (kitware.com/cmake).
87d8104d8c41:/home/docker_nonroot_user # : Merry X-mas Baaaaeeebyyy! - no more FIPS shit error now. Next I'll fix some chown on ~<user> and close my story.

SOLUTION #3: You can disable FIPS but only if you are allowed to do so, then you don't need solution #1 or solution #2.

MISC Info on OpenSSL errors:

The effects of self-test failures in the Module differ depending on the type of self-test that failed.

The FIPS_mode_set() function verifies the integrity of the runtime executable using a HMAC SHA-256 digest,
which is computed at build time. If this computed HMAC SHA-256 digest matches the stored, known digest, then
the power-up self-test (consisting of the algorithm-specific Pairwise Consistency and Known Answer tests) is
performed.

Non-fatal self-test errors transition the module into an error state. The application must be restarted to recover
from these errors. The non-fatal self-test errors are:

FIPS_R_FINGERPRINT_DOES_NOT_MATCH - The integrity verification check failed

FIPS_R_FIPS_SELFTEST_FAILED - a known answer test failed

FIPS_R_SELFTEST_FAILED - a known answer test failed

FIPS_R_TEST_FAILURE – a known answer test failed (RSA); pairwise consistency test failed (DSA)

FIPS_R_PAIRWISE_TEST_FAILED – a pairwise consistency test during DSA or RSA key generation failed

FIPS_R_FIPS_MODE_ALREADY_SET - the application initializes the FIPS mode when it is already initialized

These errors are reported through the regular ERR interface of the shared libraries and can be queried by
functions such as ERR_get_error(). See the OpenSSL Module manual page for the function description.

A fatal error occurs only when the module is in the error state (a self-test has failed) and the application calls a
crypto function of the module that cannot return an error in normal circumstances (void return functions). The
error message: 'FATAL FIPS SELFTEST FAILURE' is printed to stderr and the application is terminated with the
abort() call.

The only way to recover from a fatal error is to restart the application. If failures persist, you must reinstall the Module. If you downloaded the software, verify the package hash to confirm a proper download.

IMPORTANT NOTE:

If you installed a newer version of OpenSSL x.y.z (rpm/etc) and didn't reboot/restart your machine, some of the libraries which consume openssl will be vulnerable / error out too with similar errors.

You can test this by running:

uptime command to see how long your server/machine has been UP n running.
sudo lsof | grep libssl.so or to Find processes running with deleted OpenSSL libraries, run:

sudo lsof | grep DEL.*libssl

So, try rebooting your machine and see if the errors goes away and check the output of the above commands.

For more info, refer this link: https://raymii.org/s/snippets/Find_all_services_using_libssl_to_restart_after_an_OpenSSL_update.html

PyInstaller FATAL FIPS SELFTEST FAILURE

This usually happens, if you installed python with FIPS enabled/disabled or if Python is consuming a module/library/something, where Python/ or that particular library/python pyInstaller may be consuming an artifact (that was built on some machine previously, with FIPS Disabled); then, using such executables (built with FIPS disabled) on another machine(where FIPS is actually enabled) i.e. where FIPS setting doesn't match with the machine, where you built the app in python or that particular library, then, you'll get such errors.

Ex: Let's say a package manager ex: yum / zypper installed python without openssl (which is a pre-install/pre-requisite for python app, which needs Openssl functionality). If python is installed without Openssl first already installed, then any Python app, which needs openssl processing, Python won't work, even if you install openssl later on the machine; Python will still spit out bunch of gnarly openssl errors :--) big grin.

Similarly, if something was installed/packaged/built using FIPS disabled (on some machine), it can be an app or any library that you are consuming... then later, if you try to run an app or an app(consuming that library/package) on a machine, where FIPS = 1 (enabled) then you'll see this error.

In my case, cmake was failing for running simple cmake --version and all I did was used a cmake .tar bundle which may have been compiled on a machine, where source and target machine's FIPS settings were same i.e. in my case, cmake tar bundle came from a FIPS enabled machine, and when I used it on this other new machine where FIPS was enabled, it worked!