Verify Host Key With Pysftp

Verify host key with pysftp against known_hosts file with custom port

I guess the problem is that pysftp do not support custom ports when looking up the host key in the known_hosts file.

Add an entry like this:

data-nz.metservice.com ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC8N65MCdnbHjaEDxkZPPq1QO0RLwP3cdm9Gb9BAMS0mFH39d7/yHIerA6yFZRW27u3NClI7V1F3hDuheoCUomeF9Q9ioaeQ2dlX27hmGf611RpSfI/vGgnmipHYzzHsCIJi0LxuowCouKNw8g1v1e2VzsVWFPaq+cDeuUpDwpBKWnxQUWN7U9mzN1k0sDALimWOzhfQmXtCzPkHqERUcPpdU7/zWP8Xk9H7FQxgiPFa+EC5xuCzn01CcJppQ8VBqL9R6SNNP/d9ymQWh3cotXe6sj5gt2MdfbAUfxddQITW1rU+LSOkG21QPMq0VBDJwWf9RpqhnqcvusZIFVGyOsn

Or use Paramiko directly (pysftp is a wrapper around Paramiko), as Paramiko implements the host key lookup with custom port correctly.

Some background: The codes at the beginning of your original known_hosts are hashed host[:port] identifiers. Pysftp does not use port when looking up the entry, so it does not find the correct one. If you add fake (hashed or not) entry without the custom port, pysftp will be able to find it.

Using Python's pysftp, how do you verify a host key?

cnopts = pysftp.CnOpts()
cnopts.hostkeys.load('sftpserver.pub')

where the sftpserver.pub contains a server public key in a format like:

example.com ssh-rsa AAAAB3NzaC1yc2EAAAADAQAB...

An easy way to retrieve the host key in this format is using OpenSSH ssh-keyscan:

ssh-keyscan example.com

Though for absolute security, you should not retrieve the host key remotely, as you cannot be sure, if you are not being attacked already.

See my article Where do I get SSH host key fingerprint to authorize the server? It's for my WinSCP SFTP client, but most information there is valid in general.

If you do not want to use an external file, you can also use

cnopts.hostkeys.add(...)

For other options, see: Verify host key with pysftp.

No hostkey for ... found in pysftp code even though cnopts.hostkeys is set to None

After much trial and error, converting the following SFTP command put into the terminal:

sftp -i /some_dir/another_dir/key -oPort=12345 user@12.123.456.789

can be translated* into:

paramiko.SSHClient().connect(hostname='12.123.456.789', username='user', port=12345,
                             key_filename='/some_dir/another_dir/key')

The whole condensed code is:

#!/usr/bin/python3

import paramiko

try:
    client = paramiko.SSHClient()
    client.load_system_host_keys()
    client.set_missing_host_key_policy(paramiko.WarningPolicy)
    client.connect(hostname='12.123.456.789', username='user', port=12345, 
                   key_filename='/some_dir/another_dir/key')

    # -------------------------- [ just for testing ] --------------------------
    stdin, stdout, stderr = client.exec_command('ls -la')   # THIS IS FOR TESTING
    print(stdout.read())                                    # AND PRINTING OUT

finally:
    client.close()

PySFTP failing with No hostkey for host X found when deploying Django/Heroku

For a general discussion about the "No hostkey for host ... found", see:

Verify host key with pysftp

Regarding the implementation on Heroku: I'm not familiar with it, but afaik, and as you as well commented, it does not have a persistent file storage.

For this reason, using an implementation that has the host key hard-coded is appropriate. Two solutions from my answer to the above question suit that need:

If you do not want to use an external file, you can also use

from base64 import decodebytes
# ...

keydata = b"""AAAAB3NzaC1yc2EAAAADAQAB..."""
key = paramiko.RSAKey(data=decodebytes(keydata))
cnopts = pysftp.CnOpts()
cnopts.hostkeys.add('example.com', 'ssh-rsa', key)
 
with pysftp.Connection(host, username, password, cnopts=cnopts) as sftp:

If you need to verify the host key using its fingerprint only, see Python - pysftp / paramiko - Verify host key using its fingerprint.

This is also relevant (while about Paramiko directly, not about pysftp wrapper):

Paramiko SSH failing with "Server '...' not found in known_hosts" when run on web server

Python - pysftp / paramiko - Verify host key using its fingerprint

Depending on your needs you can use either of these two methods:

In case you need to verify only one specific host key

Use ssh-keyscan (or similar) to retrieve the host public key:

ssh-keyscan example.com > tmp.pub

The tmp.pub will look like (known_hosts file format):

example.com ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA0hVqZOvZ7yWgie9OHdTORJVI5fJJoH1yEGamAd5G3werH0z7e9ybtq1mGUeRkJtea7bzru0ISR0EZ9HIONoGYrDmI7S+BiwpDBUKjva4mAsvzzvsy6Ogy/apkxm6Kbcml8u4wjxaOw3NKzKqeBvR3pc+nQVA+SJUZq8D2XBRd4EDUFXeLzwqwen9G7gSLGB1hJkSuRtGRfOHbLUuCKNR8RV82i3JvlSnAwb3MwN0m3WGdlJA8J+5YAg4e6JgSKrsCObZK7W1R6iuyuH1zA+dtAHyDyYVHB4FnYZPL0hgz2PSb9c+iDEiFcT/lT4/dQ+kRW6DYn66lS8peS8zCJ9CSQ==

Now, you can calculate a fingerprint of that public key with ssh-keygen:

ssh-keygen -l -f tmp.pub -E md5

(use the -E md5 only with newer versions of OpenSSH that support multiple fingerprint algorithms and default to SHA256)

You will get something like:

2048 MD5:c4:26:18:cf:a0:15:9a:5f:f3:bf:96:d8:3b:19:ef:7b example.com (RSA)

If the fingerprint matches with the one you have, you can now safely assume that the tmp.pub is a legitimate public key and use it in the code:

from base64 import decodebytes
# ...

keydata = b"""AAAAB3NzaC1yc2EAAAABIwAAAQEA0hV..."""
key = paramiko.RSAKey(data=decodebytes(keydata))
cnopts = pysftp.CnOpts()
cnopts.hostkeys.add('example.com', 'ssh-rsa', key)

with pysftp.Connection(host, username, password, cnopts=cnopts) as sftp:

(based on Verify host key with pysftp)

In case you need to automate verification of a host key based on its fingerprint

E.g. because the fingerprint comes from an external configuration.

I'm not sure if a limited API of pysftp allows that. You probably would have to skip pysftp and use Paramiko library directly (pysftp uses Paramiko internally).

With Paramiko, you can cleverly implement MissingHostKeyPolicy interface.

Start with how AutoAddPolicy is implemented:

class AutoAddPolicy (MissingHostKeyPolicy):
    """
    Policy for automatically adding the hostname and new host key to the
    local `.HostKeys` object, and saving it.  This is used by `.SSHClient`.
    """

    def missing_host_key(self, client, hostname, key):
        client._host_keys.add(hostname, key.get_name(), key)
        if client._host_keys_filename is not None:
            client.save_host_keys(client._host_keys_filename)
        client._log(DEBUG, 'Adding %s host key for %s: %s' %
                    (key.get_name(), hostname, hexlify(key.get_fingerprint())))

Note that in the code you have the fingerprint available in hexlify(key.get_fingerprint()). Just compare that value against the fingerprint you have. If it matches, just return. Otherwise raise an exception,
like the RejectPolicy does.

Another solution (which would work even with pysftp) is to implement PKey in a way that it holds only the fingerprint. And implement its __cmp__ method to compare the fingerprint only. Such an instance of PKey can then be added to cnopts.hostkeys.add.

OP posted an implementation of this approach in his answer. Allegedly for Python 3, more complex implementation is needed, as seen in Connecting to an SFTP server using pysftp and Python 3 with just the server fingerprint.

Verify Host Key With Pysftp