Python Pysftp Get_R from Linux Works Fine on Linux But Not on Windows

Python pysftp get_r from Linux works fine on Linux but not on Windows

Indeed, pysftp get_r does not work on Windows. It uses os.sep and os.path functions for remote SFTP paths, what is wrong, as SFTP paths always use a forward slash.

But you can easily implement a portable replacement.

import os
from stat import S_ISDIR, S_ISREG
def get_r_portable(sftp, remotedir, localdir, preserve_mtime=False):
for entry in sftp.listdir_attr(remotedir):
remotepath = remotedir + "/" + entry.filename
localpath = os.path.join(localdir, entry.filename)
mode = entry.st_mode
if S_ISDIR(mode):
try:
os.mkdir(localpath)
except OSError:
pass
get_r_portable(sftp, remotepath, localpath, preserve_mtime)
elif S_ISREG(mode):
sftp.get(remotepath, localpath, preserve_mtime=preserve_mtime)

Use it like:

get_r_portable(sftp, '/abc/def/ghi/klm/mno', 'C:\\pqr', preserve_mtime=False) 

Note that the above code can be easily modified to work with Paramiko directly, in case you do not want to use pysftp. The Paramiko SFTPClient class also has the listdir_attr and get methods. The only difference is that the Paramiko's get does not have the preserve_mtime parameter/functionality (but it can be implemented easily, if you need it).

And you should use Paramiko instead of pysftp, as pysftp seems to be a dead project. See pysftp vs. Paramiko.


Possible modifications of the code:

  • Do not download empty folders while downloading from SFTP server using Python
  • Download files from SFTP server that are older than 5 days using Python
  • How to sync only the changed files from the remote directory using pysftp?

For a similar question about put_r, see:

Python pysftp put_r does not work on Windows


Side note: Do not "disable host key checking". You are losing a protection against MITM attacks.

For a correct solution, see Verify host key with pysftp.

Python pysftp put_r does not work on Windows

I cannot reproduce your exact problem, but indeed the recursive functions of pysftp are known to be implemented in a way that makes them fail on Windows (or any system that does not use *nix-like path syntax).

Pysftp uses os.sep and os.path functions for remote SFTP paths, what is wrong, as SFTP paths always use a forward slash.


But you can easily implement a portable replacement:

import os
def put_r_portable(sftp, localdir, remotedir, preserve_mtime=False):
for entry in os.listdir(localdir):
remotepath = remotedir + "/" + entry
localpath = os.path.join(localdir, entry)
if not os.path.isfile(localpath):
try:
sftp.mkdir(remotepath)
except OSError:
pass
put_r_portable(sftp, localpath, remotepath, preserve_mtime)
else:
sftp.put(localpath, remotepath, preserve_mtime=preserve_mtime)

Use it like:

put_r_portable(sftp, sftp_local_path, sftp_remote_path, preserve_mtime=False) 

Note that the above code can be easily modified to work with Paramiko directly, in case you do not want to use pysftp. The Paramiko SFTPClient class also has the put method. The only difference is that the Paramiko's put does not have the preserve_mtime parameter/functionality (but it can be implemented easily, if you need it).


For a similar question about get_r, see:

Python pysftp get_r from Linux works fine on Linux but not on Windows

transferring files with pysftp.get_r between linux and windows

Currently, pysftp doesn't have support for such feature.

I would recommend Python's default FTP library, which is already in the standard library (batteries included). There you can call retrbinary() giving the desired folder.

TypeError only appears when executed on Linux, works perfectly fine on Windows

As it become clear from the comments - the problem is with host/local ip for which getmac.get_mac_address() returns None. I also confirmed that on Linux.
Looking at this issue - it's known bug/limitation of the package - it does not work for host ip on Linux. Check the discussion for more info.
You can use local interface name though.

Download files from SFTP server that are older than 5 days using Python

Use the pysftp.Connection.listdir_attr to get file listing with attributes (including the file timestamp).

Then, iterate the list and pick only the files you want.

import time
from stat import S_ISDIR, S_ISREG
def get_r_portable(sftp, remotedir, localdir, preserve_mtime=False):
for entry in sftp.listdir_attr(remotedir):
remotepath = remotedir + "/" + entry.filename
localpath = os.path.join(localdir, entry.filename)
mode = entry.st_mode
if S_ISDIR(mode):
try:
os.mkdir(localpath)
except OSError:
pass
get_r_portable(sftp, remotepath, localpath, preserve_mtime)
elif S_ISREG(mode):
if (time.time() - entry.st_mtime) // (24 * 3600) >= 5:
sftp.get(remotepath, localpath, preserve_mtime=preserve_mtime)

Though the code can be much simpler, if you do not need a recursive download:

for entry in sftp.listdir_attr(remotedir):
mode = entry.st_mode
if S_ISREG(mode) and ((time.time() - entry.st_mtime) // (24 * 3600) >= 5):
remotepath = remotedir + "/" + entry.filename
localpath = os.path.join(localdir, entry.filename)
sftp.get(remotepath, localpath, preserve_mtime=True)

Though note that pysftp seems dead. Consider using Paramiko instead. It has pretty much same API, so the above code will work almost as it is (except that Paramiko does not have the preserve_mtime parameter). See also pysftp vs. Paramiko.


Based on:

  • Python pysftp get_r from Linux works fine on Linux but not on Windows

    (I have updated this source of your code to use listdir_attr as it's more effective)
  • How to sync only the changed files from the remote directory using pysftp?
  • Delete files that are older than 7 days

Specify file pattern in pysftp get

There's no function to download files matching a file mask in pysftp.

You have to:

  • list the directory, using Connection.listdir or Connection.walktree (if you need recursion)
  • iterate the list of files, filtering the files you want
  • call Connection.get individually for each.


  • For a trivial implementation, see:

    List files on SFTP server matching wildcard in Python using Paramiko

    It's about Paramiko, but the file matching part will be the same with pysftp:

    import fnmatch
    for filename in sftp.listdir('/remote/path'):
    if fnmatch.fnmatch(filename, "*.txt"):
    sftp.get("/remote/path/" + filename, "/local/path/" + filename)

    Though your should really be using Paramiko anyway: pysftp vs. Paramiko

  • For a recursive example (you have to add the file matching), see:

    Python pysftp get_r from Linux works fine on Linux but not on Windows.

  • See also how Connection.get_d or Connection.get_r are implemented.



Related Topics



Leave a reply



Submit