Python pysftp get_r from Linux works fine on Linux but not on Windows
Indeed, pysftp get_r
does not work on Windows. It uses os.sep
and os.path
functions for remote SFTP paths, what is wrong, as SFTP paths always use a forward slash.
But you can easily implement a portable replacement.
import os
from stat import S_ISDIR, S_ISREG
def get_r_portable(sftp, remotedir, localdir, preserve_mtime=False):
for entry in sftp.listdir_attr(remotedir):
remotepath = remotedir + "/" + entry.filename
localpath = os.path.join(localdir, entry.filename)
mode = entry.st_mode
if S_ISDIR(mode):
try:
os.mkdir(localpath)
except OSError:
pass
get_r_portable(sftp, remotepath, localpath, preserve_mtime)
elif S_ISREG(mode):
sftp.get(remotepath, localpath, preserve_mtime=preserve_mtime)
Use it like:
get_r_portable(sftp, '/abc/def/ghi/klm/mno', 'C:\\pqr', preserve_mtime=False)
Note that the above code can be easily modified to work with Paramiko directly, in case you do not want to use pysftp. The Paramiko SFTPClient
class also has the listdir_attr
and get
methods. The only difference is that the Paramiko's get
does not have the preserve_mtime
parameter/functionality (but it can be implemented easily, if you need it).
And you should use Paramiko instead of pysftp, as pysftp seems to be a dead project. See pysftp vs. Paramiko.
Possible modifications of the code:
- Do not download empty folders while downloading from SFTP server using Python
- Download files from SFTP server that are older than 5 days using Python
- How to sync only the changed files from the remote directory using pysftp?
For a similar question about put_r
, see:
Python pysftp put_r does not work on Windows
Side note: Do not "disable host key checking". You are losing a protection against MITM attacks.
For a correct solution, see Verify host key with pysftp.
Python pysftp put_r does not work on Windows
I cannot reproduce your exact problem, but indeed the recursive functions of pysftp are known to be implemented in a way that makes them fail on Windows (or any system that does not use *nix-like path syntax).
Pysftp uses os.sep
and os.path
functions for remote SFTP paths, what is wrong, as SFTP paths always use a forward slash.
But you can easily implement a portable replacement:
import os
def put_r_portable(sftp, localdir, remotedir, preserve_mtime=False):
for entry in os.listdir(localdir):
remotepath = remotedir + "/" + entry
localpath = os.path.join(localdir, entry)
if not os.path.isfile(localpath):
try:
sftp.mkdir(remotepath)
except OSError:
pass
put_r_portable(sftp, localpath, remotepath, preserve_mtime)
else:
sftp.put(localpath, remotepath, preserve_mtime=preserve_mtime)
Use it like:
put_r_portable(sftp, sftp_local_path, sftp_remote_path, preserve_mtime=False)
Note that the above code can be easily modified to work with Paramiko directly, in case you do not want to use pysftp. The Paramiko SFTPClient
class also has the put
method. The only difference is that the Paramiko's put
does not have the preserve_mtime
parameter/functionality (but it can be implemented easily, if you need it).
For a similar question about get_r
, see:
Python pysftp get_r from Linux works fine on Linux but not on Windows
transferring files with pysftp.get_r between linux and windows
Currently, pysftp doesn't have support for such feature.
I would recommend Python's default FTP library, which is already in the standard library (batteries included). There you can call retrbinary() giving the desired folder.
TypeError only appears when executed on Linux, works perfectly fine on Windows
As it become clear from the comments - the problem is with host/local ip for which getmac.get_mac_address()
returns None
. I also confirmed that on Linux.
Looking at this issue - it's known bug/limitation of the package - it does not work for host ip on Linux. Check the discussion for more info.
You can use local interface name though.
Download files from SFTP server that are older than 5 days using Python
Use the pysftp.Connection.listdir_attr
to get file listing with attributes (including the file timestamp).
Then, iterate the list and pick only the files you want.
import time
from stat import S_ISDIR, S_ISREG
def get_r_portable(sftp, remotedir, localdir, preserve_mtime=False):
for entry in sftp.listdir_attr(remotedir):
remotepath = remotedir + "/" + entry.filename
localpath = os.path.join(localdir, entry.filename)
mode = entry.st_mode
if S_ISDIR(mode):
try:
os.mkdir(localpath)
except OSError:
pass
get_r_portable(sftp, remotepath, localpath, preserve_mtime)
elif S_ISREG(mode):
if (time.time() - entry.st_mtime) // (24 * 3600) >= 5:
sftp.get(remotepath, localpath, preserve_mtime=preserve_mtime)
Though the code can be much simpler, if you do not need a recursive download:
for entry in sftp.listdir_attr(remotedir):
mode = entry.st_mode
if S_ISREG(mode) and ((time.time() - entry.st_mtime) // (24 * 3600) >= 5):
remotepath = remotedir + "/" + entry.filename
localpath = os.path.join(localdir, entry.filename)
sftp.get(remotepath, localpath, preserve_mtime=True)
Though note that pysftp seems dead. Consider using Paramiko instead. It has pretty much same API, so the above code will work almost as it is (except that Paramiko does not have the preserve_mtime
parameter). See also pysftp vs. Paramiko.
Based on:
- Python pysftp get_r from Linux works fine on Linux but not on Windows
(I have updated this source of your code to uselistdir_attr
as it's more effective) - How to sync only the changed files from the remote directory using pysftp?
- Delete files that are older than 7 days
Specify file pattern in pysftp get
There's no function to download files matching a file mask in pysftp.
You have to:
- list the directory, using
Connection.listdir
orConnection.walktree
(if you need recursion) - iterate the list of files, filtering the files you want
- call
Connection.get
individually for each.
For a trivial implementation, see:
List files on SFTP server matching wildcard in Python using ParamikoIt's about Paramiko, but the file matching part will be the same with pysftp:
import fnmatch
for filename in sftp.listdir('/remote/path'):
if fnmatch.fnmatch(filename, "*.txt"):
sftp.get("/remote/path/" + filename, "/local/path/" + filename)Though your should really be using Paramiko anyway: pysftp vs. Paramiko
For a recursive example (you have to add the file matching), see:
Python pysftp get_r from Linux works fine on Linux but not on Windows.See also how
Connection.get_d
orConnection.get_r
are implemented.
Related Topics
Python Spawn Off a Child Subprocess, Detach, and Exit
How to Get Output from Subprocess.Popen(). Proc.Stdout.Readline() Blocks, No Data Prints Out
Why Does My Recursive Function Return None
How to Iterate Over a List in Chunks
Convert a String Representation of a Dictionary to a Dictionary
Configure Flask Dev Server to Be Visible Across the Network
How to Import a Module Given Its Name as String
Passing HTML to Template Using Flask/Jinja2
How to Find All Occurrences of a Substring
Shebang Notation: Python Scripts on Windows and Linux
Cross-Platform Subprocess With Hidden Window
Regexp Finding Longest Common Prefix of Two Strings
Difference Between _Str_ and _Repr_
How to Dynamically Create Variables
How to Split a String into a List