Sftp Upload in "Ascii" Mode

SFTP Upload in ASCII Mode

The Firezilla wiki has a page on technical specifications. The most widely implemented version of the SFTP spec is version 3 draft 2. Notably, the Openssh SFTP client and server implement that version of the spec.

Version 3 of the SFTP protocol doesn't have a feature equivalent to FTP's ASCII transfer mode. It always transfers files verbatim.

Later versions of the SFTP protocol support transfer modes that convert line terminator characters. Hypothetically, you could find a client and server that supports the feature.

Is this a matter of what SFTP protocol version the client is using...?

I don't know FireFTP's capabilities. But the OpenSSH SFTP server doesn't support an ASCII-like transfer mode.

is it simply "accepted" that when transferring a file from Windows -> Unix that you will have the additional CR (^M) leftover...

There are various ways of dealing with the issue. There are programs to convert from one format to another. Some programs that read text files will tolerate line terminators in either format.

How to set the transfer mode to ASCII in SFTP (version 3) using Perl

Using Net::SFTP::Foreign:

$sftp->put($local_from, $remote_to, conversion => 'unix2dos');

See On the fly data conversion.

ASCII mode in SFTP transfers

Quoting SFTP specification regarding text (aka ASCII) access to remote files:

5.3. Determining Server Newline Convention
In order to correctly process text files in a cross platform
compatible way, newline sequences must be converted between client
and server conventions.
The SSH_FXF_TEXT_MODE file open flag (Section 8.1.1) makes it
possible to request that the server translate a file to a 'canonical'
wire format. This format uses CRLF as the line separator.
Servers for systems using other conventions MUST translate to and
from the canonical form.

Note that the "text mode" is supported since SFTP version 4 only. Many existing SFTP implementations (including the most widespread one, the OpenSSH) support SFTP version 3 only. So they do not support text/ASCII mode at all.

How does Apache Camel support ASCII mode file transfer with SFTP?

The documentation is most likely false.

Both the FtpEndpointConfigurer and the SftpEndpointConfigurer indeed propagate the binary parameter to FtpConfiguration.setBinary and SftpConfiguration.setBinary respectively. So in that sense, both support that parameter.

But while FtpOperations queries FtpConfiguration.isBinary and sets FtpClient mode accordingly:

client.setFileType(configuration.isBinary() ? FTP.BINARY_FILE_TYPE : FTP.ASCII_FILE_TYPE);

... the SftpConfiguration.isBinary is never called. So the configured value is never used.

Is PuTTY (PSFTP) using binary mode to transfer a file in from some server to Windows client?

The PSFTP uses an SFTP protocol version 3.

In the SFTP protocol version 3 there are no transfer modes. Or in other words, there's only a binary transfer mode.

If you need "ascii download", you need to convert the downloaded file to Windows format after you download it with PSFTP.

Or use another Windows SFTP client that can do it for you.

For example WinSCP SFTP client supports the "ascii download".

If the server supports SFTP version 4 or newer, that supports "ascii download" natively, it uses that. If not, WinSCP converts the file after download automatically.

This is a simple batch file (.bat) to download a file in ascii mode in WinSCP:

winscp.com /log="c:\path\sftp.log" /command ^
    "open sftp://username:password@example.com/" ^
    "get -transfer=ascii /remote/path/file.txt c:\local\path\file.txt" ^
    "exit"

For details see:

guide to WinSCP scripting and
guide to converting PSFTP script to WinSCP script.

(I'm the author of WinSCP)

How to set file type and file transfer mode in JSch?

First of all, it is not clear from your question, whether you are aware that JSch and Apache Commons Net FtpClient each use completely different and unrelated protocols. JSch is an SSH/SFTP client, while FtpClient is an FTP client.

JSch uses an SFTP protocol version 3.

In the SFTP protocol version 3, there are no transfer modes. Or in other words, there is only the binary transfer mode.

And even if JSch used a newer version of SFTP protocol, which supports the ascii mode, the binary mode is the default anyway in SFTP.

So you do not have to do anything.

This is in a contrast to the FTP protocol, which defaults to the ascii mode. So you have explicitly switch to the binary mode with FTP (using the FTPClient.setFileType in case of the Apache Commons Net).

As for the FTPClient.setFileTransferMode – Your FTP code is wrong. The method accepts one of the *_TRANSFER_MODE constants. Never the *_FILE_TYPE. If you use BINARY_FILE_TYPE, it results in sending an invalid MODE I command to the server. Which for sure fails, and consequently has no effect at all. Check the method result code, it for sure returns false.

Remove the call from your FTP code. There is no need to call FTPClient.setFileTransferMode. Apache Commons Net supports only the default "stream" mode anyway (and most FTP servers do not support any other mode either).

Note the confusion in the names. The ascii/binary are commonly indeed called "transfer modes". But in FTP protocol specification (section 3.1.1 of RFC 959), they are actually called "data types" and are governed using TYPE command.

While what FTP specification calls "transmission modes", governed by MODE command (section 3.4), are completely different kind of modes: stream/block/compressed. They are hardly ever set or even mentioned.

As Apache Commons Net FtpClient confusingly names the method for setting "transmission mode" .setFileTransferMode, it frequently leads to the mistakes like you did – People incorrectly try to use FTPClient.setFileTransferMode to switch between the binary/ascii "data types".

Define transfer mode when trying to SFTP files using Python

pysftp/Paramiko uses an SFTP protocol version 3.

In the SFTP protocol version 3, there are no transfer modes. Or in other words, there is only the binary transfer mode.

Even if pysftp/Paramiko supported a newer version of SFTP, which do support the text/ascii mode, it is unlikely to help you. Most SFTP servers are OpenSSH. And OpenSSH uses SFTP 3 too.

See also How to transfer binary file in SFTP?

If you need to convert the file to Windows format, you need to do it upfront, before a file transfer.

A naive implementation would be like:

WINDOWS_LINE_ENDING = b'\r\n'
UNIX_LINE_ENDING = b'\n'

with open("/local/path/file.txt", "rb") as local_file:
    contents = local_file.read()
    contents = contents.replace(UNIX_LINE_ENDING, WINDOWS_LINE_ENDING)
    with sftp.open("/remote/path/file.txt", "wb") as remote_file:
        remote_file.write(contents)

(conversion based on How to convert CRLF to LF on a Windows machine in Python)

Sftp Upload in "Ascii" Mode