Commons VFS and Java.net.URL - Adding support for sftp:// protocol
See the list of dependencies that commons-vfs requires for sftp to work. Once JSch is in the classpath, your exception no longer happen. Have a look at Apache's Wiki for examples of connecting via sftp.
Commons VFS and Java.net.URL - Adding support for “ram://” protocol
VFS supports creating a stream handler factory which knows about all the registered schemes.
// you might want to configure a manager with less schemes
FileSystemManager fsm = VFS.getManager();
URLStreamHandlerFactory factory = fsm.getURLStreamHandlerFactory();
URL.setURLStreamHandlerFactory(factory); // VM global
URL url = new URL("ram://test.txt");
How to skip password prompt during SFTP using commons-VFS
Setting setPreferredAuthentications as "publickey,keyboard-interactive,password" would resolve this issue.
// Setup our SFTP configuration
FileSystemOptions opts = new FileSystemOptions();
SftpFileSystemConfigBuilder.getInstance().setStrictHostKeyChecking(opts, "no");
SftpFileSystemConfigBuilder.getInstance().setUserDirIsRoot(opts, true);
SftpFileSystemConfigBuilder.getInstance().setTimeout(opts, 10000);
SftpFileSystemConfigBuilder.getInstance().setPreferredAuthentications(opts, "publickey,keyboard-interactive,password");
Apache commons VFS cannot resolve files with String URI
After so many attempts I was able to found the issue myself and thought to mention it here so it might be helpful to someone.
Short Answer : remove any occurence of fileSystemManager.close()
Detailed Answer : This is how I dealt with FileSystemManager in my code.
I'm always using VFS.getManager() to get a FileSystemManager instance and nowhere else custom FileSystemManagers are created by my own.
if (fileSystemManager == null) {
fileSystemManager = VFS.getManager();
}
In one place I have closed the FileSystemManager singleton which I obtained from the above method through fileSystemManager.close() in order to release the resource. But in another method which is totally independent from the first method which it has used and closed the fileSystemManager instance, is again trying to use the same fileSystemManager.
So once the fileSystemManager is closed, most of the components like providers, fileCache, etc... which are inside the fileSystemManager are set to null but the fileSystemManager singleton is still not null. So program will skip the If condition and trying to resolve the filePath with a fileSystemManager that has no Components in it.
Even this is clearly said in the documentation itself and noticed after I discovered the issue.
Closes the given file system.
If you use VFS as singleton it is VERY dangerous to call this method.
Once fileSystemManager.close() is removed, problem solved.
SFTP Upload Download Exist and Move using Apache Commons VFS
The below SFTP Utility gives an example of Upload, Exist, Download, Move and Delete operations. I have used this in several of my projects. The below code uses api calls from Apache Commons VFS library only.
Please make sure that you put the following JAR's in your project's build path:
- commons-logging-1.1.3.jar
- commons-vfs2-2.0.jar
- hamcrest-core-1.3.jar
- jsch-0.1.50.jar
Any other compatible versions would do as well.
import java.io.File;
import org.apache.commons.vfs2.FileObject;
import org.apache.commons.vfs2.FileSystemException;
import org.apache.commons.vfs2.FileSystemOptions;
import org.apache.commons.vfs2.Selectors;
import org.apache.commons.vfs2.impl.StandardFileSystemManager;
import org.apache.commons.vfs2.provider.sftp.SftpFileSystemConfigBuilder;
/**
* The class SFTPUtil containing uploading, downloading, checking if file exists
* and deleting functionality using Apache Commons VFS (Virtual File System)
* Library
*
* @author Ashok
*
*/
public class SFTPUtility {
public static void main(String[] args) {
String hostName = "PutYourHostNameHere";
String username = "PutYourUserNameForHostHere";
String password = "PutYourPasswordForHostHere";
String localFilePath = "C:\\FakePath\\FakeFile.txt";
String remoteFilePath = "/FakeRemotePath/FakeRemoteFile.txt";
String remoteTempFilePath = "/FakeRemoteTempPath/FakeRemoteTempFile.txt";
upload(hostName, username, password, localFilePath, remoteFilePath);
exist(hostName, username, password, remoteFilePath);
download(hostName, username, password, localFilePath,remoteFilePath);
move(hostName, username, password, remoteFilePath, remoteTempFilePath);
delete(hostName, username, password, remoteFilePath);
}
/**
* Method to upload a file in Remote server
*
* @param hostName
* HostName of the server
* @param username
* UserName to login
* @param password
* Password to login
* @param localFilePath
* LocalFilePath. Should contain the entire local file path -
* Directory and Filename with \\ as separator
* @param remoteFilePath
* remoteFilePath. Should contain the entire remote file path -
* Directory and Filename with / as separator
*/
public static void upload(String hostName, String username, String password, String localFilePath, String remoteFilePath) {
File file = new File(localFilePath);
if (!file.exists())
throw new RuntimeException("Error. Local file not found");
StandardFileSystemManager manager = new StandardFileSystemManager();
try {
manager.init();
// Create local file object
FileObject localFile = manager.resolveFile(file.getAbsolutePath());
// Create remote file object
FileObject remoteFile = manager.resolveFile(createConnectionString(hostName, username, password, remoteFilePath), createDefaultOptions());
/*
* use createDefaultOptions() in place of fsOptions for all default
* options - Ashok.
*/
// Copy local file to sftp server
remoteFile.copyFrom(localFile, Selectors.SELECT_SELF);
System.out.println("File upload success");
} catch (Exception e) {
throw new RuntimeException(e);
} finally {
manager.close();
}
}
public static boolean move(String hostName, String username, String password, String remoteSrcFilePath, String remoteDestFilePath){
StandardFileSystemManager manager = new StandardFileSystemManager();
try {
manager.init();
// Create remote object
FileObject remoteFile = manager.resolveFile(createConnectionString(hostName, username, password, remoteSrcFilePath), createDefaultOptions());
FileObject remoteDestFile = manager.resolveFile(createConnectionString(hostName, username, password, remoteDestFilePath), createDefaultOptions());
if (remoteFile.exists()) {
remoteFile.moveTo(remoteDestFile);;
System.out.println("Move remote file success");
return true;
}
else{
System.out.println("Source file doesn't exist");
return false;
}
} catch (Exception e) {
throw new RuntimeException(e);
} finally {
manager.close();
}
}
/**
* Method to download the file from remote server location
*
* @param hostName
* HostName of the server
* @param username
* UserName to login
* @param password
* Password to login
* @param localFilePath
* LocalFilePath. Should contain the entire local file path -
* Directory and Filename with \\ as separator
* @param remoteFilePath
* remoteFilePath. Should contain the entire remote file path -
* Directory and Filename with / as separator
*/
public static void download(String hostName, String username, String password, String localFilePath, String remoteFilePath) {
StandardFileSystemManager manager = new StandardFileSystemManager();
try {
manager.init();
// Append _downlaod_from_sftp to the given file name.
//String downloadFilePath = localFilePath.substring(0, localFilePath.lastIndexOf(".")) + "_downlaod_from_sftp" + localFilePath.substring(localFilePath.lastIndexOf("."), localFilePath.length());
// Create local file object. Change location if necessary for new downloadFilePath
FileObject localFile = manager.resolveFile(localFilePath);
// Create remote file object
FileObject remoteFile = manager.resolveFile(createConnectionString(hostName, username, password, remoteFilePath), createDefaultOptions());
// Copy local file to sftp server
localFile.copyFrom(remoteFile, Selectors.SELECT_SELF);
System.out.println("File download success");
} catch (Exception e) {
throw new RuntimeException(e);
} finally {
manager.close();
}
}
/**
* Method to delete the specified file from the remote system
*
* @param hostName
* HostName of the server
* @param username
* UserName to login
* @param password
* Password to login
* @param localFilePath
* LocalFilePath. Should contain the entire local file path -
* Directory and Filename with \\ as separator
* @param remoteFilePath
* remoteFilePath. Should contain the entire remote file path -
* Directory and Filename with / as separator
*/
public static void delete(String hostName, String username, String password, String remoteFilePath) {
StandardFileSystemManager manager = new StandardFileSystemManager();
try {
manager.init();
// Create remote object
FileObject remoteFile = manager.resolveFile(createConnectionString(hostName, username, password, remoteFilePath), createDefaultOptions());
if (remoteFile.exists()) {
remoteFile.delete();
System.out.println("Delete remote file success");
}
} catch (Exception e) {
throw new RuntimeException(e);
} finally {
manager.close();
}
}
// Check remote file is exist function:
/**
* Method to check if the remote file exists in the specified remote
* location
*
* @param hostName
* HostName of the server
* @param username
* UserName to login
* @param password
* Password to login
* @param remoteFilePath
* remoteFilePath. Should contain the entire remote file path -
* Directory and Filename with / as separator
* @return Returns if the file exists in the specified remote location
*/
public static boolean exist(String hostName, String username, String password, String remoteFilePath) {
StandardFileSystemManager manager = new StandardFileSystemManager();
try {
manager.init();
// Create remote object
FileObject remoteFile = manager.resolveFile(createConnectionString(hostName, username, password, remoteFilePath), createDefaultOptions());
System.out.println("File exist: " + remoteFile.exists());
return remoteFile.exists();
} catch (Exception e) {
throw new RuntimeException(e);
} finally {
manager.close();
}
}
/**
* Generates SFTP URL connection String
*
* @param hostName
* HostName of the server
* @param username
* UserName to login
* @param password
* Password to login
* @param remoteFilePath
* remoteFilePath. Should contain the entire remote file path -
* Directory and Filename with / as separator
* @return concatenated SFTP URL string
*/
public static String createConnectionString(String hostName, String username, String password, String remoteFilePath) {
return "sftp://" + username + ":" + password + "@" + hostName + "/" + remoteFilePath;
}
/**
* Method to setup default SFTP config
*
* @return the FileSystemOptions object containing the specified
* configuration options
* @throws FileSystemException
*/
public static FileSystemOptions createDefaultOptions() throws FileSystemException {
// Create SFTP options
FileSystemOptions opts = new FileSystemOptions();
// SSH Key checking
SftpFileSystemConfigBuilder.getInstance().setStrictHostKeyChecking(opts, "no");
/*
* Using the following line will cause VFS to choose File System's Root
* as VFS's root. If I wanted to use User's home as VFS's root then set
* 2nd method parameter to "true"
*/
// Root directory set to user home
SftpFileSystemConfigBuilder.getInstance().setUserDirIsRoot(opts, false);
// Timeout is count by Milliseconds
SftpFileSystemConfigBuilder.getInstance().setTimeout(opts, 10000);
return opts;
}
}
Commons VFS - Cannot list the contents of a directory
According to the documentation getting "directory contents" is not supported for the SFTP filesystem.
Related Topics
Looking for a CSS Parser in Java
Compound Class Names Are Not Supported Error in Webdriver
Best Way to Convert an Arraylist to a String
When Do You Use Varargs in Java
How to Avoid Installing "Unlimited Strength" Jce Policy Files When Deploying an Application
Preparedstatement With List of Parameters in a in Clause
Platform.Runlater and Task in Javafx
How to Force Max to Return All Maximum Values in a Java Stream
Split String With Dot as Delimiter
Difference Between Getattribute() and Getparameter()
Find Oracle Jdbc Driver in Maven Repository
Could Not Reserve Enough Space For Object Heap
How to Create a Linux Cluster for Running Physics Simulations in Java