Getting a File'S Md5 Checksum in Java

Getting a File's MD5 Checksum in Java

There's an input stream decorator, java.security.DigestInputStream, so that you can compute the digest while using the input stream as you normally would, instead of having to make an extra pass over the data.

MessageDigest md = MessageDigest.getInstance("MD5");
try (InputStream is = Files.newInputStream(Paths.get("file.txt"));
     DigestInputStream dis = new DigestInputStream(is, md)) 
{
  /* Read decorated stream (dis) to EOF as normal... */
}
byte[] digest = md.digest();

Getting a FTP File's MD5 Checksum in Java

You can't do that. To compute MD5 hash you need to access every single byte on the file. Hence, you need to download it first. In Java or in any other programming language for that matter.

MD5 checksum from input stream

The method digest() returns the hash as bytes. Then you tried to turn those bytes into a string directly.

What you wanted is to convert each of those bytes into two hexadecimal digits. Here is the code:

byte[] hash = complete.digest();
StringBuilder sb = new StringBuilder();
for (byte b : hash)
  sb.append(String.format("%02x", b & 0xFF));
String hexHash = sb.toString();
System.out.println(hexHash);

Calculate md5 hash of a zip file in Java program

I got that working a few weeks ago with this Article here:

http://www.javalobby.org/java/forums/t84420.html

Just to have it a stackoveflow:

public static void main(String[] args) throws NoSuchAlgorithmException, FileNotFoundException {
MessageDigest digest = MessageDigest.getInstance("MD5");
File f = new File("c:\\myfile.txt");
InputStream is = new FileInputStream(f);                
byte[] buffer = new byte[8192];
int read = 0;
try {
    while( (read = is.read(buffer)) > 0) {
        digest.update(buffer, 0, read);
    }       
    byte[] md5sum = digest.digest();
    BigInteger bigInt = new BigInteger(1, md5sum);
    String output = bigInt.toString(16);
    System.out.println("MD5: " + output);
}
catch(IOException e) {
    throw new RuntimeException("Unable to process file for MD5", e);
}
finally {
    try {
        is.close();
    }
    catch(IOException e) {
        throw new RuntimeException("Unable to close input stream for MD5 calculation", e);
    }
}       
}

Accelerating files md5 checksum in Java

It seems to me that this code will create too many threads. Each thread creation has a relatively high cost.

In addition, too many threads reading files at the same time, will result of unefficient I/O : when one thread reads a bunch of data, the system usually load a full block in cache, to fasten the upcoming access. When many threads read big blocks simultaneously, the system will discard these caches, forcing extra disks access.

A quick and easy fix will be to use a ThreadPool, limiting the number of executable threads to a fix number. The ideal number will probably be around your number of CPU cores. Your DetectorThread will have to implement Callable.

You'll face another issue if most big files are stored in a limited number of directories : a single thread will have to parse them all, sequentially. It's probably best to have one single thread recursively scan the directories, creating a Callable for each file.

How to get a file directory's MD5 checksum in Android/Java

As far as I know there isn't an efficient way to get the checksum of a directory.

Computing hash of large file to detect duplicates in storage

You can try the file size and the beginning of the file 1MB.

---

Related Topics