Aes Gcm Implementation with Authentication Tag in Java

How come putting the GCM authentication tag at the end of a cipher stream require internal buffering during decryption?

The short answer is that update() can't distinguish the ciphertext from the tag. The final() function can.

The long answer:
Since Sun's specification requires the tag to be appended to the ciphertext, the tag needs to be stripped from the source buffer (ciphertext) during (or rather, prior to) decryption. However, because the ciphertext can be provided over the course of several update() calls, Sun's code does not know when to pull off the tag (in the context of update()). The last update() call does not know that it is the last update() call.

By waiting until the final() to actually do any crypto, it knows the full ciphertext + tag has been provided, and it can easily strip the tag off the end, given the tag length (which is provided in the parameter spec). It can't do crypto during the update because it would either treat some ciphertext as the tag or vice versa.

Basically, this is the drawback to simply appending the tag to the ciphertext. Most other implementations (e.g. OpenSSL) will provide the ciphertext and tag as separate outputs (final() returns the ciphertext, some other get() function returns the tag). Sun no doubt chose to do it this way in order to make GCM fit with their API (and not require special GCM-specific code from developers).

The reason encryption is more straightforward is that it has no need to modify its input (plaintext) like decryption does. It simply takes all data as plaintext. During the final, the tag is easily appended to the ciphertext output.

What @blaze said regarding protecting you from yourself is a possible rationale, but it is not true that nothing can be returned until all ciphertext is known. Only a single block of ciphertext is needed (OpenSSL, for example, will give it to you). Sun's implementation only waits because it cannot know that that first block of ciphertext is just the first block of ciphertext. For all it knows, you're encrypting less than a block (requiring padding) and providing the tag all at once. Of course, even if it did give you the plaintext incrementally, you could not be sure of authenticity until the final(). All ciphertext is required for that.

There are, of course, any number of ways Sun could have made this work. Passing and retrieving the tag through special functions, requiring the length of the ciphertext during init(), or requiring the tag to be passed through on the final() call would all work. But, like I said, they probably wanted to make the usage as close to the other Cipher implementations as possible and maintain API uniformity.

How to decrypt AES-GCM encoded message using AAD in Java

There were two issues with my solution (see @dave_thompson_085's comment)

The tag length parameter passed to GCMParameterSpec was wrong. It should be 128
The Java decrypt assumes the tag data is at the end of the ciphertext, so in my case I had to move the tag data bytes from the beginning.

AES GCM decryption bypassing authentication in JAVA

Warning: hazardous material. Only proceed if you understand the implications of handling the resulting plaintext without verifying the authentication tag first. This could lead not just to an attacker changing the message, but also compromising confidentiality through plaintext oracle attacks.

Yes, it is possible to decrypt the message without the authentication tag: if you read the GCM specification you can see that the IV for CTR is simply the IV, appended with four bytes 00000002 (i.e. a counter starting at zero, increased by one for calculating the authentication tag and again for the starting value of the counter for encryption).

So here's the code, where I do the inc twice as I used it to validate my counter code; it is of course possible to simply set the last byte to value 0x02 as well.

package nl.owlstead.so;

import java.nio.charset.StandardCharsets;

import javax.crypto.Cipher;
import javax.crypto.SecretKey;
import javax.crypto.spec.GCMParameterSpec;
import javax.crypto.spec.IvParameterSpec;
import javax.crypto.spec.SecretKeySpec;

import org.bouncycastle.util.Arrays;

public class DecryptGCMWithoutVerification {

    private static final int TAG_SIZE = 128;

    public DecryptGCMWithoutVerification() {
        // TODO Auto-generated constructor stub
    }

    public static void main(String[] args) throws Exception {
        
        // --- encryption using GCM
        
        Cipher gcm = Cipher.getInstance("AES/GCM/NoPadding");
        SecretKey key = new SecretKeySpec(new byte[16], "AES");
        byte[] ivBytes = new byte[12];
        GCMParameterSpec iv = new GCMParameterSpec(TAG_SIZE, ivBytes);
        gcm.init(Cipher.ENCRYPT_MODE, key, iv);
        byte[] ct = gcm.doFinal("owlstead".getBytes(StandardCharsets.US_ASCII));
        
        // --- decryption using underlying CTR mode
        
        Cipher ctr = Cipher.getInstance("AES/CTR/NoPadding");

        // WARNING: this is only correct for a 12 byte IV in GCM mode
        byte[] counter = Arrays.concatenate(ivBytes, new byte[4]);
        inc(counter);
        inc(counter);
        IvParameterSpec ctrIV = new IvParameterSpec(counter);
        ctr.init(Cipher.DECRYPT_MODE, key, ctrIV);
        
        byte[] pt = ctr.doFinal(ct, 0, ct.length - TAG_SIZE / Byte.SIZE);
        
        System.out.println(new String(pt, StandardCharsets.US_ASCII));
    }
    
    private static final byte inc(byte[] counter) {
        for (int i = counter.length - 1; i >= 0; i--) {
            if (++counter[i] != 0) {
                return 0;
            }
        }
        return 1;
    }

}

EDIT: this code is for an invalid tag or a tag that cannot be recalculated (the AAD could be missing, for instance). Remove - TAG_SIZE / Byte.SIZE from doFinal if the tag is missing entirely.

EDIT 2: note that this assumes a 12 byte / 96 bit IV, the default IV size for GCM. For any other size you need to calculate the IV first.

How to make GCM Encrypt with authentication tag for Android

You already have included the authentication tag. Java (unfortunately - see below) includes the authentication tag in the ciphertext. That means that it is generated during that last call to doFinal.

You can easily check this by viewing the size of the ciphertext. It should be the same size as the plaintext (in this case aKey) plus 128 bits, the default and sensible size of the authentication tag.

This is also why AEADBadTagException is derived from BadPaddingException : during decryption the verification takes place implicitly. So older code would still be compatible with GCM mode.

As I indicated earlier I think including the authentication tag in the ciphertext is a major design mistake for an API:

it doesn't allow the user to place the authentication tag elsewhere - at least without array (size) manipulation;
it removes the online nature of GCM where decryption of plaintext is instantaneous (for each call to update) as the authentication tag needs to be buffered;
it makes the code larger, less efficient and requires spurious buffering if the location of the authentication tag is known in advance (i.e. if the protocol specifies the tag location or the size of the ciphertext directly);

In my opinion that doesn't weigh up to the advantages of retrofitting the Cipher class without adding methods and maintaining compatibility. It would have been better if the designer had just added methods for retrieving and verifying the authentication tag separately.

Because it is currently raining I have created an example:

public static void main(String... args) throws Exception {
    int tagSize = 96;

    Cipher gcm = Cipher.getInstance("AES/GCM/NoPadding");

    SecretKey aesKey = new SecretKeySpec(new byte[16], "AES");

    GCMParameterSpec gcmSpec = new GCMParameterSpec(tagSize, new byte[gcm.getBlockSize()]);

    gcm.init(Cipher.ENCRYPT_MODE, aesKey, gcmSpec);

    byte[] pt = "Maarten Bodewes creates code".getBytes(StandardCharsets.UTF_8);
    System.out.println(pt.length);
    byte[] ctAndTag = new byte[gcm.getOutputSize(pt.length)];

    System.out.println(ctAndTag.length);

    int off = 0;
    off += gcm.update(pt, 0, pt.length, ctAndTag, off);
    // prints 16 (for the Oracle crypto provider)
    // meaning it is not online, buffering even during encryption
    System.out.println(off);
    off += gcm.doFinal(new byte[0], 0, 0, ctAndTag, off);
    // prints 40 for the Oracle crypto provider, meaning it doesn't *just*
    // output the tag during doFinal !
    System.out.println(off);

    int ctSize = ctAndTag.length - tagSize / Byte.SIZE;
    System.out.println(ctSize);

    byte[] ct = Arrays.copyOfRange(ctAndTag, 0, ctSize);
    byte[] tag = Arrays.copyOfRange(ctAndTag, ctSize, ctAndTag.length);

    System.out.println(Hex.toHexString(ct));
    System.out.println(Hex.toHexString(tag));
}

Notes:

it doesn't make sense to encrypt a key with itself, I hope that was just for demo purposes
GCM quickly loses security for smaller authentication tags, changing the size is not recommended, unless your protocol is explicitly designed for it.

Using Java-8 AES/GCM to authenticate / encrypt parts of a data block

You need to use one of the updateAAD methods.

In your case, something like this (note you need to make the updateAAD calls before the update or doFinal calls):

cipher.updateAAD(data, 0, 128);              // first 128 bits are authenticated
cipher.update(data, 128, 128, encrypted, 0); // next 128 are encrypted

How are the IV and authentication tag handled for AES/GCM/NoPadding?

Q1: Is the IV returned by cipher.getIV() safe for me to use in this way?

Yes, it is at least for the Oracle provided implementation. It is generated separately using the default SecureRandom implementation. As it is 12 bytes in size (the default for GCM) then you have 96 bits of randomness. The chance that the counter repeats is abysmally small. You can look up the source in the OpenJDK (GPL'ed) which the Oracle JDK is based on.

I would however still recommend you to generate your own 12 random bytes as other providers may behave differently.

Q2: Is that IV always 12 bytes long?

It's extremely likely as it is the GCM default, but other lengths are valid for GCM. The algorithm will however have to do additional calculations for any other size than 12 bytes. Due to weaknesses it is strongly recommended to keep it at 12 bytes / 96 bits and API's may restrict you to that choice of IV size.

Q3: Is the authentication tag always 16 bytes (128 bits) long?

No, it can have any size in bytes ranging from 64 bits to 128 bits with 8 bit increments. If it is smaller it simply consists of the leftmost bytes of the authentication tag though. You can specify another size of tag using GCMParameterSpec as third parameter for your init call.

Note that the strength of GCM is strongly dependent on the size of the tag. I would recommend keeping it to 128 bits. 96 bits should be the minimum especially if you want to generate a lot of ciphertext.

Q4: With #2 and #3, and the lack of padding, does that mean my encrypted messages are always 12 + src.length + 16 bytes long? (And so I can safely squish them into one byte array, for which I know the correct length?)

See above. For the Oracle provider this is the case. Use GCMParameterSpec to be sure of it.

Q5: Is it safe for me to display an unbounded number of src data encryptions to users, given constant src data that the users know?

Virtually unbound, yes. I would start worrying after about 2^48 encryptions. In general you should however design for key change.

Q6: Is it safe for me to display an unbounded number of src data encryptions to users, if the src data is different every time (e.g. including System.currentTimeMillis() or random numbers)?

See answer to Q5 & Q7

Q7: Would it help if I padded the src data with random numbers before encryption? Say 8 random bytes in front and back, or only on one end? Or would that not help at all / make my encryption worse?

No, it would not help at all. GCM uses CTR mode underneath, so it would just be encrypted with the key stream. It would not act as an IV. Nowadays you could look at AES-GCM-SIV if you have an ever changing message to encrypt, but note that that algorithm is not implemented in any of the JCA providers.

If you need a lot of ciphertexts (higher than 2^48!, or 2^32 - ~4 billion - for the cautious) then I would suggest you use that random number and your key for a key derivation function or KDF. HKDF is currently best of breed, but you may need to use Bouncy Castle or implement it yourself.