Sha256Withrsa What Does It Do and in What Order

SHA256withRSA what does it do and in what order?

"SHA256withRSA" implements the PKCS#1 v1.5 padding and modular exponentiation with the formal name RSASSA-PKCS1-v1_5 after calculating the hash over the data using SHA256.

So the general order is:

hashing;
padding the hash for signature generation;
modular exponentiation using the private exponent and the modulus.

The padding used for encryption and signature generation is different, so using encryption may result in erroneous signatures.

The PKCS#1 v1.5 padding scheme has been superseded by PSS. For new protocols it is advisable to use the PSS scheme instead. For RSA a very readable public standard exists. This standard has also been used as a base for RFC 3447: Public-Key Cryptography Standards (PKCS) #1: RSA Cryptography Specifications Version 2.1 (which is basically a copy).

With regards to the padding in iOS, please check this answer by Thomas Pornin. Basically you should create the SHA-256 hash, prefix a static block of data (defined in the PKCS#1 specifications) then use SecKeyRawSign using kSecPaddingPKCS1.

For your convenience, the PKCS#1 defined block of data that needs to be prefixed in hex notation for SHA-256 (it can be bit hard to find in the standard documents, it's in the notes of section 9.2):

30 31 30 0D 06 09 60 86 48 01 65 03 04 02 01 05 00 04 20

Notes:

The above steps do not include the conversion from bytes to integer and vice versa. The result of raw RSA operations are generally converted to an unsigned big endian encoding with the same size of the modulus in bytes (which is generally the same as the key size, as the key size is already a multiple of 8). These conversions are called I2OSP and OS2IP in the RFC's.

Difference between SHA256withRSA and SHA256 then RSA

The difference

The difference between signing with "SHA256withRSA" and computing the SHA256 hash and signing it with "RSA" (= "NONEwithRSA") is foremost that in the former case the calculated SHA-256 hash value is first encapsulated in a DigestInfo structure

DigestInfo ::= SEQUENCE {
    digestAlgorithm DigestAlgorithm,
    digest OCTET STRING
}

before being padded and then encrypted while in the latter case the naked SHA256 hash value is padded and encrypted.

If they are different, is there a way to modify the method 2 so that both methods give the same output?

First and foremost you will have to encapsulate the hash value in a DigestInfo structure before signing using "NONEwithRSA".

RFC 3447 Section 9.2 helps here by stating in Note 1 that

1. For the six hash functions mentioned in Appendix B.1, the DER
   encoding T of the DigestInfo value is equal to the following:
   ...
   SHA-256: (0x)30 31 30 0d 06 09 60 86 48 01 65 03 04 02 01 05 00
                04 20 || H.

Making it work

In response to the section above the OP updated his question with the updated code. Unfortunately, though, it did not yet work for him. Thus,

The OP's code

I executed the OP's code (SignInSteps.java). As he didn't provide the private key, I used a test key of my own (demo-rsa2048.p12). The result:

GreenhandOriginal:
1B9557B6A076226FA4C26A9370A0E9E91B627F14204D427B03294EC4BFC346FDEEFB3A483B1E5A0593F26E9DE87F9202E1064F4D75B24B8FA355B23A560AF263361BB94B2339C3A01952C447CAC862AA9DCAB64B09ABAA0AD50232CDB299D1E4B5F7138F448A87ED32BFF4B5B66F35FFA08F13FD98DFCEC7114710282E463245311DA7A56CBEA958D88137A8B507D8601464535978EFE36EE37EF721260DB7112484F244409F0BD64C823ACFB13D06ABA84A9A0C5AB207E19231D6A71CC80F07FDA2A9654F0F609C2C3396D6DFFBBB10EF4C3D4B5ADFC72EACC044E81F252B699F095CFEF8630B284B1F6BD7201367BD5FDF2BB4C20BD07B9CC20B214D86C729
4B9ECA6DD47C1B230D972E7DA026165F1CE743EC96825E4C13DFE2C6437FE673A13CA622047EE7D2F7C5280198D81550A1CBD17F8E8A3C4C2D53A746FA6464AA5194FC2782527B014F017008D89BB2C80B7FA367C74FE01369986B56BCE7DC573A11ED884511F0CB12160CA5E42D488451AA8961BF5A9F71E6A5E89F19BC8EFAC26DDE989A0369667EE74372F6E558887FE2561EA926B441AB8F0FD3DEDD608A671011313372084B059CAD7E4807AC852C0873C57F216349422771C089678BAC3021D054C4427EADE70219E251617B83E68640DD7D03C3F99E47F79EB71C124F59EDEA724496A4552F2E9E1F90DDE550745E85483D823F146982C6D2008FE9AA

GreenhandUpdated:
method 1: 4B9ECA6DD47C1B230D972E7DA026165F1CE743EC96825E4C13DFE2C6437FE673A13CA622047EE7D2F7C5280198D81550A1CBD17F8E8A3C4C2D53A746FA6464AA5194FC2782527B014F017008D89BB2C80B7FA367C74FE01369986B56BCE7DC573A11ED884511F0CB12160CA5E42D488451AA8961BF5A9F71E6A5E89F19BC8EFAC26DDE989A0369667EE74372F6E558887FE2561EA926B441AB8F0FD3DEDD608A671011313372084B059CAD7E4807AC852C0873C57F216349422771C089678BAC3021D054C4427EADE70219E251617B83E68640DD7D03C3F99E47F79EB71C124F59EDEA724496A4552F2E9E1F90DDE550745E85483D823F146982C6D2008FE9AA
method 2: 4B9ECA6DD47C1B230D972E7DA026165F1CE743EC96825E4C13DFE2C6437FE673A13CA622047EE7D2F7C5280198D81550A1CBD17F8E8A3C4C2D53A746FA6464AA5194FC2782527B014F017008D89BB2C80B7FA367C74FE01369986B56BCE7DC573A11ED884511F0CB12160CA5E42D488451AA8961BF5A9F71E6A5E89F19BC8EFAC26DDE989A0369667EE74372F6E558887FE2561EA926B441AB8F0FD3DEDD608A671011313372084B059CAD7E4807AC852C0873C57F216349422771C089678BAC3021D054C4427EADE70219E251617B83E68640DD7D03C3F99E47F79EB71C124F59EDEA724496A4552F2E9E1F90DDE550745E85483D823F146982C6D2008FE9AA

Thus, in contrast to the OP's observations, signatures equal in case of the updated code.

Not assuming copy&paste errors, there still might be other differences involved.

The environment

I tested using Java 8 (1.8.0_20) with unlimited jurisdiction files added and BouncyCastle 1.52, 1.49, and 1.46 (with a small test code modification due to the BC API changes).

The OP mentioned in a comment:

The Java is JRE 8 update 66. The BouncyCastle is bcprov-jdk15on-153.jar.

Thus I updated Java, still no difference.

Then I updated BouncyCastle to 1.53. And indeed, suddenly the results differed:

GreenhandOriginal:
1B9557B6A076226FA4C26A9370A0E9E91B627F14204D427B03294EC4BFC346FDEEFB3A483B1E5A0593F26E9DE87F9202E1064F4D75B24B8FA355B23A560AF263361BB94B2339C3A01952C447CAC862AA9DCAB64B09ABAA0AD50232CDB299D1E4B5F7138F448A87ED32BFF4B5B66F35FFA08F13FD98DFCEC7114710282E463245311DA7A56CBEA958D88137A8B507D8601464535978EFE36EE37EF721260DB7112484F244409F0BD64C823ACFB13D06ABA84A9A0C5AB207E19231D6A71CC80F07FDA2A9654F0F609C2C3396D6DFFBBB10EF4C3D4B5ADFC72EACC044E81F252B699F095CFEF8630B284B1F6BD7201367BD5FDF2BB4C20BD07B9CC20B214D86C729
4B9ECA6DD47C1B230D972E7DA026165F1CE743EC96825E4C13DFE2C6437FE673A13CA622047EE7D2F7C5280198D81550A1CBD17F8E8A3C4C2D53A746FA6464AA5194FC2782527B014F017008D89BB2C80B7FA367C74FE01369986B56BCE7DC573A11ED884511F0CB12160CA5E42D488451AA8961BF5A9F71E6A5E89F19BC8EFAC26DDE989A0369667EE74372F6E558887FE2561EA926B441AB8F0FD3DEDD608A671011313372084B059CAD7E4807AC852C0873C57F216349422771C089678BAC3021D054C4427EADE70219E251617B83E68640DD7D03C3F99E47F79EB71C124F59EDEA724496A4552F2E9E1F90DDE550745E85483D823F146982C6D2008FE9AA

GreenhandUpdated:
method 1: 6BAAAC1060B6D0D56AD7D45A1BEECE82391088FF47A8D8179EFBBEB0925C4AC6C9DFC56F672E99F4A6E3C106A866B70513C25AE11B267286C584A136FBC20C4D1E7B10697352DF020BA5D67029A6EF890B2674F02C496CB1F1EBB0D4DBB580EB045DBB0FA0D7D73B418FF63F345658C6C73DA742FE260C9639C94967A928F74F61DACA03310B9986C32D83CAB8C7FC13E80612CCFC0B7E3E35BEA04EAEBDAA55FB8837B4661DC71499B4A0B1D36E1D23D9927CDB55C237D5AB2E5C088F29C6FAFAD9FE64DD4851CEC113560864E9923D485D0C6E092C8EBE82D29C312E5835B38EE9BD6B8B4BCC753EF4EE4D0977B2E781B391839E3EC31C36E5B1AA0CE90227
method 2: 4B9ECA6DD47C1B230D972E7DA026165F1CE743EC96825E4C13DFE2C6437FE673A13CA622047EE7D2F7C5280198D81550A1CBD17F8E8A3C4C2D53A746FA6464AA5194FC2782527B014F017008D89BB2C80B7FA367C74FE01369986B56BCE7DC573A11ED884511F0CB12160CA5E42D488451AA8961BF5A9F71E6A5E89F19BC8EFAC26DDE989A0369667EE74372F6E558887FE2561EA926B441AB8F0FD3DEDD608A671011313372084B059CAD7E4807AC852C0873C57F216349422771C089678BAC3021D054C4427EADE70219E251617B83E68640DD7D03C3F99E47F79EB71C124F59EDEA724496A4552F2E9E1F90DDE550745E85483D823F146982C6D2008FE9AA

Interestingly only the value for method 1 in the updated code differs. Thus, I looked at the intermediary objects in that case

[BC 1.52]
hash: 03AC674216F3E15C761EE1A5E255F067953623C8B388B4459E13F978D7C846F4
algo: 2.16.840.1.101.3.4.2.1
info: 3031300D06096086480165030402010500042003AC674216F3E15C761EE1A5E255F067953623C8B388B4459E13F978D7C846F4

[BC 1.53]
hash: 03AC674216F3E15C761EE1A5E255F067953623C8B388B4459E13F978D7C846F4
algo: 2.16.840.1.101.3.4.2.1
info: 302F300B0609608648016503040201042003AC674216F3E15C761EE1A5E255F067953623C8B388B4459E13F978D7C846F4

Thus, BouncyCastle 1.53 encodes the DigestInfo object differently! And the encoding in 1.52 (and below) is the one expected by the RFC 3447 Section 9.2.

Looking at the ASN.1 dumps one sees that BC 1.52 encodes the AlgorithmIdentifier as

 2  13:   SEQUENCE {
   <06 09>
 4   9:     OBJECT IDENTIFIER sha-256 (2 16 840 1 101 3 4 2 1)
      :       (NIST Algorithm)
   <05 00>
15   0:     NULL
      :     }

while BC 1.53 creates

 2  11:   SEQUENCE {
   <06 09>
 4   9:     OBJECT IDENTIFIER sha-256 (2 16 840 1 101 3 4 2 1)
      :       (NIST Algorithm)
      :     }

So in 1.53 the algorithm parameters are missing altogether. This suggests changing the line

AlgorithmIdentifier sha256Aid = new AlgorithmIdentifier(NISTObjectIdentifiers.id_sha256, null);

AlgorithmIdentifier sha256Aid = new AlgorithmIdentifier(NISTObjectIdentifiers.id_sha256, DERNull.INSTANCE);

and suddenly it works with BouncyCastle 1.53, too, the values for method 1 and method 2 coincide! ;)

TL;DR

Don't use null as the SHA-256 parameters when instantiating the AlgorithmIdentifier, use DERNull.INSTANCE instead.

How did I...

In a comment the OP indicated that he'd like to know more about

how do you inspect the intermediate object of BouncyCastle and
how do you produce the ASN.1 dumps.

So...

... inspect the intermediate object

Quite simple. First I split up the line

rsaSignature.update(di.toASN1Primitive().getEncoded());

in the updated code as

byte[] encodedDigestInfo = di.toASN1Primitive().getEncoded();
rsaSignature.update(encodedDigestInfo);

and then added console outputs

System.out.println("    hash: " + bytesToHex(outputDigest));
System.out.println("    algo: " + sha256Aid.getAlgorithm());
System.out.println("    info: " + bytesToHex(encodedDigestInfo));

Finally I executed the code with the different BouncyCastle versions.

... produce the ASN.1 dumps

There is a well-known utility called dumpasn1 by Peter Gutmann which has become the kernel of many command line and GUI tools for creating and displaying ASN.1 dumps. I currently happen to use GUIdumpASN-ng.

In the case at hand I saved the contents of the byte[] encodedDigestInfo to a file (which can be done using e.g. Files.write) and opened these files in GUIdumpASN-ng.

Create SHA256withRSA in two steps

SHA256withRSA and NoneWithRSA use PKCS#1 v1.5 padding, more precisely RSASSA-PKCS1-v1_5. This is a deterministic padding, i.e. repeated signing with the same data will produce the same signature. Details can be found in RFC8017, 8.2.

This padding applies the DER encoding of the DigestInfo that results for SHA256 when the byte sequence (0x)30 31 30 0d 06 09 60 86 48 01 65 03 04 02 01 05 00 04 20 is prepended to the hash.

So your manual code must be modified e.g. as follows:

public static byte[] sign(String algorithm, PrivateKey privateKey, byte[] hashBytes) throws Exception {

    byte[] id = new byte[] { 0x30, 0x31, 0x30, 0x0d, 0x06, 0x09, 0x60, (byte) 0x86, 0x48, 0x01, 0x65, 0x03, 0x04, 0x02, 0x01, 0x05, 0x00, 0x04, 0x20 };
    byte[] derDigestInfo = new byte[id.length + hashBytes.length];
    System.arraycopy(id, 0, derDigestInfo, 0, id.length);
    System.arraycopy(hashBytes, 0, derDigestInfo, id.length, hashBytes.length);

    // SIGN HASH
    Signature signature = Signature.getInstance(algorithm);
    signature.initSign(privateKey);
    signature.update(derDigestInfo);
    byte[] signatureBytes = signature.sign();

    ...

With this change, both sign() methods return the same result.

Btw, java.util.Base64.Encoder.encodeToString() directly generates a Base64 encoded string.

In addition, when encoding/decoding, a charset should always be specified, e.g. dataString.getBytes(StandardCharsets.UTF_8) or new String(..., StandardCharsets.UTF_8). Otherwise the platform's default charset is used, which may differ from the intended one.

How to do signature using SHA256withRSA algorithm in C#

Do not use RSACryptoServiceProvider unless you are doing interop with CAPI, like opening a named key.

To do RSA signing with SHA-(2-)256:

byte[] signature = rsa.SignData(data, HashAlgorithmName.SHA256, RSASignaturePadding.Pkcs1);

You can get the RSA object from a cert:

using (RSA rsa = cert.GetRSAPrivateKey())
{
    ...
}

Or you can make one up from existing RSA parameters:

using (RSA rsa = RSA.Create(rsaParameters))
{
   ...
}

Or you can make up a new key from nothing:

using (RSA rsa = RSA.Create(2048))
{
    // you’ll need to save the parameters somewhere to have a stable key
    rsaParameters = rsa.ExportParameters(true));
    ...
}

Is it possible to verify a SHA256withRSA signature with a SHA256 hash of the original data?

The short answer is YES.

The long answer has to do with an encoding that wraps the signature together with the algorithm Identifier and to archive the "verify on hash only" functionality you have to do 2 changes.

First - prepend the algorithm identifier to the signature:
As @President James K. Polk wrote you have to add some extra byte to get a correct encoding of the input to verification function. As you need
the "EMSA-PKCS1-v1_5"-Padding (described here: https://www.rfc-editor.org/rfc/rfc3447#page-41) you have to prepend some bytes that represent the
algorithm that was used to calculate the hash.

I'm a bit lazy and prepending the necessary bytes as hard coded byte array and so this version does work only on SHA-256 algorithm - if
you ever use a different hashing algorithm you need to change the prepended bytes:

String prependSha256String = "3031300D060960864801650304020105000420";
byte[] prependSha256 = hexStringToByteArray(prependSha256String);
int combinedLength = prependSha256.length + documentHash.length;
byte[] documentHashFull = new byte[combinedLength];
System.arraycopy(prependSha256, 0, documentHashFull, 0, prependSha256.length);
System.arraycopy(documentHash, 0, documentHashFull, prependSha256.length, documentHash.length);

Second - use another RSA signature scheme:
As we have done the SHA-256 part already we need a "naked" RSA-scheme called "NonewithRSA", so you need to change the instantiation like:

Signature signatureVerifyHash = Signature.getInstance("NonewithRSA");

These are the results of the two RSA signature verifications (old and "new" one):

verify the signature with the full document
sigVerified: true

verify the signature with the SHA256 of the document only
sigVerifiedHash: true

Here is the full working code:

import java.security.*;

public class MainSo2 {
    public static void main(String[] args) throws NoSuchAlgorithmException, InvalidKeyException, SignatureException {
        System.out.println("Is it possible to verify a SHA256withRSA signature with a SHA256 hash of the original data?");

        // create a rsa keypair of 2048 bit keylength
        KeyPairGenerator rsaGenerator = KeyPairGenerator.getInstance("RSA");
        SecureRandom random = new SecureRandom();
        rsaGenerator.initialize(2048, random);
        KeyPair rsaKeyPair = rsaGenerator.generateKeyPair();
        PublicKey publicKey = rsaKeyPair.getPublic();
        PrivateKey privateKey = rsaKeyPair.getPrivate();

        String document = "The quick brown fox jumps over the lazy dog";
        // sign
        Signature signature = Signature.getInstance("SHA256withRSA");
        signature.initSign(privateKey);
        signature.update(document.getBytes());
        byte[] sig = signature.sign();

        // verify with full message
        System.out.println("\nverify the signature with the full document");
        Signature signatureVerify = Signature.getInstance("SHA256withRSA");
        signatureVerify.initVerify(publicKey);
        signatureVerify.update(document.getBytes());
        boolean sigVerified =  signatureVerify.verify(sig);
        System.out.println("sigVerified: " + sigVerified);

        // verify just the sha256 hash of the document
        System.out.println("\nverify the signature with the SHA256 of the document only");
        byte[] documentHash = MessageDigest.getInstance("SHA-256").digest(document.getBytes());
        // you need to prepend some bytes: 30 31 30 0D 06 09 60 86 48 01 65 03 04 02 01 05 00 04 20
        // see https://www.rfc-editor.org/rfc/rfc3447#page-41
        // warning: this string is only for SHA-256 algorithm !!
        String prependSha256String = "3031300D060960864801650304020105000420";
        byte[] prependSha256 = hexStringToByteArray(prependSha256String);
        int combinedLength = prependSha256.length + documentHash.length;
        byte[] documentHashFull = new byte[combinedLength];
        System.arraycopy(prependSha256, 0, documentHashFull, 0, prependSha256.length);
        System.arraycopy(documentHash, 0, documentHashFull, prependSha256.length, documentHash.length);
        // lets verify
        Signature signatureVerifyHash = Signature.getInstance("NonewithRSA");
        signatureVerifyHash.initVerify(publicKey);
        // signatureVerifyHash.update(document.getBytes());
        signatureVerifyHash.update(documentHashFull);
        boolean sigVerifiedHash =  signatureVerifyHash.verify(sig);
        System.out.println("sigVerifiedHash: " + sigVerifiedHash);
    }

    public static byte[] hexStringToByteArray(String s) {
        int len = s.length();
        byte[] data = new byte[len / 2];
        for (int i = 0; i < len; i += 2) {
            data[i / 2] = (byte) ((Character.digit(s.charAt(i), 16) << 4)
                    + Character.digit(s.charAt(i + 1), 16));
        }
        return data;
    }
}

Sha256Withrsa What Does It Do and in What Order