Signatures differ in web3J and web3js

[kwedrowicz]

Hello, I'm using web3J version 2.3.1 and hit following problem. When I sign the same raw message (it's simply string like "TEST" for now) I get different signatures using web3js i web3j. The signature from web3js is valid (and web3j is not), I test it using solidity ecrecover function. It's worth to mention that hashes as you can see in the snippets are the same.

Java snippet:

String privateKey1 = "fba4137335dc20dc23ad3dcd9f4ad728370b09131a6a14abf6c839748700780d";
Credentials credentials = Credentials.create(privateKey1);
System.out.println("Address: " + credentials.getAddress());
byte[] data = "TEST".getBytes();
System.out.println("Sha3: " + Numeric.toHexString(Hash.sha3(data)));
System.out.println("Data: " + Numeric.toHexString(data));
Sign.SignatureData signature = Sign.signMessage(data, credentials.getEcKeyPair());
	    
System.out.println("R: " + Numeric.toHexString(signature.getR()));
System.out.println("S: " + Numeric.toHexString(signature.getS()));
System.out.println("V: " + Integer.toString(signature.getV()));

Java result:

Address: 0x7cb2e8c3b3b9dc9d02f4a3fd92402bf63b8fc1c7
Sha3: 0x852daa74cc3c31fe64542bb9b8764cfb91cc30f9acf9389071ffb44a9eefde46
Data: 0x54455354
R: 0xa5a9f39884ca95bef5bf02bea8c2c52fe2892fa25624ad0a404c70b3e0120d3b
S: 0x4e76fc757f20bed033a3132de6cabb167150632df8780a3b81728d53007f22b0
V: 28

JS snippet:

hash = web3.sha3(web3.toHex('TEST').slice(2), {encoding: 'hex'});
sig = web3.eth.sign(web3.eth.accounts[0], hash);
r = "0x"+sig.substr(2,64);
s = "0x"+sig.substr(66,64)
v = parseInt(sig.substr(130,2))+27;

JS result (testRPC 4.1.3 console):

truffle(development)> hash
'0x852daa74cc3c31fe64542bb9b8764cfb91cc30f9acf9389071ffb44a9eefde46'
truffle(development)> r
'0x62add06b2556ac5eed510a93eb699449641707a100feaea416435d90b29885e4'
truffle(development)> s
'0x5abd8808cfeb515682daeacda36159ba674cac462d74397a9c63bc4a6f1b4ffb'
truffle(development)> v
27

[conor10]

Behind the scenes, web3.js is using the personal_sign method. If you refer to the specification, you'll see that the message has additional information added to it which invalidates the above comparisons.

sign(keccack256("\x19Ethereum Signed Message:\n" + len(message) + message)))

It doesn't help that the format of this message is also encoding the length incorrectly - see the following go-ethereum issue.

An equivalent implementation to what you've done in JavaScript should be along the lines of this:

        String privateKey1 = "fba4137335dc20dc23ad3dcd9f4ad728370b09131a6a14abf6c839748700780d";
        Credentials credentials = Credentials.create(privateKey1);
        System.out.println("Address: " + credentials.getAddress());

        String hash = Hash.sha3(Numeric.toHexStringNoPrefix("TEST".getBytes()));
        System.out.println("Hash" + hash);

        String message = "\\x19Ethereum Signed Message:\n" + hash.length() + hash;

        byte[] data = message.getBytes();

        Sign.SignatureData signature = Sign.signMessage(data, credentials.getEcKeyPair());

        System.out.println("R: " + Numeric.toHexString(signature.getR()));
        System.out.println("S: " + Numeric.toHexString(signature.getS()));
        System.out.println("V: " + Integer.toString(signature.getV()));

EIP 683 is attempting to clean up this mess.

[axcrest]

In case it helps for others, I had to use this prefix string instead:

String preamble = "\u0019Ethereum Signed Message:\n" + encodedParams.length;

\\x did not work for me.

[ChocolateSlayer]

The problem was solved according this commit 4997746 in 4.0.0 release
But I still had a problem using signPrefixedMessage. My smart contract couldn't handle the signature I get from

String hash = Hash.sha3("test");
byte[] data = message.getBytes();
Sign.SignatureData signature = Sign.signPrefixedMessage(data, credentials.getEcKeyPair());

Just because it does exactly what String message = "\\x19Ethereum Signed Message:\n" + hash.length() + hash; does.
It takes length of input data and input data.
But message.getBytes(); only returns bytes of each letter in that concatenated string.
In my case I had 66 character length of my input data while eth.sign works only with 32 length byte[]
https://github.com/ethereum/go-ethereum/blob/f4c49bc0f03910f0caa16c4a067c0f201de293b7/internal/ethapi/api.go#L380

So here is the code helped me to handle web3j created signature with smart contract

String hash = Hash.sha3("test");
byte[] data = Numeric.hexStringToByteArray(hash); //returns correct input data
Sign.SignatureData signature = Sign.signPrefixedMessage(data, ecKeyPair1);

Hope it helps someone :)

[jsdavis28]

@ChocolateSlayer thanks for the information above. I hope some of this isn't repetition of questions elsewhwere, but I noticed that the Hash.sha3() function doesn't return the same value as Geth, Web3JS or literally any of the other Ethereum libraries out there... at least not following your suggestion above. I have been able to get favorable results with the following:

byte[] testMessage = "This is a test".getBytes(StandardCharsets.UTF_8);
String hash = Hash.sha3(Numeric.toHexString(testMessage));

The resulting hex string ought to be: "0x93b90fab55adf4e98787d33a38e71106e8c016f1a124dfc784f3cca4d938b1af" and with the code I've included here, you're able to do that. If I follow your example above, the hex string is not correct compared to Geth.

That aside, I've been trying to follow your instructions to sign a message and then get the hex string value of the signature to use within a Solidity contract. Again, as above, the output from Geth and Web3JS is different from what I'm getting with Web3J. Passing either testMessage to Sign.signPrefixedMessage() or a variation using Numeric.hexStringToByteArray() to create the byte[] variable does not produce a hex string that matches Geth.

Can you go a little more in-depth on your explanation above? I'm using 4.0.0, just to be clear, and I have a feeling that there's still not quite something right going on within Web3J here (or perhaps--more likely--I'm just doing something dumb).

[jsdavis28]

Ok. I think I've got it, at least for my purposes.

I was calling:

Sign.SignatureData signature = Sign.signPrefixedMessage(hash, ecKeyPair1);

and then creating a String following the same basic steps above that I used to create the hash variable:

String hexString = Numeric.toHexString(signature.toString().getBytes(StandardCharsets.UTF_8)); 

However, this only created a bunch of junk values that didn't match the signature I'm getting from Geth.

Instead, I've been able to successfully get the hex string value by doing the following:

Numeric.toHexString(signature.getR(); 
Numeric.toHexString(signature.getS(); 
Integer.toHexString(signature.getV(); 

There ought to be a method in this library that does this for me!

Here's my full code, which creates a hashed message and a signature that match the output of Geth and Web3JS:

//hexString matches web3.sha3 in Geth
byte[] hash = Hash.sha3("This is a test".getBytes(StandardCharsets.UTF_8));
String hexString = Numeric.toHexString(hash);

//Concatenated R, S, and V variables match web3.personal.sign in Geth
Sign.SignatureData signature = Sign.signPrefixedMessage(hash, ecKeyPair1);
String r = Numeric.toHexString(signature.getR(); 
String s = Numeric.toHexString(signature.getS(); 
int v = Integer.toHexString(signature.getV(); 

In order to obtain the output from web3.personal.sign in Geth, you just need to prune off the 0x on S and then concatenate R, S, and V to achieve desired results. In short, what can be accomplished in just two lines of code in Geth takes a few more here, but at least it finally works. I hope this helps anyone who has been experiencing a similar level of misery. Also, if anyone knows an easier way, I'm listening! 😉

[ashishonmobile]

is there anyone has the solution. I am using version 4.2.0-android and I am unable to recover the address from the message and R S V component.

[alexroan]

After some jiggery-pokery I've managed to get Web3J producing the exact same signatures as Web3JS like this:

        byte[] hash = SIGNATURE_MESSAGE.getBytes(StandardCharsets.UTF_8);
        Sign.SignatureData signature = Sign.signPrefixedMessage(hash, credentials.getEcKeyPair());
        String r = Numeric.toHexString(signature.getR());
        String s = Numeric.toHexString(signature.getS());
        String v = Integer.toHexString(signature.getV()-27);
        StringBuilder builder = new StringBuilder(r);
        builder.append(s.substring(2));
        if (v.length() == 1) {
            builder.append("0");
        }
        builder.append(v);
        String fullSignature = builder.toString();

[Favorlock]

Tried all the solutions above but none of them worked for me. A lot of inconsistencies with types used. I did manage to get things working such that I could take the same message and get the same signature hex back for ether.js, web3j, and MyCrypto.

    private String sign(String message, Credentials credentials) {
        byte[] hash = message.getBytes(StandardCharsets.UTF_8);
        Sign.SignatureData signature = Sign.signPrefixedMessage(hash, credentials.getEcKeyPair());
        String r = Numeric.toHexString(signature.getR());
        String s = Numeric.toHexString(signature.getS()).substring(2);
        String v = Numeric.toHexString(signature.getV()).substring(2);
        System.out.println(r + "    " + s + "    " + v);
        return new StringBuilder(r)
                .append(s)
                .append(v)
                .toString();
    }

Message: "Hello"
Signatures:

MyCrypto:
0x7f7430e57f89fcd4d2bb8ee76ccd4ddbb9449e6d8f011a3aca514ea3404747045eb4f3b8b4e068cc2925fc7d1f69ddb5351d114e9554d9b554ebab0534a5bec91b

ether.js:
0x7f7430e57f89fcd4d2bb8ee76ccd4ddbb9449e6d8f011a3aca514ea3404747045eb4f3b8b4e068cc2925fc7d1f69ddb5351d114e9554d9b554ebab0534a5bec91b

web3j:
0x7f7430e57f89fcd4d2bb8ee76ccd4ddbb9449e6d8f011a3aca514ea3404747045eb4f3b8b4e068cc2925fc7d1f69ddb5351d114e9554d9b554ebab0534a5bec91b

[fulldecent]

We now have Sign.signPrefixedMessage at https://github.com/web3j/web3j/blob/master/crypto/src/main/java/org/web3j/crypto/Sign.java#L56

[fulldecent]

For your copy/paste joy, here is an implementation of message signing in web3j that is compatible with Ethers.js and everything else, which uses byte arrays.

public byte[] sign(String message, Credentials credentials) {
  byte[] messageBytes = message.getBytes(StandardCharsets.UTF_8);
  Sign.SignatureData signature = Sign.signPrefixedMessage(message, credentials.getEcKeyPair());

  // Match the signature output format as Ethers.js v5.0.31
  // https://github.com/ethers-io/ethers.js/blob/v5.0.31/packages/bytes/src.ts/index.ts#L444-L448
  byte[] retval = new byte[65];
  System.arraycopy(signature.getR(), 0, retval, 0, 32);
  System.arraycopy(signature.getS(), 0, retval, 32, 32);
  System.arraycopy(signature.getV(), 0, retval, 64, 1);
  return retval;
}