encoding.b64decode/encode fails on binary data

[rtolar]

The base64encode and decode functions do not correctly handle binary input.

I have a base64-encoded string that was generated elsewhere.
When I use encoding.b64decode(), and feed that into a crypto.hmac() call, I get an incorrect value.
I believe the problem to be the encoding.b64decode/encode functions.

See below for a test case that illustrates the decoding issue.

Environment

• k6 version:
  $ k6 version
  k6 v0.29.0 ((devel), go1.15.4, darwin/amd64)

• OS and version:
  MacOS Catalina 10.15.7

• Docker version and image, if applicable:
  n/a

Expected Behavior

Given a base64-encoded string, use the decode function to get the original data, and then encode() it again.
The end result should be the original string.

Actual Behavior

Round-trip encoding fails.

Steps to Reproduce the Problem

Possibly related to: #1770 ?

I created a github project that recreates this issue.
See github project : https://github.com/rtolar/2021-01-13-k6-bug-b64decode

The k6 script shows the roundtrip encoding/decode failure, and a node.js script which does the same thing using the crypto-js library.

Here's a copy of the k6 code.

import encoding from "k6/encoding";

// Test simple encode sequence
// This works
function encodeHello() {
    console.log("------encodeHello()--------------");
    const data = 'hello'; 
    let enc = encoding.b64encode(data);
    console.log("after   = " + enc);
    const expected = 'aGVsbG8='; 
    console.log("encodeHello " + (enc == expected ? "PASSED" : "FAILED"));   // PASS
}

// Test simple decode/encode roundtrip 
// This works
function decodeEncodeWorks() {
    console.log("------decodeEncodeWorks()--------------");
    const b64data = 'aGVsbG8=';  // 'hello'
    let dec = encoding.b64decode(b64data);
    let enc = encoding.b64encode(dec);
    console.log("after   = " + enc);
    console.log("encode/decode " + (enc == b64data ? "PASSED" : "FAILED"));   // PASS
}

// Test a decode/encode roundtrip, but with a binary original value
// this fails. 
function decodeEncodeFails() {
    console.log("------decodeEncodeFails()--------------");
    const b64data = 'YAmMzu2PW2vfpUj22Dli4sk8I5muWlietj/gJR46gUIuWnswfaaT6XneRmP7oS34tUokHKAyL3jalq5cw7FFeA==';
    let dec = encoding.b64decode(b64data);
    let enc = encoding.b64encode(dec);
    console.log("after   = " + enc);
    console.log("encode/decode " + (enc == b64data ? "PASSED" : "FAILED"));  // FAIL
}

export default function() {
    encodeHello();
    decodeEncodeWorks();
    decodeEncodeFails();
}
---------------------------
Runtime output:
$ make
k6 run k6-bug.js

          /\      |‾‾| /‾‾/   /‾‾/   
     /\  /  \     |  |/  /   /  /    
    /  \/    \    |     (   /   ‾‾\  
   /          \   |  |\  \ |  (‾)  | 
  / __________ \  |__| \__\ \_____/ .io

  execution: local
     script: k6-bug.js
     output: -

  scenarios: (100.00%) 1 scenario, 1 max VUs, 10m30s max duration (incl. graceful stop):
           * default: 1 iterations for each of 1 VUs (maxDuration: 10m0s, gracefulStop: 30s)

INFO[0000] ------encodeHello()--------------             source=console
INFO[0000] after   = aGVsbG8=                            source=console
INFO[0000] encodeHello PASSED                            source=console
INFO[0000] ------decodeEncodeWorks()--------------       source=console
INFO[0000] after   = aGVsbG8=                            source=console
INFO[0000] encode/decode PASSED                          source=console
INFO[0000] ------decodeEncodeFails()--------------       source=console
INFO[0000] after   = YAnvv73vv73vv73vv71ba9+lSO+/ve+/vTli77+977+9PCPvv73vv71aWO+/ve+/vT/vv70lHjrvv71CLlp7MH3vv73vv73vv71577+9RmPvv73vv70t77+977+9SiQc77+9Mi942pbvv71cw7FFeA==  source=console
INFO[0000] encode/decode FAILED                          source=console

running (00m00.0s), 0/1 VUs, 1 complete and 0 interrupted iterations
default ✓ [======================================] 1 VUs  00m00.0s/10m0s  1/1 iters, 1 per VU

    data_received........: 0 B 0 B/s
    data_sent............: 0 B 0 B/s
    iteration_duration...: avg=716.45µs min=716.45µs med=716.45µs max=716.45µs p(90)=716.45µs p(95)=716.45µs
    iterations...........: 1   33.476165/s

[mstoykov]

Hi @rtolar,
I am pretty sure you are hitting #1375 (comment) :(.

Unfortunately, this hasn't been worked on specifically as there hasn't been time and it will be a breaking change, so we will prefer if we either find a way for it to not be breaking or get it right the second time, so there isn't a third ;).

I have done some experimenting, but am hitting a strange problem where instead of == I get AA at the end and I have no idea what I am doing wrong (I did try some different calls). You can try to use it, and if that works, for you we can think of using something like it in the future API.

Also heavily related to #1020

[mstoykov]

I have found the issue (:facepalm:) in my code and it is now working as expected

[mstoykov]

I even managed to get it working directly with ArrayBuffers. This though is even more ... patched together from other sources [1][2] so it likely has more corner cases, although it also has promise so 🤷‍♂️ maybe it will be useful to someone.

We would probably need to have those fixed in k6 as that will also probably be a lot faster, but in the meantime, there is at least a way, that will also work without xk6, and so in cloud or for users who can't or don't want to use xk6.

[rtolar]

Thanks for looking at this!

It looks like the ArrayBuffer approach would also require changes to the signing functions to accept ArrayBuffers instead of uint8[]?

I tried a quick test using the Base64Binary.decodeArrayBuffer() method you provided, and it generates an error trying to pass the decoded result into a signing function.
Error: GoError: could not convert [object ArrayBuffer] to []uint8

function k6SignAndEncryptComplex() {
    console.log("------k6SignAndEncryptComplex()--------------");
    const secretb64 = 'YAmMzu2PW2vfpUj22Dli4sk8I5muWlietj/gJR46gUIuWnswfaaT6XneRmP7oS34tUokHKAyL3jalq5cw7FFeA==';

    var secret = Base64Binary.decodeArrayBuffer(secretb64)

    var data = 'hello';
    let hasher = crypto.createHMAC('sha512', secret);
    hasher.update(data);
    var result = hasher.digest('base64');

    console.log("result   = " + result);
    var expected = 'utyDLzav5JbOstYZOAuA6/oa7/JyoDI8eIFTIF3A8loogS/mtAJ33V9jY5IkMtuk0kkVxR+CqgnP+WmO7Wlt3A==';
    console.log("k6SignAndEncryptComplex " + (result == expected ? "PASSED" : "FAILED"));
}

So far, my workaround is to directly download and use the CryptoJS libraries.
e.g. https://cdnjs.com/libraries/crypto-js
or more specifically, the https://cdnjs.cloudflare.com/ajax/libs/crypto-js/4.0.0/crypto-js.min.js file directly.

Then, use the CryptoJS methods directly:

import CryptoJS from "./crypto-js.min.js";

function cryptoJSBinary() {
    console.log("------cryptoJSBinary()--------------");
    var b64 = 'YAmMzu2PW2vfpUj22Dli4sk8I5muWlietj/gJR46gUIuWnswfaaT6XneRmP7oS34tUokHKAyL3jalq5cw7FFeA==';
    var wordArray = CryptoJS.enc.Base64.parse(b64);
    
    const data = 'hello';
    var sig = CryptoJS.HmacSHA512(data,wordArray);
    var result = CryptoJS.enc.Base64.stringify(sig);
    
    console.log(result);
    var expected = 'utyDLzav5JbOstYZOAuA6/oa7/JyoDI8eIFTIF3A8loogS/mtAJ33V9jY5IkMtuk0kkVxR+CqgnP+WmO7Wlt3A==';
    console.log("test " + (result == expected ? "PASSED" : "FAILED")); //pass
}

This seems to work, and I'm getting the correct signatures I need.
However, it does print some annoying warnings:

WARN[0000] The moduleSpecifier "crypto" has no scheme but we will try to resolve it as remote module. This will be deprecated in the future and all remote modules will need to explicitly use "https" as scheme.

[mstoykov]

and feed that into a crypto.hmac() call,

I missed that part - yeah this won't work with the ArrayBuffer approach. This will need changes to the crypto library or one more xk6 extension.

I take it you didn't want to use the xk6 extension for some reason?

[rtolar]

I'm not opposed to using extensions, I'm just not familiar with xk6.
I just did a little bit of reading up on it, and it looks like it's a compile-time set of extension code that lets you build a custom version of k6 that has the extensions baked in, right?

I got it to work building the xk6 version, but the go tests themselves fail. Is that expected?

Detailed steps:

1. installed gvm on Mac, using 'brew install go' for bootstrapping, then installed gvm and go1.15, and set go1.15 as the default version of go.

$ which go
/Users/rtolar/.gvm/gos/go1.15/bin/go
$ go version
go version go1.15 darwin/amd64
$ gvm list

gvm gos (installed)

=> go1.15

2. git clone https://github.com/MStoykov/xk6-encoding.git
3. cd xk6-encoding
4. go test
   The test cases fail.

--- FAIL: TestEncodingAlgorithms (0.00s)
    --- FAIL: TestEncodingAlgorithms/Base64 (0.00s)
        --- FAIL: TestEncodingAlgorithms/Base64/DefaultEnc (0.00s)
            encoding_test.go:52: 
                	Error Trace:	encoding_test.go:52
                	Error:      	Received unexpected error:
                	            	Error: Encoding mismatch: 97,71,86,115,98,71,56,103,100,50,57,121,98,71,81,61 at <eval>:5:11(23)
                	Test:       	TestEncodingAlgorithms/Base64/DefaultEnc
        --- FAIL: TestEncodingAlgorithms/Base64/DefaultDec (0.00s)
            encoding_test.go:61: 
                	Error Trace:	encoding_test.go:61
                	Error:      	Received unexpected error:
                	            	Error: Decoding mismatch: 104,101,108,108,111,32,119,111,114,108,100 at <eval>:5:11(23)
                	Test:       	TestEncodingAlgorithms/Base64/DefaultDec
        --- FAIL: TestEncodingAlgorithms/Base64/DefaultUnicodeEnc (0.00s)
            encoding_test.go:70: 
                	Error Trace:	encoding_test.go:70
                	Error:      	Received unexpected error:
                	            	Error: Encoding mismatch: 52,52,71,84,52,52,75,84,52,52,71,114,52,52,71,104,52,52,71,118,53,76,105,87,53,53,87,77 at <eval>:5:11(24)
                	Test:       	TestEncodingAlgorithms/Base64/DefaultUnicodeEnc
        --- FAIL: TestEncodingAlgorithms/Base64/DefaultUnicodeDec (0.00s)
            encoding_test.go:79: 
                	Error Trace:	encoding_test.go:79
                	Error:      	Received unexpected error:
                	            	Error: Decoding mismatch: 227,129,147,227,130,147,227,129,171,227,129,161,227,129,175,228,184,150,231,149,140 at <eval>:5:11(23)
                	Test:       	TestEncodingAlgorithms/Base64/DefaultUnicodeDec
        --- FAIL: TestEncodingAlgorithms/Base64/StdEnc (0.00s)
            encoding_test.go:88: 
                	Error Trace:	encoding_test.go:88
                	Error:      	Received unexpected error:
                	            	Error: Encoding mismatch: 97,71,86,115,98,71,56,103,100,50,57,121,98,71,81,61 at <eval>:5:11(24)
                	Test:       	TestEncodingAlgorithms/Base64/StdEnc
        --- FAIL: TestEncodingAlgorithms/Base64/StdDec (0.00s)
            encoding_test.go:97: 
                	Error Trace:	encoding_test.go:97
                	Error:      	Received unexpected error:
                	            	Error: Decoding mismatch: 104,101,108,108,111,32,119,111,114,108,100 at <eval>:5:11(24)
                	Test:       	TestEncodingAlgorithms/Base64/StdDec
        --- FAIL: TestEncodingAlgorithms/Base64/RawStdEnc (0.00s)
            encoding_test.go:106: 
                	Error Trace:	encoding_test.go:106
                	Error:      	Received unexpected error:
                	            	Error: Encoding mismatch: 97,71,86,115,98,71,56,103,100,50,57,121,98,71,81 at <eval>:5:11(24)
                	Test:       	TestEncodingAlgorithms/Base64/RawStdEnc
        --- FAIL: TestEncodingAlgorithms/Base64/RawStdDec (0.00s)
            encoding_test.go:115: 
                	Error Trace:	encoding_test.go:115
                	Error:      	Received unexpected error:
                	            	Error: Decoding mismatch: 104,101,108,108,111,32,119,111,114,108,100 at <eval>:5:11(24)
                	Test:       	TestEncodingAlgorithms/Base64/RawStdDec
        --- FAIL: TestEncodingAlgorithms/Base64/URLEnc (0.00s)
            encoding_test.go:124: 
                	Error Trace:	encoding_test.go:124
                	Error:      	Received unexpected error:
                	            	Error: Encoding mismatch: 53,98,67,80,54,97,79,56,53,98,121,45,76,105,52,61 at <eval>:5:11(24)
                	Test:       	TestEncodingAlgorithms/Base64/URLEnc
        --- FAIL: TestEncodingAlgorithms/Base64/URLDec (0.00s)
            encoding_test.go:133: 
                	Error Trace:	encoding_test.go:133
                	Error:      	Received unexpected error:
                	            	Error: Decoding mismatch: 229,176,143,233,163,188,229,188,190,46,46 at <eval>:5:11(24)
                	Test:       	TestEncodingAlgorithms/Base64/URLDec
        --- FAIL: TestEncodingAlgorithms/Base64/RawURLEnc (0.00s)
            encoding_test.go:142: 
                	Error Trace:	encoding_test.go:142
                	Error:      	Received unexpected error:
                	            	Error: Encoding mismatch: 53,98,67,80,54,97,79,56,53,98,121,45,76,105,52 at <eval>:5:11(24)
                	Test:       	TestEncodingAlgorithms/Base64/RawURLEnc
        --- FAIL: TestEncodingAlgorithms/Base64/RawURLDec (0.00s)
            encoding_test.go:151: 
                	Error Trace:	encoding_test.go:151
                	Error:      	Received unexpected error:
                	            	Error: Decoding mismatch: 229,176,143,233,163,188,229,188,190,46,46 at <eval>:5:11(24)
                	Test:       	TestEncodingAlgorithms/Base64/RawURLDec
FAIL
exit status 1
FAIL	github.com/mstoykov/xk6-encoding	0.447s

5. I expected the test cases to pass. But maybe I'm running them incorrectly?
   Then, in a separate directory, I built a custom version of k6, using:
   go get -u github.com/k6io/xk6/cmd/xk6
   xk6 build v0.29.0 --with github.com/mstoykov/xk6-encoding

Here's the local k6 build:

$ ll
total 59104
drwxr-xr-x  4 rtolar  staff       128 Jan 20 10:38 .
drwxr-xr-x  4 rtolar  staff       128 Jan 19 16:06 ..
-rwxr-xr-x  1 rtolar  staff  30254368 Jan 20 09:54 k6
-rw-r--r--  1 rtolar  staff      1584 Jan 20 10:38 k6-bug.js
$ ./k6 version
k6 v0.29.0 ((devel), go1.15, darwin/amd64)
$ ./k6 run k6-bug.js 

          /\      |‾‾| /‾‾/   /‾‾/   
     /\  /  \     |  |/  /   /  /    
    /  \/    \    |     (   /   ‾‾\  
   /          \   |  |\  \ |  (‾)  | 
  / __________ \  |__| \__\ \_____/ .io

  execution: local
     script: k6-bug.js
     output: -

  scenarios: (100.00%) 1 scenario, 1 max VUs, 10m30s max duration (incl. graceful stop):
           * default: 1 iterations for each of 1 VUs (maxDuration: 10m0s, gracefulStop: 30s)

INFO[0000] ------encodeHello()--------------             source=console
INFO[0000] after   = aGVsbG8=                            source=console
INFO[0000] encodeHello PASSED                            source=console
INFO[0000] ------decodeEncodeWorks()--------------       source=console
INFO[0000] after   = aGVsbG8=                            source=console
INFO[0000] encode/decode PASSED                          source=console
INFO[0000] ------decodeEncodeFails()--------------       source=console
INFO[0000] before   = YAmMzu2PW2vfpUj22Dli4sk8I5muWlietj/gJR46gUIuWnswfaaT6XneRmP7oS34tUokHKAyL3jalq5cw7FFeA==  source=console
INFO[0000] after   = YAmMzu2PW2vfpUj22Dli4sk8I5muWlietj/gJR46gUIuWnswfaaT6XneRmP7oS34tUokHKAyL3jalq5cw7FFeA==  source=console
INFO[0000] encode/decode PASSED                          source=console

running (00m00.0s), 0/1 VUs, 1 complete and 0 interrupted iterations
default ✓ [======================================] 1 VUs  00m00.0s/10m0s  1/1 iters, 1 per VU

    data_received........: 0 B 0 B/s
    data_sent............: 0 B 0 B/s
    iteration_duration...: avg=319.39µs min=319.39µs med=319.39µs max=319.39µs p(90)=319.39µs p(95)=319.39µs
    iterations...........: 1   41.663195/s

$ 

[mstoykov]

Hi @rtolar , yes you are correct in what xk6 does.

And unfortunately, I did not have time to try and make the test work for xk6-encoding, as that is just a PoC ;). The tests are just a copy of the same tests that are in the k6 codebase currently. So yes ... it is expected for the tests to fail, although not desirable :). I am taking PRs though ;)

[juwatanabe]

Any update on when this might be fixed?

[mstoykov]

@juwatanabe this has been fixed, by #1800 and #1841 it was just not closed :(