Add checksum body query (md5, sha1, sha256)

[jcamiel]

It's common to provide a checksum for HTTP resources to be verified.
It could be useful to add a checksum body query in asserts section.

A possible syntax:

GET https://sample.net/toto.tar.bz2

HTTP/1.1 200
[Asserts]
checksum "md5" "4b6fb79adc54152a9e5dc6aa3222b306"

GET https://github.com/Orange-OpenSource/hurl/releases/download/0.99.14/hurl-0.99.14-x86_64-osx.tar.gz

HTTP/1.1 200
[Asserts]
checksum "sha256" equals "b63e2a825d21a3f71f6eeaa439a0b6c8b5126d85c42efc14155b7ff943107f66"

Maybe hash (vs checksum) could be a better query name :

GET https://github.com/Orange-OpenSource/hurl/releases/download/0.99.14/hurl-0.99.14-x86_64-osx.tar.gz

HTTP/1.1 200
[Asserts]
hash "sha256" equals "b63e2a825d21a3f71f6eeaa439a0b6c8b5126d85c42efc14155b7ff943107f66"

A third possible syntax: just use the hash algo type

GET https://github.com/Orange-OpenSource/hurl/releases/download/0.99.14/hurl-0.99.14-x86_64-osx.tar.gz

HTTP/1.1 200
[Asserts]
sha256 equals "b63e2a825d21a3f71f6eeaa439a0b6c8b5126d85c42efc14155b7ff943107f66"

So the possible syntax for the query:

1. checksum where can be "sha1", "sha256", "md5"
2. hash where can be "sha1", "sha256", "md5"
3. 3 new queries sha1, sha256, md5

The two first has the advantage to be flexible for future algo, the last being more static and more succinct.

Note: the return type of the hash should be a string (lowercase, only 0-9, a-f), as there is no binary type in Hurl (for the moment).
So, while this asserts is syntactically correct, it could never be true: sha256 equals "zzz"

[fabricereix]

Here are my first remarks:

I tend to prefer the third form.
The string parameter for the hash might be misleading, to make us think that is possible to use any hash.
The direct mapping query-hash makes also the error message more straightforward when the hash is not supported.

It would be interesting to distinguish clearly the string from the binary type.
We could define a binary type with the common 0x prefix notation

[Asserts]
sha256 equals 0xb63e2a825d21a3f71f6eeaa439a0b6c8b5126d85c42efc14155b7ff943107f66

[jcamiel]

Ok better, but 0xAAAA is foremost a 16 based-integer (it's more an integer representation than a binary representation)
So shouldn't we accept decimal and hexadecimal representation for integer ?

[Asserts]
jsonpath "$.books" countEquals 16
jsonpath "$.books" countEquals 0x10

Maybe a binary array type should be better (also usable with body query, but not with countEquals predicate)

[Asserts]
sha256 equals b"b63e2a825d21a3f71f6eeaa439a0b6c8b5126d85c42efc14155b7ff943107f66"

In Python:

>>> import hashlib
>>> m = hashlib.sha256()
>>> m.update(b"Nobody inspects")
>>> m.update(b" the spammish repetition")
>>> m.digest()
b'\x03\x1e\xdd}Ae\x15\x93\xc5\xfe\\\x00o\xa5u+7\xfd\xdf\xf7\xbcN\x84:\xa6\xaf\x0c\x95\x0fK\x94\x06'

or even:

>>> h = hashlib.new('ripemd160')
>>> h.update(b"Nobody inspects the spammish repetition")
>>> h.hexdigest()
'cc4a5ce1b3df48aec5d22d1f16b894a0b894eccc'

[jcamiel]

Possible formats:

[Asserts]
sha256 equals [b63e2a825d21a3f71f6eeaa439a0b6c8b5126d85c42efc14155b7ff943107f66]

[Asserts]
sha256 equals b[b63e2a825d21a3f71f6eeaa439a0b6c8b5126d85c42efc14155b7ff943107f66]

[Asserts]
sha256 equals bin,b63e2a825d21a3f71f6eeaa439a0b6c8b5126d85c42efc14155b7ff943107f66

[Asserts]
sha256 equals bin[b63e2a825d21a3f71f6eeaa439a0b6c8b5126d85c42efc14155b7ff943107f66]

[Asserts]
sha256 equals b"b63e2a825d21a3f71f6eeaa439a0b6c8b5126d85c42efc14155b7ff943107f66"

[Asserts]
sha256 equals bin(b63e2a825d21a3f71f6eeaa439a0b6c8b5126d85c42efc14155b7ff943107f66)

[Asserts]
sha256 equals 0xb63e2a825d21a3f71f6eeaa439a0b6c8b5126d85c42efc14155b7ff943107f66

[Asserts]
sha256 equals <b63e2a825d21a3f71f6eeaa439a0b6c8b5126d85c42efc14155b7ff943107f66>

[Asserts]
sha256 equals \xb63e2a825d21a3f71f6eeaa439a0b6c8b5126d85c42efc14155b7ff943107f66

https://www.postgresql.org/docs/9.0/datatype-binary.html

[Asserts]
body matches r"\d{4}"

[jcamiel]

We can also unify with body assertions

GET https://example.net

HTTP/1.1 200
base64,TG9yZW0gaXBzdW0gZG9sb3Igc2l0IGFtZXQsIGNvbnNlY3RldHVyIG
FkaXBpc2NpbmcgZWxpdC4gSW4gbWFsZXN1YWRhLCBuaXNsIHZlbCBkaWN0dW0g
aGVuZHJlcml0LCBlc3QganVzdG8gYmliZW5kdW0gbWV0dXMsIG5lYyBydXRydW
0gdG9ydG9yIG1hc3NhIGlkIG1ldHVzLiA=;

Is equivalent to:

GET https://example.net

HTTP/1.1 200
body equals base64,TG9yZW0gaXBzdW0gZG9sb3Igc2l0IGFtZXQsIGNvbnNlY3RldHVyIG
FkaXBpc2NpbmcgZWxpdC4gSW4gbWFsZXN1YWRhLCBuaXNsIHZlbCBkaWN0dW0g
aGVuZHJlcml0LCBlc3QganVzdG8gYmliZW5kdW0gbWV0dXMsIG5lYyBydXRydW
0gdG9ydG9yIG1hc3NhIGlkIG1ldHVzLiA=;

And could also be represented as:

GET https://example.net

HTTP/1.1 200
body equals hexa,abcd4584de0056dea...;

So in Hurl it's natural to have "binary array literal" :

base64,xxx;
hexa,xxx;
file,xxx;

A checksum assert becomes simply:

GET http://toto.com
HTTP/1.1 200
sha256 equals hexa,456abc09753747ecba097;

And could even be used with a file:

GET http://toto.com
HTTP/1.1 200
sha256 equals file,hash.bin;

[fabricereix]

Yes! Hurl had already literals for bytearray

base64,xxx;
file,xxx;

It makes sense indeed to extend it with hexadecimal
hex,xxx;

[fabricereix]

implemented in #203, #204, #205 and #206