msgpack filtering and display broken

[skarrok]

After updating to version v2.4.0 / 2024-01-23 I noticed that display and filtering of msgpack encoded messages and headers is broken.

Using git bisect I found commit 89f4152 that introduced this issue.

What it looked like before commit 89f4152:

Value:

Headers:

And before I could use filter like headers["X-Correlation-ID"] == "771b4325b16b4128b6cdba71ed590c13" and it worked.

How it looks now:

Value:

Headers:

And filtering by header values doesn't work.

Relevant section of config file

kafka:
  brokers: ["localhost:9092"]
  messagePack:
    enabled: true

[weeco]

Thanks for the report @skarrok , could you provide some messagepack examples with the expected JSON equivalent that we can use in our unit tests that decode the binary payload to JSON?

[skarrok]

Sure. I can use https://github.com/ludocode/msgpack-tools to convert json <-> msgpack.

For example:

$ echo '{"binary": "base64:AAAAAAo=", "num": -2.4, "negative": false, "positive": true, "array": [3, 2, "1", 0]}' \
  | json2msgpack -b > example.msgpack

$ msgpack2json -bp < example.msgpack
{
    "binary": "base64:AAAAAAo=",
    "num": -2.4,
    "negative": false,
    "positive": true,
    "array": [
        3,
        2,
        "1",
        0
    ]
}

example.json.txt
example.msgpack.txt

Please note I don't really know possible edge cases but msgpack have some differences with json where strict conversion is impossible.
Also I am not sure if base64: (for binary data) prefix needed for Go msgpack library.

redpanda-console version back from December had no problem displaying binary data as (I assume) base64 string.