bug: Store REST API returns invalid digest

[kaichaosun]

Problem

I'm writing app using Store REST API.
Example endpoint: https://waku.whisperd.tech/store/v1/messages?contentTopics=/universal/1/community/waku,/universal/1/community/status&ascending=false

Using the returned digest somethings got 400 bad request, error is Incorrect base64 string:
Examples:

https://waku.whisperd.tech/store/v1/messages?contentTopics=/universal/1/community/waku,/universal/1/community/status&ascending=false&pubsubTopic=/waku/2/rs/1/4&digest=4eM/v8m9EGWRDjaKsa+sW41D8exdVrvwj8Uz6ORWs/Q=&senderTime=1713802526605000000&storeTime=1713802526605000000

https://waku.whisperd.tech/store/v1/messages?contentTopics=/universal/1/community/waku,/universal/1/community/status&ascending=false&pubsubTopic=/waku/2/rs/1/4&digest=jXLrLT0Md0zA/YZsVlvsH0RH9yxgrV/Ref3+J/+Ju9g=&senderTime=1713802535971246300&storeTime=1713802535971246300

It sometimes got 400 with the returned cursor, sometimes not.
And seems base64 is not friendly to browser, https://stackoverflow.com/questions/1374753/passing-base64-encoded-strings-in-url

[kaichaosun]

Related issues: https://github.com/search?q=repo%3Awaku-org%2Fnwaku+Incorrect+base64+string&type=issues

[Ivansete-status]

Very good morning @kaichaosun !
Looking at the issue more carefully I have the impression that this is the expected behaviour.

The following page is super handy to know how to properly deal with REST service ( kudos to @NagyZoltanPeter ! )

https://waku-org.github.io/waku-rest-api/#get-/store/v1/messages

From there, you will see that all the store GET params should be url-encoded

Let me know if that helps and we can close that one :)

[kaichaosun]

@Ivansete-status It seems the returned base64 encoded digest is not browser friendly, wondering how to fix such a broken link?
https://waku.whisperd.tech/store/v1/messages?contentTopics=/universal/1/community/waku,/universal/1/community/status&ascending=false&pubsubTopic=/waku/2/rs/1/4&digest=4eM/v8m9EGWRDjaKsa+sW41D8exdVrvwj8Uz6ORWs/Q=&senderTime=1713802526605000000&storeTime=1713802526605000000

[Ivansete-status]

@Ivansete-status It seems the returned base64 encoded digest is not browser friendly, wondering how to fix such a broken link? https://waku.whisperd.tech/store/v1/messages?contentTopics=/universal/1/community/waku,/universal/1/community/status&ascending=false&pubsubTopic=/waku/2/rs/1/4&digest=4eM/v8m9EGWRDjaKsa+sW41D8exdVrvwj8Uz6ORWs/Q=&senderTime=1713802526605000000&storeTime=1713802526605000000

Hey @kaichaosun !
Sorry for the late response.
How did you get that digest value?

[kaichaosun]

It's returned in json of first request, https://github.com/threeproto/ferry-chat/blob/main/src/App.tsx#L27

[gabrielmer]

Heey @kaichaosun
Can you please try to reproduce it with the following image? quay.io/wakuorg/nwaku-pr:2775-rln-v1
We might have a fix for it.

Otherwise, can you please write step by step how to reproduce the error? Preferably, also with inputs that don't give error so we can compare.

[kaichaosun]

@gabrielmer will do, thanks for the updates.

[gabrielmer]

@kaichaosun were you able to test the version with my fix?

[kaichaosun]

@gabrielmer I didn't got chance to setup nwaku for the tests, feel free to close it if there is no more work to do, will open a new one if new issue pop up.

[gabrielmer]

@gabrielmer I didn't got chance to setup nwaku for the tests, feel free to close it if there is no more work to do, will open a new one if new issue pop up.

The issue is that I didn't understand how to reproduce the bug.

I think that my PR solves it but I can't merge it if I don't know if it actually works. And at the same time, we can't close this without having fixed anything or found what's going on.

So whenever you have a chance please try it, or just provide detailed steps of how to reproduce it and I do it myself.

[kaichaosun]

The test scenario can be reproduced with local REST API store query for multipage results, check an example here https://github.com/threeproto/ferry-chat/blob/main/src/App.tsx#L130.

[gabrielmer]

The test scenario can be reproduced with local REST API store query for multipage results, check an example here https://github.com/threeproto/ferry-chat/blob/main/src/App.tsx#L130.

Got it, thank you! Could you please provide detailed step by step instructions on how to reproduce it?

I get the main idea, but still need the exact details of how you got into the issue. Which API calls, in which order and with which inputs.

[gabrielmer]

Closing this issue as we lack enough information, if reproduced please reopen detailing all the steps on how it is triggered