Fixing #1163 : Inconsistent Behaviour in http server : Big Integers are getting converted to float before storing thus losing their precision

[deydebaditya]

Change description

When working with map[string]interface{} in Go, json.Marshal will automatically infer the data type for each element based on the interface{} type that we use.
However, in our case, json.Marshal is casting values incorrectly (e.g., casting integer values to float64), it may be due to Go’s internal handling of untyped constants and generic types within interface{}

To fix this, I've added an explicit check during unmarshalling of the JSON body in the request payload for int64 types.
I've also added a recursive unmarshalling to check for any such big integer values in nested JSON fields as well.

To ensure that json.Marshal preserves the original data types, I've enforced specific types in our map by using json.Number for numeric data types before marshaling again.

Lastly, instead of using a vanilla json.Unmarshal() at the end of the request map creation, the JSON decoder needs to be aware that it explicitly needs to use numbers, instead of using Go's default handling of the interface{} type.

Working

Request

Data set in DICE DB

Response from HTTP server

[AshwinKul28]

Hello @deydebaditya thanks a lot for the changes, look great. I have left few comments please have a look on them.

Also, @lucifercr07 I remember you were doing some efforts regarding the same issue, can you please have a look at it?

[AshwinKul28]

Since maps are already reference types, no need to send them as pointers.

[deydebaditya]

@AshwinKul28 When you pass a map to a function, Go passes a reference to the map, meaning any modifications to the contents of the map itself (adding, removing, or updating keys and values) are indeed reflected outside the function.
But, in this case, I'm trying to overwrite the entire reference itself of v in the last line of the method using deoder.Decode(&v), basically trying to store the decoded map reference in the input map, so that the caller can directly access the map without a need for reassignment. So, passing by reference makes sense here IMO.

[deydebaditya]

@AshwinKul28 If there are no further comments, let's merge this PR?

[AshwinKul28]

Hi @deydebaditya sorry got occupied with other stuff. I see one lint error which is what I was talking about earlier in this thread. Since map is a ref type no need to pass it as pointer again.

Once you make that change hope linter checks will be successful

[AshwinKul28]

Hey @deydebaditya can we work onto closing this PR? Just a one linter change is required as discussed above and we are good to go! Thanks

[deydebaditya]

Will work on it by tomorrow. Would need to change a bit if implementation in the method body as well, since I’m serializing the json bytes back into the map reference at the end. Like I said before, maps are reference types as in the values to map keys are references definitely, but the actual map object would still be passed by value in this method. Will make the relevant changes.

…

On Tue, 12 Nov 2024 at 9:17 PM, Ashwin Kulkarni ***@***.***> wrote: Hey @deydebaditya <https://github.com/deydebaditya> can we work onto closing this PR? Just a one linter change is required as discussed above and we are good to go! Thanks — Reply to this email directly, view it on GitHub <#1227 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACX7SWUKY425P3USK5XH33D2AIPHZAVCNFSM6AAAAABQ7TMNSKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINZQHA4DOMBUHE> . You are receiving this because you were mentioned.Message ID: ***@***.***>