Skip to content
This repository has been archived by the owner on Mar 19, 2024. It is now read-only.

Fix 64-bit upper half power #8

Closed
wants to merge 1 commit into from
Closed

Conversation

deathcap
Copy link
Contributor

Bitwise operators unfortunately only operate on 32-bit integers, so x<<32 === x, and

        return (upper << 32) + lower;

is equivalent to upper + lower. For example this NBT file: https://github.com/deathcap/playerdat/blob/master/test.dat parses as:

"firstPlayed": 2016156000

This PR changes the calculation to:

        return (upper * 4294967296) + lower;

so it can use the full range of 53-bit floating point, giving the correct value:

"firstPlayed": 1367815755810

(cross checked with NBTExplorer and max-mapper/minecraft-nbt#2)

Precision loss can still occur (above 2^53 = 9007199254740992), so this doesn't fix GH-1 completely, but it takes care of some common cases.

@deathcap
Copy link
Contributor Author

The tests fail since the double is rounded to -9151314442816848000, can't precisely represent the expected value -9151314442816847872

@deathcap
Copy link
Contributor Author

Closing; better fix (with passing tests) in #9

@deathcap deathcap closed this Mar 29, 2014
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

64-bit integers overflow
1 participant