Skip to content

Problem with number format #889

Discussion options

You must be logged in to vote

Hi Michael,

There are two problems here. This first is that you're using a 32 bit version of Python and the number 36028797018963968 cannot be represented by a 32 bit signed integer. The second problem is Nutils not telling you this, but truncating the number to the least significant 32 bits silently, which is 0 in this particular case. The second problem is fixed in master (#890). The first problem is fixed by appending a dot, which makes it a floating point number, reducing the number to something that can be represented by a 32 bit signed integer, or switching to a 64 bit build of Python (which is the default on all platforms but Windows).

Best, Joost

Replies: 1 comment 1 reply

Comment options

You must be logged in to vote
1 reply
@MichaelLoibl
Comment options

Answer selected by MichaelLoibl
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Q&A
Labels
None yet
2 participants