Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nullable(0, true) !== Nullable{Int}() false #16923

Closed
nalimilan opened this issue Jun 14, 2016 · 9 comments
Closed

Nullable(0, true) !== Nullable{Int}() false #16923

nalimilan opened this issue Jun 14, 2016 · 9 comments
Labels
missing data Base.missing and related functionality

Comments

@nalimilan
Copy link
Member

Title says it all. It's possible to create two objects which are printed as Nullable{Int}() and are not supposed to differ in any way, and yet are actually different according to ===. This is because:

julia> dump(Nullable(0, true))
Nullable{Int64}
  isnull: Bool true
  value: Int64 0
julia> dump(Nullable{Int}())
Nullable{Int64}
  isnull: Bool true
  value: Int64 1

This is particularly annoying when writing tests, but it can also be disturbing in real uses.

For non-isbits types, a solution is to discard the value in the constructor when true, i.e.:

Nullable{T}(value::T, isnull::Bool=false) = ifelse(isnull, Nullable{T}(), Nullable(value))

But this doesn't work for isbits types, for which the field contains random values when left uninitialized. I guess there's a way to fill the value field with zeros à la reinterpret, but what's the best way to do it?

Cc: @johnmyleswhite

@JeffBezanson
Copy link
Member

Related: #9147

@johnmyleswhite
Copy link
Member

Without something like #9147, this seems hard to me. The current semantics of === seem right to me: the problem is really the variability in the bits of the uninitialized field.

@nalimilan
Copy link
Member Author

No doubt === is correct here. But isn't there a way to zero-fill that field manually until #9147 is implemented, without a too big performance hit?

@johnmyleswhite
Copy link
Member

I'm not aware of one. I wish we had a mechanism for doing that in general, but I don't see how to pull it off while preserving immutability. The only thing I can imagine is that we could provide a mechanism for asking for a pseudo-value that is equivalent to a bit pattern that's filled with zeros and has the right memory size.

@TotalVerb
Copy link
Contributor

TotalVerb commented Jun 15, 2016

Zero-filling will have to also be implemented in a good amount of C code that bypasses the default constructor. I don't know if it's a good idea.

Also, a good chunk of NullableArray's autolifted code seems optimized to avoid branches on operations. Some of these operations are broken anyway (filed JuliaStats/NullableArrays.jl#116), but it's conceivable to imagine that some could produce non-zero results.

@simonbyrne
Copy link
Contributor

Also note that this is similar to NaNs, which can also compare differently under === due to different payload values.

@nalimilan nalimilan added the missing data Base.missing and related functionality label Sep 6, 2016
@JeffBezanson
Copy link
Member

@TotalVerb Zero filling can be provided by the memory allocator, at a level of abstraction below constructors. Yes, several places in the C runtime code will have to be updated, but I don't see a problem with that.

@nalimilan
Copy link
Member Author

I'm starting to think this behavior isn't really a problem. === should only be used in very specific cases where you know implementation details. It can also be useful when testing operations to check that the branch-free code was used (as in #19034). Of course it's still annoying when you want to check both the type and missingness, but not the value, but well...

Should we close the issue then?

@johnmyleswhite
Copy link
Member

Fine by me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
missing data Base.missing and related functionality
Projects
None yet
Development

No branches or pull requests

5 participants