Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JETAnalyzer: false positive errors that can be analyzed to be safe by concrete evaluation #444

Closed
Krastanov opened this issue Dec 25, 2022 · 4 comments
Labels
abstract interpretation Improvements to abstract interpretation enhancement New feature or request

Comments

@Krastanov
Copy link

On Julia nightly from today:

v = Vector{Union{UInt8,Int,Float64}}()
push!(v, 0x0)
push!(v, 1)
push!(v, 1.0)
function f(vec)
    vectail = vec[2:end]
end
rep = @report_call f(v)

reports

═════ 1 possible error found ═════
┌ @ REPL[7]:2 vec[2 : lastindex(vec)]
│┌ @ array.jl:925 copyto!(X, firstindex(X), A, first(I), lI)
││┌ @ array.jl:319 Base._copyto_impl!(dest, doffs, src, soffs, n)
│││┌ @ array.jl:327 unsafe_copyto!(dest, doffs, src, soffs, n)
││││┌ @ array.jl:281 srcp = pointer(src, soffs)
│││││┌ @ abstractarray.jl:1243 Base._memory_offset(x, i)
││││││┌ @ abstractarray.jl:1247 Base.elsize(x)
│││││││┌ @ abstractarray.jl:251 Base.elsize(Base.typeof(A))
││││││││┌ @ array.jl:213 Base.aligned_sizeof(T)
│││││││││┌ @ reflection.jl:415 al = Base.datatype_alignment(T)
││││││││││ no matching method found `datatype_alignment(::Type{Union{Float64, Int64, UInt8}})`: al = Base.datatype_alignment(T::Type{Union{Float64, Int64, UInt8}})
│││││││││└─────────────────────

Versions:

julia> versioninfo()
Julia Version 1.10.0-DEV.193
Commit 162ee48e1c3 (2022-12-25 12:36 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 8 × Intel(R) Core(TM) i7-10510U CPU @ 1.80GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-14.0.6 (ORCJIT, skylake)
  Threads: 4 on 8 virtual cores

Julia nightly installed with unreleased juliaup (patched to support nightlies)

JET version v0.6.21

Should I be reporting things like this on the JET issue tracker or on the Julia one?

@jakobnissen
Copy link
Contributor

I don't think this is an issue with JET, however, it may provide a option for improving JET. Here is a reduced MWE:

julia> using JET

julia> f(::DataType) = false
f (generic function with 1 method)

julia> g(T::Type) = Base.isbitsunion(T) ? true : f(T)
g (generic function with 1 method)

julia> @report_call g(Union{Int, UInt})
═════ 1 possible error found ═════
┌ @ REPL[3]:1 f(T)
│ no matching method found `f(::Type{Union{Int64, UInt64}})`: f(T::Type{Union{Int64, UInt64}})
└─────────────

Julia is not able to statically infer that Union{Int, UInt} is Base.isbitsunion due to a @nospecialize in the definition of the latter. Therefore, it's not able to statically infer that a Union will never hit f.

The code in your OP example is intentionally marked as @nospecialize, presumably for latency reasons.
JET could avoid this false positive if it ignored @nospecialize - but then, this might make JET latency absolutely terrible.

aviatesk added a commit to JuliaLang/julia that referenced this issue Jan 10, 2023
By allowing `hasuniquerep` to analyze `Union`-types.
Motivated by <aviatesk/JET.jl#444>.
@lgeissbauer-btig
Copy link

This seems simular

┌ @ C:\MyPkg.jl:35 result = MyPkg.DataFrame()
│┌ @ C:\.julia\packages\DataFrames\dgZn3\src\dataframe\dataframe.jl:304 DataFrames.:(var"#DataFrame#207")(pairs(NamedTuple()), #self#)
││┌ @ C:\.julia\packages\DataFrames\dgZn3\src\dataframe\dataframe.jl:306 DataFrame(Base.vect(), DataFrames.Index())
│││┌ @ C:\.julia\packages\DataFrames\dgZn3\src\dataframe\dataframe.jl:187 [quote](true, #ctor-self#, columns, colindex)
││││┌ @ C:\.julia\packages\DataFrames\dgZn3\src\dataframe\dataframe.jl:229 DataFrames.convert(Vector{AbstractVector}, columns)
│││││┌ @ array.jl:613 T(a)
││││││┌ @ array.jl:621 Base.copyto_axcheck!(Vector{AbstractVector}(Base.undef, size(x)), x)
│││││││┌ @ abstractarray.jl:1180 copyto!(dest, src)
││││││││┌ @ array.jl:339 copyto!(dest, 1, src, 1, length(src))
│││││││││┌ @ array.jl:314 Base._copyto_impl!(dest, doffs, src, soffs, n)
││││││││││┌ @ array.jl:327 unsafe_copyto!(dest, doffs, src, soffs, n)
│││││││││││┌ @ array.jl:304 Base._unsafe_copyto!(dest, doffs, src, soffs, n)
││││││││││││┌ @ array.jl:245 destp = pointer(dest, doffs)
│││││││││││││┌ @ abstractarray.jl:1243 Base._memory_offset(x, i)
││││││││││││││┌ @ abstractarray.jl:1247 Base.elsize(x)
│││││││││││││││┌ @ abstractarray.jl:251 Base.elsize(Base.typeof(A))
││││││││││││││││┌ @ array.jl:213 Base.aligned_sizeof(T)
│││││││││││││││││┌ @ reflection.jl:415 al = Base.datatype_alignment(T)
││││││││││││││││││ no matching method found `datatype_alignment(::Type{AbstractVector})`: al = Base.datatype_alignment(T::Type{AbstractVector})
│││││││││┌ @ C:\file.jl:188 requests = Base.pop!(unique_symbol_2_requests, unique_symbol, Markets._Synchronization.nothing)
││││││││││┌ @ dict.jl:620 Base._pop!(h, index)
│││││││││││┌ @ dict.jl:585 Base._delete!(h, index)
││││││││││││┌ @ dict.jl:636 Base._unsetindex!(h.keys, index)
│││││││││││││┌ @ array.jl:178 pointer(A, i)
││││││││││││││┌ @ abstractarray.jl:1243 Base._memory_offset(x, i)
│││││││││││││││┌ @ abstractarray.jl:1247 Base.elsize(x)
││││││││││││││││┌ @ abstractarray.jl:251 Base.elsize(Base.typeof(A))
│││││││││││││││││┌ @ array.jl:213 Base.aligned_sizeof(T)
││││││││││││││││││┌ @ reflection.jl:415 al = Base.datatype_alignment(T)
│││││││││││││││││││ no matching method found `datatype_alignment(::Type{SharedDataTypes.Securities.UniqueSymbol})`: al = Base.datatype_alignment(T::Type{SharedDataTypes.Securities.UniqueSymbol})

@aviatesk aviatesk changed the title JET or Julia issue with Vectors of Unions JETAnalyzer: false positive errors that can be analyzed to be safe by concrete evaluation Jan 10, 2023
@aviatesk aviatesk added enhancement New feature or request abstract interpretation Improvements to abstract interpretation labels Jan 10, 2023
@jakobnissen
Copy link
Contributor

This issue appears solved on Julia master and the latest version of JET:

julia> @report_call g(Union{Int, UInt})
No errors detected

@aviatesk
Copy link
Owner

aviatesk commented Jun 1, 2023

Yeah, #529 fixed this problem. Thanks for the reminder.

@aviatesk aviatesk closed this as completed Jun 1, 2023
aviatesk added a commit to JuliaLang/julia that referenced this issue Jan 29, 2024
By allowing `hasuniquerep` to analyze `Union`-types.
Motivated by <aviatesk/JET.jl#444>.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
abstract interpretation Improvements to abstract interpretation enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants