Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

similar and traits #18161

Open
StefanKarpinski opened this issue Aug 20, 2016 · 20 comments
Open

similar and traits #18161

StefanKarpinski opened this issue Aug 20, 2016 · 20 comments
Labels
arrays [a, r, r, a, y, s] design Design of APIs or of the language itself speculative Whether the change will be implemented is speculative

Comments

@StefanKarpinski
Copy link
Member

The situation for the similar function feels a little messy these days (perhaps I'm wrong about that and it's all under control, in which case, consider this a documentation request). It seems like the similar API could be clarified with appropriate traits, e.g.

similar(a, Dense) # same dimensions and element type as `a` but dense
similar(a, Sparse) # same dimensions and element type as `a` but sparse
similar(a, Mutable) # same dimensions and element type as `a` but fully mutable
similar(a, ElType{T}) # same dimensions and storage type as `a` but with element type `T`
similar(a, Dimensions{...}) # same storage and element type as `a` but with different dimensions

This is sort of already what we're doing but the current system is a bit more ad hoc: type arguments are treated as an implicit element type trait and a tuple of integers is treated collectively as a dimensions trait. It might be fruitful to have a more explicit trait-based core API and translate the various shorthands into that.

@StefanKarpinski StefanKarpinski added design Design of APIs or of the language itself speculative Whether the change will be implemented is speculative labels Aug 20, 2016
@StefanKarpinski StefanKarpinski added this to the 1.0 milestone Aug 20, 2016
@andyferris
Copy link
Member

This seems good! While we are thinking about this I would also love to see some solutions for arrays of fixed size (the compiler might need to know the sizes as constants, e.g. as Val{}s or preferably just using constant propagation), and also some way of dealing with immutable arrays (e.g. have similar return Ref of an array? I'm not sure how to have an API that works for both immutable and mutable arrays otherwise).

@lstagner
Copy link
Contributor

Also it would be good to start thinking about adding something like similar_type.

Having something like this would be very helpful for some packages. (JuliaDiff/ForwardDiff.jl#143, SimonDanisch/FixedSizeArrays.jl#118)

In practice similar_type's interface should mirror similar and similar_type(x,args...) == typeof(similar(x,args...)).

@andyferris
Copy link
Member

+1 for similar_type. For now we can have it just change the eltype, in Base, and extend it in StaticArrays for sizes.

I've also been thinking that more traits would be really useful for dispatch on array types. Many functions assume, e.g., mutability, while you might want separate methods for arrays of fixed size and other traits. Dimensions{...} could be a great trait for this, e.g. the dimensions of Matrix might be Dimensions{(:,:)} while for SMatrix{3,3} it could be Dimensions({3,3}). Or something.

@StefanKarpinski
Copy link
Member Author

These would all be new signatures, so this could even be added post 1.0. In the meantime, this is unlikely to undergo a massive overhaul in the next few months.

@StefanKarpinski StefanKarpinski modified the milestones: 1.x, 1.0 Aug 3, 2017
@nalimilan
Copy link
Member

How about requiring that any AbstractArray type A support similar(::Type{A{T}}, dims), and defining a fallback similar(a::A, dims) = similar(typeof(A), dims)? That would be needed to provide a promotion mechanism for concatenation (see fourth bullet at #20815), and should require very limited changes.

@Sacha0
Copy link
Member

Sacha0 commented Oct 15, 2017

Ref. #13731 (regarding similar and wrapped/structured/sparse matrices).

@Sacha0
Copy link
Member

Sacha0 commented Oct 15, 2017

Ref. #10889 (regarding traits to characterize storage).

@Sacha0
Copy link
Member

Sacha0 commented Oct 15, 2017

Ref. #23270 (regarding determining types for allocation / allocation of unwrapped storage similar to a wrapped type) and downstream #24148.

@Sacha0
Copy link
Member

Sacha0 commented Oct 15, 2017

Ref. #11573 (regarding a formal interface/contract for similar).

@Sacha0
Copy link
Member

Sacha0 commented Oct 15, 2017

Ref. #10064 (regarding a formal interface/contract AbstractArrays, which touches on storage layout and allocation).

@Sacha0
Copy link
Member

Sacha0 commented Oct 15, 2017

Ref. #12441 (regarding a formal interface/contract for convert, which touches on allocating storage for copies, generating copies with different el or container type, and copying immutables).

@Sacha0
Copy link
Member

Sacha0 commented Oct 15, 2017

Ref. #11610 (regarding the interplay between similar and immutables).

@Sacha0
Copy link
Member

Sacha0 commented Oct 15, 2017

Ref. #18772 (regarding similar and non-Base array types).

@Sacha0
Copy link
Member

Sacha0 commented Oct 15, 2017

Ref. #15198 (regarding similar and structured/wrapped matrix types).

@andyferris
Copy link
Member

andyferris commented Oct 16, 2017

Ref. #24019 (regarding similar having the same semantics for Associative as AbstractArray, e.g. currently for arrays it preserves indices while for associatives it discards indices)

@nalimilan
Copy link
Member

Another use case appeared at JuliaData/DataFrames.jl#1257: for a CategoricalArray or a PooledArray, one may want an array similar in shape, element type but also with the same levels/pool (including its custom ordering). Of course in most cases the pool shouldn't be preserved when calling similar since one could want to store different values in the newly allocated array. Not sure how this could fit in the general system.

@Sacha0
Copy link
Member

Sacha0 commented Oct 16, 2017

Ref. #22218 (regarding similar-/copy_oftype-like functionality: allocating an array of the same type as the argument array but with different eltype).

@timholy
Copy link
Member

timholy commented Dec 25, 2017

We do have some of these (e.g., #18161 (comment)), and they are documented:

julia> similar(Array{Float16}, (2, 2))
2×2 Array{Float16,2}:
 1.0e-7  0.0
 0.0     0.0

@nalimilan
Copy link
Member

Right. I suggest we document this as part of the AbstractArray interface under the "Interfaces" section of the manual, and define fallbacks for all other similar variants which use it.

@brainandforce
Copy link

I know this is an old issue, but I'd like to echo the interest in having similar_type exist in Julia Base. It is not just useful for AbstractArray types but for NTuple or other types.

Case in point: I have a package which has its own implementation of similar_type for a non-array type, but does not need to depend on StaticArraysCore (the package that is usually relied on for this functionality). It would make sense for shared functionality to be in Base, IMO.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arrays [a, r, r, a, y, s] design Design of APIs or of the language itself speculative Whether the change will be implemented is speculative
Projects
None yet
Development

No branches or pull requests

9 participants