Remove dofhandler from CellCache

[lijas]

I find it a bit awkward that the CellCache holds the dofhandler and/or grid. Sometimes I have wanted to initialize and store a CellCache at the initialization phase of my code, when the dofhandler has not been constructed yet. I think it makes more sense to move the dh/grid to the CellIterator, and only store the current dofs,coords and nodes in the CellCache:

struct CellCache{X}
    nodes::Vector{Int}
    coords::Vector{X}
    dofs::Vector{Int}
end

mutable struct CellIterator{CC<:CellCache, IC<:IntegerCollection}
    set::IC
    cc::CC
    const flags::UpdateFlags
    const grid::Grid
    const dh::DofHandler
    cellid::Int
end

[termi-official]

I heavily rely on having access to the cellid (and grid) in the cell cache. However, I wanted to investigate the cell cache in more detail with my student soon, because we want to be able to use it on the GPU. My current suggestion is that the cell iterator has some prototype to derive the cache information from. My first sketch was essentially this

struct CellCachePrototype{...}
    flags::UpdateFlags
    grid::G
    nodes_buffer::VectorType{Int}
    coords_buffer::VectorType{X}
    dh::DH
    dofs_buffer::VectorType{Int}
end

struct CellIterator{CC<:CellCachePrototype, IC<:IntegerCollection}
    cc::CC
    set::IC
end

function Base.iterate(iterator::GridIterators, state_in...)
    it = iterate(_getset(iterator), state_in...)
    it === nothing && return nothing
    item, state_out = it
    cache = _makecache(iterator, item)
    return (cache, state_out)
end

struct CellCache{...}
    grid::G
    nodes::VectorType{Int}
    coords::VectorType{X}
    dh::DH
    dofs::VectorType{Int}
    cellid::Int
end

function _makecache(iterator, item)
   cache = CellCache(iterator.cc.grid, ..., item)
   reinit!(cache, item)
   return cache
end

[Abdelrahman912]

function Base.iterate(iterator::GridIterators, state_in...)
    it = iterate(_getset(iterator), state_in...)
    it === nothing && return nothing
    item, state_out = it
    cache = _makecache(iterator, item)
    return (cache, state_out)
end

struct CellCache{...}
    grid::G
    nodes::VectorType{Int}
    coords::VectorType{X}
    dh::DH
    dofs::VectorType{Int}
    cellid::Int
end

function _makecache(iterator, item)
   cache = CellCache(iterator.cc.grid, ..., item)
   reinit!(cache, item)
   return cache
end

Just some worries about this, this might not be ideal for GPU, I mean because here you allocate the CellCache based on an iterator which is an object that stores the current state and in order for this to work you have to iterate sequentially, and this model of work is not ideal for GPU because each kernel has its standalone state and they work simultanously (each cell maps to thread), so CellIterator is of no benefit for GPU and in that case we still want to create CellCache without the need of CellIterator, right? or there is something that I am missing .

[fredrikekre]

So you wouldn't be able to reinit!(cc::CellCache, cellid::Int)?

[termi-official]

The idea is to actually eliminate the reinit and, instead of mutating, just generate a new CellCache struct from some prototype struct.

[fredrikekre]

Right, I was responding to the suggestion in the OP.

I think it will be difficult to satisfy every need with a single struct. Perhaps we should just consider the existing CellCache to be directly coupled to the CellIterator. If you need something custom it is very easy to create your own struct and define method on it after all.

[termi-official]

Should we open an issue to document custom cellcache/iterator stuff?