We can do the following: * https://github.com/JuliaArrays/PaddedViews.jl/ to wrap the underlying buffer * When compiling, we peel off the padded dimensions