Fancy rules for traversing with PrimMonad

[treeowl]

This isn't ready yet, but it'll probably be easier to discuss as a PR.

[treeowl]

@andrewthad, these rules actually do seem to work, but I haven't figured out the non-PrimMonad base monad case yet. It seems like you have a better sense of mmorph and the like, so maybe you'll have some ideas.

[treeowl]

Note: if it would help, we could add another wonky parameter to this function to handle Maybe, Either, Identity, and the like. I just haven't yet been able to figure out how that part should work at all.

[andrewthad]

I've been mulling over an approach that reifies the dictionary instead of passing it around as a class constraint. I think this would permit base monads other than IO and ST, but I haven't had the chance to actually try writing it out yet.

[bgamari]

I'll admit, I'm a bit concerned that this is raising the level of abstraction a bit higher than is appropriate in primitive. Historically primitive has essentially been nothing more than a set of wrappers around GHC's primops. I think this is a very useful point in the design space since its behavior is clear, predictable, and not subject to the whims of the simplifier.

I can certainly see the value in having a set of higher-level interfaces, but I do wonder whether this might not be better explored in a new library.

[treeowl]

I think the instances we have should be as good as we can make them. Perhaps those instances should never have been added, but stripping them out would break things.

…

On Sun, Apr 29, 2018, 12:32 PM Ben Gamari ***@***.***> wrote: I'll admit, I'm a bit concerned that this is raising the level of abstraction a bit higher than is appropriate in primitive. Historically primitive has essentially been nothing more than a set of wrappers around GHC's primops. I think this is a very useful point in the design space since its behavior is clear, predictable, and not subject to the whims of the simplifier. I can certainly see the value in having a set of higher-level interfaces, but I do wonder whether this might not be better explored in a new library. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#146 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABzi_dzer6CxQXE1_-MTeSLbqlwNGRRNks5ttesUgaJpZM4Tg3n6> .

[cartazio]

I’m ok with breaking changes. But I also have to emphatically agree with Ben here. There is zero room to unpredictable rules based optimization/performance in traditional primitive. The 2-3 specialization rules that are currently in master / recently are about the maximum I’m comfortable with complexity wise for primitive. In that they’re very simple. I guess phrased differently: the harder it is to reason about the perf characteristics of code using primitive... the more all Haskell libs on top suffer. Some of the tools going on in the stack of monads traversable stuff looks pretty fiendishly clever. I guess what I mean is that first and foremost that primitive is supposed to be a lib / Data structure author friendly easy to use transparent wrapper ghc pritmives (hence the name ) Eg there’s a really clear case for adding Mvar# wrappers to the lib. Anything that complicates a see through to ghc cost model of the operations needs to be considered very carefully. Hand in hand, any type classes we add immediately infect a huge number of down stream code based if we’re not careful. This latter point being any new abstractions need to be super mature or high roi wrt impact. On Sun, Apr 29, 2018 at 12:35 PM David Feuer <notifications@github.com> wrote:

…

I think the instances we have should be as good as we can make them. Perhaps those instances should never have been added, but stripping them out would break things. On Sun, Apr 29, 2018, 12:32 PM Ben Gamari ***@***.***> wrote: > I'll admit, I'm a bit concerned that this is raising the level of > abstraction a bit higher than is appropriate in primitive. Historically > primitive has essentially been nothing more than a set of wrappers around > GHC's primops. I think this is a very useful point in the design space > since its behavior is clear, predictable, and not subject to the whims of > the simplifier. > > I can certainly see the value in having a set of higher-level interfaces, > but I do wonder whether this might not be better explored in a new library. > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > <#146 (comment)>, > or mute the thread > < https://github.com/notifications/unsubscribe-auth/ABzi_dzer6CxQXE1_-MTeSLbqlwNGRRNks5ttesUgaJpZM4Tg3n6 > > . > — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#146 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAQwheANG9ozYy5GjGefcegTM5wR6yIks5ttevWgaJpZM4Tg3n6> .

[cartazio]

Actually let me rephrase it better: I’m of the perspective that for the data structures in pritmitve that abstractions like functor and traversable should be viewed as convenience utilities. If there’s a mostly predictable constant factor / asymptotic improvement we can do where the performance pays for the the code complexity, I’m all for it. But I guess I want us to keep in mind that those operations are not going to be the mind share of most users. I would love to see some of this clever work adapted to vector though. As much clever wizardry there as you want that improves performance is golden. Primitive needs to be simple to reason about with respect to the primops that power it. On Sun, Apr 29, 2018 at 3:36 PM Carter Schonwald <carter.schonwald@gmail.com> wrote:

…

I’m ok with breaking changes. But I also have to emphatically agree with Ben here. There is zero room to unpredictable rules based optimization/performance in traditional primitive. The 2-3 specialization rules that are currently in master / recently are about the maximum I’m comfortable with complexity wise for primitive. In that they’re very simple. I guess phrased differently: the harder it is to reason about the perf characteristics of code using primitive... the more all Haskell libs on top suffer. Some of the tools going on in the stack of monads traversable stuff looks pretty fiendishly clever. I guess what I mean is that first and foremost that primitive is supposed to be a lib / Data structure author friendly easy to use transparent wrapper ghc pritmives (hence the name ) Eg there’s a really clear case for adding Mvar# wrappers to the lib. Anything that complicates a see through to ghc cost model of the operations needs to be considered very carefully. Hand in hand, any type classes we add immediately infect a huge number of down stream code based if we’re not careful. This latter point being any new abstractions need to be super mature or high roi wrt impact. On Sun, Apr 29, 2018 at 12:35 PM David Feuer ***@***.***> wrote: > I think the instances we have should be as good as we can make them. > Perhaps those instances should never have been added, but stripping them > out would break things. > > On Sun, Apr 29, 2018, 12:32 PM Ben Gamari ***@***.***> > wrote: > > > I'll admit, I'm a bit concerned that this is raising the level of > > abstraction a bit higher than is appropriate in primitive. Historically > > primitive has essentially been nothing more than a set of wrappers > around > > GHC's primops. I think this is a very useful point in the design space > > since its behavior is clear, predictable, and not subject to the whims > of > > the simplifier. > > > > I can certainly see the value in having a set of higher-level > interfaces, > > but I do wonder whether this might not be better explored in a new > library. > > > > — > > You are receiving this because you authored the thread. > > Reply to this email directly, view it on GitHub > > <#146 (comment)>, > > or mute the thread > > < > https://github.com/notifications/unsubscribe-auth/ABzi_dzer6CxQXE1_-MTeSLbqlwNGRRNks5ttesUgaJpZM4Tg3n6 > > > > . > > > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub > <#146 (comment)>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AAAQwheANG9ozYy5GjGefcegTM5wR6yIks5ttevWgaJpZM4Tg3n6> > . >

[treeowl]

@cartazio, I generally agree with your last statement. With regard to traverse, I think the best approach is likely to optimize a few good categories of use and document both what gets optimized and what to do about cases that will need to be specialized by hand. But I also think we should probably suspend a lot of the detailed discussion of yes or no until @andrewthad and I have a better sense of the bounds of what we can do well. This particular PR is pretty small and conservative in my opinion. I'd like to see if it's possible to get similarly good results for the pure cases Andrew has focused on in a similarly simple way, and to look also at compositions.

As for vector: I suspect everything will be somewhat harder, with more complicated trade-offs. We can only do things that fit sufficiently well into the fusion framework. There is surely room for improvement there (of a similar general nature), but I think that layers additional complexity on top of what we're looking at here.

[andrewthad]

I spent some time this weekend trying to explore this space further, and I cannot come up with anything easy to reason about that makes types like Maybe, Identity, and Either work as the base monad for transformer stacks.

I have similar concerns about trying to do something like this in vector. The stream fusion framework is difficult to work with, and I'm not sure if this would work in that setting.

Concerning whether or not it's good to even have the instances, I have found that having them is useful. As I have started using Array in my own libraries' data structures more and more, I've found that these instances allow GHC's powerful deriving mechanism to do a lot of work for me (especially concerning Eq, Show, Functor, and Traversable instances).

I'm content to hold off on committing to anything here until the space is better understood. In practice, I think that having IO and ST be rewritten is useful, and I think that having Maybe be rewritten is useful (I use Maybe a lot). For any (sufficiently affine) monad transformer stack on top of IO or ST, people can just use traverseArrayP and not have to rely on a rule. For Maybe, it's trickier to do manually, but it's still possible.

[andrewthad]

Thinking about this more, I think that the direction I would prefer this to take would be to just include explicitly specialized traversals for various monads. That is, just having:

• traverseArrayMaybe
• traverseArrayEither
• traverseArrayIO
• etc.

This is tedious, but at least it's really clear to the user what kind of behavior they are going to get.

[cartazio]

That allows for much more predictable perf / optimization I think.

…

On Fri, Jun 12, 2020 at 3:29 PM Andrew Martin ***@***.***> wrote: Thinking about this more, I think that the direction I would prefer this to take would be to just include explicitly specialized traversals for various monads. That is, just having: - traverseArrayMaybe - traverseArrayEither - traverseArrayIO - etc. This is tedious, but at least it's really clear to the user what kind of behavior they are going to get. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#146 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAABBQR7K3A7YQLHW2IOIF3RWJ62RANCNFSM4E4DPH5A> .