Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider making distinct types for pinned and unpinned arrays #405

Open
Rotaerk opened this issue Dec 23, 2023 · 6 comments
Open

Consider making distinct types for pinned and unpinned arrays #405

Rotaerk opened this issue Dec 23, 2023 · 6 comments

Comments

@Rotaerk
Copy link

Rotaerk commented Dec 23, 2023

There are operations that can't support unpinned arrays, but currently the types don't actually prevent such arrays from being provided. There are constructive operations that should be able to produce either pinned or unpinned arrays, but currently only exist for unpinned arrays, such as primArrayFromList (there's no pinnedPrimArrayFromList).

One potential option would be to make a type parameter that is a data kind, indicating Pinned or Unpinned. Then operations that apply to both can just leave that type parameter unspecified...

@andrewthad
Copy link
Collaborator

I don't see a way to do this without breaking the API, and API stability is important for this library. I've thought before about what you suggest. Something like this (ignoring element polymorphism):

data Pinnedness = Pinned | Unpinned
data ByteArray :: Pinnedness -> Type
indexByteArray :: ByteArray p -> Int -> Word8
pin :: ByteArray p -> ByteArray 'Pinned -- copies an unpinned array or aliases a pinned one, detecting with `isByteArrayPinned#`
byteArrayContents :: ByteArray 'Pinned -> Ptr Word8
newByteArray :: Int -> ST s (MutableByteArray p s)

This is a fine API, but I don't see how to introduce this without breaking a bunch of programs that already use this library.

@konsumlamm
Copy link
Contributor

Perhaps this would best fit into a separate library that provides thin wrappers over Data.Primitive.ByteArray and Data.Primitive.PrimArray with type parameters for "pinnedness". Such a library should be very easy to maintain (in fact, I'd be willing to do so).

@konsumlamm
Copy link
Contributor

Another reason against making this the default is that it disallows mixing arrays with different pinnedness, e.g.

arr = if makePinned then newPinnedByteArray 42 else newByteArray 42

This could be solved using existentials, but usually when using primitive, you care a lot about performance and existentials require additional boxing.

In most cases, you're only using unpinned arrays anyway (at least I never had the need to use pinned arrays), so this would probably be overkill.

However, I'd be interested to know if people feel like such a type would actually be useful to them.

@konsumlamm
Copy link
Contributor

I created a topic on the Haskell Discourse about this: https://discourse.haskell.org/t/distinguishing-pinned-and-unpinned-arrays-via-a-type-parameter/8443.

@konsumlamm
Copy link
Contributor

konsumlamm commented Jan 16, 2024

The Discourse discussion came to the conclusion that one can just use ByteString if one wants a byte array that's guaranteed to be pinned.

@andrewthad
Copy link
Collaborator

I think you intended to write "pinned" but wrote "unpinned" instead. The arrays from ByteString are guaranteed to be pinned.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants