Derive enum #2002
Derive enum #2002
Conversation
pyo3-macros-backend/src/pyenum.rs
Outdated
| #[pymethods] | ||
| impl #enum_ { | ||
| #( | ||
| #[allow(non_upper_case_globals)] | ||
| #variant_consts | ||
| )* | ||
| } |
There was a problem hiding this comment.
This is clever, but I think the downside of this is that without multiple-pymethods feature it won't be possible for users to add their own #[pymethods] block.
You should instead use gen_py_const to create "method definitions" which then need to be fed into the collection machinery via a "methods trait" - probably PyClassDescriptors can be "abused" here (that's the one which collects all the #[pyo3(get, set)] annotations from regular #[pyclass]).
Let me know if you want me to give some more pointers - this code is rather messy and not well documented as it's in a permanent state of evolution / hackery :)
There was a problem hiding this comment.
Let me try to figure that out. Meanwhile there's a new problem: I need to implement __eq__ (via __richcmp__) for the enum to be usable, but I'll need a way to do that.
There was a problem hiding this comment.
Mmm ultimately you'll need to generate the code to create a PyType_Slot struct which points to the implementation of __richcmp__ you want, which then gets fed into the collection machinery as part of the for_each_proto_slot function.
There's some code in pyo3-macros-backend/src/pymethod.rs to which handles the magic methods as part of pymethods which might be useful if you need to generate a __richcmp__ method. Or alternatively maybe you're able to hard-code an "enum protocols trait" in src/class/impl_.rs which you can then add to enum classes only using the PyClassImplBuilder.
Hopefully again those are some notes to get started reading the right code - please do ask more questions as you need!
There was a problem hiding this comment.
I have an idea, how about we derive from the enum and provide __richcmp__ in the subclass? That's less complex, and we don't have to handle the situation where the user want to provide their own __richcmp__.
Equivalent code in Python:
class Enum:
@classmethod
@property
def Var1(cls):
return EnumVar1()
...
class EnumVar1(Enum):
def __eq__(self, other):
# Return True only if `other` is an `Enum` that represents Var1There was a problem hiding this comment.
Perhaps, I think if the user provides their own __richcmp__ and it overrides our default one that's fine. Let's see how this goes.
There was a problem hiding this comment.
Another thought could be to enforce singletons for each variant, this removing need for __richcmp__
There was a problem hiding this comment.
I wanted to use subclass approach because I didn't understand how PyO3 collects methods and thought if we provided __richcmp__, the user couldn't create their own. Now I've read the implementation, I know it's not the case and I think I like introducing a new trait.
I don't know how to enforce singletons.
tests/test_enum.rs
Outdated
| #[pyenum] | ||
| #[derive(Debug, PartialEq, Clone)] | ||
| pub enum MyEnum { |
There was a problem hiding this comment.
Crazy idea, does this need to be a separate #[pyenum] macro?
This still creates a Python class, so maybe this can just be a new feature for the #[pyclass] macro?
There was a problem hiding this comment.
I think that's a great idea. I'm not sure how do that interact with customizing-the-class though. Maybe we just disable extends, subclass and dict? I'll implement this PR as an extension to #[pyclass] if you are okay with it.
There was a problem hiding this comment.
Implementing as #[pyclass] sounds great to me!
Maybe we just disable extends, subclass and dict?
I think extends makes sense to disable, however subclass and dict could reasonably be things that users want to add to customize their enum? I don't think they'll add much complexity to this PR, we can see as we go and decide that later.
There was a problem hiding this comment.
I just want to be future-proof. In the future when we add ADT support, each variant will be represented by a subclass (I explained why in another comment of #417), and we want them to be the only subclasses of this enum, otherwise pattern matching will not work. (Python interpreter cannot recognize other subclasses as a variant of the enum.)
I am now okay with enabling dict, as I cannot think of a potential reason why we would wan't to disable dict in the future.
There was a problem hiding this comment.
and we want them to be the only subclasses of this enum
Ahh good point, ok yes let's disable subclass too.
There was a problem hiding this comment.
Well, I just want to stop user from writing #[pyclass(subclass)]. We still need to secretly enable that in order to write our variant subclasses. (Will not enable that in this PR)
There was a problem hiding this comment.
I see. I'm not sure it can work like that; as far as Python is concerned either it's legal to subclass this or it isn't.
Might be able to control this a bit via a metaclass? Could be overkill.
There was a problem hiding this comment.
Python's Enum also makes use of metaclass to make sure its enums are immutable. I think that's a good reason to implement an enum metaclass.
I'll try to implement that after I finish __eq__.
There's no appropriate PyNativeType for metaclass instances to implement PyClass. I'm not familiar enough with PyO3 to change that.
There was a problem hiding this comment.
I'm sorry, for some reasons I thought PyClass::BaseNativeType must be a class created by the metaclass ... I found I was wrong after reading create_type_object and a bunch of CPython code... 😅😅
So currently PyO3 does not support subclass from PyType, because it doesn't implement PyClassBaseType... I wonder if we can implement that using pyobject_native_type_sized... Maybe we should open another issue for supporting metaclasses? We can discuss whether this use case is important enough there.
There was a problem hiding this comment.
Indeed there isn't yet a way to set metaclasses - #906.
Agreed, let's complete this PR without solving the metaclass issue and we can always explore that as a follow-up.
Can you elaborate a little more what you mean by this?
I guess that for these two it would make sense to look at how Python does it (maybe Python 3.10?) and aim for a similar end result for users. |
I think when the user writes enum Foo{
Bar = 1,
}, they will expect some relationship between
Hmmm.. I have an idea how to implement docstring. I'll push it when I'm done and you can see whether it's reasonable. |
Perhaps like Python |
https://docs.python.org/3/library/enum.html#enum.IntEnum That's implemented by subclassing I think I'll forbid that for now, and leave that for another PR. |
jovenlin0527
force-pushed
the
derive-enum
branch
2 times, most recently
from
fd73d79 to
215ee61
Compare
|
Hi, I did a major refactoring to implementations of |
jovenlin0527
force-pushed
the
derive-enum
branch
from
92af972 to
0ffc436
Compare
|
👍 will try to re-review in the next couple days when I get a chance! |
There was a problem hiding this comment.
👍 this is looking really solid to me, and nice refactoring work on the #[pyclass] implementation.
I'm leaning towards we tidy this PR up just a tiny bit and then merge it, because it's got a lot of changes which might conflict with other work going on.
We can create an issue to record all the pieces of follow-up work to do (such as __eq__ implementation and guide contents). Hopefully you'd be interested in carrying on with the future work on this? It's an exciting piece of new functionality, thank you for picking this up!
I'm okay with that, but don't we already have #417 and #834?
I'd love to implement rest of this feature. |
There was a problem hiding this comment.
Created #2013 for follow-up work. Post on there if you think anything is missing.
Would you mind rebasing and force-pushing this PR (and potentially squashing it into one or two commits)? The CI seems to have glitched out and so I can't merge it 😅
jovenlin0527
force-pushed
the
derive-enum
branch
2 times, most recently
from
9aaa67e to
e381815
Compare
There's no functionality since it does not generate __richcmp__. Also it only works on enums with only variants, and does not support C-like enums.
jovenlin0527
force-pushed
the
derive-enum
branch
from
e381815 to
b7419b5
Compare
|
Rebase done. Sorry there are some hiccups... |
|
Brilliant, thanks again! |
This PR is about #834. Actually it's just #1045 rebased on the current main branch.
However, due to PyO3 internal API changes, I have to rewrite how
pyenumimplements PyO3 traits, and I am not sure whether I got it right. I am sure I omitted docstrings and descriptors, but maybe there are some other traits that I need to implement, or maybe I implemented something wrong.The origin implement tries to create a subclass for each variant. This isn't compatible with current PyO3 API, because we don't have a way to convert the subclass back to the enum. To make PyO3 support this is beyond my capability, so this PR will use a simpler approach. But to support #417 and #1714 we will eventually have to do that.
This PR will only support enums with unit variants, by implement the enum as a
PyClasswith an__eq__method, and add aclassattrfor each variant.Unsolved issues:
How should we treat custom discriminant values? (Add anWill not touch it at this PR.__int__implementation?)__repr__?