Migrating #[pyproto] to #[pymethods]

[davidhewitt]

This is a tracking issue for the work remaining to complete the support for __dunder__ (aka "protocol") methods in #[pymethods].

As per the proposal in #1864, the current intention is to have a working but experimental implementation of this in 0.15. If any feedback from this implementation is ok, I'd suggest to deprecate #[pyproto] in 0.16 or 0.17. If we're happy with the whole implementation earlier / later we can always adjust these aims.

The items outstanding after #1864

• Documentation
• Some protocols which need design discussion:
  • The GC protocol - doesn't have a well-specified __dunder__ at the Python level, so we'll need to design one
  • The buffer protocol - same thing for this
  • The sequence protocol - it overlaps messily with the mapping protocol; we might want to do the same as what happens at the Python level, or we might want to make __sequence_getitem__ methods etc to be explicit. See also Allow taking slices when implementing PySequenceProtocol __getitem__ #1855
• Suppport for multiple-pymethods feature
• tests for
  • __delete__
  • __anext__
  • __aiter__
  • __index__
  • __int__
  • __float__
  • __invert__
• (maybe) better argument extraction errors (if there's an error, the generated code won't say which argument failed, athough that's probably obvious as most these methods take at most one argument).
• __call__
• __new__ / __init__ ?
• __getitem__ iteration. #1927 ?

[birkenfeld]

How do we make progress here? Brainstorming for missing protocols?

[davidhewitt]

Yes. I have a reasonably complete plan for the sequence protocol which I just need to find time to write up.

GC and Buffer I currently have no better idea than copying the pyproto design exactly. Maybe that's fine, however both those traits are quite clunky in my opinion so I was hoping we can come up with new proposals for them while we're breaking stuff. Ideas very welcome!

At the same time it's probably fine to just copy those two traits in, and then we just might end up wanting to make breaking improvements to them eventually.

[davidhewitt]

Just as a note, I'm hoping to get an implementation for the GC protocol done soon (maybe within the next two weeks). That's the last one we need to cover before marking pyproto deprecated and releasing 0.16 🎉

[davidhewitt]

With the merge of #2173 I think we can now call this complete. There's a lot of cleanup to remove #[pyproto] completely we can do further when #[pyproto] has been deprecated for long enough for users to migrate. I'd suggest doing this two breaking releases after 0.16, i.e. 0.18.

[birkenfeld]

__new__ and __init__ were still in the checklist. Did you have any thoughts about those?

[davidhewitt]

I think there's definitely an open question whether we want to support users writing __new__ and __init__. At the moment our #[new] function is essentially a wrapper around part of the full __new__. There's no way for users to define __init__. I don't know what the design needs are exactly; one to consider for the future.

[birkenfeld]

Hm, I don't know how useful a separate __init__ would be, since there's no way to take an "empty" class and populate it with attribute, as is common in Python.

[davidhewitt]

I think there are edge cases - e.g. in inheritance situations you might want to run __init__ initialisation based on the final class instance created. I think #[new] wouldn't be able to meet that at current. Anyway, yes, I would agree this is not a pressing concern.