Project status?

[merwok]

Hey @hadrien, long time no see 😎

Do you think this project still has value (convention-over-configuration scaffolding for RESTful web services on top of Pyramid), or that truly good generic lib can’t be done and each project should define its mini-framework (like the one we both know that parses swagger spec to derive request validation schema)?

[hadrien]

👋@merwok

I believe project idea is valuable. But current implementation is not solving the problem it aims to fix:
Ease writing a REST API.

One particular thing I dislike is that it forces user to write classes. It seems it sits between the user and pyramid framework by settings weird opinionated conventions.

While I was in my flight from YUL to CDG last night, I was not able to sleep and came up with another way.

Example:

import royal
from pyramid.authentication import RemoteUserAuthenticationPolicy
from pyramid.authorization import ACLAuthorizationPolicy
from pyramid.config import Configurator
from pyramid.renderers import JSON
from pyramid.security import ALL_PERMISSIONS, Allow, Everyone


def main(_, **settings):
    config = Configurator(settings=settings)
    config.include(".")
    return config.make_wsgi_app()


def includeme(config):
    config.include("royal")

    config.set_authentication_policy(RemoteUserAuthenticationPolicy())
    config.set_authorization_policy(ACLAuthorizationPolicy())

    config.add_renderer(None, JSON())  # None = default renderer
    config.scan()


collection = royal.resource("/users")


@collection.acl
def __acl__(request):
    return [(Allow, "admin", ("get", "post"))]


@collection.get
def get_collection(request):
    return "get_collection"


@collection.post
def create_item(request):
    return "create_item"


item = royal.resource("/users/{user_id}")


@item.acl
def __acl__(request):
    user_id = request.matchdict["user_id"]
    return [(Allow, f"user:{user_id}", "get")]


@item.get
def get_item(request):
    return "get_item"

Implementation:

from dataclasses import dataclass
from typing import Callable, List, Tuple, Type, TypeVar

from pyramid.config import Configurator
from pyramid.request import Request

import venusian

_category = "Royal resources - 👑"


def includeme(config: Configurator):
    config.add_directive("add_resource", add_resource, action_wrap=True)
    config.add_directive("add_resource_acl", add_resource_acl, action_wrap=True)
    config.add_directive("add_resource_view", add_resource_view, action_wrap=True)


class _Context:
    def __init__(self, request):
        self.request = request

    def __acl__(self, request):
        raise NotImplementedError()


C = TypeVar("C", bound=_Context)


@dataclass
class Resource:
    path: str
    kwargs: dict
    context_factory: Type[C] = None

    def acl(self, acl_callable: Callable[[Request], List[Tuple[str, str, str]]]):
        def callback(context, name, action):
            config = context.config.with_package(info.module)
            config.add_resource_acl(self, acl_callable, **settings)

        info = venusian.attach(acl_callable, callback)
        settings = {"_info": info.codeinfo}

        self.context_factory = type(
            "Context",
            tuple([_Context]),
            {"__acl__": lambda self: acl_callable(self.request)},
        )

        return acl_callable

    def get(self, view: Callable = None, **kwargs):
        return view_decorator(self, "GET", view, **kwargs)

    def post(self, view: Callable = None, **kwargs):
        return view_decorator(self, "POST", view, **kwargs)

    def patch(self, view: Callable = None, **kwargs):
        return view_decorator(self, "PATCH", view, **kwargs)

    def put(self, view: Callable = None, **kwargs):
        return view_decorator(self, "PUT", view, **kwargs)

    def delete(self, view: Callable = None, **kwargs):
        return view_decorator(self, "DELETE", view, **kwargs)


def resource(path: str, **kwargs) -> Resource:
    def callback(context, name, action):
        config = context.config.with_package(info.module)
        config.add_resource(resource, **settings)

    resource = Resource(path, kwargs)
    info = venusian.attach(resource, callback)

    settings = {"_info": info.codeinfo}
    return resource


def view_decorator(
    resource: Resource, request_method: str, view: Callable = None, **kwargs
):
    wrapper = wrap_view(resource, request_method, **kwargs)
    # allows for optional decorator args
    if view is None:
        return wrapper
    else:
        return wrapper(view, depth=3)


class wrap_view:
    def __init__(self, resource: Resource, request_method: str, **kwargs):
        self.resource = resource
        self.request_method = request_method
        self.kwargs = kwargs

    def __call__(self, view: Callable, depth=1):
        def callback(context, name, action):
            config = context.config.with_package(info.module)
            config.add_resource_view(
                self.resource, self.request_method, view, self.kwargs, **settings
            )

        info = venusian.attach(view, callback, depth=depth)
        settings = {"_info": info.codeinfo}
        return view


def add_resource(config: Configurator, resource: Resource):
    def add_route():
        config.add_route(
            name=resource.path,
            pattern=resource.path,
            factory=resource.context_factory,
            **resource.kwargs,
        )

    introspectable = config.introspectable(
        category_name=_category,
        discriminator=(__name__, "resource", resource.path),
        title=resource.path,
        type_name="resource:",
    )
    config.action(
        introspectable.discriminator,
        callable=add_route,
        order=-10,
        introspectables=(introspectable,),
    )


def add_resource_acl(config, resource, acl_callable):
    introspectable = config.introspectable(
        category_name=_category,
        discriminator=(__name__, "acl", resource.path),
        title=f"{resource.path}",
        type_name="acl:",
    )
    config.action(
        introspectable.discriminator, None, order=-20, introspectables=(introspectable,)
    )


def add_resource_view(
    config: Configurator,
    resource: Resource,
    request_method: str,
    view: Callable,
    kwargs,
):
    introspectable = config.introspectable(
        category_name=_category,
        discriminator=(__name__, "view", resource.path, request_method),
        title=f"{request_method} {resource.path}",
        type_name="view:",
    )
    config.action(introspectable.discriminator, None, introspectables=(introspectable,))

    permission = request_method.lower() if resource.context_factory else None
    config.add_view(
        route_name=resource.path,
        view=view,
        request_method=request_method,
        permission=permission,
        **kwargs,
    )

I did not have internet so ... it is just a proof of concept with no more usage of traversal: hence acl have to be defined on all resource if needed.

[merwok]

Short answer: I do like the class idioms. It looks like stock Pyramid’s class views, only with different decorators than view_config. Having to learn and use another way of using decorators on loose functions looks like… something different for no developer benefit?

[hadrien]

I agree with you. Also, I am far from being motivated to work on this anyway. :-(

[merwok]

I’ve been doing a small personal project recently, using it as a playground to test Pyramid 2 (authn and authz policies are replaced by a security policy), some frontend stuff, devbuddy as command runner, mercurial as VCS with good UI, etc. I use traversal, with simple context classes that represent collections and resources (but not any base class or wrapper or decorator), and views defined in another module using view_config.

I don’t mind writing classes with __init__ and __getitem__, it maps directly to Pyramid docs; I like to keep layers, so the views call some custom methods I defined on the resources (e.g. ThingCollection.create), so that only the resources need to know how to work with the models. I can see that with other collections added over time, this may become repetitive and I would want a way to automate these links.

When I was in the train without internet, I implemented a register_resource directive with introspection and a presources command, as I was annoyed that proutes/pviews/debug toolbar were only showing me part of my URLs. (It’s always fun to see how straightforward and powerful Pyramid tools are once you have the time to play with them!) But now I’m not satisfied with the duplication in my code (the URL paths and parent-child relationships are in the classes and in the register_resource calls), so that’s one more reason for abstraction / automation.

So I think that this project would be super useful if it let people follow Pyramid ways (classes to traverse, view_config) but automated the boring parts (__init__ methods, setting __parent__ and __name__, having default views) and added value with stuff like presources. I think I’ll try to use pyramid-royal in a branch of my project to see how far it helps me remove boilerplate, without removing the options that I need, and see if that gives me ideas for what’s missing in this project.