Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Users RPC transition #4026

Merged
merged 50 commits into from
Nov 19, 2024
Merged

Users RPC transition #4026

merged 50 commits into from
Nov 19, 2024

Conversation

Anish9901
Copy link
Member

@Anish9901 Anish9901 commented Nov 13, 2024

Fixes #3993
Fixes #4000
Fixes #4001
Fixes #4002
Fixes #4003
Fixes #4004
Fixes #4005
Fixes #4006

Checklist

  • My pull request has a descriptive title (not a vague title like Update index.md).
  • My pull request targets the develop branch of the repository
  • My commit messages follow best practices.
  • My code follows the established code style of the repository.
  • I added tests for the changes I made (if applicable).
  • I added or updated documentation (if applicable).
  • I tried running the project locally and verified that there are no
    visible errors.

Developer Certificate of Origin

Developer Certificate of Origin
Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129

Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.


Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
    have the right to submit it under the open source license
    indicated in the file; or

(b) The contribution is based upon previous work that, to the best
    of my knowledge, is covered under an appropriate open source
    license and I have the right under that license to submit that
    work with modifications, whether created in whole or in part
    by me, under the same open source license (unless I am
    permitted to submit under a different license), as indicated
    in the file; or

(c) The contribution was provided directly to me by some other
    person who certified (a), (b) or (c) and I have not modified
    it.

(d) I understand and agree that this project and the contribution
    are public and that a record of the contribution (including all
    personal information I submit with it, including my sign-off) is
    maintained indefinitely and may be redistributed consistent with
    this project or the open source license(s) involved.

@Anish9901
Copy link
Member Author

Anish9901 commented Nov 14, 2024

@mathemancer I am planning on adding tests and api documentation in a separate PR, since this one is getting too big.

Copy link
Contributor

@seancolsen seancolsen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ooo how fun! I love seeing a PR with some front end work too from you @Anish9901 🙂

This is mostly great, but I pushed b34300e to fix a small issue...

In WritableUsersStore, the request property was previously a CancellablePromise because that's was was returned from api.users.list(). The fact that it's a CancellablePromise is pretty much the whole reason we're storing at as a class variable here — so that we can cancel it later. But the new api.users.list() function doesn't return a CancellablePromise. You don't get that until you call run() on it. So I changed some things to make sure that request stayed as a CancellablePromise and then I added the cancellation call back in which you removed.

(As a side note, this .run() calling was my design and after working with it, I don't actually like this design, but that's another story — it's just to say that I think my poor design is part of what led you astray here.)

I've reviewed the front end code and done a small amount of playing with the app to verify that things work okay. Looks good to me!

Copy link
Contributor

@mathemancer mathemancer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some changes to request. See specific comments. The overall points are:

  • Try to use the built-in django-modern-rpc auth system for the is_self check.
  • Avoid spreading "auth-like" checks across multiple files.

Question: Was doing this full-stack the reason that you needed to do all endpoints at once? I could see how that would happen if there are dependencies in the front end between the old REST endpoints. I'd generally prefer reducing scope on a PR like this by reducing the number of endpoints transitioned at once, rather than skipping tests and docs.

mathesar/rpc/users.py Show resolved Hide resolved
**kwargs
) -> UserInfo:
user = kwargs.get(REQUEST_KEY).user
if not (user.id == user_id or user.is_superuser):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you try to use the set_authentication_predicate decorator for this? I think it would be cleaner and more reusable to try to use that, or set up a custom auth decorator for the "is self or superuser" case.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would also ensure that if the upstream package changes the response for failure of these decorators, that this function doesn't behave differently.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is next to impossible to use set_authentication_predicate for testing is_self because, for is_self, we would require user_id which is a parameter that we pass through the request. The way django_modern_rpc is set up, the permission checks set up through set_authentication_predicate happen before the RPC function's args and kwargs are populated for RPC function execution.

So, even something like this doesn't work:

@wraps(func)
    def wrap(*args, **kwargs):
        user_id = kwargs.get('user_id')
        wrapper = auth.set_authentication_predicate(http_basic_auth_check_user, [user_is_self_or_superuser, user_id])
        return wrapper(func)(*args, **kwargs)

I also tried getting the user_id using the request, but request.POST results in None when printed inside http_basic_auth_check_user().

I finally settled on writing a custom decorator but without using set_authentication_predicate.

Comment on lines 126 to 127
user = kwargs.get(REQUEST_KEY).user
if not user.id == user_id:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here. Try to use a custom auth decorator. You can just add one that guarantees that there's a user.id in the request that equals the user_id. Then, we have an added benefit of having a failure if a developer for some reason names the parameter something unexpected (i.e., something other than user_id)

def update_user_info(user_id, user_info, requesting_user):
if not requesting_user.is_superuser:
user_info.pop("is_superuser", None)
User.objects.filter(id=user_id).update(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm suspicious of having an auth-related step (making sure a non-superuser can't elevate their privileges) in this function, rather than the RPC layer function. Did you consider putting it there? We don't want to have to look in multiple files to validate privilege logic in this case, unless there's a really good reason.

Also, it seems like User.objects.get be better here. Why did you go with the filter function? Specifically, I think we'd want to throw an Exception if there's no matching object. Maybe I'm missing something.

Finally, can't you just explode the user_info dict for this update call? E.g.,

User.objects.get(id=user_id).update(**user_info)

Copy link
Member Author

@Anish9901 Anish9901 Nov 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm suspicious of having an auth-related step (making sure a non-superuser can't elevate their privileges) in this function, rather than the RPC layer function. Did you consider putting it there?

I did consider putting it there but thought that it would be better to add all the logic related to updating user in a single function instead of splitting it, this way we won't forget to check whether the requesting_user is a superuser, if we decided to use the update_user_info function elsewhere. But oh well..., we could also use the RPC functions directly instead. I've updated it as per your request.

Also, it seems like User.objects.get be better here. Why did you go with the filter function? Specifically, I think we'd want to throw an Exception if there's no matching object. Maybe I'm missing something.

Django advises against doing this: https://docs.djangoproject.com/en/4.2/ref/models/querysets/#django.db.models.query.QuerySet.update
The return of the update_user_info() function is get_user() which will raise an error anyway if the user doesn't exist.

Finally, can't you just explode the user_info dict for this update call? E.g.,

User.objects.get(id=user_id).update(**user_info)

We could explode the user_info dict but I decided to avoid it since someone could potentially add additional arguments in that dict to alter other fields of User model. e.g. fields like is_active, is_staff, password_change_needed etc.

We can't use update on a User object or any object we receive with get, we can only use it with filter. Yep, I know... kinda stupid. But I think its because update performs a SQL update instead of loading the result in memory first, contrary to the way get works.

Comment on lines 49 to 53
def change_password(user_id, old_password, new_password):
user = get_user(user_id)
if not user.check_password(old_password):
raise Exception('Old password is not correct')
user.set_password(new_password)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here. I don't think this layer should be authenticating against the user's old password. That should be the RPC layer (unless I'm missing something).

@Anish9901
Copy link
Member Author

Question: Was doing this full-stack the reason that you needed to do all endpoints at once? I could see how that would happen if there are dependencies in the front end between the old REST endpoints.

Nope, They seemed to be small and less complex to implement thats why I did it in one shot.

I'd generally prefer reducing scope on a PR like this by reducing the number of endpoints transitioned at once, rather than skipping tests and docs.

I'll keep that in mind next time.

@Anish9901
Copy link
Member Author

Thanks for the detailed explanation @seancolsen, that was really helpful!

(As a side note, this .run() calling was my design and after working with it, I don't actually like this design, but that's another story — it's just to say that I think my poor design is part of what led you astray here.)

I wouldn't call it bad design, when I first started reading the frontend code for this PR, admittedly my brain was turning into mush looking at the amount of interfaces there were. Mind you the only experience I had with JS before working on this PR is writing console.log. But once I figured out how the RPC code was structured it got easier to figure out where everything should go. So, I'd say the frontend code is structured/designed well enough that a complete beginner like me is also able to figure things out.

Nonetheless, you can expect more frontend contributions from me in the future!

Copy link
Contributor

@mathemancer mathemancer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I think I figured out what's bothering me about this PR w.r.t. the janky auth stuff. The functions need to change.

As designed, they're kind of left over from the REST style, but a major goal of the RPC change is to leave that stuff behind. Instead of the current setup for patch, we should emulate the password change functions: One for 'self', and another for 'other'. Crucially, we don't need a user_id passed to patch our own user: That is leftover thinking from the REST endpoint. So, we should have a patch_self function that doesn't take a user_id, and which any user can access, and a patch_other function that does take a user_id, and which only a superuser can access. Also, the replace_own function shouldn't take a user_id at all! This is completely unneeded, and only serves to force us to make that awkward auth check to make sure that a user can only modify their own password. Finally, I think these functions should take primitive arguments rather than a SettableUserInfo object. The reason is that it makes it a bit clearer what the function is actually allowed to modify, and it's a bit safer. The patch_self function shouldn't even take a is_superuser parameter.

The front end change should be pretty easy for the patch stuff, since the function that calls is already aware of whether the user is modifying themselves or another. You just need to call the appropriate RPC function at that point.

Sorry to keep going around on this stuff, but I think it's pretty important to avoid adding any ad-hoc auth pieces if we can avoid it.

Bonus idea: Given that we'd be splitting out patch_self and patch_other, we should be able to wrap up the password change logic into the same function, since that's actually just a patch. The only reason we had it separate in the first place is because of the self vs. other distinction. This is optional, but it would simplify the module a bit.

@Anish9901
Copy link
Member Author

@seancolsen I've made some changes for patching users as suggested by Brent, in 7b31e81, can you please review them?

Copy link
Contributor

@mathemancer mathemancer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, thanks for all your work on this. I'm happy with it once @seancolsen signs off.

Copy link
Contributor

@seancolsen seancolsen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! I pushed 57857cf with some additional cleanup to simplify the types.

@seancolsen seancolsen added this pull request to the merge queue Nov 19, 2024
Merged via the queue into develop with commit cbabca3 Nov 19, 2024
48 checks passed
@seancolsen seancolsen deleted the transition_user_endpoints_to_RPC branch November 19, 2024 13:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment