Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add creator field to document and allow filtering #429

Merged
merged 10 commits into from
Nov 28, 2024

Conversation

sampaccoud
Copy link
Member

@sampaccoud sampaccoud commented Nov 18, 2024

Purpose

We are currently working on version 2 of Docs' UI. The new UI requires more filters on the documents API endpoint.

In particular, we want to be able to limit the documents displayed on a logged-in user's list view by the documents they created or by the documents that other users created. This is different from having the "owner" role on a document because this can be acquired and even lost. What we want here is to be able to identify documents by the user who created them so we had to add a new field.

Proposal

Add all filters required for the new UI and corresponding tests:

  • ✨ annotate number of accesses on documents in list view
  • ✨ allow users to mark/unmark documents as favorite
  • ✨ allow filtering documents by their titles (basic search while waiting for the fulltext search engine)
  • ✨ allow filtering documents by link reach
  • ✨ allow filtering by documents marked as favorite
  • ✨add creator field on document and allow filtering on it

While working on this PR, I identified a flaky test and a db query optimization that I fixed:

  • 🐛 fix flaky test by clarifying user ordering
  • ⚡️optimize number of queries on document list view

@sampaccoud sampaccoud self-assigned this Nov 18, 2024
@sampaccoud sampaccoud added python Pull requests that update Python code feature backend urgent labels Nov 18, 2024
@sampaccoud sampaccoud changed the title Add created by field to document and allow filtering Add creator field to document and allow filtering Nov 19, 2024
@sampaccoud sampaccoud force-pushed the add-created-by-field-to-document-and-allow-filtering branch from c71a9b7 to 1e1fce5 Compare November 25, 2024 16:14
@AntoLC AntoLC force-pushed the add-created-by-field-to-document-and-allow-filtering branch from 1e1fce5 to 3e7e084 Compare November 28, 2024 11:21
sampaccoud and others added 10 commits November 28, 2024 15:34
I realized most of the database queries made when getting a document
list view were to include nested accesses. This detailed information
about accesses in only necessary for the document detail view.

I introduced a specific serializer for the document list view with
less fields. For a list of 20 documents with 5 accesses, we go down
from 3x5x20= 300 queries to just 3 queries.
A user can now mark/unmark documents as favorite.
This is done via a new action of the document API endpoint:
/api/v1.0/documents/{document_id}/favorite
POST to mark as favorite / DELETE to unmark
The new UI will display the number of accesses on each document.

/!\ Once team accesses will be used, this will not represent the number
    of people with access anymore and will have to be improved by
    computing the number of people in each team.
On the user search API by similarity, we had a flaky test because
2 users had the same similarity score. Adding a secondary ordering
field makes ordering deterministic between users who share the same
similarity score.
We want to be able to limit the documents displayed on a logged-in user's
list view by the documents they created or by the documents that other
users created.

This is different from having the "owner" role on a document because this
can be acquired and even lost. What we want here is to be able to
identify documents by the user who created them so we add a new field.
We recently allowed authenticated users to mark a document as favorite.
We were lacking the possibility for users to see only the documents
they marked as favorite.
We want to be able to limit document list views to only public documents,
or only restricted or authenticated documents.
This is the minimal and fast search feature, while we are working on
a full text search based on opensearch. For the moment we only search
on the title of the document.
We want to make sure that applying filters on the document view list
does not interfere with permissions.
We don't get the accesses anymore from the backeend,
instead we get the number of accesses.
We remove the list of owners in the doc header because
we don't have easily this informations anymore and
we will have to do a bigger refacto.
@AntoLC AntoLC force-pushed the add-created-by-field-to-document-and-allow-filtering branch from 3e7e084 to ac9d23d Compare November 28, 2024 14:34
@AntoLC AntoLC merged commit eec8b4d into main Nov 28, 2024
15 of 16 checks passed
@AntoLC AntoLC deleted the add-created-by-field-to-document-and-allow-filtering branch November 28, 2024 15:02
@AntoLC AntoLC mentioned this pull request Dec 11, 2024
@AntoLC AntoLC linked an issue Jan 3, 2025 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend feature python Pull requests that update Python code urgent
Projects
None yet
Development

Successfully merging this pull request may close these issues.

✨(backend) search documents
3 participants