Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[stable25] SystemTags optimizations #38015

Closed
wants to merge 3 commits into from
Closed

[stable25] SystemTags optimizations #38015

wants to merge 3 commits into from

Conversation

blizzz
Copy link
Member

@blizzz blizzz commented May 2, 2023

combined backport of #37961 and #37969, plus PHP 7.4 compatibility change

This is preliminary.

blizzz added 3 commits May 2, 2023 09:41
@blizzz
PoC: SystemTags endpoint to return tags used by a user with meta data
c91cf3b 
Target case is photos app: when visiting the tags category, all systemtags
of the whole cloud are retrieved. In subequent steps the next tag is
requested until the browser view is filled with tag tiles (i.e. previews
are requested just as well).

With this approach, we incorpoate the dav search and look for user related
tags that are used by them, and already returns the statistics (number of
files tagged with the respective tag) as well as a file id for the purpose
to load the preview. This defaults to the file with the highest id.

Call:
curl -s -u 'user:password' \
  'https://my.nc.srv/remote.php/dav/systemtags-current' \
  -X PROPFIND -H 'Accept: text/plain' \
  -H 'Accept-Language: en-US,en;q=0.5'  -H 'Depth: 1' \
  -H 'Content-Type: text/plain;charset=UTF-8' \
  --data @/home/doe/request-systemtag-props.xml

With request-systemtag-props.xml:
<?xml version="1.0" encoding="UTF-8"?>
<d:propfind xmlns:d="DAV:">
        <d:prop xmlns:oc="http://owncloud.org/ns" xmlns:nc="http://nextcloud.org/ns">
                <oc:id/>
                <oc:display-name/>
                <oc:user-visible/>
                <oc:user-assignable/>
                <oc:can-assign/>
                <nc:files-assigned/>
                <nc:reference-fileid/>
        </d:prop>
</d:propfind>

Example output:
  …
  <d:response>
    <d:href>/master/remote.php/dav/systemtags/84</d:href>
    <d:propstat>
      <d:prop>
        <oc:id>84</oc:id>
        <oc:display-name>Computer</oc:display-name>
        <oc:user-visible>true</oc:user-visible>
        <oc:user-assignable>true</oc:user-assignable>
        <oc:can-assign>true</oc:can-assign>
        <nc:files-assigned>42</nc:files-assigned>
        <nc:reference-fileid>924022</nc:reference-fileid>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
  </d:response>
  <d:response>
    <d:href>/remote.php/dav/systemtags/97</d:href>
    <d:propstat>
      <d:prop>
        <oc:id>97</oc:id>
        <oc:display-name>Bear</oc:display-name>
        <oc:user-visible>true</oc:user-visible>
        <oc:user-assignable>true</oc:user-assignable>
        <oc:can-assign>true</oc:can-assign>
        <nc:files-assigned>1</nc:files-assigned>
        <nc:reference-fileid>923422</nc:reference-fileid>
      </d:prop>
      <d:status>HTTP/1.1 200 OK</d:status>
    </d:propstat>
  </d:response>
  …

Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
@blizzz
use efficient tag retrieval on DAV report request
72d1488 
- uses DAV search approach against valid files joined by systemtag selector
- reduced table join for tag/systemtag search
- supports pagination
- no changes to the output formats or similar

Example request body:

<?xml version="1.0"?>
<oc:filter-files xmlns:d="DAV:" xmlns:oc="http://owncloud.org/ns" xmlns:nc="http://nextcloud.org/ns" xmlns:ocs="http://open-collaboration-services.org/ns">
  <d:prop>
    <d:getcontentlength/>
    <d:getcontenttype/>
    <d:getetag/>
    <d:getlastmodified/>
    <d:resourcetype/>
    <nc:face-detections/>
    <nc:file-metadata-size/>
    <nc:has-preview/>
    <nc:realpath/>
    <oc:favorite/>
    <oc:fileid/>
    <oc:permissions/>
    <nc:nbItems/>
  </d:prop>
  <oc:filter-rules>
    <oc:systemtag>32</oc:systemtag>
  </oc:filter-rules>
  <d:limit>
    <d:nresults>50</d:nresults>
    <nc:firstresult>0</nc:firstresult>
  </d:limit>
</oc:filter-files>

Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
@blizzz
adjustments to 25
1e97275 
Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
github-advanced-security[bot]
github-advanced-security bot found potential problems May 2, 2023
View reviewed changes
apps/dav/lib/Connector/Sabre/FilesReportPlugin.php
if (!empty($systemTagIds)) {
$tags = $this->tagManager->getTagsByIds($systemTagIds);
$tagName = (current($tags))->getName();
$nodes = $this->userFolder->searchBySystemTag($tagName, $this->userSession->getUser()->getUID(), $limit ?? 0, $offset ?? 0);

Check notice

Code scanning / Psalm

PossiblyNullReference

Cannot call method getUID on possibly null value
apps/dav/lib/Connector/Sabre/FilesReportPlugin.php
}
}

return $results;
}

protected function wrapNode(\OCP\Files\Node $node): \Sabre\DAV\INode {

Check failure

Code scanning / Psalm

InvalidReturnType

Not all code paths of OCA\DAV\Connector\Sabre\FilesReportPlugin::wrapNode end in a return statement, return type Sabre\DAV\INode expected
apps/dav/lib/SystemTag/SystemTagsInUseCollection.php
}

$userFolder = $this->rootFolder->getUserFolder($user->getUID());
$result = $userFolder->getSystemTags('image');

Check failure

Code scanning / Psalm

UndefinedInterfaceMethod

Method OCP\Files\Folder::getSystemTags does not exist
lib/private/Files/Node/Folder.php
throw new \InvalidArgumentException('searching by owner is only allows on the users home folder');
}

protected function getCachesAndMountpointsForSearch(bool $limitToHome = false): array {

Check failure

Code scanning / Psalm

InvalidDocblock

Unexpected brace character in docblock for OC\Files\Node\Folder::getCachesAndMountpointsForSearch
apps/dav/lib/Connector/Sabre/FilesReportPlugin.php
if (!empty($systemTagIds)) {
$tags = $this->tagManager->getTagsByIds($systemTagIds);
$tagName = (current($tags))->getName();
$nodes = $this->userFolder->searchBySystemTag($tagName, $this->userSession->getUser()->getUID(), $limit ?? 0, $offset ?? 0);

Check failure

Code scanning / Psalm

UndefinedInterfaceMethod

Method OCP\Files\Folder::searchBySystemTag does not exist
lib/private/Files/Cache/QuerySearchHelper.php
$searchExpr = $this->searchBuilder->searchOperatorToDBExpr($builder, $filter);
if ($searchExpr) {
$query->andWhere($searchExpr);
if ($searchQuery->getSearchOperation() instanceof ISearchComparison && $searchQuery->getSearchOperation()->getField() === 'systemtag') {

Check failure

Code scanning / Psalm

UndefinedInterfaceMethod

Method OCP\Files\Search\ISearchOperator::getField does not exist
@blizzz blizzz closed this Jul 11, 2023
@skjnldsv skjnldsv deleted the backport/37961+37969/stable25 branch March 14, 2024 07:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@blizzz