Skip to content

Conversation

@ShGKme
Copy link
Contributor

@ShGKme ShGKme commented Jun 28, 2024

📝 Summary

Fixes a performance regression for instances with many external storages.

Steps to reproduce

  1. Add 300-500 s3 external storages and mount to the root subfolder
  2. Enable text app
  3. See 10-15s loading time on /apps/files/files for PROPFIND request for local s3.
    With external s3, like Amazon, it could be 10 times slower.

That was a regression in 28.

s3 28 27
0 0.1s 0.1s
100 3s 0.2s
200 6s 0.5s
300 9s 0.9s
400 12s 1-2s
500 15s 1.1-2.4s

The problem

There is a new <nc:rich_workspace> property in each Node in the response. To add this property (even empty), text checks for a file in every child folder, which could be an external remote storage.

Having a hundred external storages mounted in one parent, it results in a hundred requests to remotes for the data that is not really required to have in advance.

🚧 TODO

  • Only add property for parent ignoring deep nodes as it was in the past

💥 Breaking changes

This breaks the Android Files client a bit—it uses a response from a parent to know about description in children.

Broken scenario:

  1. Open root
  2. Open a child without README
  3. Go back to parent
  4. Open a child with README
    • Before: README is shown as a description
    • After: README is not shown and a swipe-reload is required

Alternative solution

Only ignore external storages, which has the most performance impact.

🏁 Checklist

  • Code is properly formatted (npm run lint / npm run stylelint / composer run cs:check)
  • Sign-off message is added to all commits
  • Tests (unit, integration and/or end-to-end) passing and the changes are covered with tests
  • Documentation (README or documentation) has been updated or is not required

@juliusknorr
Copy link
Member

So the test actually has a comment on this that proofs what we discussed in the call:

// Android app relies on this when navigating nested folders
it('adds rich workspace property to nested folders', function() {
cy.createFolder('/workspace')
// FIXME: Ideally we do not need a page context for those tests at all
// For now the dashboard avoids that we have failing requests due to conflicts when updating the file
cy.visit('/apps/dashboard')
cy.propfindFolder('/', 1)
.then(results => results.pop().propStat[0].properties)
.should('have.property', richWorkspace, '')
cy.uploadFile('test.md', 'text/markdown', '/workspace/Readme.md')
cy.propfindFolder('/', 1)
.then(results => results.pop().propStat[0].properties)
.should('have.property', richWorkspace, '## Hello world\n')
})

@alperozturk96 Can you keep us posted here on if that is possible to address on Android?

@juliusknorr
Copy link
Member

Other failure is unrelated.

@ShGKme ShGKme force-pushed the perf/dont-check-rich-workspace-for-children-in-propfind branch from b55ef41 to 4a48aa3 Compare June 28, 2024 12:26
@marinofaggiana
Copy link
Member

marinofaggiana commented Jun 28, 2024

Good to know ! @mpivchev

@juliusknorr
Copy link
Member

Discussed with @tobiasKaminsky as we couldn't think of a proper way to make the root only result working with android:

  • Check if we can just cache the existence of a rich workspace
  • alternative would be to use a special property for web

@ShGKme
Copy link
Contributor Author

ShGKme commented Jul 2, 2024

  • alternative would be to use a special property for web

Then performance regression is not fixed on Android, having ~10-100 times slower root update than it was in 27

@juliusknorr
Copy link
Member

Then performance regression is not fixed on Android, having ~10-100 times slower root update than it was in 27

True, but this should already have been the case before 28. Though only considered as an alternative, caching is probably the most reasonable

@ShGKme
Copy link
Contributor Author

ShGKme commented Jul 2, 2024

True, but this should already have been the case before 28. Though only considered as an alternative, caching is probably the most reasonable

Why? It was added only in 28, or I got it wrong?

@juliusknorr
Copy link
Member

No, the clients already request this property since a long time. 28 only introduced a change where the web frontend now also fetches those, so the performance issue is also there earlier, just was never noticed.

@ShGKme ShGKme marked this pull request as draft July 15, 2024 15:06
@ShGKme
Copy link
Contributor Author

ShGKme commented Jul 15, 2024

  • Check if we can just cache the existence of a rich workspace

Working on this solution

This was referenced Apr 15, 2025
@Altahrim Altahrim mentioned this pull request Apr 17, 2025
14 tasks
@blizzz blizzz mentioned this pull request May 5, 2025
10 tasks
@nextcloud-bot nextcloud-bot mentioned this pull request May 15, 2025
20 tasks
@icewind1991
Copy link
Member

Can this be finished as it's preventing workspace from being enabled on c.nc.c

@max-nextcloud
Copy link
Collaborator

max-nextcloud commented Sep 4, 2025

Can you test the clients with this specific commit e5efd64 ? That would help to decide if we actually need the extra one.

@tobiasKaminsky Could you test? (see #5963 (comment))

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants