Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache fetched apps in update check #7264

Merged
merged 3 commits into from
Nov 27, 2017
Merged

Cache fetched apps in update check #7264

merged 3 commits into from
Nov 27, 2017

Conversation

MorrisJobke
Copy link
Member

The code tried to find the apps with updates and thus was called for every available app. This caused to get the full appstore content as often as apps are available. The appstore request itself was cached nevertheless in an appdata dir, but with an object storage this is still a lot of round trips to read this cached result. Thus the instantiated list is now cached in a static variable (because it's a static method call).

Fixes #7235

Diff of the before and after code base: https://blackfire.io/profiles/compare/0b59e81d-4ea4-4a22-ba14-640cf091d487/graph

Copy link
Member

@nickvergessen nickvergessen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Static.... ⚱️

@nickvergessen
Copy link
Member

Actually it should be cached already?

// File does already exists
$file = $rootFolder->getFile($this->fileName);
$jsonBlob = json_decode($file->getContent(), true);
if (is_array($jsonBlob)) {
// No caching when the version has been updated
if (isset($jsonBlob['ncversion']) && $jsonBlob['ncversion'] === $this->getVersion()) {
// If the timestamp is older than 300 seconds request the files new
if ((int)$jsonBlob['timestamp'] > ($this->timeFactory->getTime() - self::INVALIDATE_AFTER_SECONDS)) {
return $jsonBlob['data'];

@nickvergessen
Copy link
Member

Ah reading helps...

@nickvergessen
Copy link
Member

There was 1 failure:

1) Test\InstallerTest::testIsUpdateAvailable with data set #1 (array(array('files', array(array('1.0')))), false)
Failed asserting that '1111.0' is identical to false.

/drone/src/github.com/nextcloud/server/tests/lib/InstallerTest.php:157

Static fun

@MorrisJobke
Copy link
Member Author

Static fun

And I have no idea what this is :/

@MorrisJobke
Copy link
Member Author

And I have no idea what this is :/

Ah - because it is cached and not cleared between those jobs :/

@codecov
Copy link

codecov bot commented Nov 24, 2017

Codecov Report

Merging #7264 into master will increase coverage by 0.02%.
The diff coverage is 29.72%.

@@             Coverage Diff              @@
##             master    #7264      +/-   ##
============================================
+ Coverage     50.85%   50.88%   +0.02%     
- Complexity    24551    24552       +1     
============================================
  Files          1585     1585              
  Lines         93833    93791      -42     
  Branches       1354     1354              
============================================
+ Hits          47719    47725       +6     
+ Misses        46114    46066      -48
Impacted Files Coverage Δ Complexity Δ
core/ajax/update.php 0% <0%> (ø) 11 <0> (ø) ⬇️
settings/ajax/updateapp.php 0% <0%> (ø) 0 <0> (ø) ⬇️
lib/base.php 3.09% <0%> (-0.04%) 167 <0> (ø)
core/Command/Maintenance/Install.php 0% <0%> (ø) 21 <0> (ø) ⬇️
core/Command/Upgrade.php 0% <0%> (ø) 28 <1> (ø) ⬇️
core/Command/App/Install.php 0% <0%> (ø) 5 <0> (ø) ⬇️
core/register_command.php 0% <0%> (ø) 0 <0> (ø) ⬇️
lib/private/Setup.php 12.76% <10%> (+0.42%) 50 <0> (ø) ⬇️
lib/private/Updater.php 6.6% <20%> (+0.41%) 77 <0> (ø) ⬇️
lib/private/Server.php 82.62% <25%> (-0.69%) 126 <1> (ø)
... and 14 more

@MorrisJobke
Copy link
Member Author

Ah - because it is cached and not cleared between those jobs :/

Okay ... I needed to take a bigger reordering of how it works. basically the second commit in here brings the DI version to all places where the Installer is used. And the third commit replaced the static calls to isUpdateAvailable to normal method calls on the Installer object. I tested around and it works fine (enabling/disabling/updating apps).

@nickvergessen @rullzer @LukasReschke Please review

@blizzz
Copy link
Member

blizzz commented Nov 24, 2017

Several tests are failing, but on first sight they seem unrelated… but those happen not on master.

@MorrisJobke
Copy link
Member Author

Several tests are failing, but on first sight they seem unrelated… but those happen not on master.

It was related. I forgot to mock the call in the tests, so it asked for actual updates in the appstore. 🙈

The code tried to find the apps with updates and thus was called for every available app. This caused to get the full appstore content as often as apps are available. The appstore request itself was cached nevertheless in an appdata dir, but with an object storage this is still a lot of round trips to read this cached result. Thus the instantiated list is now cached in a static variable (because it's a static method call).

Signed-off-by: Morris Jobke <hey@morrisjobke.de>
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
@LukasReschke LukasReschke merged commit 8ccb486 into master Nov 27, 2017
@LukasReschke LukasReschke deleted the cache-fetched-apps branch November 27, 2017 10:49
return false;
}

$apps = $appFetcher->get();
if ($this->apps === null) {
$apps = $this->appFetcher->get();
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Obviously I failed here too. 🙈 cc @rullzer @skjnldsv Let me fix that.

MorrisJobke added a commit that referenced this pull request Dec 14, 2017
Followup to #7264

Signed-off-by: Morris Jobke <hey@morrisjobke.de>
@MorrisJobke MorrisJobke mentioned this pull request Dec 14, 2017
MorrisJobke added a commit that referenced this pull request Dec 14, 2017
Followup to #7264

Signed-off-by: Morris Jobke <hey@morrisjobke.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3. to review Waiting for reviews
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants