Restrict autoloaded paths to loaded apps (and other enhancements)

[RobinMcCorkell]

Unloaded apps (aka not enabled apps) will not get any files autoloaded anymore, which can prevent an unloaded app becoming an attack vector if it is not regularly updated.

In addition, the autoloader will properly resolve symlinked app directories, to allow for a semi-common usecase where apps are stored elsewhere but are symlinked into the ownCloud directory. Fixes issues as noted in #18396 (comment)

@fossxplorer want to test this out with your symlinked apps?

cc @icewind1991 @MorrisJobke @PVince81 @LukasReschke

[LukasReschke]

That's cool 👍

[RobinMcCorkell]

Oh, I've inadvertently fixed some other bugs too 🚀

Fixes #18224, fixes #18317, fixes #18836, fixes #18305, fixes #18296, fixes #18322, fixes #18329, fixes #18333, ........ (yes, I included the issues that were marked as duplicates of the original one, simply to show the impact of the bug).

The line that fixes all those bugs is the move of self::$loadedApps[] = $app; from loadApps() to loadApp(). Give me a second to prepare backport PRs, assuming that's wanted @karlitschek ?

[karlitschek]

nice 👍 please backport

[icewind1991]

Looks goods 👍

[RobinMcCorkell]

CI hasn't run the full autotest suite, so I'm not merging just yet...

[scrutinizer-notifier]

A new inspection was created.

[RobinMcCorkell]

The latest commit makes the autoloader throw a unique exception class, which the background job runner catches to prevent stale jobs breaking things.

[MorrisJobke]

The line that fixes all those bugs is the move of self::$loadedApps[] = $app; from loadApps() to loadApp(). Give me a second to prepare backport PRs, assuming that's wanted @karlitschek ?

Correct. Because loadApp is also called during app upgrade 🙈

[MorrisJobke]

Code looks good, fixes my issue and CI runs fine locally 👍

[oparoz]

@Xenopathic - Last time I tried, loading the app did not work. At least it didn't bring routes up, but I'll try again to see if it solves this one.

[RobinMcCorkell]

@nickvergessen when you see this on Monday, before you get too excited and revert immediately, I will be sending out a mailing list email to clarify the updated requirements, as soon as we find an appropriate solution with @oparoz

[oparoz]

I can confirm that loading the app in the boostrap still does nothing. I suspect a bug in the loader, unless someone can show me an app which loads (1. load the app 2. check the navigation menu).

But adding this to the boostrap fixes the issue: OC::$loader->addValidRoot('../');

It just doesn't make sense to have to whitelist the app being tested, but it's better than hundreds of failing tests ;).

[oparoz]

Apart from the problems, I do applaud the move to protect the system from dormant apps 👏

[RobinMcCorkell]

@oparoz Well, for unit tests that line will work just fine, but you realise it doesn't work as expected? All paths are relative to the OC::$SERVERROOT, so you basically just disabled the autoloader verification for everything 😆

[oparoz]

Arf, yes... Missing __DIR__

[oparoz]

LoadApp() works, but there is a problem with loadApps() #18863

[nickvergessen]

@nickvergessen when you see this on Monday, before you get too excited and revert immediately, I will be sending out a mailing list email to clarify the updated requirements, as soon as we find an appropriate solution with @oparoz

Sorry @Xenopathic but this is 💩
Leaving developers with a known broken system. Please notify people in advance. This really makes developing an app a pain. Third breakage in 5 days on the same part. Please take more time on your fixes and test them, before merging them. All apps are publicly available on github. Check them out and run their unit tests locally, so you see what's the problem and stop messing it up.

So the now required code for the bootstrap.php is:

\OC::$loader->addValidRoot(OC::$SERVERROOT . '/tests');
\OC_App::loadApp('<app name here>');

[RobinMcCorkell]

@nickvergessen Actually, I was wondering why we only load a minimal set of apps in bootstrap.php: https://github.com/owncloud/core/blob/master/tests/bootstrap.php#L14-L15. We should be loading all enabled apps, which would prevent these issues?

[nickvergessen]

I'm not using that bootstrap file....
I don't want integration tests where all apps are registered and add there stuff:
https://github.com/owncloud/activity/blob/master/tests/bootstrap.php
https://github.com/owncloud/mail/blob/master/tests/bootstrap.php
https://github.com/owncloud/notifications/blob/master/tests/bootstrap.php
...

[RobinMcCorkell]

OK, so the consensus is that we should explicitly require that the app (and any dependencies of that app) are loaded with OC_App::loadApp() before unit tests are run?

[oparoz]

@Xenopathic - The boostrap still works when I remove \OC::$loader->addValidRoot(OC::$SERVERROOT . '/tests');. Is that intended?

[MorrisJobke]

@Xenopathic - The boostrap still works when I remove \OC::$loader->addValidRoot(OC::$SERVERROOT . '/tests');. Is that intended?

Yep. The root is added on appLoad()

[oparoz]

@MorrisJobke - OK, there may be a bug then:
owncloud-archive/mail#1058 (comment)

[oparoz]

@MorrisJobke - In Gallery, addValidRoot is also still required, even with loadApps();
Keep that in mind for the dev documentation.