Do not setup a session when not required on API requests #28311

juliushaertl · 2021-08-04T14:13:18Z

This would resolve #7628 as currently every API request to webdav/OCS that is using HTTP auth is generating a PHP session even though it is never used afterwards as any follow-up request is sending the auth headers again and therefore just creating another session. This reduces the amount of sessions being created and that would need to be kept by the php session handler which might otherwise grow extensively on large setups with logs of API calls happening.

As we already use a wrapper for the session returning early in the session setup will just result in the memory session implementation being used as a fallback

server/lib/private/Server.php

Line 516 in 7179002

$session = new \OC\Session\Memory('');

Ideally the check would be performed when the request is handled and the login checks are performed is handled, but the risk of breaking something when setting up the session that late in the execution flow seems much higher.

Fix #27603

Double check if authtokens get still created and shown in the security settings
Get rid of config flag and make it the default behaviour
Smoke testing with pyocclient
Still attempt to initiate a session if a session cookie is present during the request, this could happen on public share links if the browser sends both the existing session cookie as well as basic auth for the share link token/password

juliushaertl · 2021-08-05T07:43:11Z

Some integration test failures I've not seen so far. Will look into those:

integration-auth
integration-provisioning
integration-download

They all seem related to the attempt to use the session id:

{"Exception":"OCP\\Session\\Exceptions\\SessionNotAvailableException","Message":"Memory session does not have an ID","Code":0,"Trace":[{"file":"\/drone\/src\/lib\/private\/Authentication\/TwoFactorAuth\/Manager.php","line":348,"function":"getId","class":"OC\\Session\\Memory","type":"->","args":[]},{"file":"\/drone\/src\/lib\/private\/legacy\/OC_Util.php","line":1065,"function":"needsSecondFactor","class":"OC\\Authentication\\TwoFactorAuth\\Manager","type":"->","args":[{"__class__":"OC\\User\\User"}]},{"file":"\/drone\/src\/apps\/files\/recentlist.php","line":27,"function":"checkLoggedIn","class":"OC_Util","type":"::","args":[]},{"file":"\/drone\/src\/apps\/files\/lib\/Controller\/ViewController.php","line":136,"args":["\/drone\/src\/apps\/files\/recentlist.php"],"function":"include"},{"file":"\/drone\/src\/apps\/files\/lib\/Controller\/ViewController.php","line":276,"function":"renderScript","class":"OCA\\Files\\Controller\\ViewController","type":"->","args":["files","recentlist.php"]},{"file":"\/drone\/src\/lib\/private\/AppFramework\/Http\/Dispatcher.php","line":217,"function":"index","class":"OCA\\Files\\Controller\\ViewController","type":"->","args":["","",null,false,null]},{"file":"\/drone\/src\/lib\/private\/AppFramework\/Http\/Dispatcher.php","line":126,"function":"executeController","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->","args":[{"__class__":"OCA\\Files\\Controller\\ViewController"},"index"]},{"file":"\/drone\/src\/lib\/private\/AppFramework\/App.php","line":157,"function":"dispatch","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->","args":[{"__class__":"OCA\\Files\\Controller\\ViewController"},"index"]},{"file":"\/drone\/src\/lib\/private\/Route\/Router.php","line":301,"function":"main","class":"OC\\AppFramework\\App","type":"::","args":["OCA\\Files\\Controller\\ViewController","index",{"__class__":"OC\\AppFramework\\DependencyInjection\\DIContainer"},{"_route":"files.view.index"}]},{"file":"\/drone\/src\/lib\/base.php","line":1004,"function":"match","class":"OC\\Route\\Router","type":"->","args":["\/apps\/files"]},{"file":"\/drone\/src\/index.php","line":36,"function":"handleRequest","class":"OC","type":"::","args":[]}],"File":"\/drone\/src\/lib\/private\/Session\/Memory.php","Line":107,"CustomMessage":"--"}

juliushaertl · 2021-08-17T06:09:42Z

Integration tests seem to make quite some heavy use of sending requests with basic auth and a cookie for the session, which currently breaks with the new approach

server/build/integration/features/bootstrap/Auth.php

Lines 103 to 111 in 215aef3

 'auth' => [ 

 'user0', 

 $loginViaWeb ? '123456' : $this->restrictedClientToken, 

 ], 

 'form_params' => [ 

 'requesttoken' => $this->requestToken, 

 'name' => md5(microtime()), 

 ], 

 'cookies' => $this->cookieJar,

Double check if there is no better way than the @UseSession annotation on the files controller

martinlanger90 · 2021-10-03T06:30:17Z

Hopefully soon gets merged. I want to get rid of that thousands of session files.

skjnldsv · 2022-01-19T09:37:47Z

/rebase

PVince81 · 2022-02-02T15:55:58Z

this could also break tools like https://github.com/owncloud/pyocclient which also rely on the session, I imagine that there might be more out there so this PR would need to be treated as "breaking change" which we cannot backport

or if backport, have a config switch with a different default in previous versions for those setups where the admin knows what kind of clients are being used

and maybe such switch could be turned on for the integration tests to pass, for now @juliushaertl

clarify compatibility / backportability

PVince81 · 2022-02-04T14:10:50Z

PVince81 · 2022-02-04T14:13:10Z

The failures are all in auth.feature and happen in the preconditions:

    Given a new restricted client token is added   # FeatureContext::aNewRestrictedClientTokenIsAdded()
      Client error: `PUT http://localhost:8080/index.php/settings/personal/authtokens/10` resulted in a `412 Precondition failed` response:
      {"message":"CSRF check failed"}
       (GuzzleHttp\Exception\ClientException)

PVince81 · 2022-02-04T15:39:34Z

it looks like the tests like https://github.com/nextcloud/server/blob/enh/http-auth-session/build/integration/features/auth.feature#L19 were expecting basic auth to create a session when done on the web UI endpoints like "index.php/apps/files" as the subsequent test is relying on said session

but in this PR the session is not created any more in that case, not sure if we should keep this or if we should filter the session removal to DAV and OCS only ? @juliushaertl

depending on the latter decision we'll need to rework the tests to reflect the expected behavior

PVince81 · 2022-02-05T15:39:26Z

we need to rediscuss whether a switch is preferable: the decision about not having a switch was based on the assumption that this PR only applies to Webdav requests. however, it turns out that it also applies to other requests like OCS and even web
- should we limit the fix to only Webdav ? (and not need the switch or have it default to no-cookies)
- or should we keep it as is and have a swich
- other ideas ?
to gather more data, test with more third party tools / libraries if possible and see how they react when there's no cookies

Note: the test issues so far were not at all related to the missing session, it was only that some test code was written in the wrong way but did work because a session happened to be there (ex: missing cookie jar usage)

nickvergessen · 2022-02-07T17:57:14Z

Breaks integration tests on notifications app:

  Scenario: Successful registration                            # /home/nickv/Nextcloud/24/server/appsbabies/notifications/tests/Integration/features/push-registration.feature:34
[Mon Feb  7 18:54:20 2022] 127.0.0.1:34172 Accepted
[Mon Feb  7 18:54:20 2022] could not get login credentials because session is unavailable
[Mon Feb  7 18:54:20 2022] 127.0.0.1:34172 [403]: GET /ocs/v2.php/core/getapppassword?format=json
[Mon Feb  7 18:54:20 2022] 127.0.0.1:34172 Closing
    Given user "test1" creates an app password                 # FeatureContext::createAppPassword()
      │ array(0) {
      │ }
      │ 
      Notice: Undefined index: apppassword in …/apps/notifications/tests/Integration/features/bootstrap/FeatureContext.php line 325
    Given user "test1" registers for push notifications with   # FeatureContext::registerForPushNotifications()
      | pushTokenHash   | 12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678 |
      | devicePublicKey | VALID_KEY                                                                                                                        |
      | proxyServer     | https://push-notifications.nextcloud.com/                                                                                        |
    Then status code is 201

Problem is: GET /ocs/v2.php/core/getapppassword?format=json does not work anymore.
It returns a 403 with the exception being logged:

server/lib/private/Authentication/LoginCredentials/Store.php

Lines 100 to 101 in 215aef3

 } catch (SessionNotAvailableException $ex) { 

 $this->logger->debug('could not get login credentials because session is unavailable', ['app' => 'core']);

nickvergessen · 2022-02-07T18:02:20Z

Maybe we can check the controller and see if it uses @UseSession and add that to

server/core/Controller/AppPasswordController.php

Lines 77 to 94 in 215aef3

 /** 

  * @NoAdminRequired 

  * 

  * @return DataResponse 

  * @throws OCSForbiddenException 

  */ 

 public function getAppPassword(): DataResponse { 

 // We do not allow the creation of new tokens if this is an app password 

 if ($this->session->exists('app_password')) { 

 throw new OCSForbiddenException('You cannot request an new apppassword with an apppassword'); 

 } 

 try { 

 $credentials = $this->credentialStore->getLoginCredentials(); 

 } catch (CredentialsUnavailableException $e) { 

 throw new OCSForbiddenException(); 

 }

But then again it shows exactly why this PR is kind of dangerous.

IIRC this endpoint is used by the android apps when they copy credentials from each other?

Also used in iOS

https://github.com/nextcloud/ios-communication-library/blob/c8e3eac61a846775d570b1d252612a8d2d02930d/NCCommunication/NCCommunication%2BLogin.swift#L34

nickvergessen · 2022-02-07T18:20:55Z

Talk is also completely red CI.
We write the talk session into the phpsession. that is needed so people can join the same room multiple times.

Maybe we first limit this to webdav endpoints? OCS stuff needs a lot more looking first:
https://drone.nextcloud.com/nextcloud/spreed/6617/1/3

PVince81 · 2022-08-15T17:02:27Z

lib/base.php

@@ -412,6 +412,15 @@ private static function printUpgradePage(\OC\SystemConfig $systemConfig) {
 }

 public static function initSession() {
+ $request = self::$server->getRequest();
+ $isDavRequest = strpos($request->getRequestUri(), '/remote.php/dav') === 0 || strpos($request->getRequestUri(), '/remote.php/webdav') === 0;


what about public webdav ? basic auth is used there with share token as user and share password as password

juliushaertl · 2022-10-14T05:40:10Z

🏓 for reviews now for 26

tobiasKaminsky · 2022-10-14T07:14:38Z

Is this something that affects clients?
Shall we give it a smoke test?

juliushaertl · 2022-10-14T12:05:56Z

It should not change anything, I did a smoke test at the last performance hackweek with desktop, but smoke testing with iOS/Android would still be highly appreciated :)

tobiasKaminsky · 2022-10-28T08:27:09Z

Smoke test on Android works.
Only one app token is used.
Not sure, but feels a bit faster…

juliushaertl · 2022-12-21T15:05:25Z

Rebased. Still looking for another review 😉

ChristophWurst

Looks good otherwise

lib/base.php

If basic auth is used on WebDAV endpoints, we will not setup a session by default but instead set a test cookie. Clients which handle session cookies properly will send back the cookie then on the second request and a session will be initialized which can be resued for authentication. Signed-off-by: Julius Härtl <jus@bitgrid.net>

juliushaertl · 2022-12-21T21:57:24Z

Checker failure unrelated, fix is in #35855

juliushaertl added enhancement 3. to review Waiting for reviews labels Aug 4, 2021

juliushaertl requested review from LukasReschke, ChristophWurst and a team August 4, 2021 14:13

Pytal previously approved these changes Aug 4, 2021

View reviewed changes

CarlSchwan previously approved these changes Aug 4, 2021

View reviewed changes

juliushaertl force-pushed the enh/http-auth-session branch from b33554f to 77aee15 Compare August 17, 2021 05:56

juliushaertl added 2. developing Work in progress and removed 3. to review Waiting for reviews labels Aug 17, 2021

martinlanger90 mentioned this pull request Oct 28, 2021

Session handling: api and webdav requests shall not start a session #7628

Closed

skjnldsv added this to the Nextcloud 24 milestone Jan 19, 2022

nextcloud-command force-pushed the enh/http-auth-session branch from bdaa565 to 16052db Compare January 19, 2022 09:40

juliushaertl force-pushed the enh/http-auth-session branch from 16052db to 0471bbd Compare February 3, 2022 16:46

CarlSchwan force-pushed the enh/http-auth-session branch from 0471bbd to 55dd770 Compare February 4, 2022 14:01

juliushaertl force-pushed the enh/http-auth-session branch from aa1fe70 to 01d5aca Compare February 7, 2022 15:59

nickvergessen mentioned this pull request Feb 7, 2022

Test/enh/http auth session nextcloud/spreed#6878

Closed

This comment was marked as outdated.

Sign in to view

juliushaertl requested a review from icewind1991 August 15, 2022 10:56

PVince81 reviewed Aug 15, 2022

View reviewed changes

skjnldsv mentioned this pull request Aug 18, 2022

25.0.0 beta 2 #33592

Merged

This was referenced Aug 24, 2022

25.0.0 beta 3 #33674

Merged

25.0.0 beta 4 #33762

Merged

This was referenced Sep 6, 2022

25.0.0 beta 5 #33913

Merged

25.0.0 beta 6 #34002

Merged

skjnldsv mentioned this pull request Sep 15, 2022

25.0.0 beta 7 #34095

Merged

This was referenced Sep 20, 2022

25.0.0 beta 8 #34164

Closed

25.0.0 RC 1 #34185

Merged

blizzz modified the milestones: Nextcloud 25, Nextcloud 26 Sep 22, 2022

skjnldsv approved these changes Oct 14, 2022

View reviewed changes

juliushaertl force-pushed the enh/http-auth-session branch from d0b0706 to d4e994e Compare December 21, 2022 15:05

ChristophWurst reviewed Dec 21, 2022

View reviewed changes

lib/base.php Outdated Show resolved Hide resolved

lib/base.php Outdated Show resolved Hide resolved

juliushaertl force-pushed the enh/http-auth-session branch from d4e994e to 6abb373 Compare December 21, 2022 20:17

ChristophWurst approved these changes Dec 21, 2022

View reviewed changes

juliushaertl merged commit d56ee81 into master Dec 21, 2022

juliushaertl deleted the enh/http-auth-session branch December 21, 2022 21:57

juliushaertl mentioned this pull request Dec 22, 2022

fix(session): Always setup the session if a session cookie is passed #35866

Merged

This was referenced Jul 21, 2023

No session creation on API calls #39529

Closed

No session on DAV API call nextmcloud/server#251

Open

[backport25] No session on DAV API call nextmcloud/server#252

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Do not setup a session when not required on API requests #28311

Do not setup a session when not required on API requests #28311

juliushaertl commented Aug 4, 2021 •

edited

Loading

juliushaertl commented Aug 5, 2021 •

edited

Loading

juliushaertl commented Aug 17, 2021 •

edited

Loading

martinlanger90 commented Oct 3, 2021

skjnldsv commented Jan 19, 2022

PVince81 commented Feb 2, 2022

PVince81 commented Feb 4, 2022 •

edited

Loading

PVince81 commented Feb 4, 2022

PVince81 commented Feb 4, 2022

PVince81 commented Feb 5, 2022 •

edited by juliushaertl

Loading

nickvergessen commented Feb 7, 2022

nickvergessen commented Feb 7, 2022 •

edited

Loading

This comment was marked as outdated.

nickvergessen commented Feb 7, 2022 •

edited

Loading

PVince81 Aug 15, 2022

juliushaertl commented Oct 14, 2022

tobiasKaminsky commented Oct 14, 2022

juliushaertl commented Oct 14, 2022

tobiasKaminsky commented Oct 28, 2022

juliushaertl commented Dec 21, 2022

ChristophWurst left a comment

juliushaertl commented Dec 21, 2022

Do not setup a session when not required on API requests #28311

Do not setup a session when not required on API requests #28311

Conversation

juliushaertl commented Aug 4, 2021 • edited Loading

juliushaertl commented Aug 5, 2021 • edited Loading

juliushaertl commented Aug 17, 2021 • edited Loading

martinlanger90 commented Oct 3, 2021

skjnldsv commented Jan 19, 2022

PVince81 commented Feb 2, 2022

PVince81 commented Feb 4, 2022 • edited Loading

PVince81 commented Feb 4, 2022

PVince81 commented Feb 4, 2022

PVince81 commented Feb 5, 2022 • edited by juliushaertl Loading

nickvergessen commented Feb 7, 2022

nickvergessen commented Feb 7, 2022 • edited Loading

This comment was marked as outdated.

nickvergessen commented Feb 7, 2022 • edited Loading

PVince81 Aug 15, 2022

Choose a reason for hiding this comment

juliushaertl commented Oct 14, 2022

tobiasKaminsky commented Oct 14, 2022

juliushaertl commented Oct 14, 2022

tobiasKaminsky commented Oct 28, 2022

juliushaertl commented Dec 21, 2022

ChristophWurst left a comment

Choose a reason for hiding this comment

juliushaertl commented Dec 21, 2022

juliushaertl commented Aug 4, 2021 •

edited

Loading

juliushaertl commented Aug 5, 2021 •

edited

Loading

juliushaertl commented Aug 17, 2021 •

edited

Loading

PVince81 commented Feb 4, 2022 •

edited

Loading

PVince81 commented Feb 5, 2022 •

edited by juliushaertl

Loading

nickvergessen commented Feb 7, 2022 •

edited

Loading

nickvergessen commented Feb 7, 2022 •

edited

Loading