Before installing new shipped apps we need to load authentication apps #22584

DeepDiver1975 · 2016-02-22T21:02:25Z

mention-bot · 2016-02-22T21:02:28Z

By analyzing the blame information on this pull request, we identified @PVince81, @LukasReschke and @icewind1991 to be potential reviewers

MorrisJobke · 2016-02-23T07:01:54Z

lib/private/updater.php

 			// update all shipped apps
 			$disabledApps = $this->checkAppsRequirements();
 			$this->doAppUpgrade();

+			// install new shipped apps on upgrade
+			OC_Installer::installShippedApps();


I can't find code that runs eventual code for shipped apps, that just need an update. Or aren't they installed then?

Never code too late in the night ..... updated

I don't know it - was just a gut feeling.

…s - fixes #22508

nickvergessen · 2016-02-23T07:54:16Z

And assume that they are still working 🙊

I'd say we should change the order here? First update the apps, then enable authentication apps and then install the others?

Otherwise when ldap or others need to be updated themselves, they might run into a problem here, because they are executed before they are updated themselves?

nickvergessen · 2016-02-23T07:55:41Z

And since we disable 3rdparty apps on the update, this would kick any non-default authentication app and the users would not be upgraded again?

DeepDiver1975 · 2016-02-23T09:28:54Z

And since we disable 3rdparty apps on the update, this would kick any non-default authentication app and the users would not be upgraded again?

this is a pain ... we need to address the central user account table in 9.1

@karlitschek @cmonteroluque just FYI once more

DeepDiver1975 · 2016-02-23T09:29:01Z

I'd say we should change the order here? First update the apps, then enable authentication apps and then install the others?

agreed

PVince81 · 2016-02-23T09:31:33Z

Ref: central user account table: #21282

nickvergessen · 2016-02-23T10:04:10Z

👍 in the 🙈 mode, it's better then before but still has potential to fail I guess

LukasReschke · 2016-02-23T13:25:43Z

lib/private/updater.php

@@ -334,6 +331,9 @@ private function doUpgrade($currentVersion, $installedVersion) {
 			// upgrade appstore apps
 			$this->upgradeAppStoreApps($disabledApps);

+			// install new shipped apps on upgrade
+			OC_App::loadApps('authentication');


https://scrutinizer-ci.com/g/owncloud/core/inspections/66ee62e3-218f-4eb6-8fdb-a8ddc5d01060/issues/files/lib/private/updater.php?status=new&orderField=path&order=asc&honorSelectedPaths=0

Also all other usages indicate OC_App::loadApps(array('authentication'))

(alternatively we should adjust the PHPDoc if that is supposed to work as well)

Will loadApps automatically update these authentication apps too if needed ? (I don't remember)

load will not, but that is why we do the upgrades before, see above

DeepDiver1975 · 2016-02-23T15:08:35Z

@owncloud/qa please add contact and calendar migration for ldap users - THX

DeepDiver1975 · 2016-02-23T15:10:05Z

@LukasReschke PHPDoc addressed

DeepDiver1975 · 2016-02-24T12:53:41Z

@p5n you might want to test this and let us know if this fixes your issue #22508

jvillafanez · 2016-02-25T13:13:56Z

Calendar doesn't seem to work with ldap users after the upgrade.

jvillafanez@ubuntu1404:~$ curl -X PROPFIND http://aaliyah_adams:secret@10.0.2.6:10080/remote.php/dav/calendars/aaliya_adams/personal
<?xml version="1.0" encoding="utf-8"?>
<d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns">
  <s:exception>Sabre\DAV\Exception\NotFound</s:exception>
  <s:message>Principal with name aaliya_adams not found</s:message>
</d:error>
jvillafanez@ubuntu1404:~$ curl -X PROPFIND http://aaliyah_adams:secret@10.0.2.6:10080/remote.php/dav/calendars/a566ba42-7629-1034-93a2-e5c403d3d31b/personal
<?xml version="1.0" encoding="utf-8"?>
<d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns">
  <s:exception>Sabre\DAV\Exception\NotFound</s:exception>
  <s:message>Principal with name aaliyah_adams not found</s:message>
</d:error>
jvillafanez@ubuntu1404:~$ curl -X PROPFIND http://a566ba42-7629-1034-93a2-e5c403d3d31b:secret@10.0.2.6:10080/remote.php/dav/calendars/a566ba42-7629-1034-93a2-e5c403d3d31b/personal
<?xml version="1.0" encoding="utf-8"?>
<d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns">
  <s:exception>Sabre\DAV\Exception\NotAuthenticated</s:exception>
  <s:message>Username or password was incorrect</s:message>
</d:error>

Local user (in this case admin) works fine:

jvillafanez@ubuntu1404:~$ curl -X PROPFIND http://admin:Password@10.0.2.6:10080/remote.php/dav/calendars/admin/personal
<?xml version="1.0"?>
<d:multistatus xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns" xmlns:cal="urn:ietf:params:xml:ns:caldav" xmlns:cs="http://calendarserver.org/ns/" xmlns:card="urn:ietf:params:xml:ns:carddav" xmlns:oc="http://owncloud.org/ns">
 <d:response>
  <d:href>/remote.php/dav/calendars/admin/personal/</d:href>
  <d:propstat>
   <d:prop>
    <d:resourcetype>
     <d:collection/>
     <cal:calendar/>
    </d:resourcetype>
   </d:prop>
   <d:status>HTTP/1.1 200 OK</d:status>
  </d:propstat>
 </d:response>
 <d:response>
  <d:href>/remote.php/dav/calendars/admin/personal/owncloud-229eda6d7eabd8545ed47f2f865ec8dc.ics</d:href>
  <d:propstat>
   <d:prop>
    <d:getlastmodified>Thu, 25 Feb 2016 11:37:54 GMT</d:getlastmodified>
    <d:getcontentlength>327</d:getcontentlength>
    <d:resourcetype/>
    <d:getetag>&quot;4138ad1ef8570acf274878eb0fdb5955&quot;</d:getetag>
    <d:getcontenttype>text/calendar; charset=utf-8; component=vevent</d:getcontenttype>
   </d:prop>
   <d:status>HTTP/1.1 200 OK</d:status>
  </d:propstat>
 </d:response>
</d:multistatus>

The data has been migrated, but seems to use have issue with the ldap user's id:

mysql> select * from calendars;
+----+-------------------------------------------------------+-------------------+-------------------+-----------+-------------+---------------+---------------+----------+--------------+-------------+
| id | principaluri                                          | displayname       | uri               | synctoken | description | calendarorder | calendarcolor | timezone | components   | transparent |
+----+-------------------------------------------------------+-------------------+-------------------+-----------+-------------+---------------+---------------+----------+--------------+-------------+
|  1 | principals/users/admin                                | Personal          | personal          |         2 | Personal    |             0 | NULL          | NULL     | VEVENT,VTODO |           0 |
|  2 | principals/users/a566ba42-7629-1034-93a2-e5c403d3d31b | Personal          | personal          |         3 | Personal    |             0 | NULL          | NULL     | VEVENT,VTODO |           0 |
|  3 | principals/users/a5673fe4-7629-1034-93a3-e5c403d3d31b | Personal          | personal          |         3 | Personal    |             0 | NULL          | NULL     | VEVENT,VTODO |           0 |
|  4 | principals/system/system                              | contact_birthdays | contact_birthdays |         1 | NULL        |             0 | NULL          | NULL     | VEVENT,VTODO |           0 |
+----+-------------------------------------------------------+-------------------+-------------------+-----------+-------------+---------------+---------------+----------+--------------+-------------+
4 rows in set (0.00 sec)

mysql> select * from ldap_user_mapping where ldap_dn like '%aaliyah_adams%';
+------------------------------------------------------+--------------------------------------+--------------------------------------+
| ldap_dn                                              | owncloud_name                        | directory_uuid                       |
+------------------------------------------------------+--------------------------------------+--------------------------------------+
| uid=aaliyah_adams,ou=people,dc=owncloudqa,dc=com     | a566ba42-7629-1034-93a2-e5c403d3d31b | a566ba42-7629-1034-93a2-e5c403d3d31b |
| uid=aaliyah_adams_ii,ou=people,dc=owncloudqa,dc=com  | a5673fe4-7629-1034-93a3-e5c403d3d31b | a5673fe4-7629-1034-93a3-e5c403d3d31b |
| uid=aaliyah_adams_iii,ou=people,dc=owncloudqa,dc=com | a56776bc-7629-1034-93a4-e5c403d3d31b | a56776bc-7629-1034-93a4-e5c403d3d31b |
| uid=aaliyah_adams_iv,ou=people,dc=owncloudqa,dc=com  | a567a998-7629-1034-93a5-e5c403d3d31b | a567a998-7629-1034-93a5-e5c403d3d31b |
| uid=aaliyah_adams_v,ou=people,dc=owncloudqa,dc=com   | a56815cc-7629-1034-93a6-e5c403d3d31b | a56815cc-7629-1034-93a6-e5c403d3d31b |
+------------------------------------------------------+--------------------------------------+--------------------------------------+
5 rows in set (0.01 sec)

jvillafanez · 2016-02-25T13:23:31Z

The same happens with contacts.

jvillafanez@ubuntu1404:~$ curl -X PROPFIND http://admin:Password@10.0.2.6:10080/remote.php/dav/addressbooks/users/admin/contacts
<?xml version="1.0"?>
<d:multistatus xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns" xmlns:cal="urn:ietf:params:xml:ns:caldav" xmlns:cs="http://calendarserver.org/ns/" xmlns:card="urn:ietf:params:xml:ns:carddav" xmlns:oc="http://owncloud.org/ns">
 <d:response>
  <d:href>/remote.php/dav/addressbooks/users/admin/contacts/</d:href>
  <d:propstat>
   <d:prop>
    <d:resourcetype>
     <d:collection/>
     <card:addressbook/>
    </d:resourcetype>
   </d:prop>
   <d:status>HTTP/1.1 200 OK</d:status>
  </d:propstat>
 </d:response>
 <d:response>
  <d:href>/remote.php/dav/addressbooks/users/admin/contacts/a6c71645-30e4-4f9f-be96-3a5ff8235ebb.vcf</d:href>
  <d:propstat>
   <d:prop>
    <d:getlastmodified>Thu, 25 Feb 2016 11:37:21 GMT</d:getlastmodified>
    <d:getcontentlength>240</d:getcontentlength>
    <d:resourcetype/>
    <d:getetag>&quot;30f0e5358238e8100f89f43af43bd12d&quot;</d:getetag>
    <d:getcontenttype>text/vcard; charset=utf-8</d:getcontenttype>
   </d:prop>
   <d:status>HTTP/1.1 200 OK</d:status>
  </d:propstat>
 </d:response>
</d:multistatus>
jvillafanez@ubuntu1404:~$ curl -X PROPFIND http://aaliyah_adams:secret@10.0.2.6:10080/remote.php/dav/addressbooks/users/aaliyah_adams
<?xml version="1.0" encoding="utf-8"?>
<d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns">
  <s:exception>Sabre\DAV\Exception\NotFound</s:exception>
  <s:message>Principal with name aaliyah_adams not found</s:message>
</d:error>
jvillafanez@ubuntu1404:~$ curl -X PROPFIND http://aaliyah_adams:secret@10.0.2.6:10080/remote.php/dav/addressbooks/users/a566ba42-7629-1034-93a2-e5c403d3d31b
<?xml version="1.0" encoding="utf-8"?>
<d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns">
  <s:exception>Sabre\DAV\Exception\NotFound</s:exception>
  <s:message>Principal with name aaliyah_adams not found</s:message>
</d:error>
jvillafanez@ubuntu1404:~$ curl -X PROPFIND http://aaliyah_adams:secret@10.0.2.6:10080/remote.php/dav/addressbooks/users/a566ba42-7629-1034-93a2-e5c403d3d31b/contacts
<?xml version="1.0" encoding="utf-8"?>
<d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns">
  <s:exception>Sabre\DAV\Exception\NotFound</s:exception>
  <s:message>Principal with name aaliyah_adams not found</s:message>
</d:error>

Notice that the url to access is diferent: contacts has a "users" part in the url that calendar doesn't have

curl -X PROPFIND http://admin:Password@10.0.2.6:10080/remote.php/dav/addressbooks/users/admin/contacts
curl -X PROPFIND http://admin:Password@10.0.2.6:10080/remote.php/dav/calendars/admin/personal

DeepDiver1975 · 2016-02-25T13:39:08Z

Okay - looks like we have some issues with the new dav endpoints with respect to LDAP auth - @blizzz can I ask you to have a look? Do I miss some username convertion?

@jvillafanez the issue of this PR is that upgrades for ldap users have not been executed properly - did this work out now here? THX a olot

jvillafanez · 2016-02-25T14:09:34Z

The upgrade went fine, and I don't see any relevant error in the logs.

There is a lot of noise with the ldap.

{"reqId":"bfxRlVSG9woUqUz9UP2Y","remoteAddr":"","app":"user_ldap","message":"No DN found for a575909e-7629-1034-93c4-e5c403d3d31b on 10.0.2.6","level":0,"time":"2016-02-25T11:37:22+00:00","method":"--","url":"--"}
{"reqId":"bfxRlVSG9woUqUz9UP2Y","remoteAddr":"","app":"user_ldap","message":"initializing paged search for  Filter objectClass=* base Array\n(\n    [0] => uid=aaliyah_bosco,ou=people,dc=owncloudqa,dc=com\n)\n attr Array\n(\n    [0] => entryuuid\n)\n limit 500 offset 0","level":0,"time":"2016-02-25T11:37:22+00:00","method":"--","url":"--"}
{"reqId":"bfxRlVSG9woUqUz9UP2Y","remoteAddr":"","app":"user_ldap","message":"Ready for a paged search","level":0,"time":"2016-02-25T11:37:22+00:00","method":"--","url":"--"}
{"reqId":"bfxRlVSG9woUqUz9UP2Y","remoteAddr":"","app":"user_ldap","message":"initializing paged search for  Filter objectClass=* base Array\n(\n    [0] => uid=aaliyah_bosco,ou=people,dc=owncloudqa,dc=com\n)\n attr Array\n(\n    [0] => displayName\n)\n limit 500 offset 0","level":0,"time":"2016-02-25T11:37:22+00:00","method":"--","url":"--"}
{"reqId":"bfxRlVSG9woUqUz9UP2Y","remoteAddr":"","app":"user_ldap","message":"Ready for a paged search","level":0,"time":"2016-02-25T11:37:22+00:00","method":"--","url":"--"}
{"reqId":"bfxRlVSG9woUqUz9UP2Y","remoteAddr":"","app":"user_ldap","message":"No DN found for a575d7de-7629-1034-93c5-e5c403d3d31b on 10.0.2.6","level":0,"time":"2016-02-25T11:37:22+00:00","method":"--","url":"--"}
{"reqId":"bfxRlVSG9woUqUz9UP2Y","remoteAddr":"","app":"user_ldap","message":"initializing paged search for  Filter objectClass=* base Array\n(\n    [0] => uid=aaliyah_brakus,ou=people,dc=owncloudqa,dc=com\n)\n attr Array\n(\n    [0] => entryuuid\n)\n limit 500 offset 0","level":0,"time":"2016-02-25T11:37:22+00:00","method":"--","url":"--"}

{"reqId":"bfxRlVSG9woUqUz9UP2Y","remoteAddr":"","app":"user_ldap","message":"Ready for a paged search","level":0,"time":"2016-02-25T11:37:30+00
:00","method":"--","url":"--"}
{"reqId":"bfxRlVSG9woUqUz9UP2Y","remoteAddr":"","app":"user_ldap","message":"readAttribute: uid=aaliyah_abernathy,ou=people,dc=owncloudqa,dc=co
m found","level":0,"time":"2016-02-25T11:37:30+00:00","method":"--","url":"--"}
{"reqId":"bfxRlVSG9woUqUz9UP2Y","remoteAddr":"","app":"user_ldap","message":"initializing paged search for  Filter (|(objectclass=inetOrgPerson
)) base Array\n(\n    [0] => uid=aaliyah_adams,ou=people,dc=owncloudqa,dc=com\n)\n attr Array\n(\n    [0] => \n)\n limit 500 offset 0","level":
0,"time":"2016-02-25T11:37:30+00:00","method":"--","url":"--"}
{"reqId":"bfxRlVSG9woUqUz9UP2Y","remoteAddr":"","app":"user_ldap","message":"Ready for a paged search","level":0,"time":"2016-02-25T11:37:30+00
:00","method":"--","url":"--"}
{"reqId":"bfxRlVSG9woUqUz9UP2Y","remoteAddr":"","app":"user_ldap","message":"readAttribute: uid=aaliyah_adams,ou=people,dc=owncloudqa,dc=com fo
und","level":0,"time":"2016-02-25T11:37:30+00:00","method":"--","url":"--"}
{"reqId":"bfxRlVSG9woUqUz9UP2Y","remoteAddr":"","app":"user_ldap","message":"initializing paged search for  Filter (|(objectclass=inetOrgPerson
)) base Array\n(\n    [0] => uid=aaliyah_adams_ii,ou=people,dc=owncloudqa,dc=com\n)\n attr Array\n(\n    [0] => \n)\n limit 500 offset 0","leve
l":0,"time":"2016-02-25T11:37:43+00:00","method":"--","url":"--"}

Ldap users (both old and new) can enter ownCloud without problem after the upgrade.

If there is anything more specific to check, ping me

DeepDiver1975 · 2016-02-25T14:12:30Z

If there is anything more specific to check, ping me

The addressbooks and calendars need to be copied over from the old tables to the new.
No idea if this is happening.

jvillafanez · 2016-02-25T14:25:33Z

addressbookchanges
addressbooks
calendarchanges
calendarobjects
calendars
cards
cards_properties

All of those tables contains data. I can recognize data that I input in 8.2.2 in some of those tables. The only exception is the calendarsubcriptions table which is empty. I haven't added any subscription in 8.2.2 (not sure if it's possible though)

I think DB is fine

DeepDiver1975 · 2016-02-25T14:37:59Z

I haven't added any subscription in 8.2.2 (not sure if it's possible though)

this is not possible afaik

THX a lot - so this looks good to me

DeepDiver1975 · 2016-02-25T14:39:19Z

@jvillafanez did you test this in a docker? If so please tell me which one so I can look up some things. THX

blizzz · 2016-02-25T16:31:54Z

@jvillafanez do the issues stated in #22584 (comment) and #22584 (comment) still persist?

DeepDiver1975 · 2016-02-25T20:43:30Z

tested by bug reporter: #22508 (comment)

Before installing new shipped apps we need to load authentication apps

lock · 2019-08-07T12:01:58Z

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

DeepDiver1975 added the 3 - To Review label Feb 22, 2016

DeepDiver1975 added this to the 9.0-current milestone Feb 22, 2016

DeepDiver1975 added the in progress label Feb 22, 2016

MorrisJobke reviewed Feb 23, 2016
View reviewed changes

Before installing new shipped apps we need to load authentication app…

7bf658a

…s - fixes #22508

DeepDiver1975 force-pushed the fix-app-load-order-fixes-22508 branch from d51ef3d to 7bf658a Compare February 23, 2016 07:41

DeepDiver1975 changed the title ~~Before installing new shipped apps we better upgrade apps first which…~~ Before installing new shipped apps we need to load authentication apps Feb 23, 2016

Install new shipped apps after upgrading all existing apps

ad6238c

nickvergessen added the feature:update label Feb 23, 2016

LukasReschke reviewed Feb 23, 2016
View reviewed changes

Fix PHPDoc of OC_App::loadApps()

bfde96a

DeepDiver1975 added a commit that referenced this pull request Feb 25, 2016

Merge pull request #22584 from owncloud/fix-app-load-order-fixes-22508

cb41b1a

Before installing new shipped apps we need to load authentication apps

DeepDiver1975 merged commit cb41b1a into master Feb 25, 2016

DeepDiver1975 deleted the fix-app-load-order-fixes-22508 branch February 25, 2016 20:43

DeepDiver1975 removed the in progress label Feb 25, 2016

lock bot locked as resolved and limited conversation to collaborators Aug 7, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Before installing new shipped apps we need to load authentication apps #22584

Before installing new shipped apps we need to load authentication apps #22584

DeepDiver1975 commented Feb 22, 2016

mention-bot commented Feb 22, 2016

MorrisJobke Feb 23, 2016

DeepDiver1975 Feb 23, 2016

MorrisJobke Feb 23, 2016

nickvergessen commented Feb 23, 2016

nickvergessen commented Feb 23, 2016

DeepDiver1975 commented Feb 23, 2016

DeepDiver1975 commented Feb 23, 2016

PVince81 commented Feb 23, 2016

nickvergessen commented Feb 23, 2016

LukasReschke Feb 23, 2016

LukasReschke Feb 23, 2016

LukasReschke Feb 23, 2016

PVince81 Feb 23, 2016

nickvergessen Feb 23, 2016

DeepDiver1975 commented Feb 23, 2016

DeepDiver1975 commented Feb 23, 2016

DeepDiver1975 commented Feb 24, 2016

jvillafanez commented Feb 25, 2016

jvillafanez commented Feb 25, 2016

DeepDiver1975 commented Feb 25, 2016

jvillafanez commented Feb 25, 2016

DeepDiver1975 commented Feb 25, 2016

jvillafanez commented Feb 25, 2016

DeepDiver1975 commented Feb 25, 2016

DeepDiver1975 commented Feb 25, 2016

blizzz commented Feb 25, 2016

DeepDiver1975 commented Feb 25, 2016

lock bot commented Aug 7, 2019

Before installing new shipped apps we need to load authentication apps #22584

Before installing new shipped apps we need to load authentication apps #22584

Conversation

DeepDiver1975 commented Feb 22, 2016

mention-bot commented Feb 22, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nickvergessen commented Feb 23, 2016

nickvergessen commented Feb 23, 2016

DeepDiver1975 commented Feb 23, 2016

DeepDiver1975 commented Feb 23, 2016

PVince81 commented Feb 23, 2016

nickvergessen commented Feb 23, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DeepDiver1975 commented Feb 23, 2016

DeepDiver1975 commented Feb 23, 2016

DeepDiver1975 commented Feb 24, 2016

jvillafanez commented Feb 25, 2016

jvillafanez commented Feb 25, 2016

DeepDiver1975 commented Feb 25, 2016

jvillafanez commented Feb 25, 2016

DeepDiver1975 commented Feb 25, 2016

jvillafanez commented Feb 25, 2016

DeepDiver1975 commented Feb 25, 2016

DeepDiver1975 commented Feb 25, 2016

blizzz commented Feb 25, 2016

DeepDiver1975 commented Feb 25, 2016

lock bot commented Aug 7, 2019