10.0.10 - Changelog Test Results

[patrickjahns]

Changelog

All notable changes to this project will be documented in this file.

Items based on https://github.com/owncloud/core/blob/stable10/CHANGELOG.md

[Unreleased]

Added

• - occ decrypt-all command can now read password from an environment variable - #32252 #Pending issue, under investigation @davitol
• - Command for resetting password now supports sending reset email and outputting link - #32500 @davitol
• N/A- Roave Security Advisories as a development dependency - #31818 @davitol
• - Store timestamp when ownCloud was first installed - #32000 @davitol
• ⚙️ - Symfony events for login action with token or Apache - #31985 @davitol (Contains unit tests)
• ⚙️ - Search API for files using Webdav REPORT and underlying search provider - #31946 #32328 @davitol (Unit Tests)
• Store user name in oc_preferences when provided by backend, use in external storage save in session mode #32579 - @cdamken
• Support JSON format for settings passed to occ system:config:set - #32524 @davitol
• ⚙️ - Add information whether user can share to capabilities API - #31824 @davitol
• - Reload the filelist view when accepting or rejecting a share - #31798 @davitol
• - Allow different language in public link share email - #31767 @davitol
• - Hook "loadAdditionalScripts" now also available for public link page - #31944 @davitol (Unit tests)
• - Command files:scan now outputs items per second - #32093 @davitol
• - New option to prevent users to share with specific system groups - #31740 #32533 #32501 @davitol
• - Add url parameter to files app which opens a specific sidebar tab - #32202 @davitol (Only works with files)
• - Retry chunks in web UI on stalled or timed out uploads - #32170 #32335 @davitol (Tested with IE uploading several files/sizes etc and it went fine. PR also contains debug code)
• - Add log entry for each migration that is run - #32461 @davitol
• - Ability to create users and send them an email for password creation - #32466 @davitol Fixed in [stable10] Backport of Fix the redirection url when oC run in localhost #32648 ###WIP the users are created, but when creating the users, the webUI looks like it got stuck instead of redirecting to login page
• N/A - Added Phan static code analyzer to improve code quality - #32492
• ⚙️ - Added method in PHP share API to set password hashes directly - #32572 It contains Unit tests. It will be tested deeper when Data exporter with we ready. Checked that the hash changes in the DB @davitol
• - Experimental support for asynchronous MOVE operations - #32414, will be partly 🤖 - @individual-it @PVince81

Changed

• - Handle SSL certificate verifications for others than Let's Encrypt - #31858 - @PVince81
• - Insufficient storage exception now logged with "debug" log level - #31978 - tested using quota - @PVince81
• - Skip filecache repair step for version greater than 10.0.4 - #31803 - @PVince81
• - Allow slashes in generated resource routes in app framework - #31939 @davitol
• - Email field is now default in user management page - #32466 @davitol
• - Bump swiftmailer/swiftmailer from 5.4.9 to 5.4.10 - #32200
• - Split of config.sample.php into two files for core and apps - #32554 new file is missing in RC1 build - @PVince81 => [stable10] Include new config sample for apps in dist build #32634. ✅ now fixed in RC2

Karma PRs bumping versions (no need to be tested, just check the version)

• - Bump karma from 2.0.2 to 3.0.0 in /build - #31892 #32197 #32317
• - Bump karma from 2.0.4 to 2.0.5 in /build - #32197
• - Bump behat/behat from 3.4.3 to 3.5.0 - #32318
• - Bump sinon from 2.4.1 to 6.1.5 in /build - #32319

Symfony PRs bumping versions (no need to be tested, just check the version)

• - Bump paragonie/random_compat v2.0.15 to v2.0.17 - #32107
• - Bump symfony/event-dispatcher from 3.4.12 to 3.4.13 - #32199
• - Bump symfony/console from 3.4.12 to 3.4.13 - #32140
• - Bump symfony/routing from 3.4.12 to 3.4.13 - #32137
• - Bump symfony/process from 3.4.12 to 3.4.13 - #32135
• - Bump symfony/translation from 3.4.12 to 3.4.13 - #32198
• - Bump symfony polyfill 1.8.0 to 1.9.0 - #32255
• - Minor dependency bumps 2018-08-26 - #32439
• - Bump symfony 3.4.15 and zend-stdlib 3.2.1 - #32499

Removed

Fixed

Encryption fixes

• - Fix master key recreation - #32504 @PVince81
• - Use OC_DEFAULT_MODULE constant for encryption in core - #31838 => since encryption still works, this proves the constant worked @PVince81
• - Unset encrypted flag in file cache when running decrypt-all command - #32027 - checked oc_filecache before and after decryption, flag is reset to 0 correctly - @PVince81
• - Fix decrypt of single user in decrypt-all command - #32168 - tested with master key and user-key, works - @PVince81
• - Fix login exception in decrypt-all command - #31986 - @PVince81
• - Properly clean up encryption keys after file deletion - #31959 - tested user-key, deleting file or parent folder - @PVince81

Public link PRs

• - Lock public link share dialog while processing - #31928 - hit "Save" repeatedly, only one operation - @PVince81
• - Accept email addresses with subdomains with hyphens for public link emails - #32281 @davitol
• - Don't strip linebreaks in personal note of public link share - #32331 - @PVince81

Calendar PRs

• - Fix calendar or reminder insertion error via CalDAV on MacOS - #32024 - 🤖 - @DeepDiver1975
• - Prevent share access to birthday calendar - #31882 - @PVince81 with steps from birthday calendar exposes no access share state and is… #31833 (comment)
• - Added space in display names of shared calendar/contact - #31877 - 🤖 - @DeepDiver1975

Fixes for PUT method

• - Optimize file uploads with PUT method, with custom mtime, use storage instead of view - #31891 - 🤖 covered by upload test in API tests - @PVince81
• - Optimize file uploads with PUT, don't fetch and update checksum again, reuse the one from part file - #31768 - 🤖 covered by upload test in API tests - @PVince81

Fixes related to UIDs

• - Compare UIDs instead of objects when changing displayname - #32409 - @PVince81
• - Compare UIDs instead of objects when changing email address - #32391 - @PVince81

Fixes easy to test (estimation 0,5 md)

• - Improve text about logging in config.sample.php - #32049 - @PVince81
• - Fix not allowed to share message - #32429 - @PVince81
• - AppManager text typo and PHPdoc return tags - #31918
• - Do not throw an error when the same theme is enabled twice - #31783 - @PVince81
• - Fix repair step that removes duplicate sub shares - #31146 - tested with Update error when duplicate sub-shares exist #27990 (comment) - @PVince81
• - Adjust code to follow coding standard - #32116 - @PVince81
• - Fix overriding for gif images in themes for CLI scripts - #32131 - @PVince81
• - Fix wording on password change page - #32146 @davitol
• - Fixed mount config in frontend to only load once to avoid side effects - #32095 davitol - Tested Mounting and using Personal Mount and no issue appear and it worked fine
• - Don't urlencode group id to make it work with "/" and "%" - #31109 @davitol
• - Fix version previews to fall back to icon when no preview provider is available - #32474 @davitol
• - Fix typos in config.sample.php - #32496 - @PVince81

WIP other fixes

• - Return correct status when IMip email delivery fails - #32489 - 🤖 - @DeepDiver1975
• - Don't check for avatar folder if not enabled - #32490 - @PVince81
• - Add missing ILogger declaration in MigrationService - #32473 #32475
• - Fix JS tests for future Sinon JS update - #32488 - the PR is only tests - @PVince81
• - Command to verify checksums is now more robust - #32360 - @PVince81
• - Update php doc to reflect proper return type - #32427 - nothing to test - @PVince81
• - Catch more errors in SMB storage - #32416
• - Don't crash on filescan where folder has symlink - #32408 - @PVince81
• - Fix issue with some special characters in queries - #32412 - @PVince81 - not testable, hardening, no regression
• - Use the core exception logger functionality in cron.php - #32404 @davitol
• - Improve performance when propagating size updates in file cache - #32304 - @PVince81
• - Prevent current chunk assembly failing by setting the exclusive file lock earlier - #32334 - @PVince81
• - Let files be overwritten by rename operations on local storage instead of pre-deleting - #32273 - @PVince81
• - Continue with upgrade even if the market app cannot be disabled - #32324 - confirmed by update-testing - @PVince81
• - Versions app now works also when comments app is disabled - #32208 - @PVince81
• - Fix two factor challenge page for when password has expired - #32058 - @PVince81
• - Scanner now properly resets checksum whenever a file has changed remotely - #32284 @davitol - Tested with test provided in https://github.com/owncloud/enterprise/issues/2561#issuecomment-392531205 and installing samba 4.8.2. provided via https://launchpad.net/~bnd-acc/+archive/ubuntu/samba-latest. Everything was synced fine.
• - Fix checksums not being updated on modifying shared file for objectstore - #32364 @individual-it
• - Properly set installed_version flag when enabling app via provisioning api - #32214 - @PVince81
• - Fix API response of pending shares when the state did not change - #32156 - @PVince81
• - Read mtime from both JS properties in web UI upload for browser compatibility - #32013 @davitol - Tested uploading in Chrome / Firefox and small file and a big file and checking mtime is preserved
• - Fix warning in logs while moving FutureFile after chunk assembly - #32166 - @PVince81
• - Allow null in "Origin" header for third party clients that send it with WebDAV - #32189 confirmed by community WebDAV not working with other clients #32090 (comment)
• - Properly log failed message when token based authentication is enforced - #31948 - @PVince81 - raised unrelated Missing "Login failed" message when two factor app enabled #32654
• - Deleting a user now also properly deletes their external storages and storage assignations - #32069 🚫 - @PVince81 raised [10.0.10RC1] Deleting user doesn't delete them from external storages #32637
• - Remove sensitive shared_secret data from occ config:list output - #31997 @davitol
• - Fix file cache update function to properly handle empty string and nulls with Oracle - #31996 - 🤖 - @PVince81 tested indirectly through the ScanTest of occ files:scan which triggers checksum resets (this is how the issue was found)
• - Fix bogus etag update when propagating etag for federated shares - #31992 - @PVince81
• 🚫 - Display all failed recipients when sending link share email - #31935 @davitol Fixed in RC2
• - Prevent logging LDAP password in case of failure - #32592 @davitol - Tested shutting down the already configured LDAP configuration and checking password is no longer shown in the stack trace
• - Prevent passwords to be set to empty strings - #32581 - @PVince81
• - Fix update issue related to oc_jobs when automatically enabling market app to assist for update in OC 10 - #32573 - @PVince81
• - Trigger missing migrations in files_sharing app, adds indices and can speed up some instances - #32562 - @PVince81
• - Fix issue with spam filters when sending public link emails - #32542 @davitol

Fixes related to notifications app

• Added translations - #233 - @PVince81
• Clear user notifications after user deletion - #223 - @PVince81
• Include the product name in the mail for the notifications - #211 - @PVince81
• Add "hello" in the mail templates - #209 - @PVince81
• Add common footer to notifications - #207 - @PVince81

RC2 items

• changelog diff here: [stable10] Changelog update for 10.0.10 RC2 #32679
• decrypt-all has default for "--continue" - #32677 - @PVince81
• - No more zsync in capabilities - #32603 - tested with curl on capabilities endpoint - @PVince81
• - Minimum desktop client version is 2.3.3 - #32657 @davitol - Tested with 2.3.0 and a warning is shown. 'Forbidden (Unsupported client version)'
• Fix PHP 7.2 issue with ini_set - #32538 🤖 - @PVince81
• OC redirect in password setting page - #32648 @davitol - Tested creating a user setting using a mail.
• Replace add-password with set_password - #32636 @davitol - ErrorException of Undefined index is no longer thrown
• Do not delete token if token mismatch happens - #32672 @davitol - Tested with the steps provided in the PR. Note: Step 'user sees that token is invalid', does not happen. It never shows invalid token page
• Concat all email addresses that had sending problems - #32633 @davitol - Tested reproducing dialog 'an error occured while sending email' and check that both mails are shown
• recheck failed items from RC1

RC3 items

• Config report now contains list of all migrations that have run, for easier debugging of update issues - configreport/#68 - @PVince81
• Update CA bundle - 2018-06-20 - #32688 @davitol checked in the code the changes
• Email field is now default in user management page, users receive an email with token to set initial password - #32466 [[stable10] Backport of Fix the redirection url when oC run in localhost #32648] [stable10] Backport of Fix the token exception classes to avoid infin… #32690 @davitol
• recheck failed items from RC2

[ownclouders]

GitMate.io thinks possibly related issues are #31579 (test), #22492 (test), #22485 (test), #30152 (Testing 10.0.5RC3 Changelog), and #32127 ([stable10] Update changelog for upcoming 10.0.10).

[PVince81]

raised owncloud/encryption#49 while testing encryption

[PVince81]

I've tested "Fix master key recreation - #32504" and it worked fine.

While testing, raised owncloud/encryption#50 regarding master key regeneration in maintenance mode.

[PVince81]

Tested "Unset encrypted flag in file cache when running decrypt-all command - #32027"

Checked oc_filecache before and after decryption, flag is reset to 0 correctly.

While testing, I found owncloud/encryption#51

[PVince81]

Added changelog entries from notifications app.

[PVince81]

Raised #32619 related to the new "create by email" changes from #32466.

It's about the toggle not working properly if you leave a value in the field.

[PVince81]

I've tested all the notifications items

[PVince81]

Don't strip linebreaks in personal note of public link share - #32331 => works

[PVince81]

Improve performance when propagating size updates in file cache - #32304 => size and etag propagation still works as observed in filecache.

[PVince81]

• Let files be overwritten by rename operations on local storage instead of pre-deleting - #32273 - works

• tested by renaming a file to another existing one

• test overwrite a folder with a file

• test overwrite a file with a folder

[PVince81]

• Update php doc to reflect proper return type - #32427 - nothing to test
• Fix typos in config.sample.php - #32496 - file from tarball looks the same

[PVince81]

• Improve text about logging in config.sample.php - #32049 => text is present in tarball

[PVince81]

• Fix not allowed to share message - #32429 - tested as recipient with a share that doesn't grant reshare permissions

[PVince81]

Both already covered by upload API tests:

• Optimize file uploads with PUT method, with custom mtime, use storage instead of view - [stable10] Optimize PUT - with custom mtime, use storage instead of view #31891
• Optimize file uploads with PUT, don't fetch and update checksum again, reuse the one from part file - [stable10] Optimize PUT - don't fetch and update checksum again, reuse the one from part file #31768

[PVince81]

• TODO: add items from last changelog PR: [stable10] Changelog for 10.0.10 RC1 #32582 @davitol

[PVince81]

• Versions app now works also when comments app is disabled - #32208 => works

[PVince81]

• Properly set installed_version flag when enabling app via provisioning api - #32214 - confirmed that it did not work in 10.0.9 and works now

[PVince81]

• Split of config.sample.php into two files for core and apps 🚫

I'll have a quick look, need to include the new file in the Makefile.

=> fix is here #32634

[phil-davis]

• Display all failed recipients when sending link share email - [stable10] return all failed recipients in sendLinkShareMailFromBody #31935 had regression in a later PR. Proposed fix in PR Concat all email addresses that had sending problems #32632

[PVince81]

• Deleting a user now also properly deletes their external storages and storage assignations - [stable10] Backport of Remove user from the storage setting or storag… #32069 no_entry_sign - => raised [10.0.10RC1] Deleting user doesn't delete them from external storages #32637

[PVince81]

• Fix API response of pending shares when the state did not change - #32156 - works

[PVince81]

• Fix warning in logs while moving FutureFile after chunk assembly

Tested and confirmed that the warning log was there in v10.0.9 and gone in v10.0.10RC1 for a chunked upload initiated in the browser.

[PVince81]

• Do not throw an error when the same theme is enabled twice - [Stable10] Do not throw an error when the same theme enabled twice #31783 => works

[PVince81]

• Fix repair step that removes duplicate sub shares - #31146 - tested with Update error when duplicate sub-shares exist #27990 (comment)

[PVince81]

• Don't check for avatar folder if not enabled - #32490

I've disabled avatars and checked the files app, users page and also a public link page. All work fine.
Tested this because the change is in the template layout code.

[PVince81]

• Properly log failed message when token based authentication is enforced - [stable10] If token based authentication is mandatory a login failed … #31948

Works fine with "token_auth_enforced' => true" as expected in the ticket and PR.

However while testing I noticed that if you enable any two factor app without enforcing token auth in config.php, the two factor mode will automatically enforce tokens and the client will not be able to login. In this case, the "Login failed" message is missing in the log. I've raised this separately here #32654

[PVince81]

• Fix two factor challenge page for when password has expired - #32058

Challenge page properly appears instead of white page.

[cdamken]

Store user name in oc_preferences when provided by backend, use in external storage save in session mode #32579

Checked -> works

[PVince81]

• Fix update issue related to oc_jobs when automatically enabling market app to assist for update in OC 10 - [Stable10] Fix update cases from 9.0.9 and below #32573

Confirmed working by update-testing and also rechecked manually.

[PVince81]

• Trigger missing migrations in files_sharing app, adds indices and can speed up some instances - [stable10] Trigger missing migrations in files_sharing #32562

Tested on my own server which was missing three migrations. They are now present after the upgrade:

| files_sharing         | 20170830112305 |
| files_sharing         | 20171115154900 |
| files_sharing         | 20171215103657 |

[PVince81]

• Command to verify checksums is now more robust - [stable10] Verify checksum fixes  #32360

Tested by deleting a file from disk but leaving in file cache.
Then ran occ files:checksums:verify: the command skipped the entry instead of crashing.

[PVince81]

• Fix bogus etag update when propagating etag for federated shares - [stable10] empty path has no parents #31992

Tested with steps from #31988 and etags are stable for the root and for the one byte file as recipient in 10.0.10 RC1.

Note: I tried the same in 10.0.9 but could not reproduce the original issue. @butonic

[PVince81]

• Fix issue with some special characters in queries - [stable10] Use createPositionalParameter() - see https://github.com/o… #32412

Could not find a way to cause an issue. Seems not testable, hardening.

[PVince81]

• Fix overriding for gif images in themes for CLI scripts - [Stable10] Fix overriding for gif images in themes for CLI scripts #32131

Tested with steps from #31804.
Logo in email is correct.

[PVince81]

• Compare UIDs instead of objects when changing displayname - [stable10] compare UIDs instead of objects when changing displayname #32409

Setting display name in personal page works and users page works.

• Compare UIDs instead of objects when changing email address - [stable10] compare UIDs instead of objects when changing email address #32391

Setting email in personal page works after using link.
Also works when set from users page without link.

[PVince81]

I've added the RC2 items in the original post.
Please note that most aren't listed directly in the changelog bu are part of existing entries as these bugs only exist since the addition of said entries, so they belong there. I've expanded that list into checkboxes for verification.

[PVince81]

• Skip filecache repair step for version greater than 10.0.4

Increased fourth digit in version.php and ran occ upgrade -vvvvvv. The filecache repair step did not appear in the list of executed steps.

Also tested that occ files:scan --repair still runs as it uses the repair step code path.

[individual-it]

• "Fix checksums not being updated on modifying shared file for objectstore" - [stable10] Fix checksums not being updated on modifying shared file for objectstore #32364

1. start server with OWNCLOUD_FOLDER=~/www/owncloud-10.0.10RC2 OWNCLOUD_LICENSE_KEY=xxxxxxx OWNCLOUD_MARKETPLACE_APIKEY=xxxxx OWNCLOUD_APPS_INSTALL=enterprise_key,objectstore docker-compose -f owncloud-base.yml -f owncloud-mount-core.yml -f owncloud-exported-ports.yml -f storage/scality.yml -f storage/scality-primary-objectstore.yml up
2. as user1 create a txt file and check checksum
3. as user1 share file to user2
4. as user2 verify checksum is still the same
5. as user2 overwrite file with same content, verify checksum didn't change
6. as user2 overwrite file with new content
7. as user1 and as user2 check the checksum, verify checksum changed and is the same for both users

[PVince81]

• Experimental support for asynchronous MOVE operations

did a quick test to see if the web UI uses the job status endpoint and reports back for upload and overwrite+upload.

the automated tests were already merged in #32594

[PVince81]

• Prevent share access to birthday calendar

Tested with steps from #31833 (comment).
Works, no event is added to birthday calendar.

[PVince81]

• Handle SSL certificate verifications for others than Let's Encrypt

I don't have the setup to test this but as I can see the directives in the htaccess file are present in my test setup.

[davitol]

• Catch more errors in SMB storage - [stable10] Smb unexpected errors stable10 #32416
  To take into account Owncloud Server Unsyncs all data from client and then resyncs #32552 (comment) . I was not able to get StorageNotAvailable exception but worked for other people.(to be investigated) I guess I should get a 503 and the files should not be deleted

[PVince81]

• Config report now contains list of all migrations that have run, for easier debugging of update issues - configreport/v4.5.0: unsharing directories does not make them disappear #68

Tested on RC3, the migrations appear in the report.

[davitol]

I've added the RC3 items in the original post.

[davitol]

• Email field is now default in user management page, users receive an email with token to set initial password - Invalid Token page is not shown when creating a new user via mail with some configuration #32684

Tested on RC3, invalid token page appears and no infinity loop occurs

[PVince81]

• Prevent current chunk assembly failing by setting the exclusive file lock earlier - #32334

Tested by adding sleep(300) in both locations specified in #32226 (comment), then uploading a big file. Then wait for 10+ minutes. In the end the file finishes the upload.

Observing the access log: the first MOVE has the client time out. Subsequent MOVE all fail with 423 Locked, which is expected as the first MOVE is still working. Once the first MOVE finishes, the file appears on the server.