No Previews/Thumbnails with Primary Storage S3 + Encryption

[gagaha]

Steps to reproduce

1. Use object store s3 as primary storage
2. Enable encryption for home storage
3. Upload files

Expected behaviour

Thumbnails and previews will be generated and displayed

Actual behaviour

I don't get any thumbnails and previews for new files with radosgw S3 as primary storage and Nextclouds server-side encryption for home storage. I can log in and everything looks fine but when I upload files they won't get a thumbnail and I can only download the files. I tried several different file-types but with no success. The sample files (which I suspect to be not encrypted) have thumbnails and previews. I already tried the setup steps in different order. First S3 as primary storage, then enabling encryption and then the other way around. But I always get the same behaviour. The log shows nothing when I upload a file even in debug mode.

Server configuration

Operating system: Ubuntu 16.04

Web server: apache2 + php-fpm

Database: mariaDB

PHP version: 7.0.13

Nextcloud version: 11.0.1

Updated from an older Nextcloud/ownCloud or fresh install: No

Where did you install Nextcloud from: tarball archive

Signing status:

Signing status

No errors have been found.

List of activated apps:

App list

Enabled:
  - activity: 2.4.1
  - calendar: 1.5.0
  - comments: 1.1.0
  - contacts: 1.5.2
  - dav: 1.1.1
  - encryption: 1.4.1
  - federatedfilesharing: 1.1.1
  - federation: 1.1.1
  - files: 1.6.1
  - files_pdfviewer: 1.0.1
  - files_sharing: 1.1.1
  - files_texteditor: 2.2
  - files_trashbin: 1.1.0
  - files_versions: 1.4.0
  - files_videoplayer: 1.0.0
  - firstrunwizard: 2.0
  - gallery: 16.0.0
  - logreader: 2.0.0
  - lookup_server_connector: 1.0.0
  - nextcloud_announcements: 1.0
  - notifications: 1.0.1
  - password_policy: 1.1.0
  - provisioning_api: 1.1.0
  - serverinfo: 1.1.1
  - sharebymail: 1.0.1
  - survey_client: 0.1.5
  - systemtags: 1.1.3
  - theming: 1.1.1
  - twofactor_backupcodes: 1.0.0
  - updatenotification: 1.1.1
  - workflowengine: 1.1.1

The content of config/config.php:

Config report

{
    "system": {
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "localhost"
        ],
        "datadirectory": "\/data",
        "logfile": "\/var\/log\/nextcloud.log",
        "loglevel": "0",
        "overwrite.cli.url": "http:\/\/localhost",
        "dbtype": "mysql",
        "version": "11.0.1.2",
        "dbname": "nextcloud",
        "dbhost": "nextcloud_db",
        "dbport": "",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "logtimezone": "UTC",
        "installed": true,
        "instanceid": "ocsaqvw7qdgx",
        "objectstore": {
            "class": "OC\\Files\\ObjectStore\\S3",
            "arguments": {
                "bucket": "nextcloud-test-gabriel",
                "autocreate": true,
                "key":  "***REMOVED SENSITIVE VALUE***",
                "secret":  "***REMOVED SENSITIVE VALUE***",
                "hostname":  "***REMOVED SENSITIVE VALUE***",
                "use_ssl": true,
                "use_path_style": true
            }
        }
    }
}

Are you using external storage, if yes which one: No

Are you using encryption: yes

Are you using an external user-backend, if yes which one: No

Client configuration

Browser: Firefox

Operating system: Ubuntu 16.04

Logs

Web server error log

Web server error log

``` _:443 172.20.0.1 - - [26/Jan/2017:16:42:02 +0000] "GET /ocs/v2.php/apps/notifications/api/v2/notifications HTTP/1.1" 200 2122 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:50.0) Gecko/20100101 Firefox/50.0" _:443 172.20.0.1 - - [26/Jan/2017:16:42:02 +0000] "PUT /remote.php/webdav/serverwebui.png HTTP/1.1" 201 898 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:50.0) Gecko/20100101 Firefox/50.0" _:443 172.20.0.1 - - [26/Jan/2017:16:42:03 +0000] "PROPFIND /remote.php/webdav/serverwebui.png HTTP/1.1" 207 1760 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:50.0) Gecko/20100101 Firefox/50.0" _:443 172.20.0.1 - - [26/Jan/2017:16:42:04 +0000] "GET /index.php/apps/files/ajax/getstoragestats.php?dir=%2F HTTP/1.1" 200 1147 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:50.0) Gecko/20100101 Firefox/50.0" ```

[gagaha]

I found a solution!
After finishing the nextcloud installation DON'T enable encryption or S3 as primary storage.
Create a new folder in the directory where your key-storage-root is located - by default your data directory. Then set www-data as owner:
mkdir /data/keys && chown -R www-data:www-data /data/keys/

Now change the key-storage-root to this folder:
sudo -u www-data php occ encryption:change-key-storage-root keys

Then change the key-storage-root to /
sudo -u www-data php occ encryption:change-key-storage-root /

Now enable primary storage with object store S3 (insert credentials in config.php) but don't enable encryption yet. You may also remove the previously created 'keys' folder.

Change the key-storage-root again to /
sudo -u www-data php occ encryption:change-key-storage-root /

Now you may enable encryption and thumbnails and previews for new files should be ok.

It would be cool though if you could set the key-storage-root to a local path and not to store them in S3.

[nickvergessen]

@schiessle maybe you can comment on:

It would be cool though if you could set the key-storage-root to a local path and not to store them in S3.

[ghost]

@gagaha
I have the same problem.
But when i following this solution, Encryption not work. All file i uploaded in decrypted form.

[an-ca]

Any update on this?

Just encountered this issue and this is pretty much preventing me from deploying.

[Miaourt]

Any news ? I have the same problem as @hungnvz , @gagaha solution result in unencrypted file on the bucket :/

[gaudenz]

Any news? I have the same problem on a fresh install of 12.0.3. I'm happy to help with debugging.

[LarsLWO]

Same Issue here, but I can't download the files. No way.

After ~15-20 seconds of loading it ends in an HTTP500 error. The requested URL was:
http://localhost:8080/remote.php/webdav/b807c2282ab0a491bd5c5c1051c6d312_k4PiHxO.jpg

Tested with Nextcloud 12.0.3.

[chriswayg]

Same issue here. Previews stop working once I use Nextcloud encryption together with S3 storage:

Error | gallery | Exception: Preview generation has failed

Nextcloud version: 13.0.2 - 13.0.2.1

[schiessle]

We recently supplied some fixes for encryption in combination with s3. It will be released with 13.0.6. Please re-open if the problem still exists after the release/update.

[FlorentCoppint]

It would be cool though if you could set the key-storage-root to a local path and not to store them in S3.

Any news about it ? #17561