Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Frequent error 502 and high server load since update to 16.0.1 #16020

Closed
djermann opened this issue Jun 20, 2019 · 3 comments
Closed

Frequent error 502 and high server load since update to 16.0.1 #16020

djermann opened this issue Jun 20, 2019 · 3 comments
Labels
0. Needs triage Pending check for reproducibility or if it fits our roadmap bug needs info
Milestone
Nextcloud 17

Comments

@djermann
Copy link

Steps to reproduce

  1. Upgrade to 16.0.1

Expected behaviour

Load should be normal and no 502 error

Actual behaviour

From time to time Nextcloud becomes unavailable with a 502 - Bad Gateway error and the load is between 10 - 15.
I checked the processlist of MariaDB/MySQL and there seem to be a lot of queries going on for many hours:

MySQL processlist 
+--------+--------------+-----------+--------------+---------+------+----------------+------------------------------------------------------------------------------------------------------+----------+
| Id     | User         | Host      | db           | Command | Time | State          | Info                                                                                                 | Progress |
+--------+--------------+-----------+--------------+---------+------+----------------+------------------------------------------------------------------------------------------------------+----------+
| 324404 | <REMOVED> | localhost | <REMOVED> | Query   |  187 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (1, 23, 1, 8, 1, 1, 30)) AND ((`f`.`mim |    0.000 |
| 326496 | <REMOVED> | localhost | <REMOVED> | Query   |  200 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (1, 23, 1, 8, 1, 1, 30)) AND ((`f`.`mim |    0.000 |
| 334566 | <REMOVED> | localhost | <REMOVED> | Query   |   99 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (1, 165)) AND ((`f`.`mimetype` <> 2) OR |    0.000 |
| 334646 | <REMOVED> | localhost | <REMOVED> | Query   |   99 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (1, 165)) AND ((`f`.`mimetype` <> 2) OR |    0.000 |
| 334847 | <REMOVED> | localhost | <REMOVED> | Query   |   67 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (1, 20, 1, 1, 55)) AND ((`f`.`mimetype` |    0.000 |
| 339223 | <REMOVED> | localhost | <REMOVED> | Query   |   53 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (1, 88)) AND ((`f`.`mimetype` <> 2) OR  |    0.000 |
| 341354 | <REMOVED> | localhost | <REMOVED> | Query   |   20 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (20, 1, 8, 1, 1, 61)) AND ((`f`.`mimety |    0.000 |
| 341418 | <REMOVED> | localhost | <REMOVED> | Query   |   51 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (20, 1, 8, 1, 1, 61)) AND ((`f`.`mimety |    0.000 |
| 341527 | <REMOVED> | localhost | <REMOVED> | Query   |   51 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (20, 1, 8, 1, 1, 61)) AND ((`f`.`mimety |    0.000 |
| 341617 | <REMOVED> | localhost | <REMOVED> | Query   |   20 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (20, 1, 8, 1, 1, 61)) AND ((`f`.`mimety |    0.000 |
| 341643 | <REMOVED> | localhost | <REMOVED> | Query   |   20 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (20, 1, 8, 1, 1, 61)) AND ((`f`.`mimety |    0.000 |
| 341685 | <REMOVED> | localhost | <REMOVED> | Query   |   20 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (20, 1, 8, 1, 1, 61)) AND ((`f`.`mimety |    0.000 |
| 341714 | <REMOVED> | localhost | <REMOVED> | Query   |   54 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (20, 1, 8, 1, 1, 61)) AND ((`f`.`mimety |    0.000 |
| 341730 | <REMOVED> | localhost | <REMOVED> | Query   |   20 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (20, 1, 8, 1, 1, 61)) AND ((`f`.`mimety |    0.000 |
| 341835 | <REMOVED> | localhost | <REMOVED> | Query   |   20 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (20, 1, 8, 1, 1, 61)) AND ((`f`.`mimety |    0.000 |
| 341875 | <REMOVED> | localhost | <REMOVED> | Query   |   51 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (20, 1, 8, 1, 1, 61)) AND ((`f`.`mimety |    0.000 |
| 341938 | <REMOVED> | localhost | <REMOVED> | Query   |   20 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (20, 1, 8, 1, 1, 61)) AND ((`f`.`mimety |    0.000 |
| 342183 | <REMOVED> | localhost | <REMOVED> | Query   |   20 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (20, 1, 8, 1, 1, 61)) AND ((`f`.`mimety |    0.000 |
| 343104 | <REMOVED> | localhost | <REMOVED> | Query   |  204 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (1, 8, 1, 1, 224)) AND ((`f`.`mimetype` |    0.000 |
| 343386 | <REMOVED> | localhost | <REMOVED> | Query   |   20 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (20, 1, 8, 1, 1, 61)) AND ((`f`.`mimety |    0.000 |
Server configuration

Operating system:
Debian 9.9
Web server:
Nginx
Database:
MariaDB 10.1
PHP version:
7.2
Nextcloud version: (see Nextcloud admin page)
16.0.1
Updated from an older Nextcloud/ownCloud or fresh install:
Updated from 13.0.4
Where did you install Nextcloud from:
Website

List of activated apps:

App list 
Enabled:
  - accessibility: 1.2.0
  - activity: 2.9.1
  - announcementcenter: 3.5.1
  - apporder: 0.7.1
  - bookmarks: 1.0.6
  - bruteforcesettings: 1.3.0
  - calendar: 1.7.0
  - circles: 0.17.3
  - cloud_federation_api: 0.2.0
  - comments: 1.6.0
  - contacts: 3.1.3
  - dav: 1.9.2
  - encryption: 2.4.0
  - external: 3.3.0
  - federatedfilesharing: 1.6.0
  - federation: 1.6.0
  - files: 1.11.0
  - files_pdfviewer: 1.5.0
  - files_rightclick: 0.13.0
  - files_sharing: 1.8.0
  - files_texteditor: 2.8.0
  - files_trashbin: 1.6.0
  - files_versions: 1.9.0
  - files_videoplayer: 1.5.0
  - firstrunwizard: 2.5.0
  - gallery: 18.3.0
  - logreader: 2.1.0
  - lookup_server_connector: 1.4.0
  - music: 0.9.5
  - nextcloud_announcements: 1.5.0
  - notifications: 2.4.1
  - oauth2: 1.4.2
  - password_policy: 1.6.0
  - privacy: 1.0.0
  - provisioning_api: 1.6.0
  - quota_warning: 1.5.0
  - radio: 0.6.5
  - recommendations: 0.4.0
  - serverinfo: 1.6.0
  - sharebymail: 1.6.0
  - support: 1.0.0
  - survey_client: 1.4.0
  - systemtags: 1.6.0
  - theming: 1.7.0
  - twofactor_backupcodes: 1.5.0
  - updatenotification: 1.6.0
  - viewer: 1.0.0
  - workflowengine: 1.6.0
Disabled:
  - admin_audit
  - files_external
  - user_ldapr

Nextcloud configuration:

Config report 
{
    "system": {
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "***REMOVED SENSITIVE VALUE***"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "overwrite.cli.url": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "16.0.1.1",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "logtimezone": "Europe\/Zurich",
        "installed": true,
        "activity_expire_days": 180,
        "trashbin_retention_obligation": "auto,90",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "loglevel": 2,
        "log_type": "errorlog",
        "mail_smtpmode": "smtp",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpauth": 1,
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "25",
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "updater.release.channel": "stable",
        "maintenance": false,
        "theme": ""
    }
}

Are you using external storage, if yes which one: local/smb/sftp/...
no
Are you using encryption: yes/no
yes
Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/...
no

Logs

Nothing interesting in the logs.
@djermann djermann added 0. Needs triage Pending check for reproducibility or if it fits our roadmap bug labels Jun 20, 2019
@kesselb
Copy link
Contributor

kesselb commented Jun 20, 2019

Would be good to see the complete query. Looks similar to #15995.

@kesselb kesselb mentioned this issue Jul 16, 2019
Merged
@kesselb
Copy link
Contributor

kesselb commented Jul 16, 2019

Mind to try this patch? #16424

@MorrisJobke
Copy link
Member

Mind to try this patch? #16424

This will be included in all next maintenance releases.

@MorrisJobke MorrisJobke added this to the Nextcloud 17 milestone Jul 17, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
0. Needs triage Pending check for reproducibility or if it fits our roadmap bug needs info
Projects
None yet
Milestone
Nextcloud 17
Development

No branches or pull requests
3 participants
@MorrisJobke @kesselb @djermann