Marketplace not working after upgrade from 10.8 to 10.9

[esackbauer]

Steps to reproduce

1. Install owncloud base files 10.9 via Ubuntu repository
2. Do occ upgrade as in documentation

Expected behaviour

Marketplace would be accessible, list all apps, show updates etc.

Actual behaviour

Marketplace frame opens, but stays empty, at the bottom in red this message appears:
No marketplace connection: Client error response [url] https://marketplace.owncloud.com/api/v1/categories.json [status code] 403 [reason phrase] Forbidden

per ssh:
root@xxxx:/var/www/owncloud# sudo -u www-data php occ market:list
No marketplace connection: Client error response [url] https://marketplace.owncloud.com/api/v1/platform/10.9.0/apps.json [status code] 403 [reason phrase] Forbidden

Double checked with wget and same user, this works and downloads the json files on that machine.

Server configuration

Operating system:
Ubuntu 20.04.3 LTS

Web server:
Apache2 via Ubuntu

Database:
MariaDB on external database server

PHP version:
7.4.3

ownCloud version: (see ownCloud admin page)
10.9.

Updated from an older ownCloud or fresh install:
10.8

Where did you install ownCloud from:
deb http://download.opensuse.org/repositories/isv:/ownCloud:/server:/10/Ubuntu_20.04/

Signing status (ownCloud 9.0 and above):

Login as admin user into your ownCloud and access 
http://example.com/index.php/settings/integrity/failed 
paste the results into https://gist.github.com/ and puth the link here.

No errors have been found.

The content of config/config.php:
see attached files

Log in to the web-UI with an administrator account and click on
'admin' -> 'Generate Config Report' -> 'Download ownCloud config report'
This report includes the config.php settings, the list of activated apps
and other details in a well sanitized form.

or 

If you have access to your command line run e.g.:
sudo -u www-data php occ config:list system
from within your ownCloud installation folder

*ATTENTION:* Do not post your config.php file in public as is. Please use one of the above
methods whenever possible. Both, the generated reports from the web-ui and from occ config:list
consistently remove sensitive data. You still may want to review the report before sending.
If done manually then it is critical for your own privacy to dilligently
remove *all* host names, passwords, usernames, salts and other credentials before posting.
You should assume that attackers find such information and will use them against your systems.

List of activated apps:

If you have access to your command line run e.g.:
sudo -u www-data php occ app:list
from within your ownCloud installation folder.

Enabled:

• activity:
  • Version: 2.6.1
  • Path: /var/www/owncloud/apps/activity
• bookmarks:
  • Version: 0.10.6
  • Path: /var/www/owncloud/apps/bookmarks
• comments:
  • Version: 0.3.0
  • Path: /var/www/owncloud/apps/comments
• configreport:
  • Version: 0.2.0
  • Path: /var/www/owncloud/apps/configreport
• dav:
  • Version: 0.7.0
  • Path: /var/www/owncloud/apps/dav
• duo:
  • Version: 2.5.2
  • Path: /var/www/owncloud/apps/duo
• federatedfilesharing:
  • Version: 0.5.0
  • Path: /var/www/owncloud/apps/federatedfilesharing
• federation:
  • Version: 0.1.0
  • Path: /var/www/owncloud/apps/federation
• files:
  • Version: 1.5.2
  • Path: /var/www/owncloud/apps/files
• files_external:
  • Version: 0.8.0
  • Path: /var/www/owncloud/apps/files_external
• files_mediaviewer:
  • Version: 1.0.5
  • Path: /var/www/owncloud/apps/files_mediaviewer
• files_pdfviewer:
  • Version: 1.0.1
  • Path: /var/www/owncloud/apps/files_pdfviewer
• files_sharing:
  • Version: 0.14.0
  • Path: /var/www/owncloud/apps/files_sharing
• files_texteditor:
  • Version: 2.4.1
  • Path: /var/www/owncloud/apps/files_texteditor
• files_trashbin:
  • Version: 0.9.1
  • Path: /var/www/owncloud/apps/files_trashbin
• files_versions:
  • Version: 1.3.0
  • Path: /var/www/owncloud/apps/files_versions
• firstrunwizard:
  • Version: 1.2.0
  • Path: /var/www/owncloud/apps/firstrunwizard
• gallery:
  • Version: 16.1.2
  • Path: /var/www/owncloud/apps/gallery
• market:
  • Version: 0.6.1
  • Path: /var/www/owncloud/apps/market
• notifications:
  • Version: 0.5.4
  • Path: /var/www/owncloud/apps/notifications
• provisioning_api:
  • Version: 0.5.0
  • Path: /var/www/owncloud/apps/provisioning_api
• systemtags:
  • Version: 0.3.0
  • Path: /var/www/owncloud/apps/systemtags
• templateeditor:
  • Version: 0.4.0
  • Path: /var/www/owncloud/apps/templateeditor
• updatenotification:
  • Version: 0.2.1
  • Path: /var/www/owncloud/apps/updatenotification
• user_ldap:
  • Version: 0.16.0
  • Path: /var/www/owncloud/apps/user_ldap
    Disabled:
• admin_audit:
  • Path: /var/www/owncloud/apps/admin_audit
• announcementcenter:
  • Path: /var/www/owncloud/apps/announcementcenter
• customgroups:
  • Path: /var/www/owncloud/apps/customgroups
• encryption:
  • Path: /var/www/owncloud/apps/encryption
• enterprise_key:
  • Path: /var/www/owncloud/apps/enterprise_key
• external:
  • Path: /var/www/owncloud/apps/external
• files_antivirus:
  • Path: /var/www/owncloud/apps/files_antivirus
• files_classifier:
  • Path: /var/www/owncloud/apps/files_classifier
• files_external_dropbox:
  • Path: /var/www/owncloud/apps/files_external_dropbox
• files_external_ftp:
  • Path: /var/www/owncloud/apps/files_external_ftp
• files_ldap_home:
  • Path: /var/www/owncloud/apps/files_ldap_home
• files_lifecycle:
  • Path: /var/www/owncloud/apps/files_lifecycle
• firewall:
  • Path: /var/www/owncloud/apps/firewall
• graphapi:
  • Path: /var/www/owncloud/apps/graphapi
• guests:
  • Path: /var/www/owncloud/apps/guests
• metrics:
  • Path: /var/www/owncloud/apps/metrics
• oauth2:
  • Path: /var/www/owncloud/apps/oauth2
• openidconnect:
  • Path: /var/www/owncloud/apps/openidconnect
• password_policy:
  • Path: /var/www/owncloud/apps/password_policy
• ransomware_protection:
  • Path: /var/www/owncloud/apps/ransomware_protection
• sharepoint:
  • Path: /var/www/owncloud/apps/sharepoint
• systemtags_management:
  • Path: /var/www/owncloud/apps/systemtags_management
• theme-enterprise:
  • Path: /var/www/owncloud/apps/theme-enterprise
• user_external:
  • Path: /var/www/owncloud/apps/user_external
• user_shibboleth:
  • Path: /var/www/owncloud/apps/user_shibboleth
• web:
  • Path: /var/www/owncloud/apps/web
• windows_network_drive:
  • Path: /var/www/owncloud/apps/windows_network_drive
• wopi:
  • Path: /var/www/owncloud/apps/wopi
• workflow:
  • Path: /var/www/owncloud/apps/workflow

Are you using external storage, if yes which one: local/smb/sftp/...
no

Are you using encryption: yes/no
no

Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/...

With access to your command line run e.g.:
sudo -u www-data php occ ldap:show-config
from within your ownCloud installation folder

Without access to your command line download the data/owncloud.db to your local
computer or access your SQL server remotely and run the select query:
SELECT * FROM `oc_appconfig` WHERE `appid` = 'user_ldap';


Eventually replace sensitive data as the name/IP-address of your LDAP server or groups.

Client configuration

Browser:
Chrome, Firefox, wget

Operating system:
Windows 10, Ubuntu 20.04.3 LTS

Logs

Web server error log

Insert your webserver log here

ownCloud log (data/owncloud.log)

Insert your ownCloud log here

Browser log

Insert your browser log here, this could for example include:

a) The javascript console log
b) The network log 
c) ...

[J0HAN85]

Just upgrade to ownCloud 10.9 as well. I have the exact same issue.

[patrikhall]

I confirm same issue, upgraded from 10.8 to 10.9 om CentOS 7 with php 7.4.27.

[phil-davis]

I can browse to https://marketplace.owncloud.com/api/v1/platform/10.9.0/apps.json and see the JSON, similar to https://marketplace.owncloud.com/api/v1/platform/10.8.0/apps.json

Maybe the existence and protection of the marketplace end-point for 10.9.0 has been fixed now?

[vco1]

Browsing is not the problem, as is also stated in the initial post:

Double checked with wget and same user, this works and downloads the json files on that machine.

Opening from OC is. The error still exists here.

[vedatbotuk]

It works, when you create an Account on https://marketplace.owncloud.com.
You should add the API-Key in market in your owncloud web.

[vco1]

Adding the API-key seems like a workaround, not a solution.
A Marketplace account is for OC app developers, not for regular users. At least, according to the webpage.

[esackbauer]

It works, when you create an Account on https://marketplace.owncloud.com. You should add the API-Key in market in your owncloud web.

Why wasn't this mentioned in release notes of 10.9 as breaking change? I still have to consider it broken.

[hodyroff]

Sorry for this, Workaround seems to work. We will look into this in the next days after holiday period. Not a change by purpose.

[JammingBen]

The only way I see this happening is in case oC includes a wrong API key in the request to the marketplace. This key can either be defined in the config.php file via marketplace.key, or in the oc_appconfig table via key. Assuming your market app worked before the update, I see 2 different scenarios:

1. A wrong API key has somehow sneaked its way into the config during the update.
2. The key exists in the oc_appconfig table, but is set to null. This worked in the past (< 10.9), but due to Fix so that a null appconfig value is treated as an empty string #39554, it won't work anymore.

2 seems far more likely to me. It would also explain why this error is happening only to a few users and why this comes up with 10.9.

Since the market now works again for you, I assume there is no way you can reproduce this? It would be really helpful to see if one of these config values is set while this error is happening. Maybe someone runs into the same issue and can provide this information.

[darootler]

The only way I see this happening is in case oC includes a wrong API key in the request to the marketplace. This key can either be defined in the config.php file via marketplace.key, or in the oc_appconfig table via key. Assuming your market app worked before the update, I see 2 different scenarios:

1. A wrong API key has somehow sneaked its way into the config during the update.
2. The key exists in the oc_appconfig table, but is set to `null'. This worked in the past (< 10.9), but due to Fix so that a null appconfig value is treated as an empty string #39554, it won't work anymore.

2 seems far more likely to me. It would also explain why this error is happening only to a few users and why this comes up with 10.9.

Since the market now works again for you, I assume there is no way you can reproduce this? It would be really helpful to see if one of these config values is set while this error is happening. Maybe someone runs into the same issue and can provide this information.

Hi, i am hitting the same issue. I have not configured any api key within config.php and i don't see any api key in the table oc_appconfig.

Let me know if can help further.

Regards
Richard

[vco1]

@JammingBen:

1. I don't have a marketplace.key in any of my config files (not even in the sample ones).
2. Key does exist for appid market in the oc_appconfig table and is indeed set to NULL.

I don't understand why this explains that the error is only happening for "a few" (?) users. I have a standard installation.

As I actually have 2 OC instances, I also checked the other (more recent) one. It hasn't been updated to 10.9 yet and doesn't heva the key value in the database (nor in the config). So I assume deleting that row from the database should fix the issue?

[vco1]

I can confirm that this fixed the issue for me:

delete from oc_appconfig where appid = 'market' and configkey = 'key';

[JammingBen]

I don't understand why this explains that the error is only happening for "a few" (?) users. I have a standard installation.

As I actually have 2 OC instances, I also checked the other (more recent) one. It hasn't been updated to 10.9 yet and doesn't heva the key value in the database (nor in the config).

I (and many other users) ran through the update process several times, no issues whatsoever. Therefore I assume that only instances with this null value for the market key are affected. I don't know how the value got in there, but I don't think it's related to the update itself. I rather think it has been there for a long time, probably some action/process in the past set it.

Is updating your second oC instance currently an option?

So I assume deleting that row from the database should fix the issue?

As far as I can tell, yes.

@darootler

Hi, i am hitting the same issue. I have not configured any api key within config.php and i don't see any api key in the table oc_appconfig.

Okay, that's strange... So there is no entry for key in the table at all? Or is the value for key just empty/null?

[darootler]

@JammingBen

I don't see anything that looks lika an api key. Could you please post a sql query or the exact key name?

Regards
Richard

[vco1]

Could you please post a sql query or the exact key name?

select * from oc_appconfig where appid = 'market' and configkey = 'key';

[darootler]

Okay, my bad. Seems like the value is NULL:

Regards
Richard

[JammingBen]

For anyone who runs into this issue currently:

Please check if null is set for the market API key in the app config. You can do so by accessing your database (either via CLI or phpMyAdmin) and running the following command:

select * from oc_appconfig where appid = 'market' and configkey = 'key';

If this command returns the following:

+--------+-----------+-------------+
| appid  | configkey | configvalue |
+--------+-----------+-------------+
| market | key       | NULL        |
+--------+-----------+-------------+
1 row in set (0.000 sec)

Then simply remove this entry to solve the problem. This can be done in your database again via:

delete from oc_appconfig where appid = 'market' and configkey = 'key';

Post running this command, your market place will be fully available again.

---

Thank you all for your help! We will decide what to do with the change in behavior which was introduced by #39554 and provide a fix soon.

[darootler]

Okay, so that seems to be the problem indeed. To sum it up, for anyone who runs into this issue currently: please check if null is set for the market API key in the app config (select * from oc_appconfig where appid = 'market' and configkey = 'key';). If so, simply remove this value (delete from oc_appconfig where appid = 'market' and configkey = 'key';).

Thanks for your help! We will decide what to do with the change in behavior which was introduced by #39554 and provide a fix soon.

I can confirm that deleting the key via delete from oc_appconfig where appid = 'market' and configkey = 'key'; fixes the issue with the marketplace.

Thank you!

Regards
Richard

[phil-davis]

With the change in behavior, I guess that the code is now seeing that oc_appconfig setting as "" (empty string) rather than null and there is probably some test that checks !== null and so it thinks that there is a market key set, and uses the empty string as the key. Maybe such code should just check if the length of the string is > 0.

[JammingBen]

The issue has been fixed in the new marketplace app 0.6.2: https://github.com/owncloud/market/releases/tag/v0.6.2 -> closing here.