Issue while upgrading to 18 Beta1 (SQL, related to Flow) #18265

ArtificialOwl · 2019-12-06T10:50:57Z

Steps to reproduce

Install Nextcloud 17.0.1
Download Nextcloud 18-Beta1 from download.nextcloud.com, and unzip the file
Run the upgrade from the webclient.

Expected behaviour

Nextcloud should upgrade to 18-Beta1

Actual behaviour

SQL Issue:

Doctrine\DBAL\Exception\TableNotFoundException: An exception occurred while executing 'SELECT `o`.`id` FROM `oc_flow_operations` `o` 
LEFT JOIN `oc_flow_operations_scope` `s` ON `o`.`id` = `s`.`operation_id` 
WHERE `s`.`operation_id` IS NULL': 
SQLSTATE[42S02]: Base table or view not found: 1146 Table 'nextcloud_test.oc_flow_operations_scope' doesn't exist

Server configuration

Operating system:
Linux

Web server:
Apache

Database:
MariaDB

PHP version:
7.3.4

Nextcloud version: (see Nextcloud admin page)
17.0.1 -> 18-beta1

Updated from an older Nextcloud/ownCloud or fresh install:
fresh and clean install

The text was updated successfully, but these errors were encountered:

jospoortvliet · 2019-12-06T16:18:21Z

So my upgrade exploded completely in what looks like the same way...

2019-12-06T16:12:19+00:00 Updated database
2019-12-06T16:12:19+00:00 Updating <workflowengine> ...
2019-12-06T16:12:19+00:00 Repair step: Populating added database structures for workflows
2019-12-06T16:12:19+00:00 Doctrine\DBAL\Exception\DriverException: An exception occurred while executing 'UPDATE "oc_flow_operations" SET "entity" = ? WHERE "entity" = ''' with params ["OCA\\WorkflowEngine\\Entity\\File"]:

SQLSTATE[HY000]: General error: 1 no such column: entity
2019-12-06T16:12:19+00:00 Update failed
2019-12-06T16:12:19+00:00 Maintenance mode is kept active
2019-12-06T16:12:19+00:00 Reset log level

💣
I then tried to run the occ maintenance:repair command:

In AbstractSQLiteDriver.php line 70:
                                                                                                                                                               
  An exception occurred while executing 'UPDATE "oc_flow_operations" SET "entity" = ? WHERE "entity" = ''' with params ["OCA\\WorkflowEngine\\Entity\\File"]:  
                                                                                                                                                               
  SQLSTATE[HY000]: General error: 1 no such column: entity                                                                                                     

In PDOConnection.php line 63:
  SQLSTATE[HY000]: General error: 1 no such column: entity  
                                                            
In PDOConnection.php line 61:
  SQLSTATE[HY000]: General error: 1 no such column: entity

💥
I tried to disable the workflow-engine app:

sudo -u wwwrun php occ app:disable workflowengine
Nextcloud or one of the apps require upgrade - only a limited number of commands are available
You may use your browser or the occ upgrade command to do the upgrade
workflowengine can't be disabled

Now I'm stuck on a broken instance... Tips would be helpful ;-)

jospoortvliet · 2019-12-06T16:19:42Z

@blizzz as I see workflow here, I'm going to put the blame on you for now. Feel free to distribute or just blame it on the user 😬

This doesn't always go wrong, clearly, as my desktop at home updated properly, the laptop test instance did not howevah.

editedweb · 2019-12-08T01:37:27Z

I had the same issue. In order to keep using nextcloud, I moved the nextcloud installation to another folder, installed version 17.0.1, and then copied my config from my old nextcloud installation. I then had to reinstall a few apps that weren't included with the basic 17.0.1 installation.
Now that I have a working nextcloud again, I will have a look at the database and see what I can find. I'll report back if I work something out.

Edit: Having looked at my database, I can see that the oc_flow_operations table only includes the columns: 'id, class, name, checks, operation'.
I added a column named 'entity' of type varchar[256] with the default value of NULL, and the update seemed to work fine.

jospoortvliet · 2019-12-09T14:05:28Z

@cabaretboy good catch. @rullzer we should make sure this makes it into the next beta to avoid breaking instances...

hapee · 2019-12-11T14:32:14Z

the above is about oc_flow_operations table but in my case and the first one it is the oc_flow_operations_scope table:

In AbstractMySQLDriver.php line 42:
An exception occurred while executing 'SELECT o.id FROM oc_flow_operations o LEFT JOIN oc_flow_operations_scope s ON o.id = s.operation_id WHERE s.operation_id IS NULL':

SQLSTATE[42S02]: Base table or view not found: 1146 Table 'nextcloud.oc_flow_operations_scope' doesn't exist

In PDOConnection.php line 90:
SQLSTATE[42S02]: Base table or view not found: 1146 Table 'nextcloud.oc_flow_operations_scope' doesn't exist

In PDOConnection.php line 88:
SQLSTATE[42S02]: Base table or view not found: 1146 Table 'nextcloud.oc_flow_operations_scope' doesn't exist

maintenance:repair [--include-expensive]

Any idea how the oc_flow_operations_scope table should look like?

rullzer · 2019-12-11T15:24:37Z

So I'm not sure how this happens as every test I throw at it it works.

but @daita is running more checks. Hopefully he can find something.

hapee · 2019-12-11T16:31:19Z

I am also looking into it, installed a clean one on localhost, passed the same message but with a repair it worked and the upgrade as well. With the information on localhost I was able to create the _scope table and than run into the same problem as @cabaretboy had so manually edited the oc_flow_operations table and added the missing entity and events rows. If I run a maintenance:repair everything looks fine but when I do an upgrade, the upgrade deletes the entity and events again and fails and when I do an maintenance:repair again it fails as well:

Doctrine\DBAL\Exception\InvalidFieldNameException: An exception occurred while executing 'UPDATE oc_flow_operations SET entity = ? WHERE entity = ''' with params ["OCA\WorkflowEngine\Entity\File"]:
SQLSTATE[42S22]: Column not found: 1054 Unknown column 'entity' in 'where clause'
Update failed

rullzer · 2019-12-11T18:52:35Z

It is almost as if for some reason on your systems the Migration is not run.

Could one of your try with a debugger (or just stupidly with a var_dump or something) in https://github.com/nextcloud/server/blob/master/apps/workflowengine/lib/Migration/Version2000Date20190808074233.php#L25

bhessen · 2020-01-03T16:05:11Z

I have the same problem with v18 RC1.
I tried changing the database table and add a column 'entity' but this gets deleted every time I run the upgrade procedure.

It is almost as if for some reason on your systems the Migration is not run.

Could one of your try with a debugger (or just stupidly with a var_dump or something) in https://github.com/nextcloud/server/blob/master/apps/workflowengine/lib/Migration/Version2000Date20190808074233.php#L25

I looked at the code and noticed that in the function 'changeSchema(...)' after 'createTable('flow_operations')' there are only column entries for 'id', 'name', 'class', 'checks', 'operation'. But no 'entity'...

I tried add this column into the file, but still no column 'entity' in my database if I run it afterwards...

Sp1l · 2020-01-07T14:08:39Z

Looks like the autoloading of the class doesn't work

# su -m www -c "php ./occ migrations:status workflowengine"
Nextcloud or one of the apps require upgrade - only a limited number of commands are available
You may use your browser or the occ upgrade command to do the upgrade

In MigrationService.php line 453:
                                                                                          
  Migration step 'OCA\WorkflowEngine\Migration\Version2000Date20190808074233' is unknown  
                                                                                          

migrations:status <app>

Sp1l · 2020-01-07T14:24:06Z

Managed to continue after commenting

server/apps/workflowengine/lib/Migration/PopulateNewlyIntroducedDatabaseFields.php

Line 60 in c7d5b8f

$qb->update('flow_operations')

Database is empty after creation, updater fails on this statement.

blizzz · 2020-01-08T14:29:55Z

I tried to reproduce it today, but to no avail. I set up latest 17 and did the upgrade routine against a DB without any entries in the flow tables, and another attempt with a flow rule. Both went through flawlessly. Same on upgrades of both the private instances of @juliushaertl and me.

blizzz · 2020-01-08T15:06:14Z

I looked at the code and noticed that in the function 'changeSchema(...)' after 'createTable('flow_operations')' there are only column entries for 'id', 'name', 'class', 'checks', 'operation'. But no 'entity'...

it's added with ensureEntityColumns which is called afterwards.

blizzz · 2020-01-08T15:11:46Z

@Sp1l in your case apps weren't loaded as the upgrade procedure was due, therefore the class could not be found indeed. This is expected (like in maintenance mode).

Sp1l · 2020-01-08T20:28:15Z

@Sp1l in your case apps weren't loaded as the upgrade procedure was due, therefore the class could not be found indeed. This is expected (like in maintenance mode).

When upgrading from 17.0.2 to 18 aren't you supposed to run the upgrade procedure?
Was there some condition that pushed it into maintenance mode that caused this?
Are any additional upgrade steps required?

Just want to prevent the "world" falling on top of me when I update the FreeBSD package...
I can add a message to alert users of additional steps.

blizzz · 2020-01-09T11:20:34Z

@Sp1l no, when in maintenance mode, or the upgrade procedure is due, the apps are not being loaded, until either is resolved. Therefore, when the upgrade routine is pending, occ migrations:status workflowengine cannot resolve anything that belongs to workflowengine, as it is also implemented as an app.

gbip · 2020-01-17T16:30:05Z

I just ran into the same issue, upgrading from Nextcloud 17.0.1 on FreeBSD to Nextcloud 18.
I copied the Nextcloud 18 files to my nextcloud folder, and launched the upgrade process which failed with the following error :

Doctrine\DBAL\Exception\TableNotFoundException: An exception occurred while executing 'SELECT `o`.`id` FROM `oc_flow_operations` `o` LEFT JOIN `oc_flow_operations_scope` `s` ON `o`.`id` = `s`.`operation_id` WHERE `s`.`operation_id` IS NULL':

SQLSTATE[42S02]: Base table or view not found: 1146 Table 'nextcloud.oc_flow_operations_scope' doesn't exist
Update failed

tgurr · 2020-01-17T17:17:10Z

I just ran into the same like @gbip posted above, I then cleaned my htdocs and only kept config, data and apps that don't come with nextcloud by default in the apps folder, afterwards the occ upgrade went through smooth so I suspect some files that came with the previous version causing the error when still being around.

tasagore · 2020-01-17T20:03:31Z

Same here, unable to upgrade from 17.0.2 to 18.0.0, error in the oc_flow_operations_scope.

ivdok · 2020-01-17T20:43:53Z

Ditto with nextcloud-18.0.0 (MD5: 4ce527afe81d08aeaa47c40bae2e1fdd), upgrade 17.0.2 => 18.0.0 fails with following logs on Debian 10 + nginx setup.

kesselb · 2020-01-17T21:19:37Z

If you have the same issue please use the issue template https://github.com/nextcloud/server/blob/master/.github/ISSUE_TEMPLATE/Bug_report.md and add a comment to this thread.

It's fine to skip expected behaviour, steps to reproduce, etc. but we need the basic system information, configuration export and logs (if they look helpful). I debugged a issue with multiple apps_paths recently. Maybe that's related and the information (multiple apps_paths yes/no) important (at least to me).

cc @ivdok @tasagore If possible please edit your posts and also add it. Thanks 👍

kesselb · 2020-03-07T12:39:25Z

@chkpnt That's the recommended solution: https://help.nextcloud.com/t/read-this-before-you-update-to-nextcloud-18/68389

If you run into the missing column problem: The current status is that this issue appears if an unsupported way of upgrading your instance is used.

For registerRuleListeners warning: #19818

chkpnt · 2020-03-07T20:16:35Z

@kesselb Thank you, that indeed worked.

therealbstern · 2020-03-08T08:24:51Z

@kesselb Thanks, I added a +1 to that issue.

vladaman · 2020-03-14T20:10:37Z

I am having same issue. I suspect some of the table updates were not performed during previous upgrades.
I was updating from version 16, then 17 and then 18 one after the other, without even running nextcloud. So I suspect something was not done during those updates.

Funny thing is, that when I go to MySQL table directly and I add column entity manually and then I execute Update procedure, this column gets dropped. And the updater will throw an error that column entity is missing.

ALTER TABLE `oc_flow_operations` ADD `entity` VARCHAR(255) NOT NULL ;

This relates to upgrade to 18.0.2

tonyshangrila · 2020-03-19T02:55:03Z

@chkpnt That's the recommended solution: https://help.nextcloud.com/t/read-this-before-you-update-to-nextcloud-18/68389

If you run into the missing column problem: The current status is that this issue appears if an unsupported way of upgrading your instance is used.

For registerRuleListeners warning: #19818

I was grateful to find this thread, but for me, running a manual upgrade threw the exact same error:
Doctrine\DBAL\Exception\InvalidFieldNameException: An exception occurred while executing 'UPDATE oc_flow_operations SET entity = ? WHERE entity = ''' with params ["OCA\WorkflowEngine\Entity\File"]:

SQLSTATE[42S22]: Column not found: 1054 Unknown column 'entity' in 'where clause'
Update failed

Going back to restore my 17.0.4 instance...

EDIT: Despite noting in this thread and others that manually adding the missing column may not work, gave it a go anyway, per MagelanM below. This workaround allowed me to upgrade, and with the web interface, no less.

MagelanM · 2020-03-19T22:06:16Z

I had the same problem while update to Nextcloud 18.0.2.
The update of the app workflowengine hangs.
The solution for me was to log into the database an add the column manually.
Steps for MariaDB:

Connect over ssh with user root
ssh <IP-Adress> -l root
Open database
mysql -uroot -p (use PW of user root)
Open database:
connect nextcloud
Check table oc_flow_operations, if column entity is present or not
describe oc_flow_operations;
If not present, add the column
alter table oc_flow_operations add entity varchar(256) default NULL;
After adding the column, rerun the completition of the update.

therealbstern · 2020-03-20T20:40:29Z

@therealbstern Most people fail at step 9. Please make sure to not replace the workflowengine app with the new one.

I'm pretty sure when I did the manual update, occ went ahead and downloaded the new version automatically, so I didn't copy anything over. It's been long enough that I don't remember for sure, so I'll try an update to 18 shortly and report back.

blizzz · 2020-03-24T15:30:45Z

@MagelanM @vladaman @tonyshangrila did you all follow the upgrade instructions and ensured that the new files were not simply copied over the existing ones?

tonyshangrila · 2020-03-24T15:35:40Z

@MagelanM @vladaman @tonyshangrila did you all follow the upgrade instructions and ensured that the new files were not simply copied over the existing ones?

Yes, I did.

blizzz · 2020-03-25T10:23:41Z

@tonyshangrila can you provide your nextcloud.log from your upgrade attempt?

tonyshangrila · 2020-03-25T20:14:39Z

@blizzz - I'm sorry to say that I can't. :(

Upon taking a look at my logs, I noticed an entirely different SQL error that been thrown 2 or 3 times a second since I upgraded to 18. My log effectively gets flushed at 200 MB, and I don't have anything dated prior to 3/24 now.

Off to chase down that issue!

MichaIng · 2020-04-15T19:07:23Z

Just to mention it, I had an error during NC18 upgrade about the same database table: #18327
But it didn't lead to any broken instance or followup issues until now, so not sure if those are related at some point.

therealbstern · 2020-07-29T02:29:47Z

@kesselb I just tried again, 17.0.6 -> 18.0.7 in preparation for upgrading to 19, and the exact same thing happened. I followed step 9 in https://docs.nextcloud.com/server/18/admin_manual/maintenance/manual_upgrade.html to a T (i.e., didn't update the workflowengine myself).

This time, once I issued:

ALTER TABLE oc_flow_operations ADD COLUMN entity CHARACTER VARYING(256) NOT NULL DEFAULT ''

and re-running php occ upgrade, everything appeared to work (though my server stayed in maintenance mode).

So something must have changed in either 17 or 18 between now and then, but at least there is an upgrade path, even if it's a little rocky.

(ETA: I'm running Postgres.)

arno01 · 2020-10-06T15:52:30Z

Here is my experience with upgrading Nextcloud from 17.0.9.2 to 18.0.9.1 (in docker). But with Postgresql 12.
It went almost smoothly.

# grep image: /srv/docker-compose.yml | grep nextcloud
    image: nextcloud:17-apache

# docker-compose stop nextcloud
# docker-compose exec -u postgres postgres12 pg_dump nextcloud > nextcloud-17.0.9.db
# rsync -avHx --progress /srv/data/nextcloud /srv/data/nextcloud-17.0.9.bkp

# vim /srv/docker-compose.yml

# grep image: /srv/docker-compose.yml | grep nextcloud
    image: nextcloud:18-apache

# docker-compose pull nextcloud
# docker-compose up -d nextcloud

# docker-compose logs -f --tail=222 nextcloud
Attaching to srv_nextcloud_1
nextcloud_1         | Initializing nextcloud 18.0.9.1 ...
nextcloud_1         | Upgrading nextcloud from 17.0.9.2 ...
nextcloud_1         | Initializing finished
nextcloud_1         | Nextcloud or one of the apps require upgrade - only a limited number of commands are available
nextcloud_1         | You may use your browser or the occ upgrade command to do the upgrade
nextcloud_1         | Set log level to debug
nextcloud_1         | Turned on maintenance mode
nextcloud_1         | Updating database schema
nextcloud_1         | Updated database
nextcloud_1         | Disabled incompatible app: gallery
nextcloud_1         | Updating <accessibility> ...
nextcloud_1         | Migrate old user config
nextcloud_1         | 
nextcloud_1         |  Starting ...
nextcloud_1         |     0/0 [>---------------------------]   0%
nextcloud_1         |  Starting ...
nextcloud_1         | 
nextcloud_1         | Updated <accessibility> to 1.4.0
nextcloud_1         | Updating <federatedfilesharing> ...
nextcloud_1         | Updated <federatedfilesharing> to 1.8.0
nextcloud_1         | Updating <files_pdfviewer> ...
nextcloud_1         | Updated <files_pdfviewer> to 1.7.0
nextcloud_1         | Updating <files_rightclick> ...
nextcloud_1         | Updated <files_rightclick> to 0.15.2
nextcloud_1         | Updating <files_videoplayer> ...
nextcloud_1         | Updated <files_videoplayer> to 1.7.0
nextcloud_1         | Updating <logreader> ...
nextcloud_1         | Updated <logreader> to 2.3.0
nextcloud_1         | Updating <password_policy> ...
nextcloud_1         | Updated <password_policy> to 1.8.0
nextcloud_1         | Updating <privacy> ...
nextcloud_1         | Updated <privacy> to 1.2.0
nextcloud_1         | Updating <provisioning_api> ...
nextcloud_1         | Updated <provisioning_api> to 1.8.0
nextcloud_1         | Updating <recommendations> ...
nextcloud_1         | Updated <recommendations> to 0.6.0
nextcloud_1         | Updating <serverinfo> ...
nextcloud_1         | Updated <serverinfo> to 1.8.0
nextcloud_1         | Updating <support> ...
nextcloud_1         | Updated <support> to 1.1.1
nextcloud_1         | Updating <survey_client> ...
nextcloud_1         | Updated <survey_client> to 1.6.0
nextcloud_1         | Updating <text> ...
nextcloud_1         | Updated <text> to 2.0.0
nextcloud_1         | Updating <twofactor_backupcodes> ...
nextcloud_1         | Updated <twofactor_backupcodes> to 1.7.0
nextcloud_1         | Updating <updatenotification> ...
nextcloud_1         | Updated <updatenotification> to 1.8.0
nextcloud_1         | Updating <federation> ...
nextcloud_1         | Updated <federation> to 1.8.0
nextcloud_1         | Updating <lookup_server_connector> ...
nextcloud_1         | Updated <lookup_server_connector> to 1.6.0
nextcloud_1         | Updating <oauth2> ...
nextcloud_1         | Updated <oauth2> to 1.6.0
nextcloud_1         | Updating <files> ...
nextcloud_1         | Updated <files> to 1.13.1
nextcloud_1         | Updating <activity> ...
nextcloud_1         | Updated <activity> to 2.11.0
nextcloud_1         | Updating <cloud_federation_api> ...
nextcloud_1         | Updated <cloud_federation_api> to 1.1.0
nextcloud_1         | Updating <dav> ...
nextcloud_1         | Fix broken values of calendar objects
nextcloud_1         | 
nextcloud_1         |     1/0 [->--------------------------]   0% Starting ...
nextcloud_1         | 
nextcloud_1         | Updated <dav> to 1.14.0
nextcloud_1         | Updating <files_external> ...
nextcloud_1         | Updated <files_external> to 1.9.0
nextcloud_1         | Updating <files_sharing> ...
nextcloud_1         | Updated <files_sharing> to 1.10.1
nextcloud_1         | Updating <files_trashbin> ...
nextcloud_1         | Updated <files_trashbin> to 1.8.0
nextcloud_1         | Updating <files_versions> ...
nextcloud_1         | Updated <files_versions> to 1.11.0
nextcloud_1         | Updating <sharebymail> ...
nextcloud_1         | Updated <sharebymail> to 1.8.0
nextcloud_1         | Updating <workflowengine> ...
nextcloud_1         | Updated <workflowengine> to 2.0.0
nextcloud_1         | Updating <comments> ...
nextcloud_1         | Updated <comments> to 1.8.0
nextcloud_1         | Updating <firstrunwizard> ...
nextcloud_1         | Updated <firstrunwizard> to 2.7.0
nextcloud_1         | Updating <nextcloud_announcements> ...
nextcloud_1         | Updated <nextcloud_announcements> to 1.7.0
nextcloud_1         | Updating <notifications> ...
nextcloud_1         | Updated <notifications> to 2.6.0
nextcloud_1         | Updating <systemtags> ...
nextcloud_1         | Updated <systemtags> to 1.8.0
nextcloud_1         | Updating <theming> ...
nextcloud_1         | Updated <theming> to 1.9.0
nextcloud_1         | Checking for update of app accessibility in appstore
nextcloud_1         | Checked for update of app "accessibility" in appstore 
nextcloud_1         | Checking for update of app activity in appstore
nextcloud_1         | Checked for update of app "activity" in appstore 
nextcloud_1         | Checking for update of app bruteforcesettings in appstore
nextcloud_1         | Update app bruteforcesettings from appstore
nextcloud_1         | Checked for update of app "bruteforcesettings" in appstore 
nextcloud_1         | Checking for update of app calendar in appstore
nextcloud_1         | Checked for update of app "calendar" in appstore 
nextcloud_1         | Checking for update of app cloud_federation_api in appstore
nextcloud_1         | Checked for update of app "cloud_federation_api" in appstore 
nextcloud_1         | Checking for update of app comments in appstore
nextcloud_1         | Checked for update of app "comments" in appstore 
nextcloud_1         | Checking for update of app contacts in appstore
nextcloud_1         | Update app contacts from appstore
nextcloud_1         | Checked for update of app "contacts" in appstore 
nextcloud_1         | Checking for update of app dav in appstore
nextcloud_1         | Checked for update of app "dav" in appstore 
nextcloud_1         | Checking for update of app federatedfilesharing in appstore
nextcloud_1         | Checked for update of app "federatedfilesharing" in appstore 
nextcloud_1         | Checking for update of app federation in appstore
nextcloud_1         | Checked for update of app "federation" in appstore 
nextcloud_1         | Checking for update of app files in appstore
nextcloud_1         | Checked for update of app "files" in appstore 
nextcloud_1         | Checking for update of app files_external in appstore
nextcloud_1         | Checked for update of app "files_external" in appstore 
nextcloud_1         | Checking for update of app files_pdfviewer in appstore
nextcloud_1         | Checked for update of app "files_pdfviewer" in appstore 
nextcloud_1         | Checking for update of app files_rightclick in appstore
nextcloud_1         | Checked for update of app "files_rightclick" in appstore 
nextcloud_1         | Checking for update of app files_sharing in appstore
nextcloud_1         | Checked for update of app "files_sharing" in appstore 
nextcloud_1         | Checking for update of app files_trashbin in appstore
nextcloud_1         | Checked for update of app "files_trashbin" in appstore 
nextcloud_1         | Checking for update of app files_versions in appstore
nextcloud_1         | Checked for update of app "files_versions" in appstore 
nextcloud_1         | Checking for update of app files_videoplayer in appstore
nextcloud_1         | Checked for update of app "files_videoplayer" in appstore 
nextcloud_1         | Checking for update of app firstrunwizard in appstore
nextcloud_1         | Checked for update of app "firstrunwizard" in appstore 
nextcloud_1         | Checking for update of app logreader in appstore
nextcloud_1         | Checked for update of app "logreader" in appstore 
nextcloud_1         | Checking for update of app lookup_server_connector in appstore
nextcloud_1         | Checked for update of app "lookup_server_connector" in appstore 
nextcloud_1         | Checking for update of app nextcloud_announcements in appstore
nextcloud_1         | Checked for update of app "nextcloud_announcements" in appstore 
nextcloud_1         | Checking for update of app notifications in appstore
nextcloud_1         | Checked for update of app "notifications" in appstore 
nextcloud_1         | Checking for update of app oauth2 in appstore
nextcloud_1         | Checked for update of app "oauth2" in appstore 
nextcloud_1         | Checking for update of app password_policy in appstore
nextcloud_1         | Checked for update of app "password_policy" in appstore 
nextcloud_1         | Checking for update of app privacy in appstore
nextcloud_1         | Checked for update of app "privacy" in appstore 
nextcloud_1         | Checking for update of app provisioning_api in appstore
nextcloud_1         | Checked for update of app "provisioning_api" in appstore 
nextcloud_1         | Checking for update of app recommendations in appstore
nextcloud_1         | Checked for update of app "recommendations" in appstore 
nextcloud_1         | Checking for update of app serverinfo in appstore
nextcloud_1         | Checked for update of app "serverinfo" in appstore 
nextcloud_1         | Checking for update of app settings in appstore
nextcloud_1         | Checked for update of app "settings" in appstore 
nextcloud_1         | Checking for update of app sharebymail in appstore
nextcloud_1         | Checked for update of app "sharebymail" in appstore 
nextcloud_1         | Checking for update of app support in appstore
nextcloud_1         | Checked for update of app "support" in appstore 
nextcloud_1         | Checking for update of app survey_client in appstore
nextcloud_1         | Checked for update of app "survey_client" in appstore 
nextcloud_1         | Checking for update of app systemtags in appstore
nextcloud_1         | Checked for update of app "systemtags" in appstore 
nextcloud_1         | Checking for update of app text in appstore
nextcloud_1         | Checked for update of app "text" in appstore 
nextcloud_1         | Checking for update of app theming in appstore
nextcloud_1         | Checked for update of app "theming" in appstore 
nextcloud_1         | Checking for update of app twofactor_backupcodes in appstore
nextcloud_1         | Checked for update of app "twofactor_backupcodes" in appstore 
nextcloud_1         | Checking for update of app twofactor_totp in appstore
nextcloud_1         | Update app twofactor_totp from appstore
nextcloud_1         | Checked for update of app "twofactor_totp" in appstore 
nextcloud_1         | Checking for update of app updatenotification in appstore
nextcloud_1         | Checked for update of app "updatenotification" in appstore 
nextcloud_1         | Checking for update of app viewer in appstore
nextcloud_1         | Checked for update of app "viewer" in appstore 
nextcloud_1         | Checking for update of app workflowengine in appstore
nextcloud_1         | Checked for update of app "workflowengine" in appstore 
nextcloud_1         | Checking for update of app gallery in appstore
nextcloud_1         | Checked for update of app "gallery" in appstore 
nextcloud_1         | Starting code integrity check...
nextcloud_1         | Finished code integrity check
nextcloud_1         | Update successful
nextcloud_1         | Turned off maintenance mode
nextcloud_1         | Reset log level
nextcloud_1         | The following apps have been disabled:
nextcloud_1         |  accessibility
nextcloud_1         |  activity
nextcloud_1         |  bruteforcesettings
nextcloud_1         |  cloud_federation_api
nextcloud_1         |  comments
nextcloud_1         |  contacts
nextcloud_1         |  dav
nextcloud_1         |  federatedfilesharing
nextcloud_1         |  federation
nextcloud_1         |  files
nextcloud_1         |  files_external
nextcloud_1         |  files_pdfviewer
nextcloud_1         |  files_rightclick
nextcloud_1         |  files_sharing
nextcloud_1         |  files_trashbin
nextcloud_1         |  files_versions
nextcloud_1         |  files_videoplayer
nextcloud_1         |  firstrunwizard
nextcloud_1         |  gallery
nextcloud_1         |  logreader
nextcloud_1         |  lookup_server_connector
nextcloud_1         |  nextcloud_announcements
nextcloud_1         |  notifications
nextcloud_1         |  oauth2
nextcloud_1         |  password_policy
nextcloud_1         |  privacy
nextcloud_1         |  provisioning_api
nextcloud_1         |  recommendations
nextcloud_1         |  serverinfo
nextcloud_1         |  sharebymail
nextcloud_1         |  support
nextcloud_1         |  survey_client
nextcloud_1         |  systemtags
nextcloud_1         |  text
nextcloud_1         |  theming
nextcloud_1         |  twofactor_backupcodes
nextcloud_1         |  twofactor_totp
nextcloud_1         |  updatenotification
nextcloud_1         |  workflowengine
nextcloud_1         |     0/0 [>---------------------------]   0%

Go to https://mynextcloudhost/settings/apps and make sure the Calendar & Contact apps are enabled & updated.

Then look at the Administration tab for the Security & setup warnings https://mynextcloudhost/settings/admin

The database is missing some indexes. Due to the fact that adding indexes on big tables could take some time they were not added automatically. By running "occ db:add-missing-indices" those missing indexes could be added manually while the instance keeps running. Once the indexes are added queries to those tables are usually much faster.

- Missing index "calendarobject_calid_index" in table "oc_calendarobjects_props".
- Missing index "schedulobj_principuri_index" in table "oc_schedulingobjects".

Fix:

# docker-compose exec -u www-data nextcloud php occ db:add-missing-indices
Check indices of the share table.
Check indices of the filecache table.
Check indices of the twofactor_providers table.
Check indices of the login_flow_v2 table.
Check indices of the whats_new table.
Check indices of the cards table.
Check indices of the cards_properties table.
Check indices of the calendarobjects_props table.
Adding calendarobject_calid_index index to the calendarobjects_props table, this can take some time...
calendarobjects_props table updated successfully.
Check indices of the schedulingobjects table.
Adding schedulobj_principuri_index index to the schedulingobjects table, this can take some time...
schedulingobjects table updated successfully.

MichaIng · 2020-10-06T16:12:36Z

Jep that is expected. The admin panel would have explained that missing indices as well as missing columns are not added automatically by the updater, as this can be very time consuming. So the following two steps can actually always be appended, when the possible additional offline/maintenance time is okay:

occ db:add-missing-columns
occ db:add-missing-indices

including the sudo/docker/PHP prefix of course.

Maingron · 2020-10-10T17:18:48Z

#18265 (comment) this one worked for me, but I had to create varchar(256) (not varchar[256]) using SQlite.

ALTER TABLE oc_flow_operations
ADD entity varchar(256) DEFAULT NULL;

Tie-fighter · 2020-10-10T18:05:13Z

I have the same issue upgrading from 19.0.4.2 to 20.0.0.9

I have the column

dbadmin@localhost [nextcloud]> DESCRIBE oc_flow_operations;
+-----------+--------------+------+-----+---------+----------------+
| Field     | Type         | Null | Key | Default | Extra          |
+-----------+--------------+------+-----+---------+----------------+
| id        | int(11)      | NO   | PRI | NULL    | auto_increment |
| class     | varchar(256) | NO   |     | NULL    |                |
| name      | varchar(256) | NO   |     | NULL    |                |
| checks    | longtext     | YES  |     | NULL    |                |
| operation | longtext     | YES  |     | NULL    |                |
| entity    | varchar(256) | YES  |     | NULL    |                |
+-----------+--------------+------+-----+---------+----------------+
6 rows in set (0.00 sec)

but I still receive the error :/

Doctrine\DBAL\Exception\InvalidFieldNameException: An exception occurred while executing 'UPDATE `oc_flow_operations` SET `entity` = ? WHERE `entity` = ''' with params ["OCA\\WorkflowEngine\\Entity\\File"]:

SQLSTATE[42S22]: Column not found: 1054 Unknown column 'entity' in 'where clause'
Update failed
Maintenance mode is kept active
Resetting log level
root@nextcloud:~ #

PS: This #19058 (comment) and #19058 (comment) resolved my issues.

nordurljosahvida · 2020-11-14T23:26:20Z

Hi, I too have had to manually add the entity column in mariadb, now however I get multiple of these errors a second:

Doctrine\DBAL\Exception\InvalidFieldNameException: An exception occurred while executing 'SELECT `class`, `entity`, `events` FROM `oc_flow_operations` WHERE `events` <> ? GROUP BY `class`, `entity`, `events`' with params ["[]"]: SQLSTATE[42S22]: Column not found: 1054 Unknown column 'events' in 'field list'

Already ran repair, add-missing-indices and add-missing-columns.

Everything seems to be working, I can only see the error from the logs.

Thanks

MichaIng · 2020-11-17T13:17:19Z

Is apps/workflowengine/appinfo/databases.xml present in your case? If so, remove it.
Although the "events" field should be there, at least it is in my case. Can you check your database for it?

mysql -e 'describe nextcloud.oc_flow_operations'

-u root p might be required for authentication.

bjornfor · 2020-12-10T19:24:47Z

Delete apps/workflowengine/appinfo/database.xml before running ./occ upgrade

+1. Fixed it for me (17.0.10 -> 18.0.12 update). (Thanks @kesselb!)

szaimen · 2021-06-18T13:28:54Z

Is this Issue still valid in NC21.0.2? If not, please close this issue. Thanks! :)

ArtificialOwl added bug 0. Needs triage Pending check for reproducibility or if it fits our roadmap labels Dec 6, 2019

jospoortvliet added 1. to develop Accepted and waiting to be taken care of 18-feedback feature: install and update and removed 0. Needs triage Pending check for reproducibility or if it fits our roadmap labels Dec 22, 2019

This comment has been minimized.

Sign in to view

KaiFra mentioned this issue Mar 6, 2020

NC Webupdater failed from 17.0.3 to 18.0.1, no root access #19818

Closed

This comment has been minimized.

Sign in to view

aol-nnov mentioned this issue Mar 22, 2020

upgrade to 18.0.2 failing with error #20097

Closed

This was referenced Apr 13, 2020

Updating from 17.0.5.0 to 18.0.3 failed #20440

Closed

[18.0.0 Beta1] [Update] InvalidFieldNameException in oc_flow_operations during update/repair #18327

Closed

nickvergessen removed the 18-feedback label Jul 10, 2020

kesselb mentioned this issue Aug 9, 2020

Update 17->18 Fail! How to get back? #22160

Closed

szaimen added the needs info label Jun 18, 2021

ArtificialOwl closed this as completed Jun 18, 2021

Issue while upgrading to 18 Beta1 (SQL, related to Flow) #18265

Issue while upgrading to 18 Beta1 (SQL, related to Flow) #18265

Comments

ArtificialOwl commented Dec 6, 2019

Steps to reproduce

Expected behaviour

Actual behaviour

Server configuration

jospoortvliet commented Dec 6, 2019 • edited Loading

jospoortvliet commented Dec 6, 2019 • edited Loading

editedweb commented Dec 8, 2019 • edited Loading

jospoortvliet commented Dec 9, 2019

hapee commented Dec 11, 2019

rullzer commented Dec 11, 2019

hapee commented Dec 11, 2019

rullzer commented Dec 11, 2019

bhessen commented Jan 3, 2020

Sp1l commented Jan 7, 2020

Sp1l commented Jan 7, 2020

blizzz commented Jan 8, 2020 • edited Loading

blizzz commented Jan 8, 2020

blizzz commented Jan 8, 2020

Sp1l commented Jan 8, 2020

blizzz commented Jan 9, 2020

gbip commented Jan 17, 2020

tgurr commented Jan 17, 2020

tasagore commented Jan 17, 2020

ivdok commented Jan 17, 2020 • edited Loading

kesselb commented Jan 17, 2020 • edited Loading

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

kesselb commented Mar 7, 2020

chkpnt commented Mar 7, 2020

therealbstern commented Mar 8, 2020

vladaman commented Mar 14, 2020 • edited Loading

tonyshangrila commented Mar 19, 2020 • edited Loading

MagelanM commented Mar 19, 2020

therealbstern commented Mar 20, 2020

blizzz commented Mar 24, 2020

tonyshangrila commented Mar 24, 2020

blizzz commented Mar 25, 2020

tonyshangrila commented Mar 25, 2020

MichaIng commented Apr 15, 2020

therealbstern commented Jul 29, 2020 • edited Loading

arno01 commented Oct 6, 2020

MichaIng commented Oct 6, 2020

Maingron commented Oct 10, 2020

Tie-fighter commented Oct 10, 2020 • edited Loading

nordurljosahvida commented Nov 14, 2020

MichaIng commented Nov 17, 2020 • edited Loading

bjornfor commented Dec 10, 2020 • edited Loading

szaimen commented Jun 18, 2021

jospoortvliet commented Dec 6, 2019 •

edited

Loading

jospoortvliet commented Dec 6, 2019 •

edited

Loading

editedweb commented Dec 8, 2019 •

edited

Loading

blizzz commented Jan 8, 2020 •

edited

Loading

ivdok commented Jan 17, 2020 •

edited

Loading

kesselb commented Jan 17, 2020 •

edited

Loading

vladaman commented Mar 14, 2020 •

edited

Loading

tonyshangrila commented Mar 19, 2020 •

edited

Loading

therealbstern commented Jul 29, 2020 •

edited

Loading

Tie-fighter commented Oct 10, 2020 •

edited

Loading

MichaIng commented Nov 17, 2020 •

edited

Loading

bjornfor commented Dec 10, 2020 •

edited

Loading