[Windows] Drush9 updb fails: unable to decode output into JSON

[sunlix]

Describe the bug
I am upgrading my Drupal 8.7 site to Drupal 8.8.
On Windows using mintty/cygwin env (as part of git for windows).
Performing drush updb -y all updates were performed. By finishing the update process, drush aborted abnormally.

To Reproduce

• Install drupal 8.7.10
• upgrade code base to drupal 8.8.0 using composer
• try drush updb -y (On Windows 10)

Expected behavior
drush updb should finish the database updates an exit normally without an error.

Actual behavior
It seems like the database updates were performed normally. Only the output can't decode correctly into JSON due to a syntax error.
There are unescaped quotes at the update messages.

"8803": {
    "results": {
        "query": "The "path_alias" entity type has been installed.",
        "success": true
    },
    "type": "update"
},

FULL OUTPUT

$ drush updb -y
 ----------------------------- ----------- --------------- -------------------------------------------------------------
  Module                        Update ID   Type            Description                                             
 ----------------------------- ----------- --------------- -------------------------------------------------------------
  system                        8801        hook_update_n   Remove 'path.temporary' config if redundant.            
  system                        8802        hook_update_n   Fix system.theme:admin when the default theme is used as
                                                            the admin theme.                                        
  system                        8803        hook_update_n   Install the 'path_alias' entity type.                   
  system                        8804        hook_update_n   Convert path aliases to entities.                       
  system                        8805        hook_update_n   Change the provider of the 'path_alias' entity type and its
                                                            base fields.                                            
  layout_builder_restrictions   8202        hook_update_n   Relocate all block_content configuration settings under 
                                                            "Custom blocks".                                        
  lightning_layout              8011        hook_update_n   Installs the layout_builder_styles module.              
  locale                        8800        hook_update_n   Clear Locale project storage to use new 'all' instead of
                                                            8.x in URLs.                                            
  media_library                 8703        hook_update_n   This update has been removed and will not run.          
  media_library                 8704        hook_update_n   Creates the media_library.settings config object.       
  content_moderation            views_fi    post-update     Update the moderation state views field plugin ID.      
                                eld_plug                                                                            
                                in_id                                                                               
  field_group                   0001        post-update     Assign a region to Field Groups.                        
  media_library                 add_butt    post-update     Add edit and delete button to media library view page   
                                ons_to_p                    display.                                                
                                age_view                                                                            
  media_library                 add_lang    post-update     Add langcode filters to media library view displays.    
                                code_fil                                                                            
                                ters                                                                                
  media_library                 add_stat    post-update     Add a status extra filter to the media library view default
                                us_extra                    display.                                                
                                _filter                                                                             
  media_library                 default_    post-update     Sets admincontentmedia to the table display of the 'media'
                                administ                    view.                                                   
                                rative_l                                                                            
                                ist_to_t                                                                            
                                able_dis                                                                            
                                play                                                                                
  media_library                 update_8    post-update     Add non js prefixed classes to checkboxes if not present.
                                001_chec                    Note the inclusion of "update_8001" in the function name.
                                kbox_cla                    This ensures the  function is executed after            
                                sses                        media_library_post_update_table_display(), as           
                                                            hook_post_update_NAME() implementations within the same 
                                                            file are executed in  alphanumeric order.               
  media                         add_stat    post-update     Add a status extra filter to the media view default     
                                us_extra                    display.                                                
                                _filter                                                                             
  path                          create_l    post-update     Create the language content settings configuration object
                                anguage_                    for path aliases.                                       
                                content_                                                                            
                                settings                                                                            
  system                        entity_r    post-update     Populate the new 'match_limit' setting for the ER       
                                eference                    autocomplete widget.                                    
                                _autocom                                                                            
                                plete_ma                                                                            
                                tch_limi                                                                            
                                t                                                                                   
  system                        layout_p    post-update     Clear the schema cache.                                 
                                lugin_sc                                                                            
                                hema_cha                                                                            
                                nge                                                                                 
  taxonomy                      configur    post-update     Add status with settings to all form displays for taxonomy
                                e_status                    entities.                                               
                                _field_w                                                                            
                                idget                                                                               
  text                          add_requ    post-update     Update text_with_summary fields and widgets to add summary
                                ired_sum                    required flags.                                         
                                mary_fla                                                                            
                                g                                                                                   
  views                         limit_op    post-update     Define default values for limit operators settings in all
                                erator_d                    filters.                                                
                                efaults                                                                             
  views                         remove_c    post-update     Remove core key from views configuration.               
                                ore_key                                                                             
 ----------------------------- ----------- --------------- -------------------------------------------------------------


 // Do you wish to run the specified pending updates?: yes.

>  [notice] Update started: system_update_8801
>  [notice] Update completed: system_update_8801
>  [notice] Update started: system_update_8802
>  [notice] Update completed: system_update_8802
>  [notice] Update started: system_update_8803
>  [notice] The "path_alias" entity type has been installed.
>  [notice] Update completed: system_update_8803
>  [notice] Update started: system_update_8804
>  [notice] Path aliases have been converted to entities.
>  [notice] Update completed: system_update_8804
>  [notice] Update started: media_library_update_8703
>  [notice] Update completed: media_library_update_8703
>  [notice] Update started: system_update_8805
>  [notice] Update completed: system_update_8805
>  [notice] Update started: layout_builder_restrictions_update_8202
>  [notice] Update completed: layout_builder_restrictions_update_8202
>  [notice] Update started: lightning_layout_update_8011
>  [notice] Update completed: lightning_layout_update_8011
>  [notice] Update started: locale_update_8800
>  [notice] Update completed: locale_update_8800
>  [notice] Update started: media_library_update_8704
>  [notice] Update completed: media_library_update_8704
>  [notice] Update started: content_moderation_post_update_views_field_plugin_id
>  [notice] Update completed: content_moderation_post_update_views_field_plugin_id
>  [notice] Update started: field_group_post_update_0001
>  [notice] Update completed: field_group_post_update_0001
>  [notice] Update started: media_library_post_update_add_buttons_to_page_view
>  [notice] Update completed: media_library_post_update_add_buttons_to_page_view
>  [notice] Update started: media_library_post_update_add_langcode_filters
>  [notice] The 'Language' filter was added to the default display of the Media library view and the 'Default translation' filter was added to the following displays: Widget, Widget (table)
>  [notice] Update completed: media_library_post_update_add_langcode_filters
>  [notice] Update started: media_library_post_update_add_status_extra_filter
>  [notice] The 'Published status or admin user' filter was added to the Media library view.
>  [notice] Update completed: media_library_post_update_add_status_extra_filter
>  [notice] Update started: media_library_post_update_default_administrative_list_to_table_display
>  [notice] Update completed: media_library_post_update_default_administrative_list_to_table_display
>  [notice] Update started: media_library_post_update_update_8001_checkbox_classes
>  [notice] Update completed: media_library_post_update_update_8001_checkbox_classes
>  [notice] Update started: media_post_update_add_status_extra_filter
>  [notice] The 'Published status or admin user' filter was added to the Media view.
>  [notice] Update completed: media_post_update_add_status_extra_filter
>  [notice] Update started: path_post_update_create_language_content_settings
>  [notice] Update completed: path_post_update_create_language_content_settings
>  [notice] Update started: system_post_update_entity_reference_autocomplete_match_limit
>  [notice] Update completed: system_post_update_entity_reference_autocomplete_match_limit
>  [notice] Update started: system_post_update_layout_plugin_schema_change
>  [notice] Update completed: system_post_update_layout_plugin_schema_change
>  [notice] Update started: taxonomy_post_update_configure_status_field_widget
>  [notice] Update completed: taxonomy_post_update_configure_status_field_widget
>  [notice] Update started: text_post_update_add_required_summary_flag
>  [notice] Update completed: text_post_update_add_required_summary_flag
>  [notice] Update started: views_post_update_limit_operator_defaults
>  [notice] Update completed: views_post_update_limit_operator_defaults
>  [notice] Update started: views_post_update_remove_core_key
>  [notice] Update completed: views_post_update_remove_core_key

In ProcessBase.php line 188:

  Unable to decode output into JSON: Syntax error

  {
      "0": {
          "system": {
              "8801": {
                  "results": {
                      "query": null,
                      "success": true
                  },
                  "type": "update"
              },
              "8802": {
                  "results": {
                      "query": null,
                      "success": true
                  },
                  "type": "update"
              },
              "8803": {
                  "results": {
                      "query": "The "path_alias" entity type has been installed.",
                      "success": true
                  },
                  "type": "update"
              },
              "8804": {
                  "results": {
                      "query": "Path aliases have been converted to entities.",
                      "success": true
                  },
                  "type": "update"
              },
              "8805": {
                  "results": {
                      "query": null,
                      "success": true
                  },
                  "type": "update"
              },
              "entity_reference_autocomplete_match_limit": {
                  "results": {
                      "query": null,
                      "success": true
                  },
                  "type": "post_update"
              },
              "layout_plugin_schema_change": {
                  "results": {
                      "query": null,
                      "success": true
                  },
                  "type": "post_update"
              }
          },
          "media_library": {
              "8703": {
                  "results": {
                      "query": null,
                      "success": true
                  },
                  "type": "update"
              },
              "8704": {
                  "results": {
                      "query": null,
                      "success": true
                  },
                  "type": "update"
              },
              "add_buttons_to_page_view": {
                  "results": {
                      "query": null,
                      "success": true
                  },
                  "type": "post_update"
              },
              "add_langcode_filters": {
                  "results": {
                      "query": "The 'Language' filter was added to the default display of the <em class="placehold
  er">Media library</em> view and the 'Default translation' filter was added to the following displays: <em class=
  "placeholder">Widget, Widget (table)</em>",
                      "success": true
                  },
                  "type": "post_update"
              },
              "add_status_extra_filter": {
                  "results": {
                      "query": "The 'Published status or admin user' filter was added to the <em class="placeholde
  r">Media library</em> view.",
                      "success": true
                  },
                  "type": "post_update"
              },
              "default_administrative_list_to_table_display": {
                  "results": {
                      "query": null,
                      "success": true
                  },
                  "type": "post_update"
              },
              "update_8001_checkbox_classes": {
                  "results": {
                      "query": null,
                      "success": true
                  },
                  "type": "post_update"
              }
          },
          "layout_builder_restrictions": {
              "8202": {
                  "results": {
                      "query": null,
                      "success": true
                  },
                  "type": "update"
              }
          },
          "lightning_layout": {
              "8011": {
                  "results": {
                      "query": null,
                      "success": true
                  },
                  "type": "update"
              }
          },
          "locale": {
              "8800": {
                  "results": {
                      "query": null,
                      "success": true
                  },
                  "type": "update"
              }
          },
          "content_moderation": {
              "views_field_plugin_id": {
                  "results": {
                      "query": null,
                      "success": true
                  },
                  "type": "post_update"
              }
          },
          "field_group": {
              "0001": {
                  "results": {
                      "query": null,
                      "success": true
                  },
                  "type": "post_update"
              }
          },
          "media": {
              "add_status_extra_filter": {
                  "results": {
                      "query": "The 'Published status or admin user' filter was added to the <em class="placeholde
  r">Media</em> view.",
                      "success": true
                  },
                  "type": "post_update"
              }
          },
          "path": {
              "create_language_content_settings": {
                  "results": {
                      "query": null,
                      "success": true
                  },
                  "type": "post_update"
              }
          },
          "taxonomy": {
              "configure_status_field_widget": {
                  "results": {
                      "query": null,
                      "success": true
                  },
                  "type": "post_update"
              }
          },
          "text": {
              "add_required_summary_flag": {
                  "results": {
                      "query": null,
                      "success": true
                  },
                  "type": "post_update"
              }
          },
          "views": {
              "limit_operator_defaults": {
                  "results": {
                      "query": null,
                      "success": true
                  },
                  "type": "post_update"
              },
              "remove_core_key": {
                  "results": {
                      "query": null,
                      "success": true
                  },
                  "type": "post_update"
              }
          }
      },
      "drush_batch_process_finished": true
  }

Workaround
Is there another way to do the desired action?

System Configuration

Q	A
Drush version?	9.7.1
Drupal version?	8.7.10 / 8.8.0
PHP version	7.3.3
OS?	Windows

Additional Information
On Linux the error don't came up. Our jenkins build env and staging env's are Linux mashines and there finishing the database updates normally.

[pfrenssen]

I can confirm that from the output it looks like the updates have in fact executed successfully, but the problem is located somewhere in the processing of the results. It is likely that this problem is located somewhere outside of the UpdateDB realm since the JSON conversion is not done inside the UpdateDBCommand. I did some research online and I found a similar report in relation to performing a migration rather than an update. There is no further information available, but perhaps this might be due to batch handling? Both migrations and database updates rely on batches.

Unfortunately I have no Windows machine available so I cannot dig deeper into this problem. I will unassign so this is free for someone else to pick up. A backtrace could also be helpful to try to pinpoint where the problem occurs.

[laradevitt]

Just ran into this, exactly as described in OP. Seeing that wall of orange was a little alarming but I'm happy to know it doesn't affect the database update.

[TechnoTim2010]

Hi I just got precisely the same error. When running drush updatedb but also (without the lead in) when running drush cr.
Environment is Ubuntu 18.04LTS PHP7.3 and Drupal 8.9

[laradevitt]

Just ran into this again with non-escaped quotes in the update message:

          "scheduler": {
              "8101": {
                  "results": {
                      "query": "Default set on for new option "Show a message after updating content"",
                      "success": true
                  },
                  "type": "update"
              }
          }

I think this may interrupt the updating process after the updates are run but before rebuilding caches since I had to manually run drush cr in order to get rid of a typeerror.

Drupal version : 8.8.8
PHP version : 7.2.13
PHP OS : WINNT
Drush version : 9.7.2

[weitzman]

Hmm. It may be that the Drush batch runner or output format systems are not handling those messages with quotes in them

[fenstrat]

Also hit this when running updatedb on drush 9.7.2. Environment is Alpine Linux v3.10, someone else hit this on Ubuntu above so it doesn't seem like a Windows only issue.

In my case there was no unescaped quotes in the messages, and the updates all returned "success": true and appear to have run ok.

EDIT: This looks like it could be logging destination related, changing monolog.channel_handlers to syslog seems to fix this.

[weitzman]

I dont see any reports of this with drush 8 or with drush 10. Since Drush 9 is unsupported, I'm going to close this. Please reply if you experience this with either drush8 or drush10. I will reopen the issue.

[leonk]

I'm running into this issue on drush 10.3.6.
I have monolog module enabled, and configured to output to php://stdout

In an update hook I am enabling a module. This is generating a message in the logs, which is getting output when running drush updb, and the formatting is causing this to fail (even though it successfully ran).

$drush updb -y
...
>  [notice] Update started: mymodule_update_8025
>  [notice] Update completed: mymodule_update_8025

In ProcessBase.php line 188:

  Unable to decode output into JSON: Syntax error

  [2021-02-10 15:56:17] system.INFO: profile_switcher module installed. [] {"referer":"","ip":"127.0.0.1","request_uri":"http://default/","uid":0,"user":""}
  {
      "0": {
          "mymodule": {
              "8025": {
                  "results": {
                      "query": null,
                      "success": true
                  },
                  "type": "update"
              }
          }
      },
      "drush_batch_process_finished": true
  }

As per previous comment from @fenstrat, changing the location of the logs fixes the issue. (But this isn't really desirable as we want our logs to output to stdout, so I'm trying to think of another solution).

[weitzman]

If you can change your log message so that it doesn't output { and } (i.e. its just text) then I think Drush will be able to parse out its JSON from the output. See https://github.com/consolidation/site-process/blob/main/src/ProcessBase.php#L164

[Ambient-Impact]

Hello there. I'm getting the same issue when attempting to run the drupal:directory command:

E:\Web\Ambient.Impact\Web (3.x -> GitLab) (ambientimpact-drupal@1.0.0)
λ drush drupal:directory devel

In ProcessBase.php line 171:

  Unable to decode output into JSON: Syntax error

  {
      "%paths": {
          "%root": "E:\Web\Ambient.Impact\Web/drupal",
          "%site": "sites/default",
          "%modules": "sites/all/modules",
          "%themes": "sites/all/themes",
          "%config-sync": "E:\Web\Ambient.Impact\Web/drupal/../drupal_config/sync",
          "%files": "sites/default/files",
          "%temp": "C:\Users\i\AppData\Local\Temp",
          "%private": "E:\Web\Ambient.Impact\Web/drupal/../drupal_private_files",
          "%devel": "E:\Web\Ambient.Impact\Web/drupal/modules/contrib/devel"
      }
  }

Output of drush status with relevant versions:

E:\Web\Ambient.Impact\Web (3.x -> GitLab) (ambientimpact-drupal@1.0.0)
λ drush status
 Drupal version   : 9.2.8
 Site URI         : http://default
 DB driver        : mysql
 DB hostname      : 127.0.0.1
 DB port          : 33067
 DB username      : drupaluser
 DB name          : ambientimpact_dev
 Database         : Connected
 Drupal bootstrap : Successful
 Default theme    : ambientimpact_site
 Admin theme      : gin
 PHP binary       : C:\Program Files (x86)\DevDesktop\php7_3_x64\php.exe
 PHP config       : C:\Program Files (x86)\DevDesktop\php7_3_x64\php.ini
 PHP OS           : WINNT
 Drush script     : E:\Web\Ambient.Impact\Web\vendor\drush\drush\drush
 Drush version    : 10.6.1
 Drush temp       : C:\Users\i\AppData\Local\Temp
 Drush configs    : E:/Web/Ambient.Impact/Web/vendor/drush/drush/drush.yml
                    E:/Web/Ambient.Impact/Web/drush/drush.yml
 Install profile  : standard
 Drupal root      : E:\Web\Ambient.Impact\Web/drupal
 Site path        : sites/default
 Files, Public    : sites/default/files
 Files, Private   : E:\Web\Ambient.Impact\Web/drupal/../drupal_private_files
 Files, Temp      : C:\Users\i\AppData\Local\Temp

[DanChadwick]

This still happens with windows, drush 10.6.1, drupal 9.3.0, running the updb command.

[thirstysix]

Yes Same Issue with windows, drush 10.6.1, drupal 9.3.3

[weitzman]

OK, reopened. Possibly a bug in JSON formatter on Windows.

[weitzman]

Actually, the problem is better tracked at consolidation/site-process#56