drush sqlc < does not work. #4305

wah22 · 2020-01-27T21:10:52Z

drush sqlc command takes me to the sql command prompt rather than importing the sql file.

In our setup, I am logged in via SSH to run drush sqlc there. We have several DBs linked up with galera cluster with MariaDB 10.4.11 . We found that we could import the db with 9.6.2 by replacing all the set autocommit=0 to be set autocommit=1 from a mysql dump from drush. e.g. drush sql-dump > test.sql

I ran drush sql-dump > test.sql (this ran successfully)
sed replaced all autocommit=0 to be autocommit=1
then I ran drush sqlc < test.sql

I expected this to import the sql file like it did in version 9.6.2

Instead, it sent me to the sql command prompt.

Drush version: 10.2.0
Drupal version: 8.7
PHP version: 7.3.14
OS: CentOS

weitzman · 2020-01-27T22:19:28Z

Pretty sure this is a dupe. @danepowell may recall details.

danepowell · 2020-01-27T22:45:24Z

I've certainly seen funkiness with sql:cli in the past:

The failure modes in those cases were either hanging on failure or silently failing. I've never seen it drop into the SQL shell, and we now have an automated test to catch those failures, so I suspect this new issue is something else.

I'm not able to reproduce this on Ubuntu, maybe it's a CentOS thing? What's the full command you are running and output with --debug? Are you using an alias, specifically over SSH? If so, can you reproduce the same behavior without the alias?

wah22 · 2020-01-28T18:55:43Z

I am researching this a bit more, and will give you the requested information.

wah22 · 2020-02-11T22:00:02Z

EDIT: To be sure it is known, we did this to the sql file as mentioned in the first (albiet edited) comment:

In our setup, I am logged in via SSH to run drush sqlc there. We have several DBs linked up with galera cluster with MariaDB 10.4.11 . We found that we could import the db with 9.6.2 by replacing all the set autocommit=0 to be set autocommit=1 from a mysql dump from drush. e.g. drush sql-dump > test.sql

@danepowell I have edited the first comment above. My --debug output is as follows:

drush sqlc < sqlfile.sql

[preflight] Config paths: /var/www/html/vendor/drush/drush/drush.yml,/var/www/html/drush/drush.yml

[preflight] Alias paths: /var/www/html/web/drush/sites,/var/www/html/drush/sites

[preflight] Commandfile search paths: /var/www/html/vendor/drush/drush/src,/var/www/html/drush

[debug] Starting bootstrap to max [0.07 sec, 8.41 MB]

[debug] Drush bootstrap phase: bootstrapDrupalRoot() [0.07 sec, 8.41 MB]

[debug] Change working directory to /var/www/html/web [0.07 sec, 8.41 MB]

[debug] Initialized Drupal 8.7.11 root directory at /var/www/html/web [0.07 sec, 8.54 MB]

[debug] Drush bootstrap phase: bootstrapDrupalSite() [0.08 sec, 8.84 MB]

[debug] Initialized Drupal site default at sites/default [0.08 sec, 9.07 MB]

[debug] Drush bootstrap phase: bootstrapDrupalConfiguration() [0.08 sec, 9.07 MB]

[debug] Add service modifier [0.09 sec, 9.59 MB]

[info] Executing: command -v mysql [0.09 sec, 10.05 MB]

[info] Executing: mysql --defaults-file=/tmp/drush_(some random chars here) --database=d8db --host=(ip here) --port=(port here) -A [0.1 sec, 10.11 MB]

Welcome to the MariaDB monitor. Commands end with ; or \g.

Your MariaDB connection id is 3864

Server version: 10.4.11-MariaDB MariaDB Server

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [d8db]>

weitzman · 2020-03-22T02:38:07Z

I cant reproduce this on OSX or Alpine Linux (wodby/php docker image).

misterjoonas · 2020-06-09T11:31:13Z

Tried all the 10.x versions back to 10.0.0 but couldn't get it to work. I tried to downgrade to 9.x but then drush won't work at all and I get a different error "TypeError: Argument 4 passed to Drush\Drupal\Commands\config\ConfigImportCommands::__construct() must implement interface Drupal\Core\Extension\ModuleHandlerInterface".

greg-1-anderson · 2020-06-09T15:24:18Z

Maybe someone who can reproduce this could do a git bisect between 9.6.2 and 10.2.0 and see where the behavior changed.

golddragon007 · 2020-06-30T15:04:02Z

@greg-1-anderson here is the result of the git bisect

dda3993 is the first bad commit

commit dda3993778f6d81e339d1ae7f0b1c8e18cc9b573
Author: ****
Date:   Tue Apr 9 13:57:13 2019 -0700

    Fix TTY detection for runserver command. (#4033)

:040000 040000 8e3ff352f57b0c96ba1fccc77bc79b7a94797175 86b73d1fdb80f4dc0f19bfdc8ecc3298aaa611af M      .scenarios.lock
:100644 100644 48407d9fbf30b99ca15c7738b5685dd009782c74 37a83ee6b683b126d98965ca53546c87cf966c27 M      composer.json
:100644 100644 099c42680e0faa539c0143ff5059db9e4aa3db22 5de3b9eefbc66e7a3d84207d4c6b1eb1b924cfaa M      composer.lock
:040000 040000 8b75248db74c6a7eb51fc3d97378f0178de35d86 346b597507ec0e8aec5cdb91321e9349d45fbc14 M      src

kindlehl · 2020-07-24T00:20:53Z

@weitzman We are also running into this issue on drush-8.3.5. We upgraded recently from 8.1.16 where this was not a problem.

greg-1-anderson · 2020-07-24T01:08:05Z

@kindlehl Drush 8.x doesn't use any of this code, so the problem must be unrelated. If you could test with git bisect and determine which commit on the 8.x branch caused the problem you are seeing, then we can track it down separately.

@golddragon007 I am wondering if perhaps dda3993 is just a do-over. It seems that the intention of this commit was to move away from using the posix commands to detect tty capability, and instead use the Symfony libraries, if available. However, it seems to me that the old code did something different than the new code. The old code specifically checked to see if stdin was connected to a tty. Drush used this check to determine whether stdin should be redirected or not. The Symfony functions do not appear to be equivalent; they are not specifically about STDIN. The comment on the Symfony function says:

Returns whether TTY is supported on the current operating system.

That's not the same thing.

kindlehl · 2020-07-24T02:58:24Z

A workaround: drush sql-sync --no-dump --source-dump=<sql_file> @self @self

kindlehl · 2020-07-28T19:27:06Z

@greg-1-anderson

@kindlehl Drush 8.x doesn't use any of this code, so the problem must be unrelated. If you could test with git bisect and determine which commit on the 8.x branch caused the problem you are seeing, then we can track it down separately.

It looks like drush 8.3.5 is picking up on a site-local installation of drush 10.3.0 and using that, causing this issue in my case. I was not aware of this functionality, but I can replicate this issue for version 10.3.0

greg-1-anderson · 2020-07-28T20:05:06Z

@kindlehl: See my comment to @golddragon007 above regarding the bug in Drush 10. Maybe @danepowell can help here as well.

golddragon007 · 2020-07-28T20:49:26Z

@greg-1-anderson I don't know how much it is a do-over or not, but definitely it breaks for me and that's the first commit that didn't work for me. Therefore if it was a rework, then it wasn't 100% equal.
What I can tell you, however, if you are running in docker this, this is not an issue as far as I heard from my colleagues, however, from LAMP stack when everything is in the local machine, and you have project base defined drush installed by composer and pulled in by drush launcher, this happens.

greg-1-anderson · 2020-07-28T20:57:49Z

Yes; what I mean by a "do-over" is that it did not do what it was supposed to do, or more specifically, the new code is not making the same check as the old code was, which changes the intention of the code.

kindlehl · 2020-07-28T23:25:24Z

I just tested this by redoing the changes from that commit and it still is broken. Also, this doesn't happen in our production environment on php 7.2.31 with drush 10.3.0. It happens in non-live where we run php 7.2.28 and drush 10.3.0. There are some other differences such as the version of the mysql driver, etc. I tried testing under php 7.2.26 and it still failed.

I mentioned above that this came from an upgrade from 8.1.16 to 8.3.5, but that also came along with fundamental changes to php configuration and other things, and I'm leaning towards thinking that is what is caused this to break. I gutted the php configuration on the VM and applied the old configuration, kept my local modified drush 10.3.0 installation, and it now works. 🤷

I'm not sure, very confused.

kindlehl · 2020-07-30T00:52:31Z

@golddragon007 The issue on our end was the lack of the php posix extension. On IUS, install php72u-process was good enough to solve this issue. Can you verify that you have a similar problem?

kindlehl · 2020-07-30T01:36:33Z

@greg-1-anderson would you support adding a warning on POSIX OS's if the POSIX extension is not installed?

golddragon007 · 2020-08-03T13:09:43Z

@kindlehl Well, it's not a 100% solution as I see. I've installed the php73-php-process and php73-process packages. Now it doesn't jump into the console if I through some data to it. However, if there is an import error, then it doesn't jump out.
$ drush sqlc < dump.sql
The error shows up here...

After the error, it just hangs and you need to kill it with ctrl + c.

$ zcat dump.sql.gz | drush sqlc

Even if an error happens, it doesn't display it and it hangs till you kill it with ctrl + c.
However, I don't remember if this was the original how it worked in the past or not. So maybe legit working solution actually.

And actually, as I'm testing now, $ zcat dump.sql.gz | drush sqlc is epic slow compared to the 'original' way of working.

greg-1-anderson · 2020-08-03T15:03:42Z

@kindlehl: @weitzman just removed a bunch of php configuration validation checks (not POSIX) that were not firing often/ever. Since POSIX is important to Drush, though, perhaps a PR to check for it in preflight or invoke would be welcome.

I think the point of dda3993 was to make Drush work better without POSIX, but I think it might have made things worse. Still not sure.

FlorentTorregrosa · 2020-10-02T09:02:47Z

Hello,

I ran into this bug too. I was upgrading from 9.6.2 to 10.3.4.

Drush version: 10.3.4
Drupal version: 8.9.6
PHP version: 7.2.25
OS: CentOS 7

I downgraded Drush to 10.3.0: same problem
9.7.2: same problem
9.6.2: ok

bioshazard · 2020-12-27T19:23:16Z

I ran into this problem very recently. @rsanzante replied to this closed issue in #3859 (comment). He identified sqlc as failing to accept STDIN and that there were a couple diffs that never made it into the codebase. On the latest 10.3.6 provided by composer require drush/drush:10.x I can replicate the failure. As well, per my comment linked below in that issue, when I apply diff 9a03ed8 as a patch, my STDIN functionality for sqlc is restored. I see this is the only remaining issue opened for sqlc STDIN failure, so I added a comment here. Please be sure to read the two comments above the last one I linked as I am not sure if there is any need for the second diff he linked or if my anecdotal success is safe to rely on to conclude the change as ideal. I plan to simply apply that patch to the latest 10.3.6 until I see this issue closed. Thanks!

Source comment on closed issue: #3859 (comment)

weitzman · 2021-01-02T23:20:50Z

FWIW I noticed this commit for Composer composer/composer@1496277. See the PR linked from there.

meanderix · 2021-10-22T23:59:25Z

We're running into this as well, specifically when importing into a cluster that puts an upper limit on transaction sizes. Error reported during drush sqlc < dump.sql:

ERROR 1180 (HY000) at line 23703: Got error 5 "Input/output error" during COMMIT

The problem is also outlined here.

ISTM this is due to Drush\Sql\SqlMysql::dumpCmd() adding --no-autocommit as an extra parameter (which is not safe for large transactions!)

Jelle-S · 2021-12-03T10:23:39Z

@meanderix we're running into the same problem, same sort of setup. Have you found a way to fix this yet?

silverham · 2023-05-05T12:30:33Z

The issue on our end was the lack of the php posix extension

I can confirm the same issue on Centos, running sudo yum install php-process fixed the issue for me.

For debugging purposes, here is some outputs sudo yum list installed | grep php and php -m (before installation of posix/process).

Debugging Output

$ sudo yum list installed | grep php
php.x86_64                           8.1.18-1.el7.remi              @remi-php81
php-cli.x86_64                       8.1.18-1.el7.remi              @remi-php81
php-common.x86_64                    8.1.18-1.el7.remi              @remi-php81
php-gd.x86_64                        8.1.18-1.el7.remi              @remi-php81
php-mbstring.x86_64                  8.1.18-1.el7.remi              @remi-php81
php-mysqlnd.x86_64                   8.1.18-1.el7.remi              @remi-php81
php-opcache.x86_64                   8.1.18-1.el7.remi              @remi-php81
php-pdo.x86_64                       8.1.18-1.el7.remi              @remi-php81
php-pecl-mcrypt.x86_64               1.0.6-1.el7.remi.8.1           @remi-php81
php-pecl-zip.x86_64                  1.21.1-1.el7.remi.8.1          @remi-php81
php-sodium.x86_64                    8.1.18-1.el7.remi              @remi-php81
php-xml.x86_64                       8.1.18-1.el7.remi              @remi-php81

$ php -m
[PHP Modules]
bz2
calendar
Core
ctype
curl
date
dom
exif
fileinfo
filter
ftp
gd
gettext
hash
iconv
json
libxml
mbstring
mcrypt
mysqli
mysqlnd
openssl
pcntl
pcre
PDO
pdo_mysql
pdo_sqlite
Phar
readline
Reflection
session
SimpleXML
sockets
sodium
SPL
sqlite3
standard
tokenizer
xml
xmlreader
xmlwriter
xsl
Zend OPcache
zip
zlib

[Zend Modules]
Zend OPcache

$ cat /etc/os-release
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

$ uname -a
Linux myhostname 3.10.0-1160.6.1.el7.x86_64 #1 SMP Tue Nov 17 13:59:11 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

carnnia · 2023-09-15T21:16:35Z

this (drush sqlc does not accept stdin) just happend to me on RHEL with php 8.1 and drush11; apache running php-fpm.
install php-process fixed it.

fengtan · 2023-09-27T18:39:10Z

I ran into this as well with PHP 8.1 and Drush 10.6.2, on a UBI 7 system.

yum install php-process fixed the problem.

terryzwt · 2024-05-31T03:45:34Z

Both not work.

PS: I'm using alpine based docker, and can't install php-process.

drupal 9, drush 11

drupal version   : 9.5.11
Site URI         : http://default
DB driver        : mysql
DB hostname      : mariadb
DB port          : 3306
DB username      : root
DB name          : dsf
Database         : Connected
Drupal bootstrap : Successful
Default theme    : dxpr_theme
Admin theme      : seven
PHP binary       : /usr/bin/php82
PHP config       : /etc/php82/php.ini
PHP OS           : Linux
PHP version      : 8.2.19
Drush script     : /var/www/html/vendor/bin/drush
Drush version    : 11.6.0
Drush temp       : /tmp
Drush configs    : /var/www/html/vendor/drush/drush/drush.yml
Install profile  : cloud_orchestrator
Drupal root      : /var/www/html/docroot
Site path        : sites/default
Files, Public    : sites/default/files
Files, Private   : sites/default/files/private
Files, Temp      : /tmp

drupal 10, drush 12

Drupal version   : 10.2.5
Site URI         : http://default
DB driver        : mysql
DB hostname      : mariadb
DB port          : 3306
DB username      : root
DB name          : davyin_cms_11x
Database         : Connected
Drupal bootstrap : Successful
Default theme    : dyniva_ui
Admin theme      : claro
PHP binary       : /usr/bin/php82
PHP config       : /etc/php82/php.ini
PHP OS           : Linux
PHP version      : 8.2.19
Drush script     : /var/www/html/bin/drush
Drush version    : 12.5.1.0
Drush temp       : /tmp
Drush configs    : /var/www/html/vendor/drush/drush/drush.yml
Install profile  : dyniva
Drupal root      : /var/www/html/docroot
Site path        : sites/default
Files, Public    : sites/default/files
Files, Temp      : /tmp

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

drush sqlc < does not work. #4305

drush sqlc < does not work. #4305

wah22 commented Jan 27, 2020 •

edited

Loading

weitzman commented Jan 27, 2020

danepowell commented Jan 27, 2020

wah22 commented Jan 28, 2020

wah22 commented Feb 11, 2020 •

edited

Loading

weitzman commented Mar 22, 2020

misterjoonas commented Jun 9, 2020

greg-1-anderson commented Jun 9, 2020

golddragon007 commented Jun 30, 2020 •

edited by weitzman

Loading

kindlehl commented Jul 24, 2020

greg-1-anderson commented Jul 24, 2020

kindlehl commented Jul 24, 2020

kindlehl commented Jul 28, 2020

greg-1-anderson commented Jul 28, 2020

golddragon007 commented Jul 28, 2020

greg-1-anderson commented Jul 28, 2020

kindlehl commented Jul 28, 2020 •

edited

Loading

kindlehl commented Jul 30, 2020

kindlehl commented Jul 30, 2020

golddragon007 commented Aug 3, 2020 •

edited

Loading

greg-1-anderson commented Aug 3, 2020

FlorentTorregrosa commented Oct 2, 2020

bioshazard commented Dec 27, 2020 •

edited

Loading

weitzman commented Jan 2, 2021

meanderix commented Oct 22, 2021

Jelle-S commented Dec 3, 2021

silverham commented May 5, 2023

carnnia commented Sep 15, 2023

fengtan commented Sep 27, 2023

terryzwt commented May 31, 2024

drush sqlc < does not work. #4305

drush sqlc < does not work. #4305

Comments

wah22 commented Jan 27, 2020 • edited Loading

weitzman commented Jan 27, 2020

danepowell commented Jan 27, 2020

wah22 commented Jan 28, 2020

wah22 commented Feb 11, 2020 • edited Loading

weitzman commented Mar 22, 2020

misterjoonas commented Jun 9, 2020

greg-1-anderson commented Jun 9, 2020

golddragon007 commented Jun 30, 2020 • edited by weitzman Loading

kindlehl commented Jul 24, 2020

greg-1-anderson commented Jul 24, 2020

kindlehl commented Jul 24, 2020

kindlehl commented Jul 28, 2020

greg-1-anderson commented Jul 28, 2020

golddragon007 commented Jul 28, 2020

greg-1-anderson commented Jul 28, 2020

kindlehl commented Jul 28, 2020 • edited Loading

kindlehl commented Jul 30, 2020

kindlehl commented Jul 30, 2020

golddragon007 commented Aug 3, 2020 • edited Loading

greg-1-anderson commented Aug 3, 2020

FlorentTorregrosa commented Oct 2, 2020

bioshazard commented Dec 27, 2020 • edited Loading

weitzman commented Jan 2, 2021

meanderix commented Oct 22, 2021

Jelle-S commented Dec 3, 2021

silverham commented May 5, 2023

carnnia commented Sep 15, 2023

fengtan commented Sep 27, 2023

terryzwt commented May 31, 2024

wah22 commented Jan 27, 2020 •

edited

Loading

wah22 commented Feb 11, 2020 •

edited

Loading

golddragon007 commented Jun 30, 2020 •

edited by weitzman

Loading

kindlehl commented Jul 28, 2020 •

edited

Loading

golddragon007 commented Aug 3, 2020 •

edited

Loading

bioshazard commented Dec 27, 2020 •

edited

Loading