State mysql_user.present fails first time on Salt 2018.3 #50744

jessebye · 2018-12-04T17:27:33Z

Description of Issue/Question

We have many mysql_user.present states that were working fine in Salt 2017.7. After upgrading to 2018.3, the states fail the first time they run with no error given. The second time they run, they succeed. Using the mysql module instead of state resolves the problem.

Setup

  mysql_user.present:
    - name: jdoe
    - password_hash: '*REDACTED'
    - host: '10.%'
    - connection_host: {{ rds_instance }}
    - connection_user: {{ rds_credentials.username }}
    - connection_pass: {{ rds_credentials.password }}

Steps to Reproduce Issue

Run the state. All mysql_user.present states fail with a message like this:

    Function: mysql_user.present
        Name: jdoe
      Result: False
     Comment: Failed to create user jdoe@10.%
     Started: 17:21:25.583470
    Duration: 38.274 ms
     Changes:

The minion debug log is not very helpful:

[INFO    ] Running state [billing-dev] at time 17:21:25.583470
[INFO    ] Executing state mysql_user.present for [billing-dev]
[DEBUG   ] Doing query: SELECT VERSION()
[DEBUG   ] Doing query: SELECT column_name from information_schema.COLUMNS WHERE table_schema=%(schema)s and table_name=%(table)s and column_name=%(column)s args: {u'column': u'Password', u'table': u'user', u'schema': u'mysql'} 
[DEBUG   ] Doing query: SELECT User,Host FROM mysql.user WHERE User = %(user)s AND Host = %(host)s AND Password = %(password)s args: {u'host': u'10.%', u'password': u'*REDACTED', u'user': u'billing-dev'} 
[DEBUG   ] Doing query: SELECT VERSION()
[DEBUG   ] Doing query: SELECT column_name from information_schema.COLUMNS WHERE table_schema=%(schema)s and table_name=%(table)s and column_name=%(column)s args: {u'column': u'Password', u'table': u'user', u'schema': u'mysql'} 
[DEBUG   ] Doing query: SELECT User,Host FROM mysql.user WHERE User = %(user)s AND Host = %(host)s args: {u'host': u'10.%', u'user': u'billing-dev'} 
[DEBUG   ] Doing query: SELECT VERSION()
[DEBUG   ] Doing query: SELECT VERSION()
[DEBUG   ] Doing query: SELECT column_name from information_schema.COLUMNS WHERE table_schema=%(schema)s and table_name=%(table)s and column_name=%(column)s args: {u'column': u'Password', u'table': u'user', u'schema': u'mysql'} 
[DEBUG   ] Doing query: SELECT User,Host FROM mysql.user WHERE User = %(user)s AND Host = %(host)s args: {u'host': u'10.%', u'user': u'billing-dev'} 
[DEBUG   ] Doing query: SELECT column_name from information_schema.COLUMNS WHERE table_schema=%(schema)s and table_name=%(table)s and column_name=%(column)s args: {u'column': u'Password', u'table': u'user', u'schema': u'mysql'} 
[DEBUG   ] Doing query: CREATE USER %(user)s@%(host)s IDENTIFIED BY %(password)s args: {u'host': u'10.%', u'password': u'*REDACTED', u'user': u'billing-dev'} 
[DEBUG   ] Doing query: SELECT VERSION()
[DEBUG   ] Doing query: SELECT User,Host FROM mysql.user WHERE User = %(user)s AND Host = %(host)s AND Password = %(password)s args: {u'host': u'10.%', u'password': u'*REDACTED', u'user': u'billing-dev'} 
[INFO    ] User 'billing-dev'@'10.%' was not created
[ERROR   ] Failed to create user billing-dev@10.%
[INFO    ] Completed state [billing-dev] at time 17:21:25.621744 (duration_in_ms=38.274)

Rerun the state. All the previously failing states succeed.

Possibly of interest, the following state doesn't exhibit this problem:

mysql_user_test:
  module.run:
    - mysql.user_create:
      - user: {{ name }}
      - password_hash: "*{{ password_hash }}"
      - host: {{ host }}
      - connection_host: {{ rds_instance }}
      - connection_user: {{ rds_credentials.username }}
      - connection_pass: {{ rds_credentials.password }}

Versions Report

Salt Version:
           Salt: 2018.3.3
 
Dependency Versions:
           cffi: 1.11.5
       cherrypy: Not Installed
       dateutil: 2.7.3
      docker-py: Not Installed
          gitdb: 2.0.3
      gitpython: 2.1.10
          ioflo: Not Installed
         Jinja2: 2.9
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: 1.0.7
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: 2.18
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.12 (default, Nov 12 2018, 14:36:49)
   python-gnupg: 0.3.8
         PyYAML: 3.12
          PyZMQ: 15.2.0
           RAET: Not Installed
          smmap: 2.0.3
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.1.4
 
System Versions:
           dist: Ubuntu 16.04 xenial
         locale: UTF-8
        machine: x86_64
        release: 4.4.0-134-generic
         system: Linux
        version: Ubuntu 16.04 xenial

The text was updated successfully, but these errors were encountered:

Ch3LL · 2018-12-04T17:41:54Z

this seems similar to #48204 (comment) but this should have been fixed in 2018.3.3. Is your minion also 2018.3.3? Also what version of mysql are you using?

jessebye · 2018-12-04T18:57:48Z

@Ch3LL ah, yes that definitely looks similar. After the first run of the state, the users are there, but I can't connect with their credentials. After the second run, the credentials work.

Minion and master are both 2018.3.3:

root@west-salt-api-dev-7777a:/data/repos/base# salt-minion --version
salt-minion 2018.3.3 (Oxygen)
root@west-salt-api-dev-7777a:/data/repos/base# salt-master --version
salt-master 2018.3.3 (Oxygen)

MySQL version is 5.6.40.

danielpardorus-allenta · 2019-01-14T10:32:55Z

Hi @jessebye,

Maybe related to saltstack-formulas/mysql-formula#194 (comment)?

jessebye · 2019-01-15T22:13:15Z

@danielpardorus-allenta thanks! that does appear to be related.

Reiner030 · 2019-04-20T17:07:26Z

Hi,

the states are running fine when users are setup where localhost is allowed but even here as in yesterday referenced issue mentionedthe buggy verify_login call also creates unnecessary ERROR output lines which shouldn't be written for such pre-checks:

[ERROR   ] MySQL Error 1045: Access denied for user 'monitor'@'localhost' (using password: YES)
[ERROR   ] MySQL Error 1045: Access denied for user 'staging'@'localhost' (using password: YES)
[ERROR   ] MySQL Error 1045: Access denied for user 'newrelic'@'localhost' (using password: YES)

Ch3LL · 2019-05-02T20:17:11Z

ping @garethgreenaway i'm pretty certain u have done some recent work around this, but i can't seem to find the PR. Can you comment here?

Reiner030 · 2019-05-08T15:26:04Z

Hello @Ch3LL and @garethgreenaway ,
here the false condition was implemented: #49918
and here was the connection used instead of user@hostmask values: #50551

vosscodes · 2019-06-20T19:47:50Z

I'm seeing this issue as well on 2019.2 and MariaDB 10.1 and 10.2.

[DEBUG   ] Doing query: SELECT column_name from information_schema.COLUMNS WHERE table_schema=%(schema)s and table_name=%(table)s and column_name=%(column)s args: {u'column': u'Password', u'table': u'user', u'schema': u'mysql'}
[ERROR   ] MySQL Error 1045: Access denied for user 'remoteuser'@'localhost' (using password: YES)

           Salt: 2019.2.0

Dependency Versions:
           cffi: 1.6.0
       cherrypy: Not Installed
       dateutil: Not Installed
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.7.2
        libgit2: 0.26.3
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: 1.2.5
      pycparser: 2.14
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: 0.26.4
         Python: 2.7.5 (default, Apr  9 2019, 14:30:50)
   python-gnupg: 0.4.3
         PyYAML: 3.11
          PyZMQ: 15.3.0
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.1.4

System Versions:
           dist: centos 7.6.1810 Core
         locale: UTF-8
        machine: x86_64
        release: 3.10.0-957.12.2.el7.x86_64
         system: Linux
        version: CentOS Linux 7.6.1810 Core

vosscodes · 2019-06-20T19:54:11Z

Looks like this is/has been dealt with, the issue in question is #51876 @Ch3LL

Ch3LL · 2019-07-02T13:42:03Z

thanks for pointing that out. Can anyone try the fixes here: #53418

karlyan17 · 2019-07-18T09:40:44Z

I was experiencing the error described by OP and using the fixed mysql module get's rid of that error: user_create runs through without errors on the first go.

Ch3LL · 2019-07-30T19:44:40Z

perfect okay to close then?

stale · 2020-01-08T06:10:27Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

Ch3LL added the info-needed waiting for more info label Dec 4, 2018

Ch3LL added this to the Blocked milestone Dec 4, 2018

Reiner030 mentioned this issue Apr 19, 2019

module.mysql.user_present and state.mysql_user.present can't properly check user passord #52165

Closed

Ch3LL added Pending-Discussion The issue or pull request needs more discussion before it can be closed or merged and removed info-needed waiting for more info labels May 2, 2019

stale bot added the stale label Jan 8, 2020

stale bot closed this as completed Jan 15, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

State mysql_user.present fails first time on Salt 2018.3 #50744

State mysql_user.present fails first time on Salt 2018.3 #50744

jessebye commented Dec 4, 2018 •

edited

Loading

Ch3LL commented Dec 4, 2018

jessebye commented Dec 4, 2018

danielpardorus-allenta commented Jan 14, 2019

jessebye commented Jan 15, 2019

Reiner030 commented Apr 20, 2019

Ch3LL commented May 2, 2019

Reiner030 commented May 8, 2019

vosscodes commented Jun 20, 2019

vosscodes commented Jun 20, 2019

Ch3LL commented Jul 2, 2019

karlyan17 commented Jul 18, 2019

Ch3LL commented Jul 30, 2019

stale bot commented Jan 8, 2020

State mysql_user.present fails first time on Salt 2018.3 #50744

State mysql_user.present fails first time on Salt 2018.3 #50744

Comments

jessebye commented Dec 4, 2018 • edited Loading

Description of Issue/Question

Setup

Steps to Reproduce Issue

Versions Report

Ch3LL commented Dec 4, 2018

jessebye commented Dec 4, 2018

danielpardorus-allenta commented Jan 14, 2019

jessebye commented Jan 15, 2019

Reiner030 commented Apr 20, 2019

Ch3LL commented May 2, 2019

Reiner030 commented May 8, 2019

vosscodes commented Jun 20, 2019

vosscodes commented Jun 20, 2019

Ch3LL commented Jul 2, 2019

karlyan17 commented Jul 18, 2019

Ch3LL commented Jul 30, 2019

stale bot commented Jan 8, 2020

jessebye commented Dec 4, 2018 •

edited

Loading