Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sql,pgwire: fix a host of pgwire/sql interaction bugs #30702

Merged
merged 4 commits into from
Oct 5, 2018

Conversation

knz
Copy link
Contributor

@knz knz commented Sep 26, 2018

First commit from #30691.

Fixes #30701.
Fixes #30692.
Fixes #30693.
Fixes #30697.
Fixes #30698.
Fixes #30703.
Informs #30717.

With special thanks to @benesch for asking me to poke at this.

@knz knz requested a review from a team September 26, 2018 23:35
@knz knz requested a review from a team as a code owner September 26, 2018 23:35
@knz knz requested review from a team September 26, 2018 23:35
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@knz knz force-pushed the 20180926-session-vars branch 3 times, most recently from f43d55e to 1f91f60 Compare September 27, 2018 11:31
@knz knz requested a review from a team as a code owner September 27, 2018 11:31
@knz knz requested a review from a team September 27, 2018 13:51
Copy link
Contributor

@benesch benesch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm_strong: that review was challenging—there's a lot crammed into the last commit—but I so appreciate you doing this. Note however that I didn't review the first commit, since Andrei appears to be already reviewing that in #30691.

Reviewed 3 of 3 files at r1, 1 of 1 files at r2, 4 of 4 files at r3, 29 of 29 files at r4.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained


pkg/sql/internal.go, line 265 at r4 (raw file):

	opName string,
	txn *client.Txn,
	sargs *SessionArgs,

Seems like this should maybe just take the username as a string argument? Perhaps I'm missing something.


pkg/sql/logictest/logic.go, line 918 at r4 (raw file):

	if _, err := db.Exec("SET extra_float_digits = 0"); err != nil {
		t.Fatal(err)
	}

🤦‍♂️ this can't have been fun to track down


pkg/sql/pgwire/conn.go, line 271 at r4 (raw file):

				return err
			}
		}

Is this enough of a performance win to be worth it? I don't love the idea of having a slightly-differently-configured client in unit tests than in production.

@knz knz force-pushed the 20180926-session-vars branch 2 times, most recently from e77c7d5 to 6211166 Compare October 3, 2018 09:50
Copy link
Contributor Author

@knz knz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained


pkg/sql/internal.go, line 265 at r4 (raw file):

Previously, benesch (Nikhil Benesch) wrote…

Seems like this should maybe just take the username as a string argument? Perhaps I'm missing something.

Good suggestion. Done.


pkg/sql/logictest/logic.go, line 918 at r4 (raw file):

Previously, benesch (Nikhil Benesch) wrote…

🤦‍♂️ this can't have been fun to track down

Of the two this was the simplest: we print out the list of session vars, their current value and their defautls in the "pg_catalog" logic test, and this was showing up as "2" where I was expecting "0". I then verified that "psql" leaves the default at 0 where "cockroach sql" turned it into "2" so I knew the problem was lib/pq.

The "not fun" one was the timestamp stuff -- that Go will format the timezone differently depending on whether it is "named" or not was a learning experience, to say the least.


pkg/sql/pgwire/conn.go, line 271 at r4 (raw file):

Previously, benesch (Nikhil Benesch) wrote…

Is this enough of a performance win to be worth it? I don't love the idea of having a slightly-differently-configured client in unit tests than in production.

This is because the test here needs to "plug in" its own query handler to intercept queries in lieu of the conn executor.

Copy link
Contributor

@benesch benesch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 32 of 32 files at r5, 1 of 1 files at r6, 2 of 4 files at r7, 29 of 29 files at r8.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained

Release note (bug fix): CockroachDB now properly ignores
non-alphanumeric characters in encoding names passed to
`convert_from()`, `client_encoding`, etc., for compatibility with
PostgreSQL.
@knz knz force-pushed the 20180926-session-vars branch 2 times, most recently from e0d2dd1 to b3e21af Compare October 5, 2018 09:23
Prior to this patch, CockroachDB:

- incorrectly ignored the `extra_float_digits` client
  parameter.
- improperly reported `ISO` as default value for server
  status parameter and session var `DateStyle` (the pg
  default is `ISO, MDY`.
- failed to report `TimeZone`, `is_superuser` and
  `session_authorization` in the server status parameters.
- failed to report the correct (and client-provided) value of
  `application_name` in the server status parameters.
- did not report the correct defaults in `pg_catalog.pg_settings`.

This patch corrects these various bugs by:

- keeping the client-provided parameters as session defaults, to be
  picked up by RESET and pg_settings. (Prior to this patch, the
  client parameters were discarded after setting up the session
  initially.)
- extending the handling of session variables to support reporting
  their current value during connection intiialization. This way
  the server status parameter can report the session variables
  that were just initialized from the client-provided defaults.
- eliminating the ad-hoc constant list of "statusServerParams"
  maintained in pgwire/conn.go, which could dangerously go out of sync
  with the session variable defaults without anyone noticing.
- adding the missing reports for `is_superuser` and
  `session_authorization`.

Because CockroachDB now properly honors the value of
extra_float_digits provided by the client, any test that uses lib/pq
must now account for the fact that lib/pq automatically sets this
parameter to 2. The logic_test code is modified accordingly.

Because CockroachDB now properly reports the current TimeZone
parameter to the client during connection initialization, any test
that uses lib/pq must now account for the fact that lib/pq will load
that zone name into any parsed timestamp value coming from the
server. The tests that print out timestamps have been modified
accordingly.

As a side effect of this fix, it is now possible to set arbitrary
session variables in the client connection URL. This has another
benefit reported in the release notes below.

Release note (bug fix): CockroachDB now properly recognizes the value
of `extra_float_digits` provided by clients as a connection parameter.

Release note (bug fix): CockroachDB now properly recognizes two-part
values for the session variable and connection parameter `DateStyle`,
for compatibility with PostgreSQL.

Release note (bug fix): CockroachDB now reports all server status
parameters supported by PostgreSQL when setting up a session. This is
expected to improve compatibility with some drivers.

Release note (bug fix): CockroachDB now properly uses the
client-provided default values when using the RESET statement (or SET
... = DEFAULT).

Release note (bug fix): CockroachDB now properly fills the columns
`boot_val` and `reset_val` in `pg_catalog.pg_settings` for better
compatibility with PostgreSQL.

Release note (cli change): It is now possible to provide
initial/default values for any customizable session variable in the
client connection URL.

Release note (sql change): CockroachDB now supports more
customizations from PostgreSQL client drivers when initially setting
up the client connection.
@knz
Copy link
Contributor Author

knz commented Oct 5, 2018

bors r+

craig bot pushed a commit that referenced this pull request Oct 5, 2018
30702: sql,pgwire: fix a host of pgwire/sql interaction bugs r=knz a=knz

First commit from #30691.

Fixes #30701.
Fixes #30692.
Fixes #30693.
Fixes #30697.
Fixes #30698.
Fixes #30703.
Informs #30717.

With special thanks to @benesch for asking me to poke at this.

Co-authored-by: Raphael 'kena' Poss <knz@cockroachlabs.com>
@craig
Copy link
Contributor

craig bot commented Oct 5, 2018

Build succeeded

@craig craig bot merged commit c89e0a9 into cockroachdb:master Oct 5, 2018
@knz knz deleted the 20180926-session-vars branch October 5, 2018 10:06
knz added a commit to knz/cockroach that referenced this pull request Nov 30, 2018
The initialization of `(*connExecutor).appStats` in `newConnExecutor`
was based off a temporary local variable (`sd`) unrelated to the object
actually holding the application name (`ex.sessionData`). This caused
the initial appStats to always be wrong when the initial application
name was not empty.

Meanwhile, the changes in cockroachdb#30702 have ensured that `appStats` is
initialized via `resetSessionVars` for normal client connections. The
extra initialization in `newConnExecutor` was not just wrong; it was
not needed any more except for internal executors.

This patch corrects/simplifies the situation.

Release note (bug fix): CockroachDB now properly records statistics
for sessions where the value of `application_name` is given by the
client during initialization instead of `SET`.
knz added a commit to knz/cockroach that referenced this pull request Nov 30, 2018
The initialization of `(*connExecutor).appStats` in `newConnExecutor`
was based off a temporary local variable (`sd`) unrelated to the object
actually holding the application name (`ex.sessionData`). This caused
the initial appStats to always be wrong when the initial application
name was not empty.

Meanwhile, the changes in cockroachdb#30702 have ensured that `appStats` is
initialized via `resetSessionVars` for normal client connections. The
extra initialization in `newConnExecutor` was not just wrong; it was
not needed any more except for internal executors.

This patch corrects/simplifies the situation.

Release note (bug fix): CockroachDB now properly records statistics
for sessions where the value of `application_name` is given by the
client during initialization instead of `SET`.
knz added a commit to knz/cockroach that referenced this pull request Dec 3, 2018
The initialization of `(*connExecutor).appStats` in `newConnExecutor`
was based off a temporary local variable (`sd`) unrelated to the object
actually holding the application name (`ex.sessionData`). This caused
the initial appStats to always be wrong when the initial application
name was not empty.

Meanwhile, the changes in cockroachdb#30702 have ensured that `appStats` is
initialized via `resetSessionVars` for normal client connections. The
extra initialization in `newConnExecutor` was not just wrong; it was
not needed any more except for internal executors.

This patch corrects/simplifies the situation.

Release note (bug fix): CockroachDB now properly records statistics
for sessions where the value of `application_name` is given by the
client during initialization instead of `SET`.
craig bot pushed a commit that referenced this pull request Dec 3, 2018
32754: sql: proprely record stmt stats for client-specified app names r=knz a=knz

Fixes #31766

The initialization of `(*connExecutor).appStats` in `newConnExecutor`
was based off a temporary local variable (`sd`) unrelated to the object
actually holding the application name (`ex.sessionData`). This caused
the initial appStats to always be wrong when the initial application
name was not empty.

Meanwhile, the changes in #30702 have ensured that `appStats` is
initialized via `resetSessionVars` for normal client connections. The
extra initialization in `newConnExecutor` was not just wrong; it was
not needed any more except for internal executors.

This patch corrects/simplifies the situation.

Release note (bug fix): CockroachDB now properly records statistics
for sessions where the value of `application_name` is given by the
client during initialization instead of `SET`.

Co-authored-by: Raphael 'kena' Poss <knz@cockroachlabs.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants