Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sql: ALTER TYPE ... DROP VALUE doesn't account for array type usages #60004

Closed
arulajmani opened this issue Feb 8, 2021 · 0 comments · Fixed by #61406
Closed

sql: ALTER TYPE ... DROP VALUE doesn't account for array type usages #60004

arulajmani opened this issue Feb 8, 2021 · 0 comments · Fixed by #61406
Assignees
Labels
C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. GA-blocker

Comments

@arulajmani
Copy link
Collaborator

Describe the problem

Currently, when validating wether an enum value can be dropped or not, we don't validate using the Array type descriptor.

To Reproduce

demo@127.0.0.1:26257/movr> CREATE TYPE abc AS ENUM('a', 'b', 'c');
CREATE TYPE

Time: 4ms total (execution 4ms / network 0ms)

demo@127.0.0.1:26257/movr> CREATE TABLE a(k INT, v abc[]);
CREATE TABLE

Time: 31ms total (execution 7ms / network 23ms)

demo@127.0.0.1:26257/movr> INSERT INTO a VALUES (1, ARRAY['a']);
INSERT 1

Time: 5ms total (execution 5ms / network 0ms)

demo@127.0.0.1:26257/movr> ALTER TYPE abc DROP VALUE 'a';
ALTER TYPE

Time: 41ms total (execution 4ms / network 37ms)

demo@127.0.0.1:26257/movr> SELECT * FROM a; ;
ERROR: internal error: unexpected error from the vectorized engine: could not find [64] in enum "public.abc" representation PhysicalReps: [[128] [192]]; LogicalReps: [b c]
SQLSTATE: XX000
DETAIL: stack trace:
github.com/cockroachdb/cockroach/pkg/sql/colexecbase/colexecerror/error.go:90: func1()
runtime/panic.go:969: gopanic()
github.com/cockroachdb/cockroach/pkg/sql/colexecbase/colexecerror/error.go:178: InternalError()
github.com/cockroachdb/cockroach/pkg/sql/colfetcher/colbatch_scan.go:85: Next()
github.com/cockroachdb/cockroach/pkg/sql/colexec/cancel_checker.go:48: Next()
github.com/cockroachdb/cockroach/pkg/sql/colexec/simple_project.go:124: Next()
github.com/cockroachdb/cockroach/pkg/sql/colexec/materializer.go:254: next()
github.com/cockroachdb/cockroach/pkg/sql/colexec/materializer.go:279: nextAdapter()
github.com/cockroachdb/cockroach/pkg/sql/colexecbase/colexecerror/error.go:93: CatchVectorizedRuntimeError()
github.com/cockroachdb/cockroach/pkg/sql/colexec/materializer.go:285: Next()
github.com/cockroachdb/cockroach/pkg/sql/execinfra/base.go:171: Run()
github.com/cockroachdb/cockroach/pkg/sql/execinfra/processorsbase.go:765: Run()
github.com/cockroachdb/cockroach/pkg/sql/flowinfra/flow.go:383: Run()
github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:366: Run()
github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:972: PlanAndRun()
github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:1035: execWithDistSQLEngine()
github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:907: dispatchToExecutionEngine()
github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:653: execStmtInOpenState()
github.com/cockroachdb/cockroach/pkg/sql/conn_executor_exec.go:121: execStmt()
github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:1474: func1()
github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:1476: execCmd()
github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:1401: run()
github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:490: ServeConn()
github.com/cockroachdb/cockroach/pkg/sql/pgwire/conn.go:627: func1()
runtime/asm_amd64.s:1374: goexit()

HINT: You have encountered an unexpected error.

Expected behavior
Drop should fail as the enum value is used inside of an array.

@arulajmani arulajmani added the C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. label Feb 8, 2021
@arulajmani arulajmani self-assigned this Feb 8, 2021
arulajmani added a commit to arulajmani/cockroach that referenced this issue Mar 3, 2021
Previously, when dropping an enum value, we weren't validating if the
enum value was used in a column of array type. This patch fixes the bug.

Fixes cockroachdb#60004

Release justification: bug fix to new functionality
Release note: None
arulajmani added a commit to arulajmani/cockroach that referenced this issue Mar 7, 2021
Previously, when dropping an enum value, we weren't validating if the
enum value was used in a column of array type. This patch fixes the bug.

Fixes cockroachdb#60004

Release justification: bug fix to new functionality
Release note: None
craig bot pushed a commit that referenced this issue Mar 15, 2021
61406: sql: validate against array type usages when dropping enum values r=ajwerner a=arulajmani

Previously, when dropping an enum value, we weren't validating if the
enum value was used in a column of array type. This patch fixes the bug.

Fixes #60004

Release justification: bug fix to new functionality
Release note: None

61650: roachtest: exclude `lost+found` directory r=knz a=rickystewart

This directory is created by the filesystem and unowned file chunks are
put there by `fsck`. The directory and its contents aren't readable by
anyone except `root`, so this can cause the `du -c /mnt/data1` that
`roachtest` performs to fail -- add an `--exclude` to handle this.

We already ignore this directory in other contexts (for example, see
`pkg/storage/mvcc.go`).

Fixes #53663.

Release justification: Non-production code change
Release note: None

61788: importccl: unskip userfile benchmark r=pbardea a=adityamaru

I've run this ~20 times and it averages ~13s to run. I suspect the fixes to linked issues mentioned in #59126 might have mitigated this. Going to unskip due to lack of reproducibility.

Fixes: #59126

Release note: None

61828: contention: store contention events on non-SQL keys r=yuzefovich a=yuzefovich

Previously, whenever we tried to add a contention event on a non-SQL
key, it would encounter an error during decoding tableID/indexID pair,
and the event was dropped. This commit extends the contention registry
to additionally store information about contention on non-SQL keys. That
information is stored in two levels:
- on the top level, all `SingleNonSQLKeyContention` objects are ordered
  by their keys
- on the bottom level, all `SingleTxnContention` objects are ordered by the
  number of times that transaction was observed to contend with other
  transactions.

`SingleTxnContention` protobuf message is moved out of
`SingleKeyContention` and is reused for non-SQL keys. This commit also
updates the status server API response. I assume that no changes are
needed with regards to backwards compatibility since the original
version was merged just a few weeks ago, and we haven't had a beta
released since then.

Fixes: #60669.

Release note (sql change): CockroachDB now also stores the information
about contention on non-SQL keys.

61862: cliccl: add `load show backups` to display backup collection r=pbardea a=Elliebababa

Previously, users can list backups created by `BACKUP INTO`
with `SHOW BACKUP IN`in a sql session. But this listing task
can be also done offline without a running cluster.

This PR updates `load show` with `backups` subcommand, 
which allows users to list backups in a backup collection 
created by `BACKUP INTO`. 
With the same purpose as other `load show` subcommands, 
this update allows users to list backups without running 
`SHOW BACKUP IN` in a sql session.

see #61131 #61829 to checkout other `load show` subcommand.

Release note (cli change): Add `load show backups` to
display backup collection. Previously, users can list backups
created by `BACKUP INTO` via `SHOW BACKUP IN`in a sql
session. But this listing task can be also done offline without a
running cluster. Now, users are able to list backups in a collection
with `cockroach load show backups <collection_url>.

61877: bench/ddl_analysis: fix test for real r=ajwerner a=ajwerner

See individual commits. Last is the critical one. 

Fixes #61856.

61937: colexec: make vectorized stats concurrency safe r=yuzefovich a=yuzefovich

**colflow: clean up vectorized stats for rowexec processors**

Previously, the wrapped row-execution KV reading processors were
implementing `execinfra.KVReader` interface, but they were never used as
such, only the ColBatchScans would get used to retrieve the KV stats.
This is the case because the row-execution processors report their
execution stats themselves, and we don't want to duplicate that info.
This commit moves `KVReader` interface into `colexecop` package and now
only the ColBatchScans implement it. This allowed for some cleanup
around the vectorized stats code, but the main reason for performing
this change is that the contract of the interface will be modified by
the follow-up commit to mention the safety under concurrent usage, and
I didn't want to change the row-execution processors for that since the
relevant methods never get called anyway.

Additionally, this commit begins emitting of rows-read and bytes-read by
the zigzagJoiners and invertedJoiners to complete the metrics picture.

Release note: None

**colexec: make vectorized stats concurrency safe**

Previously, the collection of vectorized stats was not synchronized with
the operators themselves. Namely, it was possible to call methods like
`GetBytesRead` on the ColBatchScans and Inboxes from a different
goroutine (the root materializer or the outbox) than from the main
goroutine of the operator. This is now fixed by putting mutexes in place
and updating `colexecop.KVReader` interface to require concurrency-safe
implementations.

Fixes: #61899.

Release note: None

Co-authored-by: arulajmani <arulajmani@gmail.com>
Co-authored-by: Ricky Stewart <ricky@cockroachlabs.com>
Co-authored-by: Aditya Maru <adityamaru@gmail.com>
Co-authored-by: Yahor Yuzefovich <yahor@cockroachlabs.com>
Co-authored-by: elliebababa <ellie24.huang@gmail.com>
Co-authored-by: Andrew Werner <awerner32@gmail.com>
@craig craig bot closed this as completed in fe888db Mar 15, 2021
arulajmani added a commit to arulajmani/cockroach that referenced this issue Mar 15, 2021
Previously, when dropping an enum value, we weren't validating if the
enum value was used in a column of array type. This patch fixes the bug.

Fixes cockroachdb#60004

Release justification: bug fix to new functionality
Release note: None
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. GA-blocker
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants