sql: move alter column type general support out of experimental #49329

RichardJCai · 2020-05-20T15:49:35Z

blathers-crl · 2020-05-20T15:49:38Z

Hi @RichardJCai, I've guessed the C-ategory of your issue and suitably labeled it. Please re-label if inaccurate.

While you're here, please consider adding an A- label to help keep our repository tidy.

_{🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is otan.}

timgraham · 2020-07-27T22:48:28Z

When running Django's test suite, I'm seeing FeatureNotSupported errors pointing to this issue that weren't present before 20.2 alpha 2.

ALTER COLUMN TYPE from varchar to string is only supported experimentally
ALTER COLUMN TYPE from int to int is only supported experimentally
ALTER COLUMN TYPE from int2 to int2 is only supported experimentally

Is this a regression? I'll provide the exact SQL statements if needed.

RichardJCai · 2020-07-27T22:52:15Z

When running Django's test suite, I'm seeing FeatureNotSupported errors pointing to this issue that weren't present before 20.2 alpha 2.

ALTER COLUMN TYPE from varchar to string is only supported experimentally

ALTER COLUMN TYPE from int to int is only supported experimentally

ALTER COLUMN TYPE from int2 to int2 is only supported experimentally

Is this a regression? I'll provide the exact SQL statements if needed.

Yeah this is a regression, while adding experimental support for other cases of alter column type, I wrapped all uses of alter column type with the error, will fix this soon.

RichardJCai · 2020-07-28T18:07:25Z

When running Django's test suite, I'm seeing FeatureNotSupported errors pointing to this issue that weren't present before 20.2 alpha 2.

ALTER COLUMN TYPE from varchar to string is only supported experimentally

ALTER COLUMN TYPE from int to int is only supported experimentally

ALTER COLUMN TYPE from int2 to int2 is only supported experimentally

Is this a regression? I'll provide the exact SQL statements if needed.

Actually can you provide the statements please? Curious if you're providing an expression to alter the column type or not.

timgraham · 2020-07-28T18:29:08Z

ALTER TABLE "schema_author" ALTER COLUMN "name" TYPE text USING "name"::text;
ALTER TABLE "schema_uniquetest" ALTER COLUMN "year" TYPE bigint USING "year"::bigint;
ALTER TABLE "schema_smallintegerpk" ALTER COLUMN "i" TYPE smallint USING "i"::smallint;

RichardJCai · 2020-07-30T18:31:25Z

@timgraham actually this isn't necessarily a regression - previously the logic did not take into account the USING expression at all.
ALTER TABLE "schema_author" ALTER COLUMN "name" TYPE text USING "name"::text; was equivalent to just
ALTER TABLE "schema_author" ALTER COLUMN "name" TYPE text

Now if any expression is provided, we force a change to happen. The examples you gave, the expression is ignored and the ALTER TABLE is a no-op.

How important is this?

cockroachdb/cockroach#49329 (comment)

cockroachdb/cockroach#49329

timgraham · 2020-08-01T12:09:28Z

Maybe Django doesn't need to add a USING clause (that code was copied from the PostgreSQL backend). Removing that resolves the crashes.

diff --git a/django_cockroachdb/schema.py b/django_cockroachdb/schema.py
index 11d90dc..ca2a9da 100644
--- a/django_cockroachdb/schema.py
+++ b/django_cockroachdb/schema.py
@@ -44,9 +44,6 @@ class DatabaseSchemaEditor(PostgresDatabaseSchemaEditor):

     def _alter_column_type_sql(self, model, old_field, new_field, new_type):
         self.sql_alter_column_type = 'ALTER COLUMN %(column)s TYPE %(type)s'
-        # Cast when data type changed.
-        if self._field_data_type(old_field) != self._field_data_type(new_field):
-            self.sql_alter_column_type += ' USING %(column)s::%(type)s'
         # Make ALTER TYPE with SERIAL make sense.
         # table = strip_quotes(model._meta.db_table)
         serial_fields_map = {'bigserial': 'bigint', 'serial': 'integer', 'smallserial': 'smallint'}

Incidentally, I added "SET enable_experimental_alter_column_type_general = true" and enabled some type change tests without issue.

The remaining type conversion test failures are covered by #47636.

timgraham · 2020-08-04T01:20:55Z

For what it's worth, here are the failures on PostgreSQL when I remove the USING clause. I guess CockroachDB does these casts implicitly.

======================================================================
ERROR: test_alter_text_field_to_date_field (schema.tests.SchemaTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tim/code/django/django/db/backends/utils.py", line 86, in _execute
    return self.cursor.execute(sql, params)
psycopg2.errors.DatatypeMismatch: column "info" cannot be cast automatically to type date
HINT:  You might need to specify "USING info::date".


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/tim/code/django/tests/schema/tests.py", line 778, in test_alter_text_field_to_date_field
    editor.alter_field(Note, old_field, new_field, strict=True)
  File "/home/tim/code/django/django/db/backends/base/schema.py", line 567, in alter_field
    old_db_params, new_db_params, strict)
  File "/home/tim/code/django/django/db/backends/postgresql/schema.py", line 154, in _alter_field
    new_db_params, strict,
  File "/home/tim/code/django/django/db/backends/base/schema.py", line 717, in _alter_field
    params,
  File "/home/tim/code/django/django/db/backends/base/schema.py", line 142, in execute
    cursor.execute(sql, params)
  File "/home/tim/code/django/django/db/backends/utils.py", line 68, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/home/tim/code/django/django/db/backends/utils.py", line 77, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/home/tim/code/django/django/db/backends/utils.py", line 86, in _execute
    return self.cursor.execute(sql, params)
  File "/home/tim/code/django/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/home/tim/code/django/django/db/backends/utils.py", line 86, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: column "info" cannot be cast automatically to type date
HINT:  You might need to specify "USING info::date".


======================================================================
ERROR: test_alter_text_field_to_datetime_field (schema.tests.SchemaTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tim/code/django/django/db/backends/utils.py", line 86, in _execute
    return self.cursor.execute(sql, params)
psycopg2.errors.DatatypeMismatch: column "info" cannot be cast automatically to type timestamp with time zone
HINT:  You might need to specify "USING info::timestamp with time zone".


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/tim/code/django/tests/schema/tests.py", line 794, in test_alter_text_field_to_datetime_field
    editor.alter_field(Note, old_field, new_field, strict=True)
  File "/home/tim/code/django/django/db/backends/base/schema.py", line 567, in alter_field
    old_db_params, new_db_params, strict)
  File "/home/tim/code/django/django/db/backends/postgresql/schema.py", line 154, in _alter_field
    new_db_params, strict,
  File "/home/tim/code/django/django/db/backends/base/schema.py", line 717, in _alter_field
    params,
  File "/home/tim/code/django/django/db/backends/base/schema.py", line 142, in execute
    cursor.execute(sql, params)
  File "/home/tim/code/django/django/db/backends/utils.py", line 68, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/home/tim/code/django/django/db/backends/utils.py", line 77, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/home/tim/code/django/django/db/backends/utils.py", line 86, in _execute
    return self.cursor.execute(sql, params)
  File "/home/tim/code/django/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/home/tim/code/django/django/db/backends/utils.py", line 86, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: column "info" cannot be cast automatically to type timestamp with time zone
HINT:  You might need to specify "USING info::timestamp with time zone".


======================================================================
ERROR: test_alter_text_field_to_time_field (schema.tests.SchemaTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tim/code/django/django/db/backends/utils.py", line 86, in _execute
    return self.cursor.execute(sql, params)
psycopg2.errors.DatatypeMismatch: column "info" cannot be cast automatically to type time without time zone
HINT:  You might need to specify "USING info::time without time zone".


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/tim/code/django/tests/schema/tests.py", line 810, in test_alter_text_field_to_time_field
    editor.alter_field(Note, old_field, new_field, strict=True)
  File "/home/tim/code/django/django/db/backends/base/schema.py", line 567, in alter_field
    old_db_params, new_db_params, strict)
  File "/home/tim/code/django/django/db/backends/postgresql/schema.py", line 154, in _alter_field
    new_db_params, strict,
  File "/home/tim/code/django/django/db/backends/base/schema.py", line 717, in _alter_field
    params,
  File "/home/tim/code/django/django/db/backends/base/schema.py", line 142, in execute
    cursor.execute(sql, params)
  File "/home/tim/code/django/django/db/backends/utils.py", line 68, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/home/tim/code/django/django/db/backends/utils.py", line 77, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/home/tim/code/django/django/db/backends/utils.py", line 86, in _execute
    return self.cursor.execute(sql, params)
  File "/home/tim/code/django/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/home/tim/code/django/django/db/backends/utils.py", line 86, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: column "info" cannot be cast automatically to type time without time zone
HINT:  You might need to specify "USING info::time without time zone".


======================================================================
ERROR: test_char_field_with_db_index_to_fk (schema.tests.SchemaTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tim/code/django/django/db/backends/utils.py", line 86, in _execute
    return self.cursor.execute(sql, params)
psycopg2.errors.DatatypeMismatch: column "char_field_id" cannot be cast automatically to type integer
HINT:  You might need to specify "USING char_field_id::integer".


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/tim/code/django/django/test/testcases.py", line 1229, in skip_wrapper
    return test_func(*args, **kwargs)
  File "/home/tim/code/django/tests/schema/tests.py", line 330, in test_char_field_with_db_index_to_fk
    editor.alter_field(AuthorCharFieldWithIndex, old_field, new_field, strict=True)
  File "/home/tim/code/django/django/db/backends/base/schema.py", line 567, in alter_field
    old_db_params, new_db_params, strict)
  File "/home/tim/code/django/django/db/backends/postgresql/schema.py", line 154, in _alter_field
    new_db_params, strict,
  File "/home/tim/code/django/django/db/backends/base/schema.py", line 717, in _alter_field
    params,
  File "/home/tim/code/django/django/db/backends/base/schema.py", line 142, in execute
    cursor.execute(sql, params)
  File "/home/tim/code/django/django/db/backends/utils.py", line 68, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/home/tim/code/django/django/db/backends/utils.py", line 77, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/home/tim/code/django/django/db/backends/utils.py", line 86, in _execute
    return self.cursor.execute(sql, params)
  File "/home/tim/code/django/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/home/tim/code/django/django/db/backends/utils.py", line 86, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: column "char_field_id" cannot be cast automatically to type integer
HINT:  You might need to specify "USING char_field_id::integer".


======================================================================
ERROR: test_text_field_with_db_index_to_fk (schema.tests.SchemaTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tim/code/django/django/db/backends/utils.py", line 86, in _execute
    return self.cursor.execute(sql, params)
psycopg2.errors.DatatypeMismatch: column "text_field_id" cannot be cast automatically to type integer
HINT:  You might need to specify "USING text_field_id::integer".


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/tim/code/django/django/test/testcases.py", line 1229, in skip_wrapper
    return test_func(*args, **kwargs)
  File "/home/tim/code/django/django/test/testcases.py", line 1229, in skip_wrapper
    return test_func(*args, **kwargs)
  File "/home/tim/code/django/tests/schema/tests.py", line 345, in test_text_field_with_db_index_to_fk
    editor.alter_field(AuthorTextFieldWithIndex, old_field, new_field, strict=True)
  File "/home/tim/code/django/django/db/backends/base/schema.py", line 567, in alter_field
    old_db_params, new_db_params, strict)
  File "/home/tim/code/django/django/db/backends/postgresql/schema.py", line 154, in _alter_field
    new_db_params, strict,
  File "/home/tim/code/django/django/db/backends/base/schema.py", line 717, in _alter_field
    params,
  File "/home/tim/code/django/django/db/backends/base/schema.py", line 142, in execute
    cursor.execute(sql, params)
  File "/home/tim/code/django/django/db/backends/utils.py", line 68, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/home/tim/code/django/django/db/backends/utils.py", line 77, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/home/tim/code/django/django/db/backends/utils.py", line 86, in _execute
    return self.cursor.execute(sql, params)
  File "/home/tim/code/django/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/home/tim/code/django/django/db/backends/utils.py", line 86, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: column "text_field_id" cannot be cast automatically to type integer
HINT:  You might need to specify "USING text_field_id::integer".

cockroachdb/cockroach#49329 (comment)

cockroachdb/cockroach#49329

cockroachdb/cockroach#49329 (comment)

cockroachdb/cockroach#49329

cockroachdb/cockroach#49329 (comment)

cockroachdb/cockroach#49329

cockroachdb/cockroach#49329 (comment)

cockroachdb/cockroach#49329

cockroachdb/cockroach#49329 (comment)

cockroachdb/cockroach#49329

cockroachdb/cockroach#49329 (comment)

cockroachdb/cockroach#49329

cockroachdb/cockroach#49329 (comment)

cockroachdb/cockroach#49329

cockroachdb/cockroach#49329 (comment)

cockroachdb/cockroach#49329

cockroachdb/cockroach#49329 (comment)

intech · 2022-01-26T11:41:21Z

Text mismatch in primary key error with pg dialect, in crdb:

alter table "t1" alter column "id" drop not null - column "id" is in a primary index

in pg:

alter table "t1" alter column "id" drop not null - column "id" is in a primary key

Check on CockroachDB CCL v21.2.4 (x86_64-unknown-linux-gnu, built 2022/01/10 18:50:15, go1.16.6)
DDL:

CREATE TABLE public.t1 (
    id INT8 NOT NULL,
    rowid INT8 NOT VISIBLE NOT NULL DEFAULT unique_rowid(),
    CONSTRAINT t1_pkey PRIMARY KEY (id ASC),
    FAMILY "primary" (id, rowid)
)

In query:

SET enable_experimental_alter_column_type_general = true;
alter table "t1" alter column "id" type integer using ("id"::integer);

knz · 2022-10-31T16:48:23Z

Added the following two items to the bullet list at top, as per request from @dbist 👍

Add NOTICE for changes that are data-destructive (e.g reducing field width).
Block data-destructive changes unless sql_safe_updates is false.

cornfeedhobo · 2024-03-06T17:50:13Z

Any hope?

blathers-crl bot added the C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) label May 20, 2020

RichardJCai self-assigned this May 20, 2020

RichardJCai changed the title ~~sql: move alter column type non-trivial out of experimental~~ sql: move alter column type general support out of experimental May 21, 2020

timgraham added a commit to timgraham/django-cockroachdb that referenced this issue Aug 1, 2020

drop ALTER COLUMN USING clause

72d8259

cockroachdb/cockroach#49329 (comment)

timgraham added a commit to timgraham/django-cockroachdb that referenced this issue Aug 1, 2020

enable experimential type conversion

8d808a8

cockroachdb/cockroach#49329

timgraham added a commit to timgraham/django-cockroachdb that referenced this issue Aug 6, 2020

drop ALTER COLUMN USING clause

fcc744d

cockroachdb/cockroach#49329 (comment)

timgraham added a commit to timgraham/django-cockroachdb that referenced this issue Aug 6, 2020

enable experimential type conversion

7e23d4a

cockroachdb/cockroach#49329

timgraham added a commit to timgraham/django-cockroachdb that referenced this issue Aug 19, 2020

drop ALTER COLUMN USING clause

a3679c9

cockroachdb/cockroach#49329 (comment)

timgraham added a commit to timgraham/django-cockroachdb that referenced this issue Aug 19, 2020

enable experimential type conversion

e8dbeab

cockroachdb/cockroach#49329

timgraham added a commit to timgraham/django-cockroachdb that referenced this issue Aug 25, 2020

drop ALTER COLUMN USING clause

6704cd4

cockroachdb/cockroach#49329 (comment)

timgraham added a commit to timgraham/django-cockroachdb that referenced this issue Aug 25, 2020

enable experimential type conversion

47c3878

cockroachdb/cockroach#49329

timgraham added a commit to timgraham/django-cockroachdb that referenced this issue Aug 26, 2020

drop ALTER COLUMN USING clause

a5fbfb6

cockroachdb/cockroach#49329 (comment)

timgraham added a commit to timgraham/django-cockroachdb that referenced this issue Aug 26, 2020

enable experimential type conversion

91d1aef

cockroachdb/cockroach#49329

timgraham added a commit to timgraham/django-cockroachdb that referenced this issue Aug 28, 2020

drop ALTER COLUMN USING clause

07af28b

cockroachdb/cockroach#49329 (comment)

timgraham added a commit to timgraham/django-cockroachdb that referenced this issue Aug 28, 2020

enable experimential type conversion

6b880d8

cockroachdb/cockroach#49329

timgraham added a commit to timgraham/django-cockroachdb that referenced this issue Aug 29, 2020

drop ALTER COLUMN USING clause

fc0e2f2

cockroachdb/cockroach#49329 (comment)

timgraham added a commit to timgraham/django-cockroachdb that referenced this issue Aug 29, 2020

enable experimential type conversion

2410992

cockroachdb/cockroach#49329

timgraham added a commit to timgraham/django-cockroachdb that referenced this issue Sep 1, 2020

drop ALTER COLUMN USING clause

eb9fedd

cockroachdb/cockroach#49329 (comment)

timgraham added a commit to timgraham/django-cockroachdb that referenced this issue Sep 1, 2020

enable experimential type conversion

a544e8c

cockroachdb/cockroach#49329

timgraham added a commit to timgraham/django-cockroachdb that referenced this issue Sep 3, 2020

drop ALTER COLUMN USING clause

f1d3a01

cockroachdb/cockroach#49329 (comment)

timgraham added a commit to timgraham/django-cockroachdb that referenced this issue Sep 3, 2020

enable experimential type conversion

0d5dc25

cockroachdb/cockroach#49329

timgraham added a commit to timgraham/django-cockroachdb that referenced this issue Sep 12, 2020

drop ALTER COLUMN USING clause

bcbc04f

cockroachdb/cockroach#49329 (comment)

intech mentioned this issue Dec 5, 2021

Checks Constraints Support knex/knex#4874

Merged

intech mentioned this issue Jan 26, 2022

Add alterType and update index.d.ts for alter function knex/knex#4967

Merged

cpaczek mentioned this issue Feb 3, 2022

Add Cockroach Dialect strapi/strapi#12346

Closed

exalate-issue-sync bot added the T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions) label Mar 11, 2022

rafiss removed the T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions) label Mar 15, 2022

blathers-crl bot added the T-sql-schema-deprecated Use T-sql-foundations instead label Mar 15, 2022

postamar unassigned RichardJCai Mar 16, 2022

postamar added the meta-issue Contains a list of several other issues. label Mar 23, 2022

cpaczek mentioned this issue Apr 16, 2022

Add native support for Cockroach DB strapi/strapi#11738

Closed

ajwerner mentioned this issue Apr 26, 2022

Allow multiple columns in the same table to be modified within the same ALTER statement (one operation) #80578

Open

licitdev mentioned this issue Jul 5, 2022

Can't set default value for fields directus/directus#11832

Closed

3 tasks

postamar assigned Xiang-Gu Jul 20, 2022

gnat mentioned this issue Sep 1, 2022

First class Cockroach DB support. piccolo-orm/piccolo#607

Closed

6 tasks

fabis94 mentioned this issue Sep 7, 2022

feat(postgres): default datetime/timestamp precision setting added knex/knex#5311

Open

dikshant mentioned this issue Nov 3, 2022

sql: support for Strapi content management system #91223

Open

2 tasks

postamar unassigned Xiang-Gu Nov 10, 2022

exalate-issue-sync bot added T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions) and removed T-sql-schema-deprecated Use T-sql-foundations instead labels May 10, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

sql: move alter column type general support out of experimental #49329

sql: move alter column type general support out of experimental #49329

RichardJCai commented May 20, 2020 •

edited by rafiss

Loading

blathers-crl bot commented May 20, 2020

timgraham commented Jul 27, 2020

RichardJCai commented Jul 27, 2020

RichardJCai commented Jul 28, 2020

timgraham commented Jul 28, 2020

RichardJCai commented Jul 30, 2020

timgraham commented Aug 1, 2020

timgraham commented Aug 4, 2020

intech commented Jan 26, 2022 •

edited

Loading

knz commented Oct 31, 2022

cornfeedhobo commented Mar 6, 2024

sql: move alter column type general support out of experimental #49329

sql: move alter column type general support out of experimental #49329

Comments

RichardJCai commented May 20, 2020 • edited by rafiss Loading

blathers-crl bot commented May 20, 2020

timgraham commented Jul 27, 2020

RichardJCai commented Jul 27, 2020

RichardJCai commented Jul 28, 2020

timgraham commented Jul 28, 2020

RichardJCai commented Jul 30, 2020

timgraham commented Aug 1, 2020

timgraham commented Aug 4, 2020

intech commented Jan 26, 2022 • edited Loading

knz commented Oct 31, 2022

cornfeedhobo commented Mar 6, 2024

RichardJCai commented May 20, 2020 •

edited by rafiss

Loading

intech commented Jan 26, 2022 •

edited

Loading