Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for SQL Variant fields #356

Open
wants to merge 7 commits into
base: dev
Choose a base branch
from
Open

Add support for SQL Variant fields #356

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
7bacf72
Add support for SQL Variant fields
akilude Mar 6, 2024
0e96914
Add SQL_VARIANT const
akilude Mar 6, 2024
136210e
Remove magic number
akilude Mar 6, 2024
d48f07f
Merge branch 'microsoft:dev' into patch-1
akilude Mar 19, 2024
c67001c
Added testcase
akilude Mar 29, 2024
82cb956
Merge branch 'dev' into patch-1
mShan0 Apr 2, 2024
6ada36a
Fix typos
akilude Apr 7, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion mssql/base.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@
from .client import DatabaseClient # noqa
from .creation import DatabaseCreation # noqa
from .features import DatabaseFeatures # noqa
from .introspection import DatabaseIntrospection, SQL_TIMESTAMP_WITH_TIMEZONE # noqa
from .introspection import DatabaseIntrospection, SQL_TIMESTAMP_WITH_TIMEZONE, SQL_VARIANT # noqa
from .operations import DatabaseOperations # noqa
from .schema import DatabaseSchemaEditor # noqa

Expand Down Expand Up @@ -83,6 +83,9 @@ def encode_value(v):
return '{%s}' % (v.replace('}', '}}'),)
return v

def handle_sql_variant_as_string(value):
# SQL variant of type 150 is not supported, convert it to string and return
return value.decode('utf-16le')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not too familiar with the sql_variant type but it looks like this method of decoding is likely to result in incorrect data.

Since sql_variant can contain multiple types, unpacking the data in a reliable manner seems difficult... We might be better off doing an explicit CAST in sql.

Copy link
Author

@akilude akilude Apr 7, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dauinsight We can find the base type of the field https://learn.microsoft.com/en-us/sql/t-sql/functions/sql-variant-property-transact-sql?view=sql-server-ver16

in the converter function, shall we convert whatever type to string and return.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dauinsight, please let me know the best way to proceed. Thanks.


def handle_datetimeoffset(dto_value):
# Decode bytes returned from SQL Server
Expand Down Expand Up @@ -381,6 +384,10 @@ def get_new_connection(self, conn_params):
# Handling values from DATETIMEOFFSET columns
# source: https://github.com/mkleehammer/pyodbc/wiki/Using-an-Output-Converter-function
conn.add_output_converter(SQL_TIMESTAMP_WITH_TIMEZONE, handle_datetimeoffset)

# add support for sql_variant fields
conn.add_output_converter(SQL_VARIANT, handle_sql_variant_as_string)

conn.timeout = query_timeout
if setencoding:
for entry in setencoding:
Expand Down
1 change: 1 addition & 0 deletions mssql/introspection.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
SQL_BIGAUTOFIELD = -777444
SQL_SMALLAUTOFIELD = -777333
SQL_TIMESTAMP_WITH_TIMEZONE = -155
SQL_VARIANT = -150

FieldInfo = namedtuple("FieldInfo", BaseFieldInfo._fields + ("comment",))
TableInfo = namedtuple("TableInfo", BaseTableInfo._fields + ("comment",))
Expand Down
13 changes: 13 additions & 0 deletions testapp/tests/test_fields.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
from django.test import TestCase

from ..models import UUIDModel, Customer_name, Customer_address
from django.db import connections


class TestUUIDField(TestCase):
Expand Down Expand Up @@ -34,3 +35,15 @@ def test_random_order_by(self):
names.append(list(Customer_name.objects.order_by('?')))

self.assertNotEqual(names.count(names[0]), 20)


class TestSQLVariant(TestCase):
def test_sql_variant(self):
connection = connections['default']
with connection.cursor() as cursor:
cursor.execute("CREATE TABLE sqlVariantTest(targetCol sql_variant, colB INT)")
cursor.execute("INSERT INTO sqlVariantTest values (CAST(46279.1 as decimal(8,2)), 1689)")
cursor.execute("SELECT targetCol FROM sqlVariantTest")

rows = cursor.fetchall()
self.assertEqual(len(rows), 1)
Loading