Make the SQLQueryDataSet compatible with mssql. #101

yassineAlouini · 2023-01-19T13:15:28Z

Signed-off-by: Yassine Alouini yalouini@idmog.com

Description

Ability to connect to mssql using the SQLQueryDataSet

Development notes

Tested locally with the following docker image + some SQL commands to create a table =>

docker run --name sql1 -e "ACCEPT_EULA=Y" -e "MSSQL_SA_PASSWORD=<YourStrong@Passw0rd>" -p 1433:1433 -v /data:/var/opt/mssql/data -d mcr.microsoft.com/mssql/server:2022-latest

docker exec -it sql1 bash
/opt/mssql-tools/bin/sqlcmd -S mssql -U SA -P "<YourStrong@Passw0rd>" -Q "CREATE DATABASE TestDB;"
/opt/mssql-tools/bin/sqlcmd -S localhost -U SA -P "<YourStrong@Passw0rd>" -Q "USE TestDB; CREATE TABLE DateTable (v INT, t DATETIME);"
/opt/mssql-tools/bin/sqlcmd -S localhost -U SA -P "<YourStrong@Passw0rd>" -Q "USE TestDB; INSERT INTO DateTable VALUES (10, '2022-2-2 00:00:00');"

Checklist

Opened this PR as a 'Draft Pull Request' if it is work-in-progress
Updated the documentation to reflect the code changes
Added a description of this change in the relevant RELEASE.md file
Added tests to cover my changes

kedro-datasets/kedro_datasets/pandas/sql_dataset.py

yassineAlouini · 2023-01-30T15:12:46Z

@ankatiyar Some new commits, let me know if other things are missing and/or can be improved. Thanks. :)

AhdraMeraliQB

Thanks for adding this change @yassineAlouini 🌟 I'll come back and test the example in the docstring but before that I just had a couple notes

AhdraMeraliQB · 2023-02-10T13:26:51Z

kedro-datasets/tests/pandas/test_sql_dataset.py

+        load_args = {
+            "params": ["2023-01-01", "2023-01-01T20:26", "2023", "test", 1.0, 100]
+        }
+        ds = SQLQueryDataSet(


The unit tests are failing as the init method makes a call to create_connection() (and in turn create_engine()) which results in an import error. I suspect you are missing a mock engine as you've used in the test above 🤔

Oh right, that's a good catch, thanks. It was working locally so I forgot about the mock, thanks for pointing this. 👍

AhdraMeraliQB · 2023-02-10T13:29:51Z

kedro-datasets/tests/pandas/test_sql_dataset.py

+        """Test that the adapt_mssql_date_params
+        function transforms the params as expected, i.e.
+        making datetime date into the format %Y-%m-%dT%H:%M:%S
+        and ignoring the other values.


Could you add a test that checks that other formats provided are ignored ?

These are the values tested:

["2023-01-01", "2023-01-01T20:26", "2023", "test", 1.0, 100]

Some of them aren't dates. Maybe you had something else in mind @AhdraMeraliQB? 🤔

Please let me know. 👍

@yassineAlouini I was thinking more along the lines of testing that the appropriate error is thrown when anything other than a list is passed through. This will also complete the code coverage that is preventing the unit tests from passing.

Ah, I see now. Thanks for the clarification. 👍

AhdraMeraliQB · 2023-02-13T16:45:51Z

kedro-datasets/setup.py

@@ -58,6 +58,7 @@ def _collect_requirements(requires):
    "pandas.ParquetDataSet": [PANDAS, "pyarrow>=6.0"],
    "pandas.SQLTableDataSet": [PANDAS, "SQLAlchemy~=1.2"],
    "pandas.SQLQueryDataSet": [PANDAS, "SQLAlchemy~=1.2"],
+    # TODO: Add the mssql dataset requirements here. Or maybe not needed?


Unresolved TODO - you'll need to add pyodbc here and any other dependencies

What is the best practice you use here to make it optional? 🤔

It looks like pyodbc is a relatively small package so it should be fine to include for all cases 👍

AhdraMeraliQB · 2023-02-13T16:51:13Z

kedro-datasets/tests/pandas/test_sql_dataset.py

+        """Test that the adapt_mssql_date_params
+        function transforms the params as expected, i.e.
+        making datetime date into the format %Y-%m-%dT%H:%M:%S
+        and ignoring the other values.


@yassineAlouini I was thinking more along the lines of testing that the appropriate error is thrown when anything other than a list is passed through. This will also complete the code coverage that is preventing the unit tests from passing.

AhdraMeraliQB

@yassineAlouini Thank you for adding the test and this contribution as a whole! I noticed the linter is complaining about a few dictionary declarations - I've commented what would be the fix for them but let me know if this isn't possible in the implementation.

I'll manually test the example in the docstring and after that I'll be happy to approve 🥳 Fantastic work 🌟

AhdraMeraliQB · 2023-02-17T11:22:19Z

kedro-datasets/tests/pandas/test_sql_dataset.py

+            "kedro_datasets.pandas.sql_dataset.SQLQueryDataSet.adapt_mssql_date_params"
+        )
+        mock_engine = mocker.patch("kedro_datasets.pandas.sql_dataset.create_engine")
+        ds = SQLQueryDataSet(sql=SQL_QUERY, credentials=dict(con=MSSQL_CONNECTION))


Suggested change

ds = SQLQueryDataSet(sql=SQL_QUERY, credentials=dict(con=MSSQL_CONNECTION))

ds = SQLQueryDataSet(sql=SQL_QUERY, credentials={"con":MSSQL_CONNECTION})

Oh yes, I saw the linter complaining as well but thought it wasn't a big deal. 😄
Will fix this!

That being said, many more similar patterns exist in the code base but I won't fix them in this MR. @AhdraMeraliQB

AhdraMeraliQB · 2023-02-17T11:23:04Z

kedro-datasets/tests/pandas/test_sql_dataset.py

+            "params": ["2023-01-01", "2023-01-01T20:26", "2023", "test", 1.0, 100]
+        }
+        ds = SQLQueryDataSet(
+            sql=SQL_QUERY, credentials=dict(con=MSSQL_CONNECTION), load_args=load_args


Suggested change

sql=SQL_QUERY, credentials=dict(con=MSSQL_CONNECTION), load_args=load_args

sql=SQL_QUERY, credentials={"con":MSSQL_CONNECTION}, load_args=load_args

AhdraMeraliQB · 2023-02-17T11:26:02Z

kedro-datasets/tests/pandas/test_sql_dataset.py

+        with pytest.raises(DataSetError, match=pattern):
+            SQLQueryDataSet(
+                sql=SQL_QUERY,
+                credentials=dict(con=MSSQL_CONNECTION),


Suggested change

credentials=dict(con=MSSQL_CONNECTION),

credentials={"con":MSSQL_CONNECTION},

kedro-datasets/kedro_datasets/pandas/sql_dataset.py

SajidAlamQB

I tested the docstring example manually all looks good. Thank you @yassineAlouini for this!

AhdraMeraliQB

@yassineAlouini This looks fantastic, we're happy to get this merged in!

I believe the final hurdle is the DCO - something in the new commits must've triggered it. The instructions for resolution are here: https://github.com/kedro-org/kedro-plugins/pull/101/checks.

Awesome work! 🥳

yassineAlouini · 2023-02-24T13:19:15Z

@yassineAlouini This looks fantastic, we're happy to get this merged in!

I believe the final hurdle is the DCO - something in the new commits must've triggered it. The instructions for resolution are here: https://github.com/kedro-org/kedro-plugins/pull/101/checks.

Awesome work! partying_face

Yes, I think the suggestion I have merged wasn't signed-off. I tried to revert back but it didn't work. I will fix this very shortly. Thanks for the different review rounds. 🎉

* [kedro-docker] Layers size optimization Signed-off-by: Mariusz Strzelecki <mariusz.strzelecki@getindata.com> * Adjust test requirements Signed-off-by: Mariusz Strzelecki <mariusz.strzelecki@getindata.com> * Skip coverage check on tests dir (some do not execute on Windows) Signed-off-by: Mariusz Strzelecki <mariusz.strzelecki@getindata.com> * Update .coveragerc with the setup Signed-off-by: Mariusz Strzelecki <mariusz.strzelecki@getindata.com> * Fix bandit so it does not scan kedro-datasets Signed-off-by: Mariusz Strzelecki <mariusz.strzelecki@getindata.com> * Fixed existence test Signed-off-by: Mariusz Strzelecki <mariusz.strzelecki@getindata.com> * Check why dir is not created Signed-off-by: Mariusz Strzelecki <mariusz.strzelecki@getindata.com> * Kedro starters are fixed now Signed-off-by: Mariusz Strzelecki <mariusz.strzelecki@getindata.com> * Increased no-output-timeout for long spark image build Signed-off-by: Mariusz Strzelecki <mariusz.strzelecki@getindata.com> * Spark image optimized Signed-off-by: Mariusz Strzelecki <szczeles@gmail.com> * Linting Signed-off-by: Mariusz Strzelecki <szczeles@gmail.com> * Switch to slim image always Signed-off-by: Mariusz Strzelecki <szczeles@gmail.com> * Trigger build Signed-off-by: Mariusz Strzelecki <szczeles@gmail.com> * Use textwrap.dedent for nicer indentation Signed-off-by: Mariusz Strzelecki <szczeles@gmail.com> * Revert "Use textwrap.dedent for nicer indentation" This reverts commit 3a1e3f8. Signed-off-by: Mariusz Strzelecki <szczeles@gmail.com> * Revert "Revert "Use textwrap.dedent for nicer indentation"" This reverts commit d322d35. Signed-off-by: Mariusz Strzelecki <szczeles@gmail.com> * Make tests read more lines (to skip all deprecation warnings) Signed-off-by: Mariusz Strzelecki <szczeles@gmail.com> Signed-off-by: Mariusz Strzelecki <mariusz.strzelecki@getindata.com> Signed-off-by: Mariusz Strzelecki <szczeles@gmail.com> Signed-off-by: Yassine Alouini <yalouini@idmog.com>

* Add release notes for kedro-docker 0.3.1 Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Update version in kedro_docker module Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> Signed-off-by: Yassine Alouini <yalouini@idmog.com>

Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com> Signed-off-by: Yassine Alouini <yalouini@idmog.com>

Signed-off-by: Yassine Alouini <yalouini@idmog.com>

…mog.com) Signed-off-by: Yassine Alouini <yalouini@idmog.com>

Signed-off-by: Yassine Alouini <yalouini@idmog.com>

Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com> Signed-off-by: Yassine Alouini <yalouini@idmog.com>

…org#99) * Add non-spark related test changes Replace kedro.pipeline.Pipeline with kedro.pipeline.modular_pipeline.pipeline factory. This is for symmetry with changes made to the main kedro library. Signed-off-by: Adam Farley <adamfrly@gmail.com> Signed-off-by: Yassine Alouini <yalouini@idmog.com>

* fix links * fix dill links Signed-off-by: Yassine Alouini <yalouini@idmog.com>

* Fix docs formatting and phrasing for some datasets Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> * Manually fix files not resolved with patch command Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> * Apply fix from kedro-org#98 Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> --------- Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> Signed-off-by: Yassine Alouini <yalouini@idmog.com>

* bump version and update release notes * fix pylint errors Signed-off-by: Yassine Alouini <yalouini@idmog.com>

Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com> Signed-off-by: Yassine Alouini <yalouini@idmog.com>

* Prefix Docker plugin name with "Kedro-" in usage message Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> Signed-off-by: Yassine Alouini <yalouini@idmog.com>

…o-org#56) * Keep Kedro-Docker plugin docstring from appearing in `kedro -h` Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> Signed-off-by: Yassine Alouini <yalouini@idmog.com>

Signed-off-by: wmoreiraa <walber3@gmail.com> Signed-off-by: Yassine Alouini <yalouini@idmog.com>

…dro-org#54) Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> Signed-off-by: Yassine Alouini <yalouini@idmog.com>

Signed-off-by: Yassine Alouini <yalouini@idmog.com>

yassineAlouini · 2023-02-24T13:38:19Z

@AhdraMeraliQB Alright, I've fixed the DCO issue. It seems there are conflicts with the main branch but can't see them locally. Could you please fix them? Thanks. 👍

* [kedro-docker] Layers size optimization (kedro-org#92) * [kedro-docker] Layers size optimization Signed-off-by: Mariusz Strzelecki <mariusz.strzelecki@getindata.com> * Adjust test requirements Signed-off-by: Mariusz Strzelecki <mariusz.strzelecki@getindata.com> * Skip coverage check on tests dir (some do not execute on Windows) Signed-off-by: Mariusz Strzelecki <mariusz.strzelecki@getindata.com> * Update .coveragerc with the setup Signed-off-by: Mariusz Strzelecki <mariusz.strzelecki@getindata.com> * Fix bandit so it does not scan kedro-datasets Signed-off-by: Mariusz Strzelecki <mariusz.strzelecki@getindata.com> * Fixed existence test Signed-off-by: Mariusz Strzelecki <mariusz.strzelecki@getindata.com> * Check why dir is not created Signed-off-by: Mariusz Strzelecki <mariusz.strzelecki@getindata.com> * Kedro starters are fixed now Signed-off-by: Mariusz Strzelecki <mariusz.strzelecki@getindata.com> * Increased no-output-timeout for long spark image build Signed-off-by: Mariusz Strzelecki <mariusz.strzelecki@getindata.com> * Spark image optimized Signed-off-by: Mariusz Strzelecki <szczeles@gmail.com> * Linting Signed-off-by: Mariusz Strzelecki <szczeles@gmail.com> * Switch to slim image always Signed-off-by: Mariusz Strzelecki <szczeles@gmail.com> * Trigger build Signed-off-by: Mariusz Strzelecki <szczeles@gmail.com> * Use textwrap.dedent for nicer indentation Signed-off-by: Mariusz Strzelecki <szczeles@gmail.com> * Revert "Use textwrap.dedent for nicer indentation" This reverts commit 3a1e3f8. Signed-off-by: Mariusz Strzelecki <szczeles@gmail.com> * Revert "Revert "Use textwrap.dedent for nicer indentation"" This reverts commit d322d35. Signed-off-by: Mariusz Strzelecki <szczeles@gmail.com> * Make tests read more lines (to skip all deprecation warnings) Signed-off-by: Mariusz Strzelecki <szczeles@gmail.com> Signed-off-by: Mariusz Strzelecki <mariusz.strzelecki@getindata.com> Signed-off-by: Mariusz Strzelecki <szczeles@gmail.com> Signed-off-by: Yassine Alouini <yalouini@idmog.com> * Release Kedro-Docker 0.3.1 (kedro-org#94) * Add release notes for kedro-docker 0.3.1 Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Update version in kedro_docker module Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> Signed-off-by: Yassine Alouini <yalouini@idmog.com> * Bump version and update release notes (kedro-org#96) Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com> Signed-off-by: Yassine Alouini <yalouini@idmog.com> * Make the SQLQueryDataSet compatible with mssql. Signed-off-by: Yassine Alouini <yalouini@idmog.com> * Add one test + update RELEASE.md. Signed-off-by: Yassine Alouini <yalouini@idmog.com> * Add missing pyodbc for tests. Signed-off-by: Yassine Alouini <yalouini@idmog.com> * Mock connection as well. Signed-off-by: Yassine Alouini <yalouini@idmog.com> * Add more dates parsing for mssql backend (thanks to fgaudindelrieu@idmog.com) Signed-off-by: Yassine Alouini <yalouini@idmog.com> * Fix an error in docstring of MetricsDataSet (kedro-org#98) Signed-off-by: Yassine Alouini <yalouini@idmog.com> * Bump relax pyarrow version to work the same way as Pandas (kedro-org#100) * Bump relax pyarrow version to work the same way as Pandas We only use PyArrow for `pandas.ParquetDataSet` as such I suggest we keep our versions pinned to the same range as [Pandas does](https://github.com/pandas-dev/pandas/blob/96fc51f5ec678394373e2c779ccff37ddb966e75/pyproject.toml#L100) for the same reason. As such I suggest we remove the upper bound as we have users requesting later versions in [support channels](https://kedro-org.slack.com/archives/C03RKP2LW64/p1674040509133529) * Updated release notes Signed-off-by: Yassine Alouini <yalouini@idmog.com> * Add missing type in catalog example. Signed-off-by: Yassine Alouini <yalouini@idmog.com> * Add one more unit tests for adapt_mssql. Signed-off-by: Yassine Alouini <yalouini@idmog.com> * [FIX] Add missing mocker from date test. Signed-off-by: Yassine Alouini <yalouini@idmog.com> * [TEST] Add a wrong input test. Signed-off-by: Yassine Alouini <yalouini@idmog.com> * Add pyodbc dependency. Signed-off-by: Yassine Alouini <yalouini@idmog.com> * [FIX] Remove dict() in tests. Signed-off-by: Yassine Alouini <yalouini@idmog.com> * Change check to check on plugin name (kedro-org#103) Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com> Signed-off-by: Yassine Alouini <yalouini@idmog.com> * Set coverage in pyproject.toml (kedro-org#105) Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com> Signed-off-by: Yassine Alouini <yalouini@idmog.com> * Move coverage settings to pyproject.toml (kedro-org#106) Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com> Signed-off-by: Yassine Alouini <yalouini@idmog.com> * Replace kedro.pipeline with modular_pipeline.pipeline factory (kedro-org#99) * Add non-spark related test changes Replace kedro.pipeline.Pipeline with kedro.pipeline.modular_pipeline.pipeline factory. This is for symmetry with changes made to the main kedro library. Signed-off-by: Adam Farley <adamfrly@gmail.com> Signed-off-by: Yassine Alouini <yalouini@idmog.com> * Fix outdated links in Kedro Datasets (kedro-org#111) * fix links * fix dill links Signed-off-by: Yassine Alouini <yalouini@idmog.com> * Fix docs formatting and phrasing for some datasets (kedro-org#107) * Fix docs formatting and phrasing for some datasets Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> * Manually fix files not resolved with patch command Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> * Apply fix from kedro-org#98 Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> --------- Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> Signed-off-by: Yassine Alouini <yalouini@idmog.com> * Release `kedro-datasets` `version 1.0.2` (kedro-org#112) * bump version and update release notes * fix pylint errors Signed-off-by: Yassine Alouini <yalouini@idmog.com> * Bump pytest to 7.2 (kedro-org#113) Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com> Signed-off-by: Yassine Alouini <yalouini@idmog.com> * Prefix Docker plugin name with "Kedro-" in usage message (kedro-org#57) * Prefix Docker plugin name with "Kedro-" in usage message Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> Signed-off-by: Yassine Alouini <yalouini@idmog.com> * Keep Kedro-Docker plugin docstring from appearing in `kedro -h` (kedro-org#56) * Keep Kedro-Docker plugin docstring from appearing in `kedro -h` Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> Signed-off-by: Yassine Alouini <yalouini@idmog.com> * [kedro-datasets ] Add `Polars.CSVDataSet` (kedro-org#95) Signed-off-by: wmoreiraa <walber3@gmail.com> Signed-off-by: Yassine Alouini <yalouini@idmog.com> * Remove deprecated `test_requires` from `setup.py` in Kedro-Docker (kedro-org#54) Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> Signed-off-by: Yassine Alouini <yalouini@idmog.com> * [FIX] Fix ds to data_set. Signed-off-by: Yassine Alouini <yalouini@idmog.com> --------- Signed-off-by: Mariusz Strzelecki <mariusz.strzelecki@getindata.com> Signed-off-by: Mariusz Strzelecki <szczeles@gmail.com> Signed-off-by: Yassine Alouini <yalouini@idmog.com> Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com> Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> Co-authored-by: Mariusz Strzelecki <szczeles@gmail.com> Co-authored-by: Jannic <37243923+jmholzer@users.noreply.github.com> Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com> Co-authored-by: OKA Naoya <pn11@users.noreply.github.com> Co-authored-by: Joel <35801847+datajoely@users.noreply.github.com> Co-authored-by: adamfrly <45516720+adamfrly@users.noreply.github.com> Co-authored-by: Sajid Alam <90610031+SajidAlamQB@users.noreply.github.com> Co-authored-by: Deepyaman Datta <deepyaman.datta@utexas.edu> Co-authored-by: Walber Moreira <58264877+wmoreiraa@users.noreply.github.com> Signed-off-by: Danny Farah <danny_farah@mckinsey.com>

yassineAlouini changed the title ~~[DRAFT] Make the SQLQueryDataSet compatible with mssql.~~ Make the SQLQueryDataSet compatible with mssql. Jan 23, 2023

yassineAlouini mentioned this pull request Jan 23, 2023

MSSQLQueryDataSet that extends pandas.SQLQueryDataSet #97

Closed

merelcht requested review from jmholzer and ankatiyar January 23, 2023 15:57

ankatiyar reviewed Jan 26, 2023

View reviewed changes

kedro-datasets/kedro_datasets/pandas/sql_dataset.py Outdated Show resolved Hide resolved

ankatiyar reviewed Jan 26, 2023

View reviewed changes

kedro-datasets/kedro_datasets/pandas/sql_dataset.py Outdated Show resolved Hide resolved

merelcht added the Community Issue/PR opened by the open-source community label Feb 6, 2023

merelcht linked an issue Feb 6, 2023 that may be closed by this pull request

MSSQLQueryDataSet that extends pandas.SQLQueryDataSet #97

Closed

merelcht removed the Community Issue/PR opened by the open-source community label Feb 6, 2023

AhdraMeraliQB reviewed Feb 10, 2023

View reviewed changes

AhdraMeraliQB reviewed Feb 13, 2023

View reviewed changes

AhdraMeraliQB reviewed Feb 17, 2023

View reviewed changes

AhdraMeraliQB requested a review from ankatiyar February 17, 2023 16:05

AhdraMeraliQB self-assigned this Feb 20, 2023

AhdraMeraliQB requested a review from SajidAlamQB February 21, 2023 11:22

SajidAlamQB reviewed Feb 23, 2023

View reviewed changes

kedro-datasets/kedro_datasets/pandas/sql_dataset.py Outdated Show resolved Hide resolved

SajidAlamQB approved these changes Feb 23, 2023

View reviewed changes

AhdraMeraliQB approved these changes Feb 23, 2023

View reviewed changes

yassineAlouini force-pushed the mssql-read-dataset branch from f9029c0 to 5a600b8 Compare February 24, 2023 13:26

szczeles and others added 9 commits February 24, 2023 14:27

Bump version and update release notes (kedro-org#96)

580a820

Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com> Signed-off-by: Yassine Alouini <yalouini@idmog.com>

Make the SQLQueryDataSet compatible with mssql.

53f8758

Signed-off-by: Yassine Alouini <yalouini@idmog.com>

Add one test + update RELEASE.md.

c4495d6

Signed-off-by: Yassine Alouini <yalouini@idmog.com>

Add missing pyodbc for tests.

bf38e2f

Signed-off-by: Yassine Alouini <yalouini@idmog.com>

Mock connection as well.

4ad6ce4

Signed-off-by: Yassine Alouini <yalouini@idmog.com>

Add more dates parsing for mssql backend (thanks to fgaudindelrieu@id…

c121f0c

…mog.com) Signed-off-by: Yassine Alouini <yalouini@idmog.com>

Fix an error in docstring of MetricsDataSet (kedro-org#98)

2422765

Signed-off-by: Yassine Alouini <yalouini@idmog.com>

Yassine Alouini and others added 19 commits February 24, 2023 14:27

Add missing type in catalog example.

f0b0a8f

Signed-off-by: Yassine Alouini <yalouini@idmog.com>

Add one more unit tests for adapt_mssql.

3f4458f

Signed-off-by: Yassine Alouini <yalouini@idmog.com>

[FIX] Add missing mocker from date test.

4f95bb8

Signed-off-by: Yassine Alouini <yalouini@idmog.com>

[TEST] Add a wrong input test.

ab70024

Signed-off-by: Yassine Alouini <yalouini@idmog.com>

Add pyodbc dependency.

454d58a

Signed-off-by: Yassine Alouini <yalouini@idmog.com>

[FIX] Remove dict() in tests.

2bd73f7

Signed-off-by: Yassine Alouini <yalouini@idmog.com>

Change check to check on plugin name (kedro-org#103)

93a374e

Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com> Signed-off-by: Yassine Alouini <yalouini@idmog.com>

Set coverage in pyproject.toml (kedro-org#105)

4470e24

Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com> Signed-off-by: Yassine Alouini <yalouini@idmog.com>

Move coverage settings to pyproject.toml (kedro-org#106)

e503bbc

Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com> Signed-off-by: Yassine Alouini <yalouini@idmog.com>

Fix outdated links in Kedro Datasets (kedro-org#111)

60b849d

* fix links * fix dill links Signed-off-by: Yassine Alouini <yalouini@idmog.com>

Release kedro-datasets version 1.0.2 (kedro-org#112)

ff43cb5

* bump version and update release notes * fix pylint errors Signed-off-by: Yassine Alouini <yalouini@idmog.com>

Bump pytest to 7.2 (kedro-org#113)

4a338a7

Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com> Signed-off-by: Yassine Alouini <yalouini@idmog.com>

Prefix Docker plugin name with "Kedro-" in usage message (kedro-org#57)

fa7b6bb

* Prefix Docker plugin name with "Kedro-" in usage message Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> Signed-off-by: Yassine Alouini <yalouini@idmog.com>

Keep Kedro-Docker plugin docstring from appearing in kedro -h (kedr…

0b1aac5

…o-org#56) * Keep Kedro-Docker plugin docstring from appearing in `kedro -h` Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> Signed-off-by: Yassine Alouini <yalouini@idmog.com>

[kedro-datasets ] Add Polars.CSVDataSet (kedro-org#95)

758a82a

Signed-off-by: wmoreiraa <walber3@gmail.com> Signed-off-by: Yassine Alouini <yalouini@idmog.com>

Remove deprecated test_requires from setup.py in Kedro-Docker (ke…

79f99a5

…dro-org#54) Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> Signed-off-by: Yassine Alouini <yalouini@idmog.com>

[FIX] Fix ds to data_set.

552c878

Signed-off-by: Yassine Alouini <yalouini@idmog.com>

yassineAlouini force-pushed the mssql-read-dataset branch from 5a600b8 to 552c878 Compare February 24, 2023 13:28

Merge main and fix conflicts.

ab9dfa7

Signed-off-by: Yassine Alouini <yalouini@idmog.com>

Merge branch 'main' into mssql-read-dataset

6608e75

AhdraMeraliQB merged commit 4450ce6 into kedro-org:main Feb 27, 2023

yassineAlouini deleted the mssql-read-dataset branch February 27, 2023 09:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make the SQLQueryDataSet compatible with mssql. #101

Make the SQLQueryDataSet compatible with mssql. #101

yassineAlouini commented Jan 19, 2023 •

edited

Loading

yassineAlouini commented Jan 30, 2023

AhdraMeraliQB left a comment

AhdraMeraliQB Feb 10, 2023 •

edited

Loading

yassineAlouini Feb 13, 2023

AhdraMeraliQB Feb 10, 2023

yassineAlouini Feb 13, 2023

AhdraMeraliQB Feb 13, 2023

yassineAlouini Feb 13, 2023

AhdraMeraliQB Feb 13, 2023

yassineAlouini Feb 13, 2023

AhdraMeraliQB Feb 14, 2023

AhdraMeraliQB Feb 13, 2023

AhdraMeraliQB left a comment

AhdraMeraliQB Feb 17, 2023

yassineAlouini Feb 17, 2023

yassineAlouini Feb 17, 2023

AhdraMeraliQB Feb 17, 2023

AhdraMeraliQB Feb 17, 2023

SajidAlamQB left a comment

AhdraMeraliQB left a comment •

edited

Loading

yassineAlouini commented Feb 24, 2023

yassineAlouini commented Feb 24, 2023

	ds = SQLQueryDataSet(sql=SQL_QUERY, credentials=dict(con=MSSQL_CONNECTION))
	ds = SQLQueryDataSet(sql=SQL_QUERY, credentials={"con":MSSQL_CONNECTION})

	sql=SQL_QUERY, credentials=dict(con=MSSQL_CONNECTION), load_args=load_args
	sql=SQL_QUERY, credentials={"con":MSSQL_CONNECTION}, load_args=load_args

	credentials=dict(con=MSSQL_CONNECTION),
	credentials={"con":MSSQL_CONNECTION},

Make the SQLQueryDataSet compatible with mssql. #101

Make the SQLQueryDataSet compatible with mssql. #101

Conversation

yassineAlouini commented Jan 19, 2023 • edited Loading

Description

Development notes

Checklist

yassineAlouini commented Jan 30, 2023

AhdraMeraliQB left a comment

Choose a reason for hiding this comment

AhdraMeraliQB Feb 10, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AhdraMeraliQB left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SajidAlamQB left a comment

Choose a reason for hiding this comment

AhdraMeraliQB left a comment • edited Loading

Choose a reason for hiding this comment

yassineAlouini commented Feb 24, 2023

yassineAlouini commented Feb 24, 2023

yassineAlouini commented Jan 19, 2023 •

edited

Loading

AhdraMeraliQB Feb 10, 2023 •

edited

Loading

AhdraMeraliQB left a comment •

edited

Loading