move py.typed to correct places (#403)

* move py.typed to correct places

https://peps.python.org/pep-0561/ says 'For namespace packages (see PEP 420), the py.typed file should be in the submodules of the namespace, to avoid conflicts and for clarity.'. Previously, when I added the py.typed file to this project, #382 , I was unaware this was a namespace package (although, curiously, it seems I had done it right initially and then changed to the wrong way). As PEP 561 warns us, this does create conflicts; other libraries in the databricks namespace package (such as, in my case, databricks-vectorsearch) are then treated as though they are typed, which they are not. This commit moves the py.typed file to the correct places, the submodule folders, fixing that problem.
Signed-off-by: wyattscarpenter <wyattscarpenter@gmail.com>

* change target of mypy to src/databricks instead of src.

I think this might fix the CI code-quality checks failure, but unfortunately I can't replicate that failure locally and the error message is unhelpful

Signed-off-by: wyattscarpenter <wyattscarpenter@gmail.com>

* Possible workaround for bad error message 'error: --install-types failed (no mypy cache directory)'; see python/mypy#10768 (comment)

Signed-off-by: wyattscarpenter <wyattscarpenter@gmail.com>

* fix invalid yaml syntax

Signed-off-by: wyattscarpenter <wyattscarpenter@gmail.com>

* Best fix (#3)

Fixes the problem by cding and supplying a flag to mypy (that mypy needs this flag is seemingly fixed/changed in later versions of mypy; but that's another pr altogether...). Also fixes a type error that was somehow in the arguments of the program (?!) (I guess this is because you guys are still using implicit optional)

---------

Signed-off-by: wyattscarpenter <wyattscarpenter@gmail.com>

* return the old result_links default (#5)

Return the old result_links default, make the type optional, & I'm pretty sure the original problem is that add_file_links can't take a None, so these statements should be in the body of the if-statement that ensures it is not None

Signed-off-by: wyattscarpenter <wyattscarpenter@gmail.com>

* Update src/databricks/sql/utils.py

"self.download_manager is unconditionally used later, so must be created. Looks this part of code is totally not covered with tests 🤔"

Co-authored-by: Levko Kravets <levko.ne@gmail.com>
Signed-off-by: wyattscarpenter <wyattscarpenter@gmail.com>

---------

Signed-off-by: wyattscarpenter <wyattscarpenter@gmail.com>
Co-authored-by: Levko Kravets <levko.ne@gmail.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>

Author: 2 people

.github/workflows/code-quality-checks.yml

@@ -160,4 +160,7 @@ jobs:
       #              mypy the code
       #----------------------------------------------
       - name: Mypy
-        run: poetry run mypy --install-types --non-interactive src
+        run: |
+          cd src # Need to be in the actual databricks/ folder or mypy does the wrong thing.
+          mkdir .mypy_cache # Workaround for bad error message "error: --install-types failed (no mypy cache directory)"; see https://github.com/python/mypy/issues/10768#issuecomment-2178450153
+          poetry run mypy --config-file ../pyproject.toml --install-types --non-interactive --namespace-packages databricks

src/databricks/py.typed renamed to src/databricks/sql/py.typed

@@ -7,7 +7,7 @@
 from collections.abc import Iterable
 from decimal import Decimal
 from enum import Enum
-from typing import Any, Dict, List, Union
+from typing import Any, Dict, List, Optional, Union
 import re
 
 import lz4.frame
@@ -134,7 +134,7 @@ def __init__(
         schema_bytes,
         max_download_threads: int,
         start_row_offset: int = 0,
-        result_links: List[TSparkArrowResultLink] = None,
+        result_links: Optional[List[TSparkArrowResultLink]] = None,
         lz4_compressed: bool = True,
         description: List[List[Any]] = None,
     ):
@@ -168,11 +168,10 @@ def __init__(
                         result_link.startRowOffset, result_link.rowCount
                     )
                 )
-
         self.download_manager = ResultFileDownloadManager(
             self.max_download_threads, self.lz4_compressed
         )
-        self.download_manager.add_file_links(result_links)
+        self.download_manager.add_file_links(result_links or [])
 
         self.table = self._create_next_table()
         self.table_row_index = 0