docs: Review data types (#708)

kaskada-ai · Aug 28, 2023 · 31bbccb · 31bbccb
1 parent 4caa780
commit 31bbccb
Show file tree

Hide file tree

Showing 5 changed files with 15 additions and 9 deletions.
diff --git a/.github/workflows/ci_python.yml b/.github/workflows/ci_python.yml
@@ -132,7 +132,6 @@ jobs:
           source $(poetry env info --path)/bin/activate
           poetry install --with=docs
           pip install 'kaskada[plot]>=0.6.0-a.0' --find-links dist --force-reinstall
-          ls docs/source/_static
           sphinx-build docs/source docs/_build -j auto -W
           deactivate
       - name: Upload docs

diff --git a/python/docs/source/examples/time_centric.ipynb b/python/docs/source/examples/time_centric.ipynb
@@ -173,12 +173,8 @@
    "id": "c2c5a298",
    "metadata": {},
    "source": [
-    "Kaskada query language is parsed by the `fenl` extension. Query calculations are\n",
-    "defined in a code blocks starting with `%%fenl`.\n",
-    "\n",
-    "See [the `fenl`\n",
-    "documentation](https://kaskada-ai.github.io/docs-site/kaskada/main/fenl/fenl-quick-start.html)\n",
-    "for more information.\n",
+    "Kaskada queries are defined in Python, using the `{py}Timestream` class.\n",
+    "Sources are Timestreams generally containing [records](../guide/data_types.md#record-types).\n",
     "\n",
     "Let's do a simple query for events for a specific entity ID.\n"
    ]

diff --git a/python/docs/source/guide/data_types.md b/python/docs/source/guide/data_types.md
@@ -1,7 +1,7 @@
 # Data Types
 
 Kaskada operates on typed Timestreams.
-Similar to how every Pandas `DataFrame` has an associated `dtype`, every Kaskada `Timestream` has an associated type.
+Similar to how every Pandas `DataFrame` has an associated `dtype`, every Kaskada `Timestream` has an associated [PyArrow data type](https://arrow.apache.org/docs/python/api/datatypes.html) returned by {py:attr}`kaskada.Timestream.data_type`.
 The set of supported types is based on the types supported by [Apache Arrow](https://arrow.apache.org/).
 
 Each `Timestream` contains points of the corresponding type.
@@ -73,6 +73,16 @@ For example, `{name: string, age: u32 }` is a record type with two fields and `{
 
 NOTE: Record types may be nested.
 
+## Collection Types
+
+Kaskada also supports collections -- lists and maps.
+
+The type `list<T>` describes a list of elements of type `T`.
+For example, `list<i64>` is a list of 64-bit integers.
+
+Similarly, `map<K, V>` describes a map containing keys of type `K` and values of type `V`.
+For example, `map<str, i64>` is a map from strings to 64-bit integers.
+
 ## Type Coercion
 Kaskada implicitly coerces numeric types when different kinds of numbers are combined.
 For example adding a 64-bit signed integer value to a 32-bit floating point value produces a 64-point floating point value

diff --git a/python/docs/source/reference/timestream/index.md b/python/docs/source/reference/timestream/index.md
@@ -10,6 +10,8 @@ html_theme.sidebar_secondary.remove:
 .. autoclass:: kaskada.Literal
 .. autoclass:: kaskada.Timestream
     :exclude-members: __init__
+
+    .. autoproperty:: data_type
 ```
 
 ```{toctree}

diff --git a/python/docs/source/reference/timestream/misc.md b/python/docs/source/reference/timestream/misc.md
@@ -8,7 +8,6 @@
 
     Timestream.cast
     Timestream.coalesce
-    Timestream.data_type
     Timestream.else_
     Timestream.filter
     Timestream.if_