Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Review data types #708

Merged
merged 3 commits into from
Aug 28, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .github/workflows/ci_python.yml
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,6 @@ jobs:
source $(poetry env info --path)/bin/activate
poetry install --with=docs
pip install 'kaskada[plot]>=0.6.0-a.0' --find-links dist --force-reinstall
ls docs/source/_static
sphinx-build docs/source docs/_build -j auto -W
deactivate
- name: Upload docs
Expand Down
8 changes: 2 additions & 6 deletions python/docs/source/examples/time_centric.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -173,12 +173,8 @@
"id": "c2c5a298",
"metadata": {},
"source": [
"Kaskada query language is parsed by the `fenl` extension. Query calculations are\n",
"defined in a code blocks starting with `%%fenl`.\n",
"\n",
"See [the `fenl`\n",
"documentation](https://kaskada-ai.github.io/docs-site/kaskada/main/fenl/fenl-quick-start.html)\n",
"for more information.\n",
"Kaskada queries are defined in Python, using the `{py}Timestream` class.\n",
"Sources are Timestreams generally containing [records](../guide/data_types.md#record-types).\n",
bjchambers marked this conversation as resolved.
Show resolved Hide resolved
"\n",
"Let's do a simple query for events for a specific entity ID.\n"
]
Expand Down
12 changes: 11 additions & 1 deletion python/docs/source/guide/data_types.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Data Types

Kaskada operates on typed Timestreams.
Similar to how every Pandas `DataFrame` has an associated `dtype`, every Kaskada `Timestream` has an associated type.
Similar to how every Pandas `DataFrame` has an associated `dtype`, every Kaskada `Timestream` has an associated [PyArrow data type](https://arrow.apache.org/docs/python/api/datatypes.html) returned by {py:attr}`kaskada.Timestream.data_type`.
The set of supported types is based on the types supported by [Apache Arrow](https://arrow.apache.org/).

Each `Timestream` contains points of the corresponding type.
Expand Down Expand Up @@ -73,6 +73,16 @@ For example, `{name: string, age: u32 }` is a record type with two fields and `{

NOTE: Record types may be nested.

## Collection Types

Kaskada also supports collections -- lists and maps.
bjchambers marked this conversation as resolved.
Show resolved Hide resolved

The type `list<T>` describes a list of elements of type `T`.
For example, `list<i64>` is a list of 64-bit integers.

Similarly, `map<K, V>` describes a map containing keys of type `K` and values of type `V`.
For example, `map<str, i64>` is a map from strings to 64-bit integers.

## Type Coercion
Kaskada implicitly coerces numeric types when different kinds of numbers are combined.
For example adding a 64-bit signed integer value to a 32-bit floating point value produces a 64-point floating point value
Expand Down
2 changes: 2 additions & 0 deletions python/docs/source/reference/timestream/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ html_theme.sidebar_secondary.remove:
.. autoclass:: kaskada.Literal
.. autoclass:: kaskada.Timestream
:exclude-members: __init__

.. autoproperty:: data_type
```

```{toctree}
Expand Down
1 change: 0 additions & 1 deletion python/docs/source/reference/timestream/misc.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@

Timestream.cast
Timestream.coalesce
Timestream.data_type
Timestream.else_
Timestream.filter
Timestream.if_
Expand Down