From f93497eef94f64fad67b0cd11e2a4f3ae2e6936a Mon Sep 17 00:00:00 2001 From: Ian Spektor Date: Tue, 16 Jan 2024 13:23:00 -0300 Subject: [PATCH 1/3] start operator contribution in contrib guide --- CONTRIBUTING.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index bd91a57db..81076e872 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -165,3 +165,13 @@ bazel test --config=linux //temporian/test:doc_test --test_output=streamed ``` In case of unexpected outputs, the result is printed and compared to the expected values, so that they can be fixed. + +### Developing a new operator + +We provide a utility script that generates new files, modifies existing ones, and prints needed modifications to develop and make available a new operator. From the project's root, run: + +```shell +tools/create_operator.py --operator +``` + +so for example, to create the `EventSet.map()` operator, you'd run `tools/create_operator.py --operator map`. From e6b35250b1a7a0e1ecbfe807f66b9e162c4c7174 Mon Sep 17 00:00:00 2001 From: Ian Spektor Date: Tue, 16 Jan 2024 13:35:33 -0300 Subject: [PATCH 2/3] index important files in operator contrib guide --- CONTRIBUTING.md | 13 +++++++++++++ temporian/core/operators/since_last.py | 2 +- 2 files changed, 14 insertions(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 81076e872..4e744efce 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -175,3 +175,16 @@ tools/create_operator.py --operator ``` so for example, to create the `EventSet.map()` operator, you'd run `tools/create_operator.py --operator map`. + +Here are some key files you'll need to modify (and write the operator's logic in): +- [temporian/core/event_set_ops.py](temporian/core/event_set_ops.py) or [temporian/__init__.py](temporian/__init__.py), depending on if the operator is available in the `EventSet` class (like `EventSet.since_last()`) or in the global `tp` module (like `tp.glue()`). +- Write the operator's core logic in `temporian/core/operators/.py`. + - The core logic is that related to the operator's definition in the graph, checks, and normalization during initialization. It doesn't interact with the actual data contained within the `EventSet`. + - See for example [temporian/core/operators/since_last.py](temporian/core/operators/since_last.py). +- Write the operator's implementation in `temporian/implementation/numpy/operators/.py`. + - The implementation is what actually executes the operator's logic on an `EventSet`'s data. + - See for example [temporian/implementation/numpy/operators/since_last.py](temporian/implementation/numpy/operators/since_last.py). +- Write unit tests for the operator in `temporian/core/operators/test/test_.py`. + - See for example [temporian/core/operators/test/test_since_last.py](temporian/core/operators/test/test_since_last.py). + +Read the script's output to see in detail all other files that need to be modified to finish setting up the operator! diff --git a/temporian/core/operators/since_last.py b/temporian/core/operators/since_last.py index 2075bbb1b..284a09f98 100644 --- a/temporian/core/operators/since_last.py +++ b/temporian/core/operators/since_last.py @@ -12,7 +12,7 @@ # See the License for the specific language governing permissions and # limitations under the License. -"""Resample operator class and public API function definition.""" +"""Since last operator class and public API function definition.""" from typing import Optional From 0262f59cd20c0e745de2bf5ace4635356ac23179 Mon Sep 17 00:00:00 2001 From: Ian Spektor Date: Tue, 16 Jan 2024 13:54:47 -0300 Subject: [PATCH 3/3] address PR commentts --- CONTRIBUTING.md | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 4e744efce..09dc4efac 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -168,7 +168,7 @@ In case of unexpected outputs, the result is printed and compared to the expecte ### Developing a new operator -We provide a utility script that generates new files, modifies existing ones, and prints needed modifications to develop and make available a new operator. From the project's root, run: +We provide a utility script that generates placeholder files, modifies existing ones, and prints needed modifications to develop and make available a new operator. From the project's root, run: ```shell tools/create_operator.py --operator @@ -180,11 +180,17 @@ Here are some key files you'll need to modify (and write the operator's logic in - [temporian/core/event_set_ops.py](temporian/core/event_set_ops.py) or [temporian/__init__.py](temporian/__init__.py), depending on if the operator is available in the `EventSet` class (like `EventSet.since_last()`) or in the global `tp` module (like `tp.glue()`). - Write the operator's core logic in `temporian/core/operators/.py`. - The core logic is that related to the operator's definition in the graph, checks, and normalization during initialization. It doesn't interact with the actual data contained within the `EventSet`. - - See for example [temporian/core/operators/since_last.py](temporian/core/operators/since_last.py). + - Example: [temporian/core/operators/since_last.py](temporian/core/operators/since_last.py). - Write the operator's implementation in `temporian/implementation/numpy/operators/.py`. - The implementation is what actually executes the operator's logic on an `EventSet`'s data. - - See for example [temporian/implementation/numpy/operators/since_last.py](temporian/implementation/numpy/operators/since_last.py). + - Example of a Python-only operator: [temporian/implementation/numpy/operators/since_last.py](temporian/implementation/numpy/operators/since_last.py). + - Example of a C++ operator: [temporian/implementation/numpy/operators/resample.py](temporian/implementation/numpy/operators/resample.py) and [temporian/implementation/numpy_cc/operators/resample.cc](temporian/implementation/numpy_cc/operators/resample.cc). - Write unit tests for the operator in `temporian/core/operators/test/test_.py`. - - See for example [temporian/core/operators/test/test_since_last.py](temporian/core/operators/test/test_since_last.py). + - Example: [temporian/core/operators/test/test_since_last.py](temporian/core/operators/test/test_since_last.py). +- Add the operator to the docs in `docs/src/reference/temporian/operators/.md`. + - The docs are generated automatically by [mkdocstrings](https://mkdocstrings.github.io/python/) from the operator's docstring. + - Example: [docs/src/reference/temporian/operators/since_last.md](docs/src/reference/temporian/operators/since_last.md). + +Groups of operator with a similar implementation as grouped together. For instance, `temporian/core/operators/window` contains moving window operators (e.g., `EventSet.simple_moving_average()`) and `temporian/core/operators/binary` contains operators taking two features as input (e.g. `EventSet.subtract()`). Read the script's output to see in detail all other files that need to be modified to finish setting up the operator!