Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -211,12 +211,12 @@ updates:

# Topics.

- directory: "/topic/machine-learning/automl"
- directory: "/topic/machine-learning/pycaret"
package-ecosystem: "pip"
schedule:
interval: "daily"

- directory: "/topic/machine-learning/llm-langchain"
- directory: "/topic/machine-learning/langchain"
package-ecosystem: "pip"
schedule:
interval: "daily"
Expand All @@ -226,7 +226,7 @@ updates:
schedule:
interval: "daily"

- directory: "/topic/machine-learning/mlops-mlflow"
- directory: "/topic/machine-learning/mlflow"
package-ecosystem: "pip"
schedule:
interval: "daily"
Expand Down
12 changes: 6 additions & 6 deletions .github/workflows/ml-langchain.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,13 @@ on:
pull_request:
paths:
- '.github/workflows/ml-langchain.yml'
- 'topic/machine-learning/llm-langchain/**'
- 'topic/machine-learning/langchain/**'
- '/requirements.txt'
push:
branches: [ main ]
paths:
- '.github/workflows/ml-langchain.yml'
- 'topic/machine-learning/llm-langchain/**'
- 'topic/machine-learning/langchain/**'
- '/requirements.txt'

# Allow job to be triggered manually.
Expand Down Expand Up @@ -78,13 +78,13 @@ jobs:
cache: 'pip'
cache-dependency-path: |
requirements.txt
topic/machine-learning/llm-langchain/requirements.txt
topic/machine-learning/llm-langchain/requirements-dev.txt
topic/machine-learning/langchain/requirements.txt
topic/machine-learning/langchain/requirements-dev.txt

- name: Install utilities
run: |
pip install -r requirements.txt

- name: Validate topic/machine-learning/llm-langchain
- name: Validate topic/machine-learning/langchain
run: |
ngr test --accept-no-venv topic/machine-learning/llm-langchain
ngr test --accept-no-venv topic/machine-learning/langchain
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
name: "LLM"
name: "llm"

on:
pull_request:
paths:
- '.github/workflows/application-llm.yml'
- 'application/llm/**'
- '.github/workflows/ml-llm.yml'
- 'topic/machine-learning/llm/**'
push:
branches: [ main ]
paths:
- '.github/workflows/application-llm.yml'
- 'application/llm/**'
- '.github/workflows/ml-llm.yml'
- 'topic/machine-learning/llm/**'

# Allow job to be triggered manually.
workflow_dispatch:
Expand Down Expand Up @@ -53,9 +53,9 @@ jobs:
- name: Install uv
uses: astral-sh/setup-uv@v6

- name: Validate application/llm
- name: Validate topic/machine-learning/llm
run: |
# TODO: Generalize invocation into `ngr` test runner.
cd application/llm
cd topic/machine-learning/llm
uv pip install -r requirements.txt
uvx xonsh test.xsh
12 changes: 6 additions & 6 deletions .github/workflows/ml-mlflow.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,13 @@ on:
branches: ~
paths:
- '.github/workflows/ml-mlflow.yml'
- 'topic/machine-learning/mlops-mlflow/**'
- 'topic/machine-learning/mlflow/**'
- '/requirements.txt'
push:
branches: [ main ]
paths:
- '.github/workflows/ml-mlflow.yml'
- 'topic/machine-learning/mlops-mlflow/**'
- 'topic/machine-learning/mlflow/**'
- '/requirements.txt'

# Allow job to be triggered manually.
Expand Down Expand Up @@ -62,13 +62,13 @@ jobs:
cache: 'pip'
cache-dependency-path: |
requirements.txt
topic/machine-learning/mlops-mlflow/requirements.txt
topic/machine-learning/mlops-mlflow/requirements-dev.txt
topic/machine-learning/mlflow/requirements.txt
topic/machine-learning/mlflow/requirements-dev.txt

- name: Install utilities
run: |
pip install -r requirements.txt

- name: Validate topic/machine-learning/mlops-mlflow
- name: Validate topic/machine-learning/mlflow
run: |
ngr test --accept-no-venv topic/machine-learning/mlops-mlflow
ngr test --accept-no-venv topic/machine-learning/mlflow
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,13 @@ name: "Open WebUI"
on:
pull_request:
paths:
- '.github/workflows/application-open-webui.yml'
- 'application/open-webui/**'
- '.github/workflows/ml-open-webui.yml'
- 'topic/machine-learning/open-webui/**'
push:
branches: [ main ]
paths:
- '.github/workflows/application-open-webui.yml'
- 'application/open-webui/**'
- '.github/workflows/ml-open-webui.yml'
- 'topic/machine-learning/open-webui/**'

# Allow job to be triggered manually.
workflow_dispatch:
Expand Down Expand Up @@ -43,12 +43,12 @@ jobs:
- name: Acquire sources
uses: actions/checkout@v4

- name: Validate application/open-webui
- name: Validate topic/machine-learning/open-webui
run: |
# TODO: Generalize invocation into `ngr` test runner.

# Invoke software stack.
cd application/open-webui
cd topic/machine-learning/open-webui
docker compose up --detach

# Invoke validation payload.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
name: AutoML
name: PyCaret

on:
pull_request:
branches: ~
paths:
- '.github/workflows/ml-automl.yml'
- 'topic/machine-learning/automl/**'
- '.github/workflows/ml-pycaret.yml'
- 'topic/machine-learning/pycaret/**'
- '/requirements.txt'
push:
branches: [ main ]
paths:
- '.github/workflows/ml-automl.yml'
- 'topic/machine-learning/automl/**'
- '.github/workflows/ml-pycaret.yml'
- 'topic/machine-learning/pycaret/**'
- '/requirements.txt'

# Allow job to be triggered manually.
Expand Down Expand Up @@ -72,6 +72,6 @@ jobs:
run: |
pip install -r requirements.txt

- name: Validate topic/machine-learning/automl
- name: Validate topic/machine-learning/pycaret
run: |
ngr test --accept-no-venv topic/machine-learning/automl
ngr test --accept-no-venv topic/machine-learning/pycaret
Original file line number Diff line number Diff line change
Expand Up @@ -45,22 +45,22 @@ and [CrateDB].
- [requirements.txt](requirements.txt): Pulls in a patched version of LangChain,
as well as the CrateDB client driver and the `crash` command-line interface.

- `vector_store.ipynb` [![Open on GitHub](https://img.shields.io/badge/Open%20on-GitHub-lightgray?logo=GitHub)](vector_search.ipynb) [![Launch Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/crate/cratedb-examples/main?labpath=topic%2Fmachine-learning%2Fllm-langchain%2Fvector_search.ipynb) [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/crate/cratedb-examples/blob/main/topic/machine-learning/llm-langchain/vector_search.ipynb)
- `vector_store.ipynb` [![Open on GitHub](https://img.shields.io/badge/Open%20on-GitHub-lightgray?logo=GitHub)](vector_search.ipynb) [![Launch Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/crate/cratedb-examples/main?labpath=topic%2Fmachine-learning%2Flangchain%2Fvector_search.ipynb) [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/crate/cratedb-examples/blob/main/topic/machine-learning/langchain/vector_search.ipynb)

This notebook explores CrateDB's [`FLOAT_VECTOR`] and [`KNN_MATCH`] functionalities for storing and retrieving
embeddings, and for conducting similarity searches.

- `document_loader.ipynb` [![Open on GitHub](https://img.shields.io/badge/Open%20on-GitHub-lightgray?logo=GitHub)](document_loader.ipynb) [![Launch Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/crate/cratedb-examples/main?labpath=topic%2Fmachine-learning%2Fllm-langchain%2Fdocument_loader.ipynb) [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/crate/cratedb-examples/blob/main/topic/machine-learning/llm-langchain/document_loader.ipynb)
- `document_loader.ipynb` [![Open on GitHub](https://img.shields.io/badge/Open%20on-GitHub-lightgray?logo=GitHub)](document_loader.ipynb) [![Launch Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/crate/cratedb-examples/main?labpath=topic%2Fmachine-learning%2Flangchain%2Fdocument_loader.ipynb) [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/crate/cratedb-examples/blob/main/topic/machine-learning/langchain/document_loader.ipynb)

The notebook about the Document Loader demonstrates how to query a database table in CrateDB and use it as a
source provider for LangChain documents.

- `conversational_memory.ipynb` [![Open on GitHub](https://img.shields.io/badge/Open%20on-GitHub-lightgray?logo=GitHub)](conversational_memory.ipynb) [![Launch Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/crate/cratedb-examples/main?labpath=topic%2Fmachine-learning%2Fllm-langchain%2Fconversational_memory.ipynb) [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/crate/cratedb-examples/blob/main/topic/machine-learning/llm-langchain/conversational_memory.ipynb)
- `conversational_memory.ipynb` [![Open on GitHub](https://img.shields.io/badge/Open%20on-GitHub-lightgray?logo=GitHub)](conversational_memory.ipynb) [![Launch Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/crate/cratedb-examples/main?labpath=topic%2Fmachine-learning%2Flangchain%2Fconversational_memory.ipynb) [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/crate/cratedb-examples/blob/main/topic/machine-learning/langchain/conversational_memory.ipynb)

LangChain also supports managing conversation history in SQL databases. This notebook exercises
how that works with CrateDB.

- `cratedb-vectorstore-rag-openai-sql.ipynb` [![Open on GitHub](https://img.shields.io/badge/Open%20on-GitHub-lightgray?logo=GitHub)](conversational_memory.ipynb) [![Launch Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/crate/cratedb-examples/main?labpath=topic%2Fmachine-learning%2Fllm-langchain%2Fcratedb-vectorstore-rag-openai-sql.ipynb) [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/crate/cratedb-examples/blob/main/topic/machine-learning/llm-langchain/cratedb-vectorstore-rag-openai-sql.ipynb)
- `cratedb-vectorstore-rag-openai-sql.ipynb` [![Open on GitHub](https://img.shields.io/badge/Open%20on-GitHub-lightgray?logo=GitHub)](conversational_memory.ipynb) [![Launch Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/crate/cratedb-examples/main?labpath=topic%2Fmachine-learning%2Flangchain%2Fcratedb-vectorstore-rag-openai-sql.ipynb) [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/crate/cratedb-examples/blob/main/topic/machine-learning/langchain/cratedb-vectorstore-rag-openai-sql.ipynb)

This example intentionally shows how to use the CrateDB Vector Store using SQL.
There might be cases where the default parameters of the LangChain integration
Expand All @@ -74,7 +74,7 @@ and [CrateDB].
[document_loader.py](document_loader.py), and
[conversational_memory.py](conversational_memory.py).

- `cratedb_rag_customer_support_langchain.ipynb` [![Open on GitHub](https://img.shields.io/badge/Open%20on-GitHub-lightgray?logo=GitHub)](cratedb_rag_customer_support_langchain.ipynb)[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/crate/cratedb-examples/blob/main/topic/machine-learning/llm-langchain/cratedb_rag_customer_support_langchain.ipynb)
- `cratedb_rag_customer_support_langchain.ipynb` [![Open on GitHub](https://img.shields.io/badge/Open%20on-GitHub-lightgray?logo=GitHub)](cratedb_rag_customer_support_langchain.ipynb)[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/crate/cratedb-examples/blob/main/topic/machine-learning/langchain/cratedb_rag_customer_support_langchain.ipynb)

This example illustrates the RAG implementation of a customer support scenario.
The dataset used in this example is based on a collection of customer support interactions
Expand All @@ -84,7 +84,7 @@ and [CrateDB].
augmented generation (RAG) pipeline. To implement RAG we use the Python client driver for
CrateDB and vector store support in LangChain.

- `cratedb_rag_customer_support_vertexai.ipynb` [![Open on GitHub](https://img.shields.io/badge/Open%20on-GitHub-lightgray?logo=GitHub)](cratedb_rag_customer_support_vertexai.ipynb)[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/crate/cratedb-examples/blob/main/topic/machine-learning/llm-langchain/cratedb_rag_customer_support_vertexai.ipynb)
- `cratedb_rag_customer_support_vertexai.ipynb` [![Open on GitHub](https://img.shields.io/badge/Open%20on-GitHub-lightgray?logo=GitHub)](cratedb_rag_customer_support_vertexai.ipynb)[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/crate/cratedb-examples/blob/main/topic/machine-learning/langchain/cratedb_rag_customer_support_vertexai.ipynb)

This example illustrates the RAG implementation of a customer support scenario.
It is based on the previous notebook, and it illustrates how to use Vertex AI platform
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ and [CrateDB].
- [requirements.txt](requirements.txt): For pulling the required dependencies to
run the example programs.

- `automl_classification_with_pycaret.ipynb` [![Open on GitHub](https://img.shields.io/badge/Open%20on-GitHub-lightgray?logo=GitHub)](automl_classification_with_pycaret.ipynb) [![Open in Collab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/crate/cratedb-examples/blob/main/topic/machine-learning/automl/automl_classification_with_pycaret.ipynb)
- `automl_classification_with_pycaret.ipynb` [![Open on GitHub](https://img.shields.io/badge/Open%20on-GitHub-lightgray?logo=GitHub)](automl_classification_with_pycaret.ipynb) [![Open in Collab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/crate/cratedb-examples/blob/main/topic/machine-learning/pycaret/automl_classification_with_pycaret.ipynb)

This notebook explores the PyCaret framework and shows how to use it to
train different classification models - using a user churn dataset as an
Expand All @@ -62,7 +62,7 @@ and [CrateDB].
model. The notebook also shows how to use CrateDB as storage for both the raw
data and the expirement tracking and model registry data.

- `automl_timeseries_forecasting_with_pycaret.ipynb` [![Open on GitHub](https://img.shields.io/badge/Open%20on-GitHub-lightgray?logo=GitHub)](automl_timeseries_forecasting_with_pycaret.ipynb) [![Open in Collab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/crate/cratedb-examples/blob/main/topic/machine-learning/automl/automl_timeseries_forecasting_with_pycaret.ipynb)
- `automl_timeseries_forecasting_with_pycaret.ipynb` [![Open on GitHub](https://img.shields.io/badge/Open%20on-GitHub-lightgray?logo=GitHub)](automl_timeseries_forecasting_with_pycaret.ipynb) [![Open in Collab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/crate/cratedb-examples/blob/main/topic/machine-learning/pycaret/automl_timeseries_forecasting_with_pycaret.ipynb)

This notebook explores the PyCaret framework and shows how to use it to
train various timeseries forecasting models - using a real-world sales dataset
Expand Down Expand Up @@ -92,7 +92,7 @@ test runner program on the top-level folder of this repository. This is the
same code path the CI jobs are taking.
```shell
pip install -U -r requirements.txt
ngr test topic/machine-learning/automl
ngr test topic/machine-learning/pycaret
```


Expand Down
Loading