Skip to content

Conversation

yanxi0830
Copy link
Contributor

@yanxi0830 yanxi0830 commented Mar 15, 2025

What does this PR do?

  • We need to tag DatasetIO class correctly with Datasets with the endpoint change

Test Plan

Before
image

After
image

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Meta Open Source bot. label Mar 15, 2025
@yanxi0830 yanxi0830 changed the base branch from main to pr1573 March 15, 2025 21:05
@yanxi0830 yanxi0830 marked this pull request as ready for review March 15, 2025 21:05
@yanxi0830 yanxi0830 merged commit 72ccdc1 into pr1573 Mar 15, 2025
3 of 4 checks passed
@yanxi0830 yanxi0830 deleted the datasetio_dataset_openapi branch March 15, 2025 21:12
@yanxi0830 yanxi0830 changed the title feat(datasets api): (1.3/n) patch OpenAPI gen for datasetio->datasets feat(dataset api): (1.3/n) patch OpenAPI gen for datasetio->datasets Mar 15, 2025
yanxi0830 added a commit that referenced this pull request Mar 17, 2025
## PR Stack
- #1573
- #1625
- #1656
- #1657
- #1658
- #1659
- #1660

**Client SDK**
- llamastack/llama-stack-client-python#203

**CI**
- https://github.com/meta-llama/llama-stack-ops/actions/runs/13911304888
<img width="1042" alt="image"
src="https://github.com/user-attachments/assets/69636067-376d-436b-9204-896e2dd490ca"
/>
-- the test_rag_agent_with_attachments is flaky and not related to this
PR

## Doc
<img width="789" alt="image"
src="https://github.com/user-attachments/assets/b88390f3-73d6-4483-b09a-a192064e32d9"
/>


## Client Usage
```python
client.datasets.register(
    source={
        "type": "uri",
        "uri": "lsfs://mydata.jsonl",
    },
    schema="jsonl_messages",
    # optional 
    dataset_id="my_first_train_data"
)

# quick prototype debugging
client.datasets.register(
    data_reference={
        "type": "rows",
        "rows": [
                "messages": [...],
        ],
    },
    schema="jsonl_messages",
)
```

## Test Plan
- CI:
https://github.com/meta-llama/llama-stack-ops/actions/runs/13878055453

```
LLAMA_STACK_CONFIG=fireworks pytest -v tests/integration/datasets/test_datasets.py
```

```
LLAMA_STACK_CONFIG=fireworks pytest -v tests/integration/scoring/test_scoring.py
```

```
pytest -v -s --nbval-lax ./docs/notebooks/Llama_Stack_Benchmark_Evals.ipynb
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants