[BUG] fetch_size does not work when sending PPL request

[abbyhu2000]

What is the bug?
According to documentation, https://opensearch.org/docs/latest/search-plugins/sql/sql-ppl-api/#request-body-fields, fetch_size should control the size of the responses we got back for both PPL and SQL requests. However, currently when i send requests with fetch_size in the dev tool, only SQL works; PPL responses do not seem to respect the fetch_size param and still return the full size responses.

How can one reproduce the bug?
Steps to reproduce the behavior:

1. Go to dev tool
2. type

POST _plugins/_ppl/
{
  "fetch_size" : 1,
  "query" : "source = opensearch_dashboards_sample_data_flights",
}

3. Send the request
4. See responses return with full size, but not size 1

What is your host/environment?

• Version 2.18

Do you have any screenshots?

Screen.Recording.2025-02-07.at.3.00.06.PM.mov

Do you have any additional context?
Add any other context about the problem.

[ykmr1224]

This is not supported in PPL. We will update doc to reflect this.

[kavilla]

@ykmr1224 could you provide more context on why it's preferable to not support this feature? this would then mean, data limiting relies solely on the data source configuration or explicit limit clauses in queries. which would mean OSD HAS to modify the query string directly to implement pagination or cost control, which could lead to unexpected results as shown below.

# Query 1
source = opensearch_dashboards_sample_data_logs | where response > 400 | limit 5

# Query 2
source = opensearch_dashboards_sample_data_logs | limit 5 | where response > 400

these queries produce different results because:

• Query 1: filters all error responses first, then takes top 5
• Query 2: takes first 5 logs, then filters for errors within those (might return 0-5 results)

if this feature was supported there could benefits like:

• performance: avoids retrieving unnecessary data
• memory: controls client-side memory usage
• pagination: enables standard paging functionality
• cost: prevents unexpected large query execution
• efficiency: aligns with common database practices