Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Transform] Add basic stats #689

Merged
merged 1 commit into from
Apr 15, 2024
Merged

[Transform] Add basic stats #689

merged 1 commit into from
Apr 15, 2024

Conversation

prwhelan
Copy link
Member

@prwhelan prwhelan commented Apr 4, 2024

Basic Stats was introduced in 8.13.0. It is faster than Transform Stats and will skip checkpointing and other supporting API calls, making it return results more reliably.

Checklist

  • I have verified that the APIs in this pull request do not return sensitive data

Testing:

Ran changes against 8.14 snapshot:

local-diagnostics-20240404-210020 git:(main) ✗ ls commercial/transform*
commercial/transform_basic_stats.json  commercial/transform_stats.json
commercial/transform.json
➜  local-diagnostics-20240404-210020 git:(main) ✗ cat commercial/transform_basic_stats.json | jq '.'
{
  "count": 1,
  "transforms": [
    {
      "id": "last-log-from-clientip",
      "state": "stopped",
      "stats": {
        "pages_processed": 0,
        "documents_processed": 0,
        "documents_indexed": 0,
        "documents_deleted": 0,
        "trigger_count": 0,
        "index_time_in_ms": 0,
        "index_total": 0,
        "index_failures": 0,
        "search_time_in_ms": 0,
        "search_total": 0,
        "search_failures": 0,
        "processing_time_in_ms": 0,
        "processing_total": 0,
        "delete_time_in_ms": 0,
        "exponential_avg_checkpoint_duration_ms": 0,
        "exponential_avg_documents_indexed": 0,
        "exponential_avg_documents_processed": 0
      },
      "checkpointing": {
        "last": {
          "checkpoint": 0
        }
      },
      "health": {
        "status": "green"
      }
    }
  ]
}

On version 8.12:

➜  local-diagnostics-20240404-214214 git:(main) ✗ ls commercial/transform*
commercial/transform.json  commercial/transform_stats.json

Basic Stats was introduced in 8.13.0.  It is faster than Transform Stats
and will skip checkpointing and other supporting API calls, making it
return results more reliably.
@prwhelan prwhelan requested a review from a team as a code owner April 4, 2024 21:56
@sophiec20
Copy link

@ppf2 how can we get this reviewed?

@pickypg pickypg merged commit 3c7b0b5 into elastic:main Apr 15, 2024
4 checks passed
@pickypg
Copy link
Member

pickypg commented Apr 15, 2024

I happened to be out when this PR came in and I missed it upon return.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants