Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bulk API ignores ingest pipeline for bulk update items #87131

Closed
martijnvg opened this issue May 25, 2022 · 1 comment · Fixed by #87719
Closed

Bulk API ignores ingest pipeline for bulk update items #87131

martijnvg opened this issue May 25, 2022 · 1 comment · Fixed by #87719
Labels
>bug :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP Team:Data Management Meta label for data/management team

Comments

@martijnvg
Copy link
Member

The pipeline defined in the bulk update item here is ignored:

POST /_bulk
{"update":{"_id":"1","_index":"pipeline-test","pipeline":"lowercase-pipeline"}}
{"upsert":{"myField":"UPPER"},"script":{"source":"ctx._source.myField = 'CHANGED'","lang":"painless"}}

Only if the pipeline is defined as query string argument, then the pipeline is picked up and used to process the upset:

POST /_bulk?pipeline=lowercase-pipeline
{"update":{"_id":"1","_index":"pipeline-test"}}
{"upsert":{"myField":"UPPER"},"script":{"source":"ctx._source.myField = 'CHANGED'","lang":"painless"}}

The Bulk API should also use the pipeline defined in the bulk update item.

@martijnvg martijnvg added >bug :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP labels May 25, 2022
@elasticmachine elasticmachine added the Team:Data Management Meta label for data/management team label May 25, 2022
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-data-management (Team:Data Management)

martijnvg added a commit to martijnvg/elasticsearch that referenced this issue Jun 15, 2022
Don't ignore pipelines defined in bulk item for updates with upsert in bulk api.

Closes elastic#87131
martijnvg added a commit that referenced this issue Jun 27, 2022
Don't ignore pipelines defined in bulk item for updates with upsert in bulk api.

Closes #87131
martijnvg added a commit to martijnvg/elasticsearch that referenced this issue Jun 27, 2022
Don't ignore pipelines defined in bulk item for updates with upsert in bulk api.

Closes elastic#87131
martijnvg added a commit to martijnvg/elasticsearch that referenced this issue Jun 27, 2022
Don't ignore pipelines defined in bulk item for updates with upsert in bulk api.

Closes elastic#87131
martijnvg added a commit that referenced this issue Jun 27, 2022
Don't ignore pipelines defined in bulk item for updates with upsert in bulk api.

Closes #87131
elasticsearchmachine pushed a commit that referenced this issue Jun 27, 2022
* Don't ignore pipeline for upserts in bulk api (#87719)

Don't ignore pipelines defined in bulk item for updates with upsert in bulk api.

Closes #87131

* update skip version section
@exalate-issue-sync exalate-issue-sync bot reopened this Aug 23, 2022
@dakrone dakrone closed this as completed Aug 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>bug :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP Team:Data Management Meta label for data/management team
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants