Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Support for upsert in BulkIndexer Update action #207

Closed
SaiSatwik opened this issue Jan 5, 2023 · 4 comments · Fixed by #269
Closed

[FEATURE] Support for upsert in BulkIndexer Update action #207

SaiSatwik opened this issue Jan 5, 2023 · 4 comments · Fixed by #269
Labels
enhancement New feature or request good first issue Good for newcomers

Comments

@SaiSatwik
Copy link

Is your feature request related to a problem?

In openserch Go client, 'update' action on BulkIndexerItem is not supporting 'upsert' flag currently. 'upsert' flag is used.
Upsert is used to insert a new document into a database if it does not exist, or updates an existing document if it does exist.
May be support for the flag can be provided in BulkIndexerItem type
https://github.com/opensearch-project/opensearch-go/blob/v1.1.0/opensearchutil/bulk_indexer.go#L111

What solution would you like?

Support for 'upsert' flag in BulkIndexerItem type in opensearch go client library.

What alternatives have you considered?

As of now, we use index operation for update as well so if document doesn't exist it will create a new document.

Do you have any additional context?

No

@SaiSatwik SaiSatwik added the enhancement New feature or request label Jan 5, 2023
@arunx2
Copy link

arunx2 commented Jan 9, 2023

It would be a needed addition to the API to complete the implementation of https://github.com/arunx2/benthos/blob/opensearch/internal/impl/opensearch/output.go

@VachaShah VachaShah added the good first issue Good for newcomers label Mar 1, 2023
@zethuman
Copy link
Contributor

AFAIK, all bulk requests in either are combined into one interface where requests will be sent. That is, a create, delete, update request is sent in the same format. I would like to discuss with the community in which version should be refactor?

  • If add some tag, will it not turn out to be an overweight for the rest of the operations?
  • If add a new abstract parameter where you can pass additional parameters? (this option suits me better)

@zethuman
Copy link
Contributor

zethuman commented Apr 1, 2023

@SaiSatwik, I decided to dig deeper into this issue and started digging into the source code. So, my previous post was wrong, as the source code already supports this functionality. However, it would seem non-obvious, on the other hand it gives a lot of flexibility in terms of use. In order to show how to use it, I decided to slightly tweak the integration test cases #269 . Please review and provide feedback. If this solves your problem, please close the issue.

@dblock
Copy link
Member

dblock commented Apr 3, 2023

I merged #269, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants