Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bulk helper onFlush method #2553

Open
tomsquest opened this issue Dec 18, 2024 · 0 comments
Open

Bulk helper onFlush method #2553

tomsquest opened this issue Dec 18, 2024 · 0 comments

Comments

@tomsquest
Copy link

tomsquest commented Dec 18, 2024

🚀 Feature Proposal

Hello,

Using the helper is convenient, but for a long running bulk, the operation will run without any feedback.

I am suggesting to have a method, onFlush that will be called on each flush operation.

The method would be similar to onSuccess but will receive the ongoing stats (number of success, errors so far).

Motivation

Ideally, I would like to have a feedback in terms of number of docs processed, instead of processed bytes.
This would make more sense for a human.

Example

await this.elastic.helpers.bulk<Document>({
  index: this.indexName,
  datasource: iterator,
  onDocument(doc) {
    return [
      { update: { _id: doc._id } },
      { doc_as_upsert: true, doc: omit(doc, "_id") },
    ];
  },
  onFlush({ result }) {
    console.log(`onFlush: '${result.total}'`);
  },
});
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant