[ML] Parent datafeed actions to the datafeed's persistent task #81143
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The vast majority of a datafeed's actions are executed from the data extractor. This includes the heaviest actions which are the searches. This commit passes a `ParentTaskAssigningClient` to `DataExtractorFactory.create` which ensures the client used by any extractor will be setting the corresponding task id: the action task id for preview datafeed and the master operation stage of the start datafeed action, and the persistent task id for the datafeed operations after it has started.
|
Pinging @elastic/ml-core (Team:ML) |
dimitris-athanasiou
added a commit
to dimitris-athanasiou/elasticsearch
that referenced
this pull request
Nov 30, 2021
…ic#81143) The vast majority of a datafeed's actions are executed from the data extractor. This includes the heaviest actions which are the searches. This commit passes a `ParentTaskAssigningClient` to `DataExtractorFactory.create` which ensures the client used by any extractor will be setting the corresponding task id: the action task id for preview datafeed and the master operation stage of the start datafeed action, and the persistent task id for the datafeed operations after it has started.
dimitris-athanasiou
added a commit
to dimitris-athanasiou/elasticsearch
that referenced
this pull request
Nov 30, 2021
…ic#81143) The vast majority of a datafeed's actions are executed from the data extractor. This includes the heaviest actions which are the searches. This commit passes a `ParentTaskAssigningClient` to `DataExtractorFactory.create` which ensures the client used by any extractor will be setting the corresponding task id: the action task id for preview datafeed and the master operation stage of the start datafeed action, and the persistent task id for the datafeed operations after it has started.
elasticsearchmachine
pushed a commit
that referenced
this pull request
Nov 30, 2021
… (#81152) The vast majority of a datafeed's actions are executed from the data extractor. This includes the heaviest actions which are the searches. This commit passes a `ParentTaskAssigningClient` to `DataExtractorFactory.create` which ensures the client used by any extractor will be setting the corresponding task id: the action task id for preview datafeed and the master operation stage of the start datafeed action, and the persistent task id for the datafeed operations after it has started.
weizijun
added a commit
to weizijun/elasticsearch
that referenced
this pull request
Nov 30, 2021
* upstream/master: [ML] Parent datafeed actions to the datafeed's persistent task (elastic#81143) Simplify ConcreteIndices and its usage in TransportBulkAction (elastic#81098) Unmute DataStreamsSnapshotsIT#testRestoreDataStreamAliasWithConflictingIndicesAlias() test (elastic#81142) TSDB: Do not allow index splits for time series indices (elastic#81125) Reduce verbosity-increase timeout to 3m (elastic#81118) Mute DataStreamsSnapshotsIT#testRestoreDataStreamAliasWithConflictingIndicesAlias() test Fix stopping of old elasticsearch cluster (elastic#81059) Fix data stream alias validation. (elastic#81040) Add replicated field to get data stream api response. (elastic#80988)
elasticsearchmachine
pushed a commit
that referenced
this pull request
Nov 30, 2021
…#81143) (#81153) * [ML] Parent datafeed actions to the datafeed's persistent task (#81143) The vast majority of a datafeed's actions are executed from the data extractor. This includes the heaviest actions which are the searches. This commit passes a `ParentTaskAssigningClient` to `DataExtractorFactory.create` which ensures the client used by any extractor will be setting the corresponding task id: the action task id for preview datafeed and the master operation stage of the start datafeed action, and the persistent task id for the datafeed operations after it has started. * Can't parent on the start datafeed master op as we don't have task
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The vast majority of a datafeed's actions are executed from the
data extractor. This includes the heaviest actions which are the
searches. These are currently not parented correctly to the datafeed's
task. This makes it hard to trace search tasks back to their datafeeds.
This commit passes a
ParentTaskAssigningClienttoDataExtractorFactory.createwhich ensures the client used byany extractor will be setting the corresponding task id: the action
task id for preview datafeed and the master operation stage of the
start datafeed action, and the persistent task id for the datafeed
operations after it has started.