Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution][Detections] Adds Bulk edit API #120472

Merged
merged 54 commits into from
Jan 6, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
163502d
Bulk edit API
vitaliidm Dec 6, 2021
e7c23f9
add timelineId
vitaliidm Dec 6, 2021
a8bfe4e
change API implementation
vitaliidm Dec 6, 2021
9e1248f
fix tests
vitaliidm Dec 7, 2021
134ea18
add basic unit tests
vitaliidm Dec 7, 2021
8ca75f0
typings fixes
vitaliidm Dec 7, 2021
d506997
add more unit tests
vitaliidm Dec 7, 2021
0371501
fix ts
vitaliidm Dec 7, 2021
8cf5668
add functional test
vitaliidm Dec 7, 2021
475ae8d
Merge branch 'main' into security-solution/bulk-edit-api
kibanamachine Dec 8, 2021
fa2571c
Merge branch 'main' into security-solution/bulk-edit-api
kibanamachine Dec 8, 2021
305bd7e
add error wrapper
vitaliidm Dec 8, 2021
fb051bc
add additional tests
vitaliidm Dec 8, 2021
39e3f3c
add chunks to Promise processing
vitaliidm Dec 8, 2021
737296b
remove only from tests
vitaliidm Dec 9, 2021
609e41b
update chunking
vitaliidm Dec 9, 2021
0d18790
Merge branch 'main' into security-solution/bulk-edit-api
vitaliidm Dec 9, 2021
9e10f42
Merge branch 'main' into security-solution/bulk-edit-api
kibanamachine Dec 9, 2021
fbb4e15
fix merge conflict
vitaliidm Dec 9, 2021
cdd299f
add index test
vitaliidm Dec 9, 2021
1ee54ee
Merge branch 'main' into security-solution/bulk-edit-api
kibanamachine Dec 9, 2021
814ef53
revert changes for the rest of actions
vitaliidm Dec 13, 2021
990b2b1
linting
vitaliidm Dec 13, 2021
805fae6
Merge branch 'main' into security-solution/bulk-edit-api
kibanamachine Dec 13, 2021
61efff8
rewrite tests
vitaliidm Dec 13, 2021
07278d5
CR: modify function test for tags adding
vitaliidm Dec 13, 2021
865a3ed
CR: eliminate castings
vitaliidm Dec 14, 2021
81b2b61
CR: additonal functional tests
vitaliidm Dec 14, 2021
25ab66a
CR: change validation functions
vitaliidm Dec 14, 2021
8bca467
CR: change error handling
vitaliidm Dec 14, 2021
adf5a35
Merge branch 'main' into security-solution/bulk-edit-api
kibanamachine Dec 14, 2021
c5802aa
CR
vitaliidm Dec 14, 2021
c6aec7e
CR: cleanup & concurrency set
vitaliidm Dec 15, 2021
c45d5bb
CR: bulkUpdateAction
vitaliidm Dec 15, 2021
0d07087
CR: schema tests
vitaliidm Dec 15, 2021
aabb101
CR: rename bulk update to edit
vitaliidm Dec 15, 2021
e862bce
CR: further renames
vitaliidm Dec 15, 2021
acbfcac
CR: move 50 rules number to constants
vitaliidm Dec 15, 2021
c83c585
CR: errors response
vitaliidm Dec 15, 2021
5a5c8d7
Merge branch 'main' into security-solution/bulk-edit-api
kibanamachine Dec 15, 2021
35bf020
cleanup
vitaliidm Dec 15, 2021
c360cf6
CR: more tests
vitaliidm Dec 15, 2021
a863f4a
Merge branch 'main' into security-solution/bulk-edit-api
kibanamachine Dec 20, 2021
8a258db
CR: nits and tests
vitaliidm Dec 21, 2021
15d9f4a
Merge branch 'main' into security-solution/bulk-edit-api
kibanamachine Dec 22, 2021
7123f16
CR
vitaliidm Dec 22, 2021
6562010
CR: fix tests
vitaliidm Dec 22, 2021
1a28c20
CR: add immutability guard
vitaliidm Dec 23, 2021
39623bd
Merge branch 'main' into security-solution/bulk-edit-api
kibanamachine Dec 23, 2021
ffb568c
CR: add aborted signal
vitaliidm Dec 24, 2021
4cda94d
Merge branch 'main' into security-solution/bulk-edit-api
kibanamachine Jan 4, 2022
c7a9176
add limited concurrency
vitaliidm Jan 5, 2022
72f40ee
fix eslint
vitaliidm Jan 5, 2022
0a1cca2
Merge branch 'main' into security-solution/bulk-edit-api
kibanamachine Jan 5, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions x-pack/plugins/security_solution/common/constants.ts
Original file line number Diff line number Diff line change
Expand Up @@ -375,3 +375,30 @@ export const WARNING_TRANSFORM_STATES = new Set([
TRANSFORM_STATES.STOPPED,
TRANSFORM_STATES.STOPPING,
]);

/**
* How many rules to update at a time is set to 50 from errors coming from
* the slow environments such as cloud when the rule updates are > 100 we were
* seeing timeout issues.
*
* Since there is not timeout options at the alerting API level right now, we are
* at the mercy of the Elasticsearch server client/server default timeouts and what
* we are doing could be considered a workaround to not being able to increase the timeouts.
*
* However, other bad effects and saturation of connections beyond 50 makes this a "noisy neighbor"
* if we don't limit its number of connections as we increase the number of rules that can be
* installed at a time.
*
* Lastly, we saw weird issues where Chrome on upstream 408 timeouts will re-call the REST route
* which in turn could create additional connections we want to avoid.
*
* See file import_rules_route.ts for another area where 50 was chosen, therefore I chose
* 50 here to mimic it as well. If you see this re-opened or what similar to it, consider
* reducing the 50 above to a lower number.
*
* See the original ticket here:
* https://github.com/elastic/kibana/issues/94418
*/
export const MAX_RULES_TO_UPDATE_IN_PARALLEL = 50;

export const LIMITED_CONCURRENCY_ROUTE_TAG_PREFIX = `${APP_ID}:limitedConcurrency`;
Original file line number Diff line number Diff line change
Expand Up @@ -453,6 +453,53 @@ export enum BulkAction {
'export' = 'export',
'delete' = 'delete',
'duplicate' = 'duplicate',
'edit' = 'edit',
}

export const bulkAction = enumeration('BulkAction', BulkAction);

export enum BulkActionEditType {
'add_tags' = 'add_tags',
'delete_tags' = 'delete_tags',
'set_tags' = 'set_tags',
'add_index_patterns' = 'add_index_patterns',
'delete_index_patterns' = 'delete_index_patterns',
'set_index_patterns' = 'set_index_patterns',
'set_timeline' = 'set_timeline',
}

export const bulkActionEditType = enumeration('BulkActionEditType', BulkActionEditType);

const bulkActionEditPayloadTags = t.type({
type: t.union([
t.literal(BulkActionEditType.add_tags),
t.literal(BulkActionEditType.delete_tags),
t.literal(BulkActionEditType.set_tags),
]),
value: tags,
});

const bulkActionEditPayloadIndexPatterns = t.type({
type: t.union([
t.literal(BulkActionEditType.add_index_patterns),
t.literal(BulkActionEditType.delete_index_patterns),
t.literal(BulkActionEditType.set_index_patterns),
]),
value: index,
});

const bulkActionEditPayloadTimeline = t.type({
type: t.literal(BulkActionEditType.set_timeline),
value: t.type({
timeline_id,
timeline_title,
}),
});

export const bulkActionEditPayload = t.union([
bulkActionEditPayloadTags,
bulkActionEditPayloadIndexPatterns,
bulkActionEditPayloadTimeline,
]);

export type BulkActionEditPayload = t.TypeOf<typeof bulkActionEditPayload>;
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,16 @@
* 2.0.
*/

import { BulkAction } from '../common/schemas';
import { BulkAction, BulkActionEditType } from '../common/schemas';
import { PerformBulkActionSchema } from './perform_bulk_action_schema';

export const getPerformBulkActionSchemaMock = (): PerformBulkActionSchema => ({
query: '',
action: BulkAction.disable,
});

export const getPerformBulkActionEditSchemaMock = (): PerformBulkActionSchema => ({
query: '',
action: BulkAction.edit,
[BulkAction.edit]: [{ type: BulkActionEditType.add_tags, value: ['tag1'] }],
});
Loading