Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SIEM][Detections] Create Exception List API #65938

Closed
spong opened this issue May 9, 2020 · 2 comments
Closed

[SIEM][Detections] Create Exception List API #65938

spong opened this issue May 9, 2020 · 2 comments
Assignees
Labels
enhancement New value added to drive a business result Feature:Detection Rules Security Solution rules and Detection Engine Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:SIEM v7.9.0

Comments

@spong
Copy link
Member

spong commented May 9, 2020

This issue is for creating the Exception Lists implementation and API. While a SIEM/Endpoint feature, this functionality will live within the lists plugin added as part of #62552.

References:
Features dependent on this API:
Exception List / Exception Item Schema

The tentative exception-list schema is as follows:

Note: Details around adding comments are still WIP.

exceptions list

{
  "id": "test-exception",
  "created_at": "2020-04-23T00:18:56.361Z",
  "created_by": "user_name",
  "description": "This is a sample endpoint type exception",
  "name": "Sample Endpoint Exception",
  "tie_breaker_id": "6e879314-08a2-42eb-8137-3e43885a1aaf",
  "tags": ["endpoint", "process", "malware"],
  "type": "endpoint",
  "updated_at": "2020-04-23T00:18:56.361Z",
  "updated_by": "user_name",
  "exceptions_hash": "hash-here"
}

exception item

{
   "id": "[insert_uuid_here]",
   "created_at": "2020-04-23T00:19:13.289Z",
   "created_by": "user_name",
   "exception_id": "test-exception",
   "tie_breaker_id": "77fd1909-6786-428a-a671-30229a719c1f",
   "updated_at": "2020-04-23T00:19:13.289Z",
   "updated_by": "user_name",
   "exceptions_hash": "hash_here",
   "sensor_os": "windows",
   "entries": [
     {
       "field": "actingProcess.file.signer",
       "operator": "included",
       "entry": {
          "match": "Elastic, N.V."
       }
    },
    {
      "field": "event.category",
      "operator": "included",
      "entry": {
        "match_any": ["process", "malware"]
      }
    }
  ] 
}
@spong spong added enhancement New value added to drive a business result Team:SIEM v7.9.0 Feature:Detection Rules Security Solution rules and Detection Engine labels May 9, 2020
@elasticmachine
Copy link
Contributor

Pinging @elastic/siem (Team:SIEM)

FrankHassanabad added a commit that referenced this issue May 15, 2020
…ists and exception items

## Summary

See for more details:
#65938

Adds pieces of the `exception list` and `exception list item` and refactors/cleans the code up where I had parts incorrect with little things such as the javascript library io-ts. Some unit tests were added but I am holding off until more of the operations solidify before adding the unit tests. Everything is still behind a feature flag that must be enabled and not advised still at this point to use so I feel ok pushing these parts forward.

Adds to the API:
- Create exception list
- Read exception list
- Update exception list
- Delete exception list (and exception list items that are associated with it)
- Create exception list item
- Find exception list (/_find)
- Read exception list item
- Update exception list item
- Delete exception list items individually
- Find exception list item (/_find)

What is still missing from the REST and client API?
- Patch exception list
- Patch exception list item
- Bulk versions of everything
- Import/Export options for these exception lists and list items

### Manual testing and REST API endpoints

Go here:
```sh
/projects/kibana/x-pack/plugins/lists/server/scripts
```

See the files:

```sh
delete_all_exception_lists.sh
delete_exception_list.sh
delete_exception_list_by_id.sh
delete_exception_list_item.sh
delete_exception_list_item_by_id.sh
exception_lists
find_exception_list_items.sh
find_exception_lists.sh
get_exception_list.sh
get_exception_list_by_id.sh
get_exception_list_item.sh
get_exception_list_item_by_id.sh
post_exception_list.sh
post_exception_list_item.sh
update_exception_list.sh
update_exception_list_item.sh
```

Ensure you first run:

```sh
./hard_reset
```

and ensure you have setup your kibana.dev.yml to have:

```yml
# Enable lists feature
xpack.lists.enabled: true
xpack.lists.listIndex: '.lists-frank'
xpack.lists.listItemIndex: '.items-frank'
```

Then you can use the above scripts to create, read, update, and delete exception list and exception list items as well as perform find commands against them all.

### Checklist

Delete any items that are not applicable to this PR.

- [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios

Note: Some but limited unit tests at this point.
FrankHassanabad added a commit to FrankHassanabad/kibana that referenced this issue May 15, 2020
…ists and exception items

## Summary

See for more details:
elastic#65938

Adds pieces of the `exception list` and `exception list item` and refactors/cleans the code up where I had parts incorrect with little things such as the javascript library io-ts. Some unit tests were added but I am holding off until more of the operations solidify before adding the unit tests. Everything is still behind a feature flag that must be enabled and not advised still at this point to use so I feel ok pushing these parts forward.

Adds to the API:
- Create exception list
- Read exception list
- Update exception list
- Delete exception list (and exception list items that are associated with it)
- Create exception list item
- Find exception list (/_find)
- Read exception list item
- Update exception list item
- Delete exception list items individually
- Find exception list item (/_find)

What is still missing from the REST and client API?
- Patch exception list
- Patch exception list item
- Bulk versions of everything
- Import/Export options for these exception lists and list items

### Manual testing and REST API endpoints

Go here:
```sh
/projects/kibana/x-pack/plugins/lists/server/scripts
```

See the files:

```sh
delete_all_exception_lists.sh
delete_exception_list.sh
delete_exception_list_by_id.sh
delete_exception_list_item.sh
delete_exception_list_item_by_id.sh
exception_lists
find_exception_list_items.sh
find_exception_lists.sh
get_exception_list.sh
get_exception_list_by_id.sh
get_exception_list_item.sh
get_exception_list_item_by_id.sh
post_exception_list.sh
post_exception_list_item.sh
update_exception_list.sh
update_exception_list_item.sh
```

Ensure you first run:

```sh
./hard_reset
```

and ensure you have setup your kibana.dev.yml to have:

```yml
# Enable lists feature
xpack.lists.enabled: true
xpack.lists.listIndex: '.lists-frank'
xpack.lists.listItemIndex: '.items-frank'
```

Then you can use the above scripts to create, read, update, and delete exception list and exception list items as well as perform find commands against them all.

### Checklist

Delete any items that are not applicable to this PR.

- [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios

Note: Some but limited unit tests at this point.
FrankHassanabad added a commit that referenced this issue May 15, 2020
…ists and exception items (#66811)

## Summary

See for more details:
#65938

Adds pieces of the `exception list` and `exception list item` and refactors/cleans the code up where I had parts incorrect with little things such as the javascript library io-ts. Some unit tests were added but I am holding off until more of the operations solidify before adding the unit tests. Everything is still behind a feature flag that must be enabled and not advised still at this point to use so I feel ok pushing these parts forward.

Adds to the API:
- Create exception list
- Read exception list
- Update exception list
- Delete exception list (and exception list items that are associated with it)
- Create exception list item
- Find exception list (/_find)
- Read exception list item
- Update exception list item
- Delete exception list items individually
- Find exception list item (/_find)

What is still missing from the REST and client API?
- Patch exception list
- Patch exception list item
- Bulk versions of everything
- Import/Export options for these exception lists and list items

### Manual testing and REST API endpoints

Go here:
```sh
/projects/kibana/x-pack/plugins/lists/server/scripts
```

See the files:

```sh
delete_all_exception_lists.sh
delete_exception_list.sh
delete_exception_list_by_id.sh
delete_exception_list_item.sh
delete_exception_list_item_by_id.sh
exception_lists
find_exception_list_items.sh
find_exception_lists.sh
get_exception_list.sh
get_exception_list_by_id.sh
get_exception_list_item.sh
get_exception_list_item_by_id.sh
post_exception_list.sh
post_exception_list_item.sh
update_exception_list.sh
update_exception_list_item.sh
```

Ensure you first run:

```sh
./hard_reset
```

and ensure you have setup your kibana.dev.yml to have:

```yml
# Enable lists feature
xpack.lists.enabled: true
xpack.lists.listIndex: '.lists-frank'
xpack.lists.listItemIndex: '.items-frank'
```

Then you can use the above scripts to create, read, update, and delete exception list and exception list items as well as perform find commands against them all.

### Checklist

Delete any items that are not applicable to this PR.

- [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios

Note: Some but limited unit tests at this point.
@FrankHassanabad
Copy link
Contributor

"Reviewed by Frank Hassanabad on 7/29/2020", This looks complete and will be shippable at this point. Going to close this and associated tickets. Any new bugs/features should be new tickets.

@MindyRS MindyRS added the Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. label Oct 27, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New value added to drive a business result Feature:Detection Rules Security Solution rules and Detection Engine Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:SIEM v7.9.0
Projects
None yet
Development

No branches or pull requests

4 participants