Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Okta module documentation, config cleanup, _id field #18953

Merged
merged 1 commit into from
Jun 4, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.next.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
* iptables {pull}18756[18756]
* Checkpoint {pull}18754[18754]
- Preserve case of http.request.method. ECS prior to 1.6 specified normalizing to lowercase, which lost information. Affects filesets: apache/access, elasticsearch/audit, iis/access, iis/error, nginx/access, nginx/ingress_controller, aws/elb, suricata/eve, zeek/http. {issue}18154[18154] {pull}18359[18359]
- Okta module now requires objects instead of JSON strings for the `http_headers`, `http_request_body`, `pagination`, `rate_limit`, and `ssl` variables. {pull}18953[18953]

*Heartbeat*

Expand Down Expand Up @@ -163,6 +164,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
- Fix `googlecloud.audit` pipeline to only take in fields that are explicitly defined by the dataset. {issue}18465[18465] {pull}18472[18472]
- Fix `o365.audit` failing to ingest events when ip address is surrounded by square brackets. {issue}18587[18587] {pull}18591[18591]
- Fix Kubernetes Watcher goroutine leaks when input config is invalid and `input.reload` is enabled. {issue}18629[18629] {pull}18630[18630]
- Okta module now sets the Elasticsearch `_id` field to the Okta UUID value contained in each system log to minimize the possibility of duplicating events. {pull}18953[18953]

*Heartbeat*

Expand Down
96 changes: 93 additions & 3 deletions filebeat/docs/modules/okta.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -12,15 +12,105 @@ This file is generated! See scripts/docs_collector.py

beta[]

This is a filebeat module for retrieving system logs from Okta (www.okta.com) via API.
The Okta module collects events from the
https://developer.okta.com/docs/reference/[Okta API]. Specifically this supports
reading from the https://developer.okta.com/docs/reference/api/system-log/[Okta
System Log API].

:has-dashboards!:
:fileset_ex: system

include::../include/config-option-intro.asciidoc[]

[float]
==== `system` fileset settings

The Okta System Log records system events related to your organization in order
to provide an audit trail that can be used to understand platform activity and
to diagnose problems. This module is implemented using the
<<filebeat-input-httpjson,httpjson>> input and is configured to paginate through
the logs while honoring any
https://developer.okta.com/docs/reference/rate-limits/[rate-limiting] headers
sent by Okta.

NOTE: This module does not persist the timestamp of the last read event in
order to facilitate resuming on restart. This feature will be coming in a future
version. When you restart the module will read events from the beginning of the
log. To minimize duplicates documents the module uses the event's Okta UUID
value as the Elasticsearch `_id`.

This is an example configuration for the module.

[source,yaml]
----
- module okta
system:
var.url: https://yourOktaDomain/api/v1/logs
var.api_key: '00QCjAl4MlV-WPXM...0HmjFx-vbGua'
----

[float]
===== Configuration options

*`var.url`*::

Specifies the URL to the Okta System Log API. Required.
+
[source,yaml]
----
var.url: https://mycompany.okta.com/api/v1/logs
----

*`var.api_key`*::

Specifies the Okta API token to use in requests to the API. Required.
The token is used in an HTTP `Authorization` header with the `SSWS` scheme.
See https://developer.okta.com/docs/guides/create-an-api-token/create-the-token/[
Create an API token] for information on how to obtain a token.
+
[source,yaml]
----
var.api_key: '00QCjAl4MlV-WPXM...0HmjFx-vbGua'
----

This module comes with a sample dashboard. For example:
*`var.http_client_timeout`*::

Duration of the time limit on HTTP requests made by the module. Defaults to
`60s`.

*`var.interval`*::

Duration between requests to the API. Defaults to `60s`.

*`var.keep_original_message`*::

Boolean flag indicating if the original JSON event string should be included in
the `event.original` field. Defaults to `true`.

*`var.ssl`*::

Configuration options for SSL parameters like the certificate authority to use
for HTTPS-based connections. If the `ssl` section is missing, the host CAs are
used for HTTPS connections to Okta. See <<configuration-ssl>> for more
information.
+
[source,yaml]
----
var.ssl:
supported_protocols: [TLSv1.2]
----

[float]
=== Example dashboard

This module comes with a sample dashboard:

[role="screenshot"]
image::./images/filebeat-okta-dashboard.png[]

:has-dashboards!:

:fileset_ex!:

:modulename!:


Expand Down
15 changes: 4 additions & 11 deletions x-pack/filebeat/filebeat.reference.yml
Original file line number Diff line number Diff line change
Expand Up @@ -860,17 +860,10 @@ filebeat.modules:
- module: okta
system:
enabled: true
# API key to access Okta
#var.api_key

# URL of the Okta REST API
#var.url

# Disable SSL verification
#var.ssl: |-
# {
# "verification_mode": "none"
# }
# You must configure the URL with your Okta domain and provide an
# API token to access the logs API.
#var.url: https://yourOktaDomain/api/v1/logs
#var.api_key: 'yourApiTokenHere'

#------------------------------- Osquery Module -------------------------------
- module: osquery
Expand Down
15 changes: 4 additions & 11 deletions x-pack/filebeat/module/okta/_meta/config.yml
Original file line number Diff line number Diff line change
@@ -1,14 +1,7 @@
- module: okta
system:
enabled: true
# API key to access Okta
#var.api_key

# URL of the Okta REST API
#var.url

# Disable SSL verification
#var.ssl: |-
# {
# "verification_mode": "none"
# }
# You must configure the URL with your Okta domain and provide an
# API token to access the logs API.
#var.url: https://yourOktaDomain/api/v1/logs
#var.api_key: 'yourApiTokenHere'
96 changes: 93 additions & 3 deletions x-pack/filebeat/module/okta/_meta/docs.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,103 @@

beta[]

This is a filebeat module for retrieving system logs from Okta (www.okta.com) via API.
The Okta module collects events from the
https://developer.okta.com/docs/reference/[Okta API]. Specifically this supports
reading from the https://developer.okta.com/docs/reference/api/system-log/[Okta
System Log API].

:has-dashboards!:
:fileset_ex: system

include::../include/config-option-intro.asciidoc[]

[float]
==== `system` fileset settings

The Okta System Log records system events related to your organization in order
to provide an audit trail that can be used to understand platform activity and
to diagnose problems. This module is implemented using the
<<filebeat-input-httpjson,httpjson>> input and is configured to paginate through
the logs while honoring any
https://developer.okta.com/docs/reference/rate-limits/[rate-limiting] headers
sent by Okta.

NOTE: This module does not persist the timestamp of the last read event in
order to facilitate resuming on restart. This feature will be coming in a future
version. When you restart the module will read events from the beginning of the
log. To minimize duplicates documents the module uses the event's Okta UUID
value as the Elasticsearch `_id`.

This is an example configuration for the module.

[source,yaml]
----
- module okta
system:
var.url: https://yourOktaDomain/api/v1/logs
var.api_key: '00QCjAl4MlV-WPXM...0HmjFx-vbGua'
----

[float]
===== Configuration options

*`var.url`*::

Specifies the URL to the Okta System Log API. Required.
+
[source,yaml]
----
var.url: https://mycompany.okta.com/api/v1/logs
----

*`var.api_key`*::

Specifies the Okta API token to use in requests to the API. Required.
The token is used in an HTTP `Authorization` header with the `SSWS` scheme.
See https://developer.okta.com/docs/guides/create-an-api-token/create-the-token/[
Create an API token] for information on how to obtain a token.
+
[source,yaml]
----
var.api_key: '00QCjAl4MlV-WPXM...0HmjFx-vbGua'
----

This module comes with a sample dashboard. For example:
*`var.http_client_timeout`*::

Duration of the time limit on HTTP requests made by the module. Defaults to
`60s`.

*`var.interval`*::

Duration between requests to the API. Defaults to `60s`.

*`var.keep_original_message`*::

Boolean flag indicating if the original JSON event string should be included in
the `event.original` field. Defaults to `true`.

*`var.ssl`*::

Configuration options for SSL parameters like the certificate authority to use
for HTTPS-based connections. If the `ssl` section is missing, the host CAs are
used for HTTPS connections to Okta. See <<configuration-ssl>> for more
information.
+
[source,yaml]
----
var.ssl:
supported_protocols: [TLSv1.2]
----

[float]
=== Example dashboard

This module comes with a sample dashboard:

[role="screenshot"]
image::./images/filebeat-okta-dashboard.png[]

:has-dashboards!:

:fileset_ex!:

:modulename!:
41 changes: 35 additions & 6 deletions x-pack/filebeat/module/okta/system/config/input.yml
Original file line number Diff line number Diff line change
@@ -1,19 +1,48 @@
{{ if eq .input "httpjson" }}

type: httpjson

{{ if .api_key }}
api_key: {{ .api_key }}
authentication_scheme: {{.authentication_scheme}}
{{ end }}

authentication_scheme: {{ .authentication_scheme }}

{{ if .http_client_timeout }}
http_client_timeout: {{ .http_client_timeout }}
{{ end }}

{{ if .http_method }}
http_method: {{ .http_method }}
http_headers: {{ .http_headers }}
{{ end }}

{{ if .http_headers }}
http_headers: {{ .http_headers | tojson }}
{{ end }}

{{ if .http_request_body }}
http_request_body: {{ .http_request_body }}
no_http_body: {{ .no_http_body }}
{{ end }}

interval: {{ .interval }}

{{ if .json_objects_array }}
json_objects_array: {{ .json_objects_array }}
pagination: {{ .pagination }}
rate_limit: {{ .rate_limit }}
{{ end }}

no_http_body: {{ .no_http_body }}

pagination: {{ .pagination | tojson }}

rate_limit: {{ .rate_limit | tojson }}

{{ if .ssl }}
ssl: {{ .ssl | tojson }}
{{ end }}

{{ if .url }}
url: {{ .url }}
ssl: {{ .ssl }}
{{ end }}

{{ else if eq .input "file" }}

Expand Down
Loading