Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Okta module documentation, config cleanup, _id field #18953

Merged
merged 1 commit into from
Jun 4, 2020

Conversation

andrewkroh
Copy link
Member

What does this PR do?

This add documentation for the Okta module. It contains descriptions of the
configuration options and general information about the module.

I fixed an issue with the module where it was not setting the _id field for Elasticsearch events.

I also did some cleanup to the pipeline.js (indentation, semi-colons, strict equality checks).

The module's manifest was updated to not duplicate httpjson's default values.

The module was accepting configuration as JSON strings for some parameters (http_headers, http_request_body, pagination, rate_limit, ssl) which is inconsistent with how other parts of Beats are configured so I removed this. Now these options expect regular YAML objects for values. None of these options are required to use the module so the impact to users should be minimal.

Why is it important?

Documentation is required for users to know how to use the module.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Logs

I ran the module against the Okta API to test. You'll note that the @metadata._id field is now set which libbeat will use when writing the data to Elasticsearch.

{
  "@timestamp": "2020-06-03T18:10:38.118Z",
  "@metadata": {
    "beat": "filebeat",
    "type": "_doc",
    "version": "8.0.0",
    "pipeline": "filebeat-8.0.0-okta-system-pipeline",
    "_id": "870da4d0-a5c5-11ea-a40a-5d8cbb5cdb88"
  },
  "input": {
    "type": "httpjson"
  },
  "fileset": {
    "name": "system"
  },
  "client": {
    "user": {
      "full_name": "Andrew Kroh",
      "id": "00ue26zi9EdhopJgw4x6"
    },
    "geo": {
      "region_name": "Virginia",
      "country_name": "United States",
      "location": {
        "lat": 38.9637,
        "lon": -77.6099
      },
      "city_name": "Aldie"
    },
    "ip": "198.51.100.1",
    "as": {
      "number": 701,
      "organization": {
        "name": "verizon"
      }
    },
    "domain": "verizon.net"
  },
  "user_agent": {
    "original": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:76.0) Gecko/20100101 Firefox/76.0"
  },
  "source": {
    "ip": "198.51.100.1",
    "domain": "verizon.net",
    "user": {
      "full_name": "Andrew Kroh",
      "id": "00ue26zi9EdhopJgw4x6"
    }
  },
  "ecs": {
    "version": "1.5.0"
  },
  "event": {
    "category": [
      "authentication"
    ],
    "action": "system.api_token.create",
    "id": "870da4d0-a5c5-11ea-a40a-5d8cbb5cdb88",
    "kind": "event",
    "module": "okta",
    "dataset": "okta.system",
    "original": "{\"actor\":{\"alternateId\":\"id-okta@test.com\",\"detailEntry\":null,\"displayName\":\"Andrew Kroh\",\"id\":\"00ue26zi9EdhopJgw4x6\",\"type\":\"User\"},\"authenticationContext\":{\"authenticationProvider\":null,\"authenticationStep\":0,\"credentialProvider\":null,\"credentialType\":null,\"externalSessionId\":\"102NKlIOnToQGGOboI6cpvgYQ\",\"interface\":null,\"issuer\":null},\"client\":{\"device\":\"Computer\",\"geographicalContext\":{\"city\":\"Aldie\",\"country\":\"United States\",\"geolocation\":{\"lat\":38.9637,\"lon\":-77.6099},\"postalCode\":\"20105\",\"state\":\"Virginia\"},\"id\":null,\"ipAddress\":\"198.51.100.1\",\"userAgent\":{\"browser\":\"FIREFOX\",\"os\":\"Mac OS X\",\"rawUserAgent\":\"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:76.0) Gecko/20100101 Firefox/76.0\"},\"zone\":\"null\"},\"debugContext\":{\"debugData\":{\"requestId\":\"XtfnnhZLAtrLPbdLtrp-4AAABw4\",\"requestUri\":\"/api/internal/tokens\",\"threatSuspected\":\"false\",\"url\":\"/api/internal/tokens?expand=user\"}},\"displayMessage\":\"Create API token\",\"eventType\":\"system.api_token.create\",\"legacyEventType\":\"api.token.create\",\"outcome\":{\"reason\":null,\"result\":\"SUCCESS\"},\"published\":\"2020-06-03T18:10:38.118Z\",\"request\":{\"ipChain\":[{\"geographicalContext\":{\"city\":\"Aldie\",\"country\":\"United States\",\"geolocation\":{\"lat\":38.9637,\"lon\":-77.6099},\"postalCode\":\"20105\",\"state\":\"Virginia\"},\"ip\":\"198.51.100.1\",\"source\":null,\"version\":\"V4\"}]},\"securityContext\":{\"asNumber\":701,\"asOrg\":\"verizon\",\"domain\":\"verizon.net\",\"isProxy\":false,\"isp\":\"mci communications services  inc. d/b/a verizon business\"},\"severity\":\"INFO\",\"target\":[{\"alternateId\":\"unknown\",\"detailEntry\":null,\"displayName\":\"filebeat\",\"id\":\"00Tt0pmrpvAs7CHYL4x5\",\"type\":\"Token\"}],\"transaction\":{\"detail\":{},\"id\":\"XtfnnhZLAtrLPbdLtrp-4AAABw4\",\"type\":\"WEB\"},\"uuid\":\"870da4d0-a5c5-11ea-a40a-5d8cbb5cdb88\",\"version\":\"0\"}",
    "type": [
      "access"
    ],
    "outcome": "success",
    "created": "2020-06-03T18:12:23.908Z"
  },
  "okta": {
    "event_type": "system.api_token.create",
    "client": {
      "device": "Computer",
      "ip": "198.51.100.1",
      "user_agent": {
        "os": "Mac OS X",
        "raw_user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:76.0) Gecko/20100101 Firefox/76.0",
        "browser": "FIREFOX"
      },
      "zone": "null"
    },
    "authentication_context": {
      "external_session_id": "102NKlIOnToQGGOboI6cpvgYQ",
      "authentication_step": 0
    },
    "display_message": "Create API token",
    "uuid": "870da4d0-a5c5-11ea-a40a-5d8cbb5cdb88",
    "actor": {
      "display_name": "Andrew Kroh",
      "id": "00ue26zi9EdhopJgw4x6",
      "type": "User",
      "alternate_id": "id-okta@test.com"
    },
    "outcome": {
      "result": "SUCCESS"
    },
    "target": [
      {
        "type": "Token",
        "alternate_id": "unknown",
        "display_name": "filebeat",
        "id": "00Tt0pmrpvAs7CHYL4x5"
      }
    ],
    "transaction": {
      "id": "XtfnnhZLAtrLPbdLtrp-4AAABw4",
      "type": "WEB"
    },
    "debug_context": {
      "debug_data": {
        "request_id": "XtfnnhZLAtrLPbdLtrp-4AAABw4",
        "request_uri": "/api/internal/tokens",
        "threat_suspected": "false",
        "url": "/api/internal/tokens?expand=user"
      }
    },
    "security_context": {
      "as": {
        "number": 701,
        "organization": {
          "name": "verizon"
        }
      },
      "domain": "verizon.net",
      "is_proxy": false,
      "isp": "mci communications services  inc. d/b/a verizon business"
    }
  },
  "tags": [
    "forwarded"
  ],
  "service": {
    "type": "okta"
  },
  "agent": {
    "ephemeral_id": "9cd78eb3-acb9-4da6-95a3-7484450a5ad9",
    "id": "1ee8bc3e-9e78-4384-b771-a5655904db72",
    "name": "mac15.example.com",
    "type": "filebeat",
    "version": "8.0.0"
  },
  "related": {
    "user": "Andrew Kroh",
    "ip": "198.51.100.1"
  }
}

@andrewkroh andrewkroh added docs review Filebeat Filebeat needs_backport PR is waiting to be backported to other branches. Team:SIEM breaking change labels Jun 3, 2020
This add documentation for the Okta module. It contains descriptions of the
configuration options and general information about the module.

I fixed an issue with the module where it was not setting the _id field for Elasticsearch events.

I also did some cleanup to the pipeline.js (indentation, semi-colons, strict equality checks).

The module's manifest was updated to not duplicate httpjson's default values.

The module was accepting configuration as JSON strings for some parameters (http_headers, http_request_body, pagination, rate_limit, ssl) which
is inconsistent with how other parts of Beats are configured so I removed this. Now these options expect regular YAML objects for values. None
of these options are required to use the module so the impact to users should be minimal.
@andrewkroh andrewkroh marked this pull request as ready for review June 3, 2020 18:41
@elasticmachine
Copy link
Collaborator

Pinging @elastic/siem (Team:SIEM)

@elasticmachine
Copy link
Collaborator

elasticmachine commented Jun 3, 2020

💚 Build Succeeded

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [andrewkroh commented: run tests]

  • Start Time: 2020-06-03T22:32:00.048+0000

  • Duration: 57 min 16 sec

Test stats 🧪

Test Results
Failed 0
Passed 3823
Skipped 672
Total 4495

@andrewkroh
Copy link
Member Author

run tests

@andrewkroh andrewkroh merged commit 0ef20cc into elastic:master Jun 4, 2020
@andrewkroh andrewkroh added v7.9.0 and removed needs_backport PR is waiting to be backported to other branches. labels Jun 8, 2020
andrewkroh added a commit to andrewkroh/beats that referenced this pull request Jun 9, 2020
This add documentation for the Okta module. It contains descriptions of the
configuration options and general information about the module.

I fixed an issue with the module where it was not setting the _id field for Elasticsearch events.

I also did some cleanup to the pipeline.js (indentation, semi-colons, strict equality checks).

The module's manifest was updated to not duplicate httpjson's default values.

The module was accepting configuration as JSON strings for some parameters (http_headers, http_request_body, pagination, rate_limit, ssl) which
is inconsistent with how other parts of Beats are configured so I removed this. Now these options expect regular YAML objects for values. None
of these options are required to use the module so the impact to users should be minimal.

(cherry picked from commit 0ef20cc)
andrewkroh added a commit that referenced this pull request Jun 9, 2020
…9051)

This add documentation for the Okta module. It contains descriptions of the
configuration options and general information about the module.

I fixed an issue with the module where it was not setting the _id field for Elasticsearch events.

I also did some cleanup to the pipeline.js (indentation, semi-colons, strict equality checks).

The module's manifest was updated to not duplicate httpjson's default values.

The module was accepting configuration as JSON strings for some parameters (http_headers, http_request_body, pagination, rate_limit, ssl) which
is inconsistent with how other parts of Beats are configured so I removed this. Now these options expect regular YAML objects for values. None
of these options are required to use the module so the impact to users should be minimal.

(cherry picked from commit 0ef20cc)
melchiormoulin pushed a commit to melchiormoulin/beats that referenced this pull request Oct 14, 2020
This add documentation for the Okta module. It contains descriptions of the
configuration options and general information about the module.

I fixed an issue with the module where it was not setting the _id field for Elasticsearch events.

I also did some cleanup to the pipeline.js (indentation, semi-colons, strict equality checks).

The module's manifest was updated to not duplicate httpjson's default values.

The module was accepting configuration as JSON strings for some parameters (http_headers, http_request_body, pagination, rate_limit, ssl) which
is inconsistent with how other parts of Beats are configured so I removed this. Now these options expect regular YAML objects for values. None
of these options are required to use the module so the impact to users should be minimal.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants