Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[7.x] Network community_id processor for ingest pipelines (#66534) #67518

Merged

Conversation

danhermann
Copy link
Contributor

Adds a processor that computes the community_id for flow data according to the Community ID Specification.

For example:

POST _ingest/pipeline/_simulate?verbose
{
  "pipeline": {
    "processors": [
      {
        "community_id": {
        }
      }
    ]
  },
  "docs": [
    {
      "_source": {
        "source": {
          "ip": "128.232.110.120",
          "port": 34855
        },
        "destination": {
          "ip": "66.35.250.204",
          "port": 80
        },
        "network": {
          "transport": "TCP"
        }
      }
    }
  ]
}

populates the network.community_id field as below:

...
"_source" : {
  "destination" : {
    "port" : 80,
    "ip" : "66.35.250.204"
  },
  "source" : {
    "port" : 34855,
    "ip" : "128.232.110.120"
  },
  "network" : {
    "community_id" : "1:LQU9qZlK+B5F3KDmev6m5PMibrg=",
    "transport" : "TCP"
  }
}
...

Closes #55685

Backport of #66534

@danhermann danhermann added >feature :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP backport v7.12.0 labels Jan 14, 2021
@elasticmachine elasticmachine added the Team:Data Management Meta label for data/management team label Jan 14, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-features (Team:Core/Features)

@danhermann
Copy link
Contributor Author

@elasticmachine update branch

@danhermann danhermann merged commit 21c3970 into elastic:7.x Jan 14, 2021
@danhermann danhermann deleted the backport_7x_66534_community_id_processor branch January 14, 2021 20:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP >feature Team:Data Management Meta label for data/management team v7.12.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants