[1.x] Add mapping network event guidance doc (#969) (#983)

CHANGELOG.next.md

@@ -25,6 +25,7 @@ Thanks, you're awesome :-) -->
 #### Improvements
 
 * Expanded field set definitions for `source.*` and `destination.*`. #967
+* Provided better guidance for mapping network events. #969
 
 #### Deprecated
 

docs/additional.asciidoc

@@ -2,6 +2,7 @@
 == Additional Information
 
 * <<ecs-faq>>
+* <<ecs-mapping-network-events>>
 * <<ecs-glossary>>
 * <<ecs-contributing>>
 

docs/using-mapping-network-events.asciidoc

@@ -0,0 +1,267 @@
+[[ecs-mapping-network-events]]
+=== Mapping Network Events
+
+Network events capture the details of one device communicating with another. The initiator is referred to as the source, and the recipient as the destination. Depending on the data source, a network event can contain details of addresses, protocols, headers, and device roles.
+
+This guide describes the different field sets available for network-related events in ECS and provides direction on the ECS best practices for mapping to them.
+
+[float]
+==== Source and destination baseline
+
+When an event contains details about the sending and receiving hosts, the baseline for capturing these values will be the <<ecs-source,source>> and <<ecs-destination,destination>> fields.
+
+Some events may also indicate each host's role in the exchange: client or server. When this information is available, the <<ecs-client,client>> and <<ecs-server,server>> fields should be used _in addition to_ the `source` and `destination` fields. The fields and values mapped under `source`/`destination` should be copied under `client`/`server`.
+
+[float]
+==== Network event mapping example
+
+Below is a DNS network event. The source device (`192.168.86.222`) makes a DNS query, acting as the client and the DNS server is the destination (`192.168.86.1`).
+
+Note this event contains additional details that would populate additional fields (such as the <<ecs-dns>>) if this was a complete mapping example. These additional fields are omitted here to focus on the network details.
+
+[source,json]
+----
+{
+  "ts":1599775747.53056,
+  "uid":"CYqFPH3nOAa0kPxA0d",
+  "id.orig_h":"192.168.86.222",
+  "id.orig_p":54162,
+  "id.resp_h":"192.168.86.1",
+  "id.resp_p":53,
+  "proto":"udp",
+  "trans_id":28899,
+  "rtt":0.02272200584411621,
+  "query":"example.com",
+  "qclass":1,
+  "qclass_name":"C_INTERNET",
+  "qtype":1,
+  "qtype_name":"A",
+  "rcode":0,
+  "rcode_name":"NOERROR",
+  "AA":false,
+  "TC":false,
+  "RD":true,
+  "RA":true,
+  "Z":0,
+  "answers":["93.184.216.34"],
+  "TTLs":[21209.0],
+  "rejected":false
+}
+----
+
+[float]
+==== Source and destination fields
+
+First, the `source.*` and `destination.*` field sets are populated:
+
+[source,json]
+----
+  "source": {
+    "ip": "192.168.86.222",
+    "port": 54162
+  }
+----
+
+[source,json]
+----
+  "destination": {
+    "ip": "192.168.86.1",
+    "port": 53
+  }
+----
+
+[float]
+==== Client and server fields
+
+Looking back at the original event, it shows the source device is the DNS client and the destination device is the DNS server. The values mapped under `source` and `destination` are copied and mapped under `client` and `server`, respectively:
+
+[source,json]
+----
+  "client": {
+    "ip": "192.168.86.222",
+    "port": 64734
+  }
+----
+
+[source,json]
+----
+  "server": {
+    "ip": "192.168.86.1",
+    "port": 53
+  }
+----
+
+Mapping both pairs of field sets gives query visibility of the same network transaction in two ways.
+
+* `source.ip:192.168.86.222` returns all events sourced from `192.168.86.222`, regardless its role in a transaction
+* `client.ip:192.168.86.222` returns all events with host `192.168.86.222` acting as a client
+
+The same applies for the `destination` and `server` fields:
+
+* `destination.ip:192.168.86.1` returns all events destined to `192.168.86.1`
+* `server.ip:192.168.86.1` returns all events with `192.168.86.1` acting as the server
+
+It's important to note that while the values for the `source` and `destination` fields may reverse between events in a single network transaction, the values for `client` and `server` typically will not. The following two tables demonstrate how two DNS transactions involving two clients and one server would map to `source.ip`/`destination.ip` vs. `client.ip`/`server.ip`:
+
+[options="header"]
+.Source/Destination
+|=====
+| source.ip | destination.ip | event
+
+// ===============================================================
+
+| 192.168.86.222
+| 192.168.86.1
+| DNS query request 1
+
+// ===============================================================
+
+| 192.168.86.1
+| 192.168.86.222
+| DNS answer response 1
+
+// ===============================================================
+
+| 192.168.86.42
+| 192.168.86.1
+| DNS answer request 2
+
+// ===============================================================
+
+| 192.168.86.1
+| 192.168.86.42
+| DNS answer request 2
+
+|=====
+
+[options="header"]
+.Client/Server
+|=====
+| client.ip | server.ip | event
+
+// ===============================================================
+
+| 192.168.86.222
+| 192.168.86.1
+| DNS query request 1
+
+// ===============================================================
+
+| 192.168.86.222
+| 192.168.86.1
+| DNS answer response 1
+
+// ===============================================================
+
+| 192.168.86.42
+| 192.168.86.1
+| DNS query request 2
+
+// ===============================================================
+
+| 192.168.86.42
+| 192.168.86.1
+| DNS answer response 2
+
+|=====
+
+[float]
+==== Related fields
+
+The `related.ip` field captures all the IPs present in the event in a single array:
+
+[source,json]
+----
+  "related": {
+    "ip": [
+      "192.168.86.222",
+      "192.168.86.1",
+      "93.184.216.34"
+    ]
+  }
+----
+
+The <<ecs-related,related fields>> are meant to facilitate pivoting. Since these IP addresses can appear in many different fields (`source.ip`, `destination.ip`, `client.ip`, `server.ip`, etc.), you can search for the IP trivially no matter what field it appears using a single query, e.g. `related.ip:192.168.86.222`.
+
+Network events are not only limited to using `related.ip`. If hostnames or other host identifiers were present in the event, `related.hosts` should be populated too.
+
+[float]
+==== Categorization using event fields
+
+When considering the <<ecs-category-field-values-reference, event categorization fields>>, the `category` and `type` fields are populated using their respective allowed values which best classify the source network event.
+
+[source,json]
+----
+  "event": {
+    "category": [
+      "network"
+    ],
+    "type": [
+      "connection",
+      "protocol"
+    ],
+    "kind": "event"
+  }
+----
+
+Most <<ecs-allowed-values-event-category,event.category>>/<<ecs-allowed-values-event-type,event.type>> ECS pairings are complete on their own. However, the pairing of `event.category:network` and `event.type:protocol` is an exception. When these two fields/value pairs both used to categorize an event, the `network.protocol` field should also be populated:
+
+[source,json]
+----
+  "network": {
+    "protocol": "dns",
+    "type": "ipv4",
+    "transport": "udp"
+  }
+----
+
+[float]
+==== Result
+
+Putting everything together covered so far, we have a final ECS-mapped event:
+
+[source,json]
+----
+{
+  "event": {
+    "category": [
+      "network"
+    ],
+    "type": [
+      "connection",
+      "protocol"
+    ],
+    "kind": "event"
+  },
+  "network": {
+    "protocol": "dns",
+    "type": "ipv4",
+    "transport": "udp"
+  },
+  "source": {
+    "ip": "192.168.86.222",
+    "port": 54162
+  },
+  "destination": {
+    "ip": "192.168.86.1",
+    "port": 53
+  },
+  "client": {
+    "ip": "192.168.86.222",
+    "port": 64734
+  },
+  "server": {
+    "ip": "192.168.86.1",
+    "port": 53
+  },
+  "related": {
+    "ip": [
+      "192.168.86.222",
+      "192.168.86.1",
+      "93.184.216.34"
+    ]
+  },
+  "dns": { ... }, <= Again, not diving into the DNS fields here but included for completeness.
+  "zeek": { "ts":1599775747.53056, ... } <= Original fields can optionally be kept around as custom fields.
+}
+----

docs/using.asciidoc

@@ -8,13 +8,17 @@ If you're new to ECS and would like an introduction on implementing and using
 the schema, check out the <<ecs-getting-started>> guide.
 
 Whether you're trying to recall a field name, implementing a solution that
-follows ECS, or proposing a change to  the schema, the <<ecs-guidelines>> and
+follows ECS, or proposing a change to the schema, the <<ecs-guidelines>> and
 <<ecs-conventions>> will help get you there.
 
 If you're wondering how to best capture event details that don't map to existing
 ECS fields, head over to <<ecs-custom-fields-in-ecs>>.
 
+<<ecs-mapping-network-events, Mapping network events>> provides a detailed walk-through of how to best map and
+categorize an example network event to the schema.
+
 include::using-getting-started.asciidoc[][]
 include::using-guidelines.asciidoc[]
 include::using-conventions.asciidoc[]
 include::using-custom-fields.asciidoc[]
+include::using-mapping-network-events.asciidoc[]