Add network category and related event types (elastic#761)

CHANGELOG.next.md

@@ -26,6 +26,9 @@ Thanks, you're awesome :-) -->
 
 * Added fields `event.reference` and `event.url` to hold link to additional event info/actions. (#757)
 
+
+* Added `event.category` value of network and associated `event.type` values. (#761)
+
 #### Improvements
 
 * Temporary workaround for Beats templates' `default_field` growing too big. #687

docs/field-details.asciidoc

@@ -1491,7 +1491,7 @@ Note: this field should contain an array of values.
 
 *Important*: The field value must be one of the following:
 
-authentication, database, driver, file, host, iam, intrusion_detection, malware, package, process, web
+authentication, database, driver, file, host, iam, intrusion_detection, malware, network, package, process, web
 
 To learn more about when to use which value, visit the page
 <<ecs-allowed-values-event-category,allowed values for event.category>>
@@ -1840,7 +1840,7 @@ Note: this field should contain an array of values.
 
 *Important*: The field value must be one of the following:
 
-access, admin, change, creation, deletion, end, error, group, info, installation, start, user
+access, admin, allowed, change, connection, creation, deletion, denied, end, error, group, info, installation, protocol, start, user
 
 To learn more about when to use which value, visit the page
 <<ecs-allowed-values-event-type,allowed values for event.type>>

docs/field-values.asciidoc

@@ -152,6 +152,7 @@ that will require subsequent breaking changes.
 * <<ecs-event-category-iam,iam>>
 * <<ecs-event-category-intrusion_detection,intrusion_detection>>
 * <<ecs-event-category-malware,malware>>
+* <<ecs-event-category-network,network>>
 * <<ecs-event-category-package,package>>
 * <<ecs-event-category-process,process>>
 * <<ecs-event-category-web,web>>
@@ -257,7 +258,7 @@ Relating to intrusion detections from IDS/IPS systems and functions, both networ
 
 *Expected event types for category intrusion_detection:*
 
-info
+allowed, denied, info
 
 
 [float]
@@ -274,6 +275,20 @@ Malware detection events and alerts. Use this category to visualize and analyze
 info
 
 
+[float]
+[[ecs-event-category-network]]
+==== network
+
+Relating to all network activity, including network connection lifecycle, network traffic, and essentially any event that includes an IP address. Many events containing decoded network protocol transactions fit into this category. Use events in this category to visualize or analyze counts of network ports, protocols, addresses, geolocation information, etc.
+
+
+
+
+*Expected event types for category network:*
+
+access, allowed, connection, denied, end, info, protocol, start
+
+
 [float]
 [[ecs-event-category-package]]
 ==== package
@@ -334,14 +349,18 @@ that will require subsequent breaking changes.
 
 * <<ecs-event-type-access,access>>
 * <<ecs-event-type-admin,admin>>
+* <<ecs-event-type-allowed,allowed>>
 * <<ecs-event-type-change,change>>
+* <<ecs-event-type-connection,connection>>
 * <<ecs-event-type-creation,creation>>
 * <<ecs-event-type-deletion,deletion>>
+* <<ecs-event-type-denied,denied>>
 * <<ecs-event-type-end,end>>
 * <<ecs-event-type-error,error>>
 * <<ecs-event-type-group,group>>
 * <<ecs-event-type-info,info>>
 * <<ecs-event-type-installation,installation>>
+* <<ecs-event-type-protocol,protocol>>
 * <<ecs-event-type-start,start>>
 * <<ecs-event-type-user,user>>
 
@@ -365,6 +384,16 @@ The admin event type is used for the subset of events within a category that are
 
 
 
+[float]
+[[ecs-event-type-allowed]]
+==== allowed
+
+The allowed event type is used for the subset of events within a category that indicate that something was allowed. Common examples include `event.category:network AND event.type:connection AND event.type:allowed` (to indicate a network firewall event for which the firewall disposition was to allow the connection to complete) and `event.category:intrusion_detection AND event.type:allowed` (to indicate a network intrusion prevention system event for which the IPS disposition was to allow the connection to complete). You can further distinguish allowed operations using the ECS `event.action` field, populating with values of your choosing, such as "allow", "detect", or "pass".
+
+
+
+
+
 [float]
 [[ecs-event-type-change]]
 ==== change
@@ -375,6 +404,16 @@ The change event type is used for the subset of events within a category that in
 
 
 
+[float]
+[[ecs-event-type-connection]]
+==== connection
+
+Used primarily with `event.category:network` this value is used for the subset of network traffic that includes sufficient information for the event to be included in flow or connection analysis. Events in this subcategory will contain at least source and destination IP addresses, source and destination TCP/UDP ports, and will usually contain counts of bytes and/or packets transferred. Events in this subcategory may contain unidirectional or bidirectional information, including summary information. Use this subcategory to visualize and analyze network connections. Flow analysis, including Netflow, IPFIX, and other flow-related events fit in this subcategory. Note that firewall events from many Next-Generation Firewall (NGFW) devices will also fit into this subcategory.  A common filter for flow/connection information would be `event.category:network AND event.type:connection AND event.type:end` (to view or analyze all completed network connections, ignoring mid-flow reports). You can further distinguish connection events using the ECS `event.action` field, populating with values of your choosing, such as "timeout", or "reset".
+
+
+
+
+
 [float]
 [[ecs-event-type-creation]]
 ==== creation
@@ -395,6 +434,16 @@ The deletion event type is used for the subset of events within a category that
 
 
 
+[float]
+[[ecs-event-type-denied]]
+==== denied
+
+The denied event type is used for the subset of events within a category that indicate that something was denied. Common examples include `event.category:network AND event.type:denied` (to indicate a network firewall event for which the firewall disposition was to deny the connection) and `event.category:intrusion_detection AND event.type:denied` (to indicate a network intrusion prevention system event for which the IPS disposition was to deny the connection to complete). You can further distinguish denied operations using the ECS `event.action` field, populating with values of your choosing, such as "blocked", "dropped", or "quarantined".
+
+
+
+
+
 [float]
 [[ecs-event-type-end]]
 ==== end
@@ -445,6 +494,20 @@ The installation event type is used for the subset of events within a category t
 
 
 
+[float]
+[[ecs-event-type-protocol]]
+==== protocol
+
+The protocol event type is used for the subset of events within a category that indicate that they contain protocol details or analysis, beyond simply identifying the protocol. Generally, network events that contain specific protocol details will fall into this subcategory. A common example is `event.category:network AND event.type:protocol AND event.type:connection AND event.type:end` (to indicate that the event is a network connection event sent at the end of a connection that also includes a protocol detail breakdown). Note that events that only indicate the name or id of the protocol should not use the protocol value. Further note that when the protocol subcategory is used, the identified protocol is populated in the ECS `network.protocol` field.
+
+
+
+
+*Expected event types for category protocol:*
+
+access, change, end, info, start
+
+
 [float]
 [[ecs-event-type-start]]
 ==== start

generated/ecs/ecs_flat.yml

@@ -1860,6 +1860,8 @@ event.category:
 
       '
     expected_event_types:
+    - allowed
+    - denied
     - info
     name: intrusion_detection
   - description: 'Malware detection events and alerts. Use this category to visualize
@@ -1872,6 +1874,23 @@ event.category:
     expected_event_types:
     - info
     name: malware
+  - description: 'Relating to all network activity, including network connection lifecycle,
+      network traffic, and essentially any event that includes an IP address. Many
+      events containing decoded network protocol transactions fit into this category.
+      Use events in this category to visualize or analyze counts of network ports,
+      protocols, addresses, geolocation information, etc.
+
+      '
+    expected_event_types:
+    - access
+    - allowed
+    - connection
+    - denied
+    - end
+    - info
+    - protocol
+    - start
+    name: network
   - description: 'Relating to software packages installed on hosts. Use this category
       to visualize and analyze inventory of software installed on various hosts, or
       to determine host vulnerability in the absence of vulnerability scan data.
@@ -2365,6 +2384,18 @@ event.type:
 
       '
     name: admin
+  - description: 'The allowed event type is used for the subset of events within a
+      category that indicate that something was allowed. Common examples include `event.category:network
+      AND event.type:connection AND event.type:allowed` (to indicate a network firewall
+      event for which the firewall disposition was to allow the connection to complete)
+      and `event.category:intrusion_detection AND event.type:allowed` (to indicate
+      a network intrusion prevention system event for which the IPS disposition was
+      to allow the connection to complete). You can further distinguish allowed operations
+      using the ECS `event.action` field, populating with values of your choosing,
+      such as "allow", "detect", or "pass".
+
+      '
+    name: allowed
   - description: 'The change event type is used for the subset of events within a
       category that indicate that something has changed. If semantics best describe
       an event as modified, then include them in this subcategory. Common examples
@@ -2374,6 +2405,24 @@ event.type:
 
       '
     name: change
+  - description: 'Used primarily with `event.category:network` this value is used
+      for the subset of network traffic that includes sufficient information for the
+      event to be included in flow or connection analysis. Events in this subcategory
+      will contain at least source and destination IP addresses, source and destination
+      TCP/UDP ports, and will usually contain counts of bytes and/or packets transferred.
+      Events in this subcategory may contain unidirectional or bidirectional information,
+      including summary information. Use this subcategory to visualize and analyze
+      network connections. Flow analysis, including Netflow, IPFIX, and other flow-related
+      events fit in this subcategory. Note that firewall events from many Next-Generation
+      Firewall (NGFW) devices will also fit into this subcategory.  A common filter
+      for flow/connection information would be `event.category:network AND event.type:connection
+      AND event.type:end` (to view or analyze all completed network connections, ignoring
+      mid-flow reports). You can further distinguish connection events using the ECS
+      `event.action` field, populating with values of your choosing, such as "timeout",
+      or "reset".
+
+      '
+    name: connection
   - description: 'The "creation" event type is used for the subset of events within
       a category that indicate that something was created. A common example is `event.category:file
       AND event.type:creation`.
@@ -2386,6 +2435,17 @@ event.type:
 
       '
     name: deletion
+  - description: 'The denied event type is used for the subset of events within a
+      category that indicate that something was denied. Common examples include `event.category:network
+      AND event.type:denied` (to indicate a network firewall event for which the firewall
+      disposition was to deny the connection) and `event.category:intrusion_detection
+      AND event.type:denied` (to indicate a network intrusion prevention system event
+      for which the IPS disposition was to deny the connection to complete). You can
+      further distinguish denied operations using the ECS `event.action` field, populating
+      with values of your choosing, such as "blocked", "dropped", or "quarantined".
+
+      '
+    name: denied
   - description: 'The end event type is used for the subset of events within a category
       that indicate something has ended. A common example is `event.category:process
       AND event.type:end`.
@@ -2424,6 +2484,25 @@ event.type:
 
       '
     name: installation
+  - description: 'The protocol event type is used for the subset of events within
+      a category that indicate that they contain protocol details or analysis, beyond
+      simply identifying the protocol. Generally, network events that contain specific
+      protocol details will fall into this subcategory. A common example is `event.category:network
+      AND event.type:protocol AND event.type:connection AND event.type:end` (to indicate
+      that the event is a network connection event sent at the end of a connection
+      that also includes a protocol detail breakdown). Note that events that only
+      indicate the name or id of the protocol should not use the protocol value. Further
+      note that when the protocol subcategory is used, the identified protocol is
+      populated in the ECS `network.protocol` field.
+
+      '
+    expected_event_types:
+    - access
+    - change
+    - end
+    - info
+    - start
+    name: protocol
   - description: 'The start event type is used for the subset of events within a category
       that indicate something has started. A common example is `event.category:process
       AND event.type:start`.