Change device to observer (#238)

* change device to observer * Update CHANGELOG.md * Update geo, os descriptions * Update agent, related, to change device to observer * backed out change to CONTRIBUTING.md * re-fixed CONTRIBUING after re-base
elastic · Dec 7, 2018 · e0a48af · e0a48af
1 parent 983befa
commit e0a48af
Show file tree

Hide file tree

Showing 11 changed files with 162 additions and 163 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -4,6 +4,7 @@ All notable changes to this project will be documented in this file based on the
 ## Unreleased
 
 ### Breaking changes
+* Changed `device.*` fields to `observer.*` fields to eliminate user confusion. #238
 
 * Rename `network.total.bytes` to `network.bytes` and `network.total.packets`
   to `network.packets`. #179

diff --git a/README.md b/README.md
@@ -52,7 +52,6 @@ ECS defines these fields.
  * [Cloud fields](#cloud)
  * [Container fields](#container)
  * [Destination fields](#destination)
- * [Device fields](#device)
  * [ECS fields](#ecs)
  * [Error fields](#error)
  * [Event fields](#event)
@@ -63,6 +62,7 @@ ECS defines these fields.
  * [HTTP fields](#http)
  * [Log fields](#log)
  * [Network fields](#network)
+ * [Observer fields](#observer)
  * [Organization fields](#organization)
  * [Operating System fields](#os)
  * [Process fields](#process)
@@ -87,7 +87,7 @@ The base set contains all fields which are on the top level. These fields are co
 
 ## <a name="agent"></a> Agent fields
 
-The agent fields contain the data about the software entity, if any, that collects, detects, or observes events on a host, or takes measurements on a host. Examples include beats. Agents may also run on devices. ECS agent.* fields shall be populated with details of the agent running on the host or device where the event happened or the measurement was taken.
+The agent fields contain the data about the software entity, if any, that collects, detects, or observes events on a host, or takes measurements on a host. Examples include Beats. Agents may also run on observers. ECS agent.* fields shall be populated with details of the agent running on the host or observer where the event happened or the measurement was taken.
 
 
 | Field  | Description  | Level  | Type  | Example  |
@@ -151,22 +151,6 @@ Destination fields describe details about the destination of a packet/event.
 | <a name="destination.packets"></a>destination.packets | Packets sent from the destination to the source. | core | long | `12` |
 
 
-## <a name="device"></a> Device fields
-
-Device fields are used to provide additional information about the device that is the source of the information. This could be a firewall, network device, etc.
-
-
-| Field  | Description  | Level  | Type  | Example  |
-|---|---|---|---|---|
-| <a name="device.mac"></a>device.mac | MAC address of the device | core | keyword |  |
-| <a name="device.ip"></a>device.ip | IP address of the device. | core | ip |  |
-| <a name="device.hostname"></a>device.hostname | Hostname of the device. | core | keyword |  |
-| <a name="device.vendor"></a>device.vendor | Device vendor information. | core | keyword |  |
-| <a name="device.version"></a>device.version | Device version. | core | keyword |  |
-| <a name="device.serial_number"></a>device.serial_number | Device serial number. | extended | keyword |  |
-| <a name="device.type"></a>device.type | The type of the device the data is coming from.<br/>There is no predefined list of device types. Some examples are `endpoint`, `firewall`, `ids`, `ips`, `proxy`. | core | keyword | `firewall` |
-
-
 ## <a name="ecs"></a> ECS fields
 
 Meta-information specific to ECS.
@@ -241,7 +225,7 @@ A file is defined as a set of information that has been created on, or has exist
 Geo fields can carry data about a specific location related to an event or geo information derived from an IP field.
 
 
-The `geo` fields are expected to be nested at: `destination.geo`, `device.geo`, `host.geo`, `source.geo`.
+The `geo` fields are expected to be nested at: `destination.geo`, `host.geo`, `observer.geo`, `source.geo`.
 
 Note also that the `geo` fields are not expected to be used directly at the top level.
 
@@ -328,6 +312,22 @@ The network is defined as the communication path over which a host or network ev
 | <a name="network.packets"></a>network.packets | Total packets transferred in both directions.<br/>If `source.packets` and `destination.packets` are known, `network.packets` is their sum. | core | long | `24` |
 
 
+## <a name="observer"></a> Observer fields
+
+An observer is defined as a special network, security, or application device used to detect, observe, or create network, security, or application-related events and metrics. This could be a custom hardware appliance or a server that has been configured to run special network, security, or application software. Examples include firewalls, intrusion detection/prevention systems, network monitoring sensors, web application firewalls, data loss prevention systems, and APM servers. The observer.* fields shall be populated with details of the system, if any, that detects, observes and/or creates a network, security, or application event or metric. Message queues and ETL components used in processing events or metrics are not considered observers in ECS.  
+
+
+| Field  | Description  | Level  | Type  | Example  |
+|---|---|---|---|---|
+| <a name="observer.mac"></a>observer.mac | MAC address of the observer | core | keyword |  |
+| <a name="observer.ip"></a>observer.ip | IP address of the observer. | core | ip |  |
+| <a name="observer.hostname"></a>observer.hostname | Hostname of the observer. | core | keyword |  |
+| <a name="observer.vendor"></a>observer.vendor | observer vendor information. | core | keyword |  |
+| <a name="observer.version"></a>observer.version | Observer version. | core | keyword |  |
+| <a name="observer.serial_number"></a>observer.serial_number | Observer serial number. | extended | keyword |  |
+| <a name="observer.type"></a>observer.type | The type of the observer the data is coming from.<br/>There is no predefined list of observer types. Some examples are `forwarder`, `firewall`, `ids`, `ips`, `proxy`, `poller`, `sensor`, `APM server`. | core | keyword | `firewall` |
+
+
 ## <a name="organization"></a> Organization fields
 
 The organization fields enrich data with information about the company or entity the data is associated with. These fields help you arrange or filter data stored in an index by one or multiple organizations.
@@ -344,7 +344,7 @@ The organization fields enrich data with information about the company or entity
 The OS fields contain information about the operating system.
 
 
-The `os` fields are expected to be nested at: `device.os`, `host.os`, `user_agent.os`.
+The `os` fields are expected to be nested at: `host.os`, `observer.os`, `user_agent.os`.
 
 Note also that the `os` fields are not expected to be used directly at the top level.
 
@@ -379,7 +379,7 @@ These fields contain information about a process. These fields can help you corr
 
 This field set is meant to facilitate pivoting around a piece of data. Some pieces of information can be seen in many places in ECS. To facilitate searching for them, append values to their corresponding field in `related.`.
 
-A concrete example is IP addresses, which can be under host, device, source, destination, and network.forwarded_ip. If you append all IPs to `related.ip`, you can then search for a given IP trivially, no matter where it appeared, by querying `related.ip:a.b.c.d`.
+A concrete example is IP addresses, which can be under host, observer, source, destination, and network.forwarded_ip. If you append all IPs to `related.ip`, you can then search for a given IP trivially, no matter where it appeared, by querying `related.ip:a.b.c.d`.
 
 
 | Field  | Description  | Level  | Type  | Example  |

diff --git a/fields.yml b/fields.yml
@@ -9,7 +9,7 @@
       title: Agent
       group: 2
       description: >
-        The agent fields contain the data about the software entity, if any, that collects, detects, or observes events on a host, or takes measurements on a host. Examples include beats. Agents may also run on devices. ECS agent.* fields shall be populated with details of the agent running on the host or device where the event happened or the measurement was taken.
+        The agent fields contain the data about the software entity, if any, that collects, detects, or observes events on a host, or takes measurements on a host. Examples include Beats. Agents may also run on observers. ECS agent.* fields shall be populated with details of the agent running on the host or observer where the event happened or the measurement was taken.
       footnote: >
         Examples: In the case of Beats for logs, the agent.name is filebeat. For APM, it is the
         agent running in the app/service. The agent information does not change if
@@ -284,54 +284,6 @@
           description: >
             Packets sent from the destination to the source.
     
-    - name: device
-      title: Device
-      group: 2
-      description: >
-        Device fields are used to provide additional information about the device
-        that is the source of the information. This could be a firewall, network device, etc.
-      type: group
-      fields:
-        - name: mac
-          level: core
-          type: keyword
-          description: >
-            MAC address of the device
-        - name: ip
-          level: core
-          type: ip
-          description: >
-            IP address of the device.
-        - name: hostname
-          level: core
-          type: keyword
-          description: >
-            Hostname of the device.
-        - name: vendor
-          level: core
-          type: keyword
-          description: >
-            Device vendor information.
-        - name: version
-          level: core
-          type: keyword
-          description: >
-            Device version.
-        - name: serial_number
-          level: extended
-          type: keyword
-          description: >
-            Device serial number.
-        - name: type
-          level: core
-          type: keyword
-          description: >
-            The type of the device the data is coming from.
-    
-            There is no predefined list of device types. Some examples are
-            `endpoint`, `firewall`, `ids`, `ips`, `proxy`.
-          example: firewall
-
     - name: ecs
       title: ECS
       group: 2
@@ -633,7 +585,7 @@
         top_level: false
         expected:
           - destination
-          - device
+          - observer
           - host
           - source
       type: group
@@ -966,6 +918,53 @@
             If `source.packets` and `destination.packets` are known, `network.packets` is their sum.
           example: 24
 
+    - name: observer
+      title: Observer
+      group: 2
+      description: >
+        An observer is defined as a special network, security, or application device used to detect, observe, or create network, security, or application-related events and metrics. This could be a custom hardware appliance or a server that has been configured to run special network, security, or application software. Examples include firewalls, intrusion detection/prevention systems, network monitoring sensors, web application firewalls, data loss prevention systems, and APM servers. The observer.* fields shall be populated with details of the system, if any, that detects, observes and/or creates a network, security, or application event or metric. Message queues and ETL components used in processing events or metrics are not considered observers in ECS.  
+      type: group
+      fields:
+        - name: mac
+          level: core
+          type: keyword
+          description: >
+            MAC address of the observer
+        - name: ip
+          level: core
+          type: ip
+          description: >
+            IP address of the observer.
+        - name: hostname
+          level: core
+          type: keyword
+          description: >
+            Hostname of the observer.
+        - name: vendor
+          level: core
+          type: keyword
+          description: >
+            observer vendor information.
+        - name: version
+          level: core
+          type: keyword
+          description: >
+            Observer version.
+        - name: serial_number
+          level: extended
+          type: keyword
+          description: >
+            Observer serial number.
+        - name: type
+          level: core
+          type: keyword
+          description: >
+            The type of the observer the data is coming from.
+    
+            There is no predefined list of observer types. Some examples are
+            `forwarder`, `firewall`, `ids`, `ips`, `proxy`, `poller`, `sensor`, `APM server`.
+          example: firewall
+
     - name: organization
       title: Organization
       group: 2
@@ -995,7 +994,7 @@
       reusable:
         top_level: false
         expected:
-          - device
+          - observer
           - host
           - user_agent
       type: group
@@ -1124,7 +1123,7 @@
         searching for them, append values to their corresponding field in
         `related.`.
     
-        A concrete example is IP addresses, which can be under host, device, source,
+        A concrete example is IP addresses, which can be under host, observer, source,
         destination, and network.forwarded_ip. If you append all IPs to
         `related.ip`, you can then search for a given IP trivially,
         no matter where it appeared, by querying `related.ip:a.b.c.d`.

diff --git a/schema.csv b/schema.csv
@@ -27,13 +27,6 @@ destination.ip,ip,core,
 destination.mac,keyword,core,
 destination.packets,long,core,12
 destination.port,long,core,
-device.hostname,keyword,core,
-device.ip,ip,core,
-device.mac,keyword,core,
-device.serial_number,keyword,extended,
-device.type,keyword,core,firewall
-device.vendor,keyword,core,
-device.version,keyword,core,
 ecs.version,keyword,core,1.0.0-beta1
 error.code,keyword,core,
 error.id,keyword,core,
@@ -101,6 +94,13 @@ network.packets,long,core,24
 network.protocol,keyword,core,http
 network.transport,keyword,core,TCP
 network.type,keyword,core,IPv4
+observer.hostname,keyword,core,
+observer.ip,ip,core,
+observer.mac,keyword,core,
+observer.serial_number,keyword,extended,
+observer.type,keyword,core,firewall
+observer.vendor,keyword,core,
+observer.version,keyword,core,
 organization.id,keyword,extended,
 organization.name,keyword,extended,
 os.family,keyword,extended,debian

diff --git a/schemas/agent.yml b/schemas/agent.yml
@@ -3,7 +3,7 @@
   title: Agent
   group: 2
   description: >
-    The agent fields contain the data about the software entity, if any, that collects, detects, or observes events on a host, or takes measurements on a host. Examples include beats. Agents may also run on devices. ECS agent.* fields shall be populated with details of the agent running on the host or device where the event happened or the measurement was taken.
+    The agent fields contain the data about the software entity, if any, that collects, detects, or observes events on a host, or takes measurements on a host. Examples include Beats. Agents may also run on observers. ECS agent.* fields shall be populated with details of the agent running on the host or observer where the event happened or the measurement was taken.
   footnote: >
     Examples: In the case of Beats for logs, the agent.name is filebeat. For APM, it is the
     agent running in the app/service. The agent information does not change if

diff --git a/schemas/device.yml b/schemas/device.yml
diff --git a/schemas/geo.yml b/schemas/geo.yml
@@ -9,7 +9,7 @@
     top_level: false
     expected:
       - destination
-      - device
+      - observer
       - host
       - source
   type: group

diff --git a/schemas/observer.yml b/schemas/observer.yml
@@ -0,0 +1,47 @@
+---
+- name: observer
+  title: Observer
+  group: 2
+  description: >
+    An observer is defined as a special network, security, or application device used to detect, observe, or create network, security, or application-related events and metrics. This could be a custom hardware appliance or a server that has been configured to run special network, security, or application software. Examples include firewalls, intrusion detection/prevention systems, network monitoring sensors, web application firewalls, data loss prevention systems, and APM servers. The observer.* fields shall be populated with details of the system, if any, that detects, observes and/or creates a network, security, or application event or metric. Message queues and ETL components used in processing events or metrics are not considered observers in ECS.  
+  type: group
+  fields:
+    - name: mac
+      level: core
+      type: keyword
+      description: >
+        MAC address of the observer
+    - name: ip
+      level: core
+      type: ip
+      description: >
+        IP address of the observer.
+    - name: hostname
+      level: core
+      type: keyword
+      description: >
+        Hostname of the observer.
+    - name: vendor
+      level: core
+      type: keyword
+      description: >
+        observer vendor information.
+    - name: version
+      level: core
+      type: keyword
+      description: >
+        Observer version.
+    - name: serial_number
+      level: extended
+      type: keyword
+      description: >
+        Observer serial number.
+    - name: type
+      level: core
+      type: keyword
+      description: >
+        The type of the observer the data is coming from.
+
+        There is no predefined list of observer types. Some examples are
+        `forwarder`, `firewall`, `ids`, `ips`, `proxy`, `poller`, `sensor`, `APM server`.
+      example: firewall