Change device to observer
#238
Conversation
| @@ -314,6 +298,22 @@ The network is defined as the communication path over which a host or network ev | |||
| | <a name="network.total.packets"></a>network.total.packets | Network outbound packets. The sum of inbound.packets + outbound.packets | core | long | `24` | | |||
|
|
|||
|
|
|||
| ## <a name="observer"></a> Observer fields | |||
|
|
|||
| An observer is defined as a special network, security, or application device used to detect, observe, or create network, security, or application-related events and metrics. This could be a custom hardware appliance or a server that has been configured to run special network, security, or application software. Examples include firewalls, intrusion detection/prevention systems, network monitoring sensors, web application firewalls, data loss prevention systems, and APM servers. The observer.* fields shall be populated with details of the system, if any, that detects, observes and/or creates a network, security, or application event or metric. Message queues and ETL components used in processing events or metrics are not considered observers in ECS. | |||
There was a problem hiding this comment.
@graphaelli Can you leave your comment on this one? APM-Server is in here :-)
There was a problem hiding this comment.
Elastic APM server typically operates as more as a message queue using this definition. Would logstash be considered an observer?
There was a problem hiding this comment.
I would say even if APM server doesn't do a lot of work on the payload, compared to the library installed in each app, it could still be considered as the observer.
There was a problem hiding this comment.
Perhaps it makes sense for an observer simply to be an entity that works on behalf of an agent to report its data.
There was a problem hiding this comment.
@graphaelli I think our definition of observer needs to extend beyond what you suggest to include firewalls and network IDS, etc. that don't have an agent at all.
There was a problem hiding this comment.
@graphaelli Yes, LS is in many cases also the observer.
There was a problem hiding this comment.
I agree that compared to the agent the server doesn't do a lot with the payload currently. However, I am not sure that this is a general enough criteria for not considering the server an ETL component. There are some additions and changes the server does, and going forward with ECS, the server will change the structure of events even more in the future, as there need to be a mapping from the incoming events to the ES ECS structure.
My point is, I assume that
Message queues and ETL components used in processing events or metrics are not considered observers in ECS.
should clarify what to consider an observer and what not, but for me it introduces more confusion.
There was a problem hiding this comment.
LGTM.
I really prefer observer over device. Until this came up last week, it had never occurred to me that device. specifically had to be part of the monitoring pipeline. I thought its purpose was to make a distinction between full blown hosts vs single purpose devices like hardware appliances.
This change removes this ambiguity completely.
| @@ -238,7 +222,7 @@ A file is defined as a set of information that has been created on, or has exist | |||
| Geo fields can carry data about a specific location related to an event or geo information derived from an IP field. | |||
|
|
|||
|
|
|||
| The `geo` fields are expected to be nested at: `destination.geo`, `device.geo`, `host.geo`, `source.geo`. | |||
| The `geo` fields are expected to be nested at: `destination.geo`, `host.geo`, `observer.geo`, `source.geo`. | |||
| @@ -314,6 +298,22 @@ The network is defined as the communication path over which a host or network ev | |||
| | <a name="network.total.packets"></a>network.total.packets | Network outbound packets. The sum of inbound.packets + outbound.packets | core | long | `24` | | |||
|
|
|||
|
|
|||
| ## <a name="observer"></a> Observer fields | |||
|
|
|||
| An observer is defined as a special network, security, or application device used to detect, observe, or create network, security, or application-related events and metrics. This could be a custom hardware appliance or a server that has been configured to run special network, security, or application software. Examples include firewalls, intrusion detection/prevention systems, network monitoring sensors, web application firewalls, data loss prevention systems, and APM servers. The observer.* fields shall be populated with details of the system, if any, that detects, observes and/or creates a network, security, or application event or metric. Message queues and ETL components used in processing events or metrics are not considered observers in ECS. | |||
There was a problem hiding this comment.
I would say even if APM server doesn't do a lot of work on the payload, compared to the library installed in each app, it could still be considered as the observer.
MikePaquette
force-pushed
the
device-to-observer
branch
from
bce0b83 to
3f62009
Compare
CONTRIBUTING.md
Outdated
| * 1 (beta): It's clear that there is value of having the field in ECS and discussions about prefixes/naming have started. It's unlikely that the field will be removed, but naming might change at any time. | ||
| * 2 (rc): The field has been accepted and is unlikely to change. It is now tested in the field. | ||
| * 3 (GA): The field is part of ECS. Any breaking changes will happen only at a major release. | ||
|
|
There was a problem hiding this comment.
So yeah, please make sure to remove this section before merging ;-)
This changes the
device.*fields toobserver.*fields, and clarifies the definition.The
devicename continued to cause significant confusion amongst new users of ECS, so we're proposing this change before beta2, even though it is a breaking change from beta1.