Add field event.agent_id_status (#1454) (#1456)

* Add field event.agent_id_status This adds a field that can be used to reflect the status of the agent.id verification performed by the receiving system or data pipeline. If the receiving system checks that the sender is authorized for a given agent.id value then the outcome can be added to this field. For example you might implement mTLS for authenticating agents sending data to Logstash. You could add the agent's ID to the agent's client cert subject and then validate incoming events in your Logstash pipeline to ensure the data has the agent ID. Or with Elasticsearch you could provide each of your agents with an API key that has the agent.id associated to the API key metadata. Then using an Ingest Node pipeline you can validate the agent.id against the client's API key metadata. * Shorten value names, remove allowed_values Co-authored-by: Andrew Kroh <[email protected]>
elastic · Jun 14, 2021 · 829873c · 829873c
1 parent 2a372c8
commit 829873c
Show file tree

Hide file tree

Showing 18 changed files with 309 additions and 1 deletion.
diff --git a/CHANGELOG.next.md b/CHANGELOG.next.md
@@ -19,6 +19,7 @@ Thanks, you're awesome :-) -->
 * `elf.*` field set added as beta. #1410
 * Remove `beta` from `orchestrator` field set. #1417
 * Extend `threat.*` field set beta. #1438
+* Added `event.agent_id_status` field. #1454 
 
 #### Improvements
 

diff --git a/Makefile b/Makefile
@@ -69,7 +69,7 @@ generate: generator legacy_use_cases codegen
 
 # Run the new generator
 .PHONY: generator
-generator:
+generator: ve
 	$(PYTHON) scripts/generator.py --strict --include "${INCLUDE}"
 
 # Generate Go code from the schema.

diff --git a/code/go/ecs/event.go b/code/go/ecs/event.go
diff --git a/docs/field-details.asciidoc b/docs/field-details.asciidoc
@@ -2498,6 +2498,36 @@ example: `user-password-change`
 
 // ===============================================================
 
+|
+[[field-event-agent-id-status]]
+<<field-event-agent-id-status, event.agent_id_status>>
+
+| Agents are normally responsible for populating the `agent.id` field value. If the system receiving events is capable of validating the value based on authentication information for the client then this field can be used to reflect the outcome of that validation.
+
+For example if the agent's connection is authenticated with mTLS and the client cert contains the ID of the agent to which the cert was issued then the `agent.id` value in events can be checked against the certificate. If the values match then `event.agent_id_status: verified` is added to the event, otherwise one of the other allowed values should be used.
+
+If no validation is performed then the field should be omitted.
+
+The allowed values are:
+
+`verified` - The `agent.id` field value matches expected value obtained from auth metadata.
+
+`mismatch` - The `agent.id` field value does not match the expected value obtained from auth metadata.
+
+`missing` - There was no `agent.id` field in the event to validate.
+
+`auth_metadata_missing` - There was no auth metadata or it was missing information about the agent ID.
+
+type: keyword
+
+
+
+example: `verified`
+
+| extended
+
+// ===============================================================
+
 |
 [[field-event-category]]
 <<field-event-category, event.category>>

diff --git a/experimental/generated/beats/fields.ecs.yml b/experimental/generated/beats/fields.ecs.yml
@@ -1769,6 +1769,37 @@
         Examples are `group-add`, `process-started`, `file-created`. The value is
         normally defined by the implementer.'
       example: user-password-change
+    - name: agent_id_status
+      level: extended
+      type: keyword
+      ignore_above: 1024
+      description: 'Agents are normally responsible for populating the `agent.id`
+        field value. If the system receiving events is capable of validating the value
+        based on authentication information for the client then this field can be
+        used to reflect the outcome of that validation.
+
+        For example if the agent''s connection is authenticated with mTLS and the
+        client cert contains the ID of the agent to which the cert was issued then
+        the `agent.id` value in events can be checked against the certificate. If
+        the values match then `event.agent_id_status: verified` is added to the event,
+        otherwise one of the other allowed values should be used.
+
+        If no validation is performed then the field should be omitted.
+
+        The allowed values are:
+
+        `verified` - The `agent.id` field value matches expected value obtained from
+        auth metadata.
+
+        `mismatch` - The `agent.id` field value does not match the expected value
+        obtained from auth metadata.
+
+        `missing` - There was no `agent.id` field in the event to validate.
+
+        `auth_metadata_missing` - There was no auth metadata or it was missing information
+        about the agent ID.'
+      example: verified
+      default_field: false
     - name: category
       level: core
       type: keyword

diff --git a/experimental/generated/csv/fields.csv b/experimental/generated/csv/fields.csv
@@ -183,6 +183,7 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description
 1.11.0-dev+exp,true,error,error.stack_trace.text,text,extended,,,The stack trace of this error in plain text.
 1.11.0-dev+exp,true,error,error.type,wildcard,extended,,java.lang.NullPointerException,"The type of the error, for example the class name of the exception."
 1.11.0-dev+exp,true,event,event.action,keyword,core,,user-password-change,The action captured by the event.
+1.11.0-dev+exp,true,event,event.agent_id_status,keyword,extended,,verified,Validation status of the event's agent.id field.
 1.11.0-dev+exp,true,event,event.category,keyword,core,array,authentication,Event category. The second categorization field in the hierarchy.
 1.11.0-dev+exp,true,event,event.code,keyword,extended,,4648,Identification code for this event.
 1.11.0-dev+exp,true,event,event.created,date,core,,2016-05-23T08:05:34.857Z,Time when the event was first read by an agent or by your pipeline.

diff --git a/experimental/generated/ecs/ecs_flat.yml b/experimental/generated/ecs/ecs_flat.yml
@@ -2214,6 +2214,41 @@ event.action:
   normalize: []
   short: The action captured by the event.
   type: keyword
+event.agent_id_status:
+  dashed_name: event-agent-id-status
+  description: 'Agents are normally responsible for populating the `agent.id` field
+    value. If the system receiving events is capable of validating the value based
+    on authentication information for the client then this field can be used to reflect
+    the outcome of that validation.
+
+    For example if the agent''s connection is authenticated with mTLS and the client
+    cert contains the ID of the agent to which the cert was issued then the `agent.id`
+    value in events can be checked against the certificate. If the values match then
+    `event.agent_id_status: verified` is added to the event, otherwise one of the
+    other allowed values should be used.
+
+    If no validation is performed then the field should be omitted.
+
+    The allowed values are:
+
+    `verified` - The `agent.id` field value matches expected value obtained from auth
+    metadata.
+
+    `mismatch` - The `agent.id` field value does not match the expected value obtained
+    from auth metadata.
+
+    `missing` - There was no `agent.id` field in the event to validate.
+
+    `auth_metadata_missing` - There was no auth metadata or it was missing information
+    about the agent ID.'
+  example: verified
+  flat_name: event.agent_id_status
+  ignore_above: 1024
+  level: extended
+  name: agent_id_status
+  normalize: []
+  short: Validation status of the event's agent.id field.
+  type: keyword
 event.category:
   allowed_values:
   - description: Events in this category are related to the challenge and response

diff --git a/experimental/generated/ecs/ecs_nested.yml b/experimental/generated/ecs/ecs_nested.yml
@@ -2985,6 +2985,41 @@ event:
       normalize: []
       short: The action captured by the event.
       type: keyword
+    event.agent_id_status:
+      dashed_name: event-agent-id-status
+      description: 'Agents are normally responsible for populating the `agent.id`
+        field value. If the system receiving events is capable of validating the value
+        based on authentication information for the client then this field can be
+        used to reflect the outcome of that validation.
+
+        For example if the agent''s connection is authenticated with mTLS and the
+        client cert contains the ID of the agent to which the cert was issued then
+        the `agent.id` value in events can be checked against the certificate. If
+        the values match then `event.agent_id_status: verified` is added to the event,
+        otherwise one of the other allowed values should be used.
+
+        If no validation is performed then the field should be omitted.
+
+        The allowed values are:
+
+        `verified` - The `agent.id` field value matches expected value obtained from
+        auth metadata.
+
+        `mismatch` - The `agent.id` field value does not match the expected value
+        obtained from auth metadata.
+
+        `missing` - There was no `agent.id` field in the event to validate.
+
+        `auth_metadata_missing` - There was no auth metadata or it was missing information
+        about the agent ID.'
+      example: verified
+      flat_name: event.agent_id_status
+      ignore_above: 1024
+      level: extended
+      name: agent_id_status
+      normalize: []
+      short: Validation status of the event's agent.id field.
+      type: keyword
     event.category:
       allowed_values:
       - description: Events in this category are related to the challenge and response

diff --git a/experimental/generated/elasticsearch/7/template.json b/experimental/generated/elasticsearch/7/template.json
@@ -858,6 +858,10 @@
             "ignore_above": 1024,
             "type": "keyword"
           },
+          "agent_id_status": {
+            "ignore_above": 1024,
+            "type": "keyword"
+          },
           "category": {
             "ignore_above": 1024,
             "type": "keyword"

diff --git a/experimental/generated/elasticsearch/component/event.json b/experimental/generated/elasticsearch/component/event.json
@@ -12,6 +12,10 @@
               "ignore_above": 1024,
               "type": "keyword"
             },
+            "agent_id_status": {
+              "ignore_above": 1024,
+              "type": "keyword"
+            },
             "category": {
               "ignore_above": 1024,
               "type": "keyword"

diff --git a/generated/beats/fields.ecs.yml b/generated/beats/fields.ecs.yml
@@ -1581,6 +1581,37 @@
         Examples are `group-add`, `process-started`, `file-created`. The value is
         normally defined by the implementer.'
       example: user-password-change
+    - name: agent_id_status
+      level: extended
+      type: keyword
+      ignore_above: 1024
+      description: 'Agents are normally responsible for populating the `agent.id`
+        field value. If the system receiving events is capable of validating the value
+        based on authentication information for the client then this field can be
+        used to reflect the outcome of that validation.
+
+        For example if the agent''s connection is authenticated with mTLS and the
+        client cert contains the ID of the agent to which the cert was issued then
+        the `agent.id` value in events can be checked against the certificate. If
+        the values match then `event.agent_id_status: verified` is added to the event,
+        otherwise one of the other allowed values should be used.
+
+        If no validation is performed then the field should be omitted.
+
+        The allowed values are:
+
+        `verified` - The `agent.id` field value matches expected value obtained from
+        auth metadata.
+
+        `mismatch` - The `agent.id` field value does not match the expected value
+        obtained from auth metadata.
+
+        `missing` - There was no `agent.id` field in the event to validate.
+
+        `auth_metadata_missing` - There was no auth metadata or it was missing information
+        about the agent ID.'
+      example: verified
+      default_field: false
     - name: category
       level: core
       type: keyword

diff --git a/generated/csv/fields.csv b/generated/csv/fields.csv
@@ -152,6 +152,7 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description
 1.11.0-dev,false,error,error.stack_trace.text,text,extended,,,The stack trace of this error in plain text.
 1.11.0-dev,true,error,error.type,keyword,extended,,java.lang.NullPointerException,"The type of the error, for example the class name of the exception."
 1.11.0-dev,true,event,event.action,keyword,core,,user-password-change,The action captured by the event.
+1.11.0-dev,true,event,event.agent_id_status,keyword,extended,,verified,Validation status of the event's agent.id field.
 1.11.0-dev,true,event,event.category,keyword,core,array,authentication,Event category. The second categorization field in the hierarchy.
 1.11.0-dev,true,event,event.code,keyword,extended,,4648,Identification code for this event.
 1.11.0-dev,true,event,event.created,date,core,,2016-05-23T08:05:34.857Z,Time when the event was first read by an agent or by your pipeline.

diff --git a/generated/ecs/ecs_flat.yml b/generated/ecs/ecs_flat.yml
@@ -1865,6 +1865,41 @@ event.action:
   normalize: []
   short: The action captured by the event.
   type: keyword
+event.agent_id_status:
+  dashed_name: event-agent-id-status
+  description: 'Agents are normally responsible for populating the `agent.id` field
+    value. If the system receiving events is capable of validating the value based
+    on authentication information for the client then this field can be used to reflect
+    the outcome of that validation.
+
+    For example if the agent''s connection is authenticated with mTLS and the client
+    cert contains the ID of the agent to which the cert was issued then the `agent.id`
+    value in events can be checked against the certificate. If the values match then
+    `event.agent_id_status: verified` is added to the event, otherwise one of the
+    other allowed values should be used.
+
+    If no validation is performed then the field should be omitted.
+
+    The allowed values are:
+
+    `verified` - The `agent.id` field value matches expected value obtained from auth
+    metadata.
+
+    `mismatch` - The `agent.id` field value does not match the expected value obtained
+    from auth metadata.
+
+    `missing` - There was no `agent.id` field in the event to validate.
+
+    `auth_metadata_missing` - There was no auth metadata or it was missing information
+    about the agent ID.'
+  example: verified
+  flat_name: event.agent_id_status
+  ignore_above: 1024
+  level: extended
+  name: agent_id_status
+  normalize: []
+  short: Validation status of the event's agent.id field.
+  type: keyword
 event.category:
   allowed_values:
   - description: Events in this category are related to the challenge and response

diff --git a/generated/ecs/ecs_nested.yml b/generated/ecs/ecs_nested.yml
@@ -2633,6 +2633,41 @@ event:
       normalize: []
       short: The action captured by the event.
       type: keyword
+    event.agent_id_status:
+      dashed_name: event-agent-id-status
+      description: 'Agents are normally responsible for populating the `agent.id`
+        field value. If the system receiving events is capable of validating the value
+        based on authentication information for the client then this field can be
+        used to reflect the outcome of that validation.
+
+        For example if the agent''s connection is authenticated with mTLS and the
+        client cert contains the ID of the agent to which the cert was issued then
+        the `agent.id` value in events can be checked against the certificate. If
+        the values match then `event.agent_id_status: verified` is added to the event,
+        otherwise one of the other allowed values should be used.
+
+        If no validation is performed then the field should be omitted.
+
+        The allowed values are:
+
+        `verified` - The `agent.id` field value matches expected value obtained from
+        auth metadata.
+
+        `mismatch` - The `agent.id` field value does not match the expected value
+        obtained from auth metadata.
+
+        `missing` - There was no `agent.id` field in the event to validate.
+
+        `auth_metadata_missing` - There was no auth metadata or it was missing information
+        about the agent ID.'
+      example: verified
+      flat_name: event.agent_id_status
+      ignore_above: 1024
+      level: extended
+      name: agent_id_status
+      normalize: []
+      short: Validation status of the event's agent.id field.
+      type: keyword
     event.category:
       allowed_values:
       - description: Events in this category are related to the challenge and response

diff --git a/generated/elasticsearch/6/template.json b/generated/elasticsearch/6/template.json
@@ -748,6 +748,10 @@
               "ignore_above": 1024,
               "type": "keyword"
             },
+            "agent_id_status": {
+              "ignore_above": 1024,
+              "type": "keyword"
+            },
             "category": {
               "ignore_above": 1024,
               "type": "keyword"

diff --git a/generated/elasticsearch/7/template.json b/generated/elasticsearch/7/template.json
@@ -744,6 +744,10 @@
             "ignore_above": 1024,
             "type": "keyword"
           },
+          "agent_id_status": {
+            "ignore_above": 1024,
+            "type": "keyword"
+          },
           "category": {
             "ignore_above": 1024,
             "type": "keyword"

diff --git a/generated/elasticsearch/component/event.json b/generated/elasticsearch/component/event.json
@@ -12,6 +12,10 @@
               "ignore_above": 1024,
               "type": "keyword"
             },
+            "agent_id_status": {
+              "ignore_above": 1024,
+              "type": "keyword"
+            },
             "category": {
               "ignore_above": 1024,
               "type": "keyword"