diff --git a/docs/additional-info/artifacts.asciidoc b/docs/additional-info/artifacts.asciidoc deleted file mode 100644 index 77df53de17..0000000000 --- a/docs/additional-info/artifacts.asciidoc +++ /dev/null @@ -1,6 +0,0 @@ -[[ecs-artifacts]] -=== Generated Artifacts - -ECS maintains a collection of artifacts which are generated based on the schema. Examples include Elasticsearch index templates, CSV, and Beats field mappings. The maintained artifacts can be found in the {ecs_github_repo_link}/generated#artifacts-generated-from-ecs[ECS Github repo]. - -Users can generate custom versions of these artifacts using the ECS project's tooling. See the tooling {ecs_github_repo_link}/USAGE.md[usage documentation] for more detail. diff --git a/docs/additional-info/contributing.asciidoc b/docs/additional-info/contributing.asciidoc deleted file mode 100644 index 998c8d4c4d..0000000000 --- a/docs/additional-info/contributing.asciidoc +++ /dev/null @@ -1,7 +0,0 @@ -[[ecs-contributing]] -=== Contributing to {ecs} - -All information related to ECS is versioned in the https://github.com/elastic/ecs[elastic/ecs] repository. All -changes to ECS happen through Pull Requests submitted through Git. - -See the https://github.com/elastic/ecs/blob/master/CONTRIBUTING.md[Contribution Guidelines]. diff --git a/docs/additional-info/faq.asciidoc b/docs/additional-info/faq.asciidoc deleted file mode 100644 index 5b66a40942..0000000000 --- a/docs/additional-info/faq.asciidoc +++ /dev/null @@ -1,115 +0,0 @@ -[[ecs-faq]] -=== Questions and Answers - -[float] -[[ecs-benefits]] -==== What are the benefits of using ECS? - -The benefits to a user adopting these fields and names in their clusters are: - -* **Data correlation.** Ability to easily correlate data from the same or different sources, including: -** data from metrics, logs, and application performance management (apm) tools -** data from the same machines/hosts -** data from the same service -* **Ease of recall.** Improved ability to remember commonly used field names (because there is a single set, not a set per data source) -* **Ease of deduction.** Improved ability to deduce field names (because the field naming follows a small number of rules with few exceptions) -* **Reuse.** Ability to re-use analysis content (searches, visualizations, dashboards, alerts, reports, and machine learning jobs) across multiple data sources -* **Future proofing.** Ability to use any future Elastic-provided analysis content in your environment without modifications - -[float] -[[conflict]] -==== What if I have fields that conflict with ECS? - -The -{ref}/rename-processor.html[rename -processor] can help you resolve field conflicts. For example, imagine that you -already have a field called "user," but ECS employs `user` as an object. You can -use the rename processor on ingest time to rename your field to the matching ECS -field. If your field does not match ECS, you can rename your field to -`user.value` instead. - -[float] -[[addl-fields]] -==== What if my events have additional fields? - -Events may contain fields in addition to ECS fields. These fields can follow the -ECS naming and writing rules, but this is not a requirement. - -[float] -[[dot-notation]] -==== Why does ECS use a dot notation instead of an underline notation? - -There are two common key formats for ingesting data into Elasticsearch: - -* Dot notation: `user.firstname: Nicolas`, `user.lastname: Ruflin` -* Underline notation: `user_firstname: Nicolas`, `user_lastname: Ruflin` - -ECS uses the dot notation to represent nested objects. - -[float] -[[notation-diff]] -===== What is the difference between the two notations? - -Ingesting `user.firstname: Nicolas` and `user.lastname: Ruflin` is identical to ingesting the following JSON: - -[source,json] - "user": { - "firstname": "Nicolas", - "lastname": "Ruflin" - } - -In Elasticsearch, `user` is represented as an {ref}/object.html[object -datatype]. In the case of the underline notation, both are just -{ref}/mapping-types.html[string datatypes]. - -[float] -[[dot-adv]] -===== Advantages of dot notation - -With dot notation, each prefix in Elasticsearch is an object. Each object can have -{ref}/object.html#object-params[parameters] -that control how fields inside the object are treated. In the context of ECS, -for example, these parameters would allow you to disable dynamic property -creation for certain prefixes. - -Individual objects give you more flexibility on both the ingest and the event -sides. In Elasticsearch, for example, you can use the remove processor to drop -complete objects instead of selecting each key inside. You don't have to know -ahead of time which keys will be in an object. - -In Beats, you can simplify the creation of events. For example, you can treat -each object as an object (or struct in Golang), which makes constructing and -modifying each part of the final event easier. - -[float] -[[dot-disadv]] -===== Disadvantage of dot notation - -In Elasticsearch, each key can have only one type. For example, if `user` is an -`object`, you can't use it as a `keyword` type in the same index, like `{"user": -"nicolas ruflin"}`. This restriction can be an issue in certain datasets. For -the ECS data itself, this is not an issue because all fields are predefined. - -[float] -[[underline]] -===== What if I already use the underline notation? - -As long as there are no conflicts, underline notation and ECS dot notation can -coexist in the same document. - -[float] -[[type-interop]] -==== What if I want to use a different data type from the same field type family? - -In Elasticsearch, field types are grouped by family. Types in the same family support -the same search functionality but may have different space usage or performance -characteristics. For example, both `keyword` and `wildcard` types are members of the -`keyword` family, and `text` and `match_only_text` are members of the `text` family. - -The field types defined in ECS provide the best default experience for most users. -However, a different type from the same family can replace the default defined in ECS -if required for a specific use cases. Users should understand any potential performance -or storage differences before changing from a default field type. - -The Elasticsearch {ref}/mapping-types.html[mapping types] section has more information about type -families. diff --git a/docs/additional-info/glossary.asciidoc b/docs/additional-info/glossary.asciidoc deleted file mode 100644 index a1e143f981..0000000000 --- a/docs/additional-info/glossary.asciidoc +++ /dev/null @@ -1,10 +0,0 @@ -[[ecs-glossary]] -=== Glossary of {ecs} Terms - -[[ecs-glossary-ecs]] -ECS:: -*Elastic Common Schema*. The Elastic Common Schema (ECS) is a document schema -for Elasticsearch, for use cases such as logging and metrics. -ECS defines a common set of fields, their datatype, -and gives guidance on their correct usage. -ECS is used to improve uniformity of event data coming from different sources. diff --git a/docs/additional-info/index.asciidoc b/docs/additional-info/index.asciidoc deleted file mode 100644 index f6796174d8..0000000000 --- a/docs/additional-info/index.asciidoc +++ /dev/null @@ -1,13 +0,0 @@ -[[ecs-additional-information]] -== Additional Information - -* <> -* <> -* <> -* <> - -// include::use-cases.asciidoc[] -include::faq.asciidoc[] -include::glossary.asciidoc[] -include::contributing.asciidoc[] -include::artifacts.asciidoc[] diff --git a/docs/docset.yml b/docs/docset.yml new file mode 100644 index 0000000000..cd15a88dff --- /dev/null +++ b/docs/docset.yml @@ -0,0 +1,490 @@ +project: 'ECS reference' +exclude: + - fields/usage/README.md +cross_links: + - beats + - docs-content + - elasticsearch + - logstash +toc: + - toc: reference +subs: + ref: "https://www.elastic.co/guide/en/elasticsearch/reference/current" + ref-bare: "https://www.elastic.co/guide/en/elasticsearch/reference" + ref-8x: "https://www.elastic.co/guide/en/elasticsearch/reference/8.1" + ref-80: "https://www.elastic.co/guide/en/elasticsearch/reference/8.0" + ref-7x: "https://www.elastic.co/guide/en/elasticsearch/reference/7.17" + ref-70: "https://www.elastic.co/guide/en/elasticsearch/reference/7.0" + ref-60: "https://www.elastic.co/guide/en/elasticsearch/reference/6.0" + ref-64: "https://www.elastic.co/guide/en/elasticsearch/reference/6.4" + xpack-ref: "https://www.elastic.co/guide/en/x-pack/6.2" + logstash-ref: "https://www.elastic.co/guide/en/logstash/current" + kibana-ref: "https://www.elastic.co/guide/en/kibana/current" + kibana-ref-all: "https://www.elastic.co/guide/en/kibana" + beats-ref-root: "https://www.elastic.co/guide/en/beats" + beats-ref: "https://www.elastic.co/guide/en/beats/libbeat/current" + beats-ref-60: "https://www.elastic.co/guide/en/beats/libbeat/6.0" + beats-ref-63: "https://www.elastic.co/guide/en/beats/libbeat/6.3" + beats-devguide: "https://www.elastic.co/guide/en/beats/devguide/current" + auditbeat-ref: "https://www.elastic.co/guide/en/beats/auditbeat/current" + packetbeat-ref: "https://www.elastic.co/guide/en/beats/packetbeat/current" + metricbeat-ref: "https://www.elastic.co/guide/en/beats/metricbeat/current" + filebeat-ref: "https://www.elastic.co/guide/en/beats/filebeat/current" + functionbeat-ref: "https://www.elastic.co/guide/en/beats/functionbeat/current" + winlogbeat-ref: "https://www.elastic.co/guide/en/beats/winlogbeat/current" + heartbeat-ref: "https://www.elastic.co/guide/en/beats/heartbeat/current" + journalbeat-ref: "https://www.elastic.co/guide/en/beats/journalbeat/current" + ingest-guide: "https://www.elastic.co/guide/en/ingest/current" + fleet-guide: "https://www.elastic.co/guide/en/fleet/current" + apm-guide-ref: "https://www.elastic.co/guide/en/apm/guide/current" + apm-guide-7x: "https://www.elastic.co/guide/en/apm/guide/7.17" + apm-app-ref: "https://www.elastic.co/guide/en/kibana/current" + apm-agents-ref: "https://www.elastic.co/guide/en/apm/agent" + apm-android-ref: "https://www.elastic.co/guide/en/apm/agent/android/current" + apm-py-ref: "https://www.elastic.co/guide/en/apm/agent/python/current" + apm-py-ref-3x: "https://www.elastic.co/guide/en/apm/agent/python/3.x" + apm-node-ref-index: "https://www.elastic.co/guide/en/apm/agent/nodejs" + apm-node-ref: "https://www.elastic.co/guide/en/apm/agent/nodejs/current" + apm-node-ref-1x: "https://www.elastic.co/guide/en/apm/agent/nodejs/1.x" + apm-rum-ref: "https://www.elastic.co/guide/en/apm/agent/rum-js/current" + apm-ruby-ref: "https://www.elastic.co/guide/en/apm/agent/ruby/current" + apm-java-ref: "https://www.elastic.co/guide/en/apm/agent/java/current" + apm-go-ref: "https://www.elastic.co/guide/en/apm/agent/go/current" + apm-dotnet-ref: "https://www.elastic.co/guide/en/apm/agent/dotnet/current" + apm-php-ref: "https://www.elastic.co/guide/en/apm/agent/php/current" + apm-ios-ref: "https://www.elastic.co/guide/en/apm/agent/swift/current" + apm-lambda-ref: "https://www.elastic.co/guide/en/apm/lambda/current" + apm-attacher-ref: "https://www.elastic.co/guide/en/apm/attacher/current" + docker-logging-ref: "https://www.elastic.co/guide/en/beats/loggingplugin/current" + esf-ref: "https://www.elastic.co/guide/en/esf/current" + kinesis-firehose-ref: "https://www.elastic.co/guide/en/kinesis/{{kinesis_version}}" + estc-welcome-current: "https://www.elastic.co/guide/en/starting-with-the-elasticsearch-platform-and-its-solutions/current" + estc-welcome: "https://www.elastic.co/guide/en/starting-with-the-elasticsearch-platform-and-its-solutions/current" + estc-welcome-all: "https://www.elastic.co/guide/en/starting-with-the-elasticsearch-platform-and-its-solutions" + hadoop-ref: "https://www.elastic.co/guide/en/elasticsearch/hadoop/current" + stack-ref: "https://www.elastic.co/guide/en/elastic-stack/current" + stack-ref-67: "https://www.elastic.co/guide/en/elastic-stack/6.7" + stack-ref-68: "https://www.elastic.co/guide/en/elastic-stack/6.8" + stack-ref-70: "https://www.elastic.co/guide/en/elastic-stack/7.0" + stack-ref-80: "https://www.elastic.co/guide/en/elastic-stack/8.0" + stack-ov: "https://www.elastic.co/guide/en/elastic-stack-overview/current" + stack-gs: "https://www.elastic.co/guide/en/elastic-stack-get-started/current" + stack-gs-current: "https://www.elastic.co/guide/en/elastic-stack-get-started/current" + javaclient: "https://www.elastic.co/guide/en/elasticsearch/client/java-api/current" + java-api-client: "https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/current" + java-rest: "https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current" + jsclient: "https://www.elastic.co/guide/en/elasticsearch/client/javascript-api/current" + jsclient-current: "https://www.elastic.co/guide/en/elasticsearch/client/javascript-api/current" + es-ruby-client: "https://www.elastic.co/guide/en/elasticsearch/client/ruby-api/current" + es-dotnet-client: "https://www.elastic.co/guide/en/elasticsearch/client/net-api/current" + es-php-client: "https://www.elastic.co/guide/en/elasticsearch/client/php-api/current" + es-python-client: "https://www.elastic.co/guide/en/elasticsearch/client/python-api/current" + defguide: "https://www.elastic.co/guide/en/elasticsearch/guide/2.x" + painless: "https://www.elastic.co/guide/en/elasticsearch/painless/current" + plugins: "https://www.elastic.co/guide/en/elasticsearch/plugins/current" + plugins-8x: "https://www.elastic.co/guide/en/elasticsearch/plugins/8.1" + plugins-7x: "https://www.elastic.co/guide/en/elasticsearch/plugins/7.17" + plugins-6x: "https://www.elastic.co/guide/en/elasticsearch/plugins/6.8" + glossary: "https://www.elastic.co/guide/en/elastic-stack-glossary/current" + upgrade_guide: "https://www.elastic.co/products/upgrade_guide" + blog-ref: "https://www.elastic.co/blog/" + curator-ref: "https://www.elastic.co/guide/en/elasticsearch/client/curator/current" + curator-ref-current: "https://www.elastic.co/guide/en/elasticsearch/client/curator/current" + metrics-ref: "https://www.elastic.co/guide/en/metrics/current" + metrics-guide: "https://www.elastic.co/guide/en/metrics/guide/current" + logs-ref: "https://www.elastic.co/guide/en/logs/current" + logs-guide: "https://www.elastic.co/guide/en/logs/guide/current" + uptime-guide: "https://www.elastic.co/guide/en/uptime/current" + observability-guide: "https://www.elastic.co/guide/en/observability/current" + observability-guide-all: "https://www.elastic.co/guide/en/observability" + siem-guide: "https://www.elastic.co/guide/en/siem/guide/current" + security-guide: "https://www.elastic.co/guide/en/security/current" + security-guide-all: "https://www.elastic.co/guide/en/security" + endpoint-guide: "https://www.elastic.co/guide/en/endpoint/current" + sql-odbc: "https://www.elastic.co/guide/en/elasticsearch/sql-odbc/current" + ecs-ref: "https://www.elastic.co/guide/en/ecs/current" + ecs-logging-ref: "https://www.elastic.co/guide/en/ecs-logging/overview/current" + ecs-logging-go-logrus-ref: "https://www.elastic.co/guide/en/ecs-logging/go-logrus/current" + ecs-logging-go-zap-ref: "https://www.elastic.co/guide/en/ecs-logging/go-zap/current" + ecs-logging-go-zerolog-ref: "https://www.elastic.co/guide/en/ecs-logging/go-zap/current" + ecs-logging-java-ref: "https://www.elastic.co/guide/en/ecs-logging/java/current" + ecs-logging-dotnet-ref: "https://www.elastic.co/guide/en/ecs-logging/dotnet/current" + ecs-logging-nodejs-ref: "https://www.elastic.co/guide/en/ecs-logging/nodejs/current" + ecs-logging-php-ref: "https://www.elastic.co/guide/en/ecs-logging/php/current" + ecs-logging-python-ref: "https://www.elastic.co/guide/en/ecs-logging/python/current" + ecs-logging-ruby-ref: "https://www.elastic.co/guide/en/ecs-logging/ruby/current" + ml-docs: "https://www.elastic.co/guide/en/machine-learning/current" + eland-docs: "https://www.elastic.co/guide/en/elasticsearch/client/eland/current" + eql-ref: "https://eql.readthedocs.io/en/latest/query-guide" + extendtrial: "https://www.elastic.co/trialextension" + wikipedia: "https://en.wikipedia.org/wiki" + forum: "https://discuss.elastic.co/" + xpack-forum: "https://discuss.elastic.co/c/50-x-pack" + security-forum: "https://discuss.elastic.co/c/x-pack/shield" + watcher-forum: "https://discuss.elastic.co/c/x-pack/watcher" + monitoring-forum: "https://discuss.elastic.co/c/x-pack/marvel" + graph-forum: "https://discuss.elastic.co/c/x-pack/graph" + apm-forum: "https://discuss.elastic.co/c/apm" + enterprise-search-ref: "https://www.elastic.co/guide/en/enterprise-search/current" + app-search-ref: "https://www.elastic.co/guide/en/app-search/current" + workplace-search-ref: "https://www.elastic.co/guide/en/workplace-search/current" + enterprise-search-node-ref: "https://www.elastic.co/guide/en/enterprise-search-clients/enterprise-search-node/current" + enterprise-search-php-ref: "https://www.elastic.co/guide/en/enterprise-search-clients/php/current" + enterprise-search-python-ref: "https://www.elastic.co/guide/en/enterprise-search-clients/python/current" + enterprise-search-ruby-ref: "https://www.elastic.co/guide/en/enterprise-search-clients/ruby/current" + elastic-maps-service: "https://maps.elastic.co" + integrations-docs: "https://docs.elastic.co/en/integrations" + integrations-devguide: "https://www.elastic.co/guide/en/integrations-developer/current" + time-units: "https://www.elastic.co/guide/en/elasticsearch/reference/current/api-conventions.html#time-units" + byte-units: "https://www.elastic.co/guide/en/elasticsearch/reference/current/api-conventions.html#byte-units" + apm-py-ref-v: "https://www.elastic.co/guide/en/apm/agent/python/current" + apm-node-ref-v: "https://www.elastic.co/guide/en/apm/agent/nodejs/current" + apm-rum-ref-v: "https://www.elastic.co/guide/en/apm/agent/rum-js/current" + apm-ruby-ref-v: "https://www.elastic.co/guide/en/apm/agent/ruby/current" + apm-java-ref-v: "https://www.elastic.co/guide/en/apm/agent/java/current" + apm-go-ref-v: "https://www.elastic.co/guide/en/apm/agent/go/current" + apm-ios-ref-v: "https://www.elastic.co/guide/en/apm/agent/swift/current" + apm-dotnet-ref-v: "https://www.elastic.co/guide/en/apm/agent/dotnet/current" + apm-php-ref-v: "https://www.elastic.co/guide/en/apm/agent/php/current" + ecloud: "Elastic Cloud" + esf: "Elastic Serverless Forwarder" + ess: "Elasticsearch Service" + ece: "Elastic Cloud Enterprise" + eck: "Elastic Cloud on Kubernetes" + serverless-full: "Elastic Cloud Serverless" + serverless-short: "Serverless" + es-serverless: "Elasticsearch Serverless" + es3: "Elasticsearch Serverless" + obs-serverless: "Elastic Observability Serverless" + sec-serverless: "Elastic Security Serverless" + serverless-docs: "https://docs.elastic.co/serverless" + cloud: "https://www.elastic.co/guide/en/cloud/current" + ess-utm-params: "?page=docs&placement=docs-body" + ess-baymax: "?page=docs&placement=docs-body" + ess-trial: "https://cloud.elastic.co/registration?page=docs&placement=docs-body" + ess-product: "https://www.elastic.co/cloud/elasticsearch-service?page=docs&placement=docs-body" + ess-console: "https://cloud.elastic.co?page=docs&placement=docs-body" + ess-console-name: "Elasticsearch Service Console" + ess-deployments: "https://cloud.elastic.co/deployments?page=docs&placement=docs-body" + ece-ref: "https://www.elastic.co/guide/en/cloud-enterprise/current" + eck-ref: "https://www.elastic.co/guide/en/cloud-on-k8s/current" + ess-leadin: "You can run Elasticsearch on your own hardware or use our hosted Elasticsearch Service that is available on AWS, GCP, and Azure. https://cloud.elastic.co/registration{ess-utm-params}[Try the Elasticsearch Service for free]." + ess-leadin-short: "Our hosted Elasticsearch Service is available on AWS, GCP, and Azure, and you can https://cloud.elastic.co/registration{ess-utm-params}[try it for free]." + ess-icon: "image:https://doc-icons.s3.us-east-2.amazonaws.com/logo_cloud.svg[link=\"https://cloud.elastic.co/registration{ess-utm-params}\", title=\"Supported on Elasticsearch Service\"]" + ece-icon: "image:https://doc-icons.s3.us-east-2.amazonaws.com/logo_cloud_ece.svg[link=\"https://cloud.elastic.co/registration{ess-utm-params}\", title=\"Supported on Elastic Cloud Enterprise\"]" + cloud-only: "This feature is designed for indirect use by https://cloud.elastic.co/registration{ess-utm-params}[Elasticsearch Service], https://www.elastic.co/guide/en/cloud-enterprise/{ece-version-link}[Elastic Cloud Enterprise], and https://www.elastic.co/guide/en/cloud-on-k8s/current[Elastic Cloud on Kubernetes]. Direct use is not supported." + ess-setting-change: "image:https://doc-icons.s3.us-east-2.amazonaws.com/logo_cloud.svg[link=\"{ess-trial}\", title=\"Supported on {ess}\"] indicates a change to a supported https://www.elastic.co/guide/en/cloud/current/ec-add-user-settings.html[user setting] for Elasticsearch Service." + ess-skip-section: "If you use Elasticsearch Service, skip this section. Elasticsearch Service handles these changes for you." + api-cloud: "https://www.elastic.co/docs/api/doc/cloud" + api-ece: "https://www.elastic.co/docs/api/doc/cloud-enterprise" + api-kibana-serverless: "https://www.elastic.co/docs/api/doc/serverless" + es-feature-flag: "This feature is in development and not yet available for use. This documentation is provided for informational purposes only." + es-ref-dir: "'{{elasticsearch-root}}/docs/reference'" + apm-app: "APM app" + uptime-app: "Uptime app" + synthetics-app: "Synthetics app" + logs-app: "Logs app" + metrics-app: "Metrics app" + infrastructure-app: "Infrastructure app" + siem-app: "SIEM app" + security-app: "Elastic Security app" + ml-app: "Machine Learning" + dev-tools-app: "Dev Tools" + ingest-manager-app: "Ingest Manager" + stack-manage-app: "Stack Management" + stack-monitor-app: "Stack Monitoring" + alerts-ui: "Alerts and Actions" + rules-ui: "Rules" + rac-ui: "Rules and Connectors" + connectors-ui: "Connectors" + connectors-feature: "Actions and Connectors" + stack-rules-feature: "Stack Rules" + user-experience: "User Experience" + ems: "Elastic Maps Service" + ems-init: "EMS" + hosted-ems: "Elastic Maps Server" + ipm-app: "Index Pattern Management" + ingest-pipelines: "ingest pipelines" + ingest-pipelines-app: "Ingest Pipelines" + ingest-pipelines-cap: "Ingest pipelines" + ls-pipelines: "Logstash pipelines" + ls-pipelines-app: "Logstash Pipelines" + maint-windows: "maintenance windows" + maint-windows-app: "Maintenance Windows" + maint-windows-cap: "Maintenance windows" + custom-roles-app: "Custom Roles" + data-source: "data view" + data-sources: "data views" + data-source-caps: "Data View" + data-sources-caps: "Data Views" + data-source-cap: "Data view" + data-sources-cap: "Data views" + project-settings: "Project settings" + manage-app: "Management" + index-manage-app: "Index Management" + data-views-app: "Data Views" + rules-app: "Rules" + saved-objects-app: "Saved Objects" + tags-app: "Tags" + api-keys-app: "API keys" + transforms-app: "Transforms" + connectors-app: "Connectors" + files-app: "Files" + reports-app: "Reports" + maps-app: "Maps" + alerts-app: "Alerts" + crawler: "Enterprise Search web crawler" + ents: "Enterprise Search" + app-search-crawler: "App Search web crawler" + agent: "Elastic Agent" + agents: "Elastic Agents" + fleet: "Fleet" + fleet-server: "Fleet Server" + integrations-server: "Integrations Server" + ingest-manager: "Ingest Manager" + ingest-management: "ingest management" + package-manager: "Elastic Package Manager" + integrations: "Integrations" + package-registry: "Elastic Package Registry" + artifact-registry: "Elastic Artifact Registry" + aws: "AWS" + stack: "Elastic Stack" + xpack: "X-Pack" + es: "Elasticsearch" + kib: "Kibana" + esms: "Elastic Stack Monitoring Service" + esms-init: "ESMS" + ls: "Logstash" + beats: "Beats" + auditbeat: "Auditbeat" + filebeat: "Filebeat" + heartbeat: "Heartbeat" + metricbeat: "Metricbeat" + packetbeat: "Packetbeat" + winlogbeat: "Winlogbeat" + functionbeat: "Functionbeat" + journalbeat: "Journalbeat" + es-sql: "Elasticsearch SQL" + esql: "ES|QL" + elastic-agent: "Elastic Agent" + k8s: "Kubernetes" + log-driver-long: "Elastic Logging Plugin for Docker" + security: "X-Pack security" + security-features: "security features" + operator-feature: "operator privileges feature" + es-security-features: "Elasticsearch security features" + stack-security-features: "Elastic Stack security features" + endpoint-sec: "Endpoint Security" + endpoint-cloud-sec: "Endpoint and Cloud Security" + elastic-defend: "Elastic Defend" + elastic-sec: "Elastic Security" + elastic-endpoint: "Elastic Endpoint" + swimlane: "Swimlane" + sn: "ServiceNow" + sn-itsm: "ServiceNow ITSM" + sn-itom: "ServiceNow ITOM" + sn-sir: "ServiceNow SecOps" + jira: "Jira" + ibm-r: "IBM Resilient" + webhook: "Webhook" + webhook-cm: "Webhook - Case Management" + opsgenie: "Opsgenie" + bedrock: "Amazon Bedrock" + gemini: "Google Gemini" + hive: "TheHive" + monitoring: "X-Pack monitoring" + monitor-features: "monitoring features" + stack-monitor-features: "Elastic Stack monitoring features" + watcher: "Watcher" + alert-features: "alerting features" + reporting: "X-Pack reporting" + report-features: "reporting features" + graph: "X-Pack graph" + graph-features: "graph analytics features" + searchprofiler: "Search Profiler" + xpackml: "X-Pack machine learning" + ml: "machine learning" + ml-cap: "Machine learning" + ml-init: "ML" + ml-features: "machine learning features" + stack-ml-features: "Elastic Stack machine learning features" + ccr: "cross-cluster replication" + ccr-cap: "Cross-cluster replication" + ccr-init: "CCR" + ccs: "cross-cluster search" + ccs-cap: "Cross-cluster search" + ccs-init: "CCS" + ilm: "index lifecycle management" + ilm-cap: "Index lifecycle management" + ilm-init: "ILM" + dlm: "data lifecycle management" + dlm-cap: "Data lifecycle management" + dlm-init: "DLM" + search-snap: "searchable snapshot" + search-snaps: "searchable snapshots" + search-snaps-cap: "Searchable snapshots" + slm: "snapshot lifecycle management" + slm-cap: "Snapshot lifecycle management" + slm-init: "SLM" + rollup-features: "data rollup features" + ipm: "index pattern management" + ipm-cap: "Index pattern" + rollup: "rollup" + rollup-cap: "Rollup" + rollups: "rollups" + rollups-cap: "Rollups" + rollup-job: "rollup job" + rollup-jobs: "rollup jobs" + rollup-jobs-cap: "Rollup jobs" + dfeed: "datafeed" + dfeeds: "datafeeds" + dfeed-cap: "Datafeed" + dfeeds-cap: "Datafeeds" + ml-jobs: "machine learning jobs" + ml-jobs-cap: "Machine learning jobs" + anomaly-detect: "anomaly detection" + anomaly-detect-cap: "Anomaly detection" + anomaly-job: "anomaly detection job" + anomaly-jobs: "anomaly detection jobs" + anomaly-jobs-cap: "Anomaly detection jobs" + dataframe: "data frame" + dataframes: "data frames" + dataframe-cap: "Data frame" + dataframes-cap: "Data frames" + watcher-transform: "payload transform" + watcher-transforms: "payload transforms" + watcher-transform-cap: "Payload transform" + watcher-transforms-cap: "Payload transforms" + transform: "transform" + transforms: "transforms" + transform-cap: "Transform" + transforms-cap: "Transforms" + dataframe-transform: "transform" + dataframe-transform-cap: "Transform" + dataframe-transforms: "transforms" + dataframe-transforms-cap: "Transforms" + dfanalytics-cap: "Data frame analytics" + dfanalytics: "data frame analytics" + dataframe-analytics-config: "'{dataframe} analytics config'" + dfanalytics-job: "'{dataframe} analytics job'" + dfanalytics-jobs: "'{dataframe} analytics jobs'" + dfanalytics-jobs-cap: "'{dataframe-cap} analytics jobs'" + cdataframe: "continuous data frame" + cdataframes: "continuous data frames" + cdataframe-cap: "Continuous data frame" + cdataframes-cap: "Continuous data frames" + cdataframe-transform: "continuous transform" + cdataframe-transforms: "continuous transforms" + cdataframe-transforms-cap: "Continuous transforms" + ctransform: "continuous transform" + ctransform-cap: "Continuous transform" + ctransforms: "continuous transforms" + ctransforms-cap: "Continuous transforms" + oldetection: "outlier detection" + oldetection-cap: "Outlier detection" + olscore: "outlier score" + olscores: "outlier scores" + fiscore: "feature influence score" + evaluatedf-api: "evaluate {dataframe} analytics API" + evaluatedf-api-cap: "Evaluate {dataframe} analytics API" + binarysc: "binary soft classification" + binarysc-cap: "Binary soft classification" + regression: "regression" + regression-cap: "Regression" + reganalysis: "regression analysis" + reganalysis-cap: "Regression analysis" + depvar: "dependent variable" + feature-var: "feature variable" + feature-vars: "feature variables" + feature-vars-cap: "Feature variables" + classification: "classification" + classification-cap: "Classification" + classanalysis: "classification analysis" + classanalysis-cap: "Classification analysis" + infer-cap: "Inference" + infer: "inference" + lang-ident-cap: "Language identification" + lang-ident: "language identification" + data-viz: "Data Visualizer" + file-data-viz: "File Data Visualizer" + feat-imp: "feature importance" + feat-imp-cap: "Feature importance" + nlp: "natural language processing" + nlp-cap: "Natural language processing" + apm-agent: "APM agent" + apm-go-agent: "Elastic APM Go agent" + apm-go-agents: "Elastic APM Go agents" + apm-ios-agent: "Elastic APM iOS agent" + apm-ios-agents: "Elastic APM iOS agents" + apm-java-agent: "Elastic APM Java agent" + apm-java-agents: "Elastic APM Java agents" + apm-dotnet-agent: "Elastic APM .NET agent" + apm-dotnet-agents: "Elastic APM .NET agents" + apm-node-agent: "Elastic APM Node.js agent" + apm-node-agents: "Elastic APM Node.js agents" + apm-php-agent: "Elastic APM PHP agent" + apm-php-agents: "Elastic APM PHP agents" + apm-py-agent: "Elastic APM Python agent" + apm-py-agents: "Elastic APM Python agents" + apm-ruby-agent: "Elastic APM Ruby agent" + apm-ruby-agents: "Elastic APM Ruby agents" + apm-rum-agent: "Elastic APM Real User Monitoring (RUM) JavaScript agent" + apm-rum-agents: "Elastic APM RUM JavaScript agents" + apm-lambda-ext: "Elastic APM AWS Lambda extension" + project-monitors: "project monitors" + project-monitors-cap: "Project monitors" + private-location: "Private Location" + private-locations: "Private Locations" + pwd: "YOUR_PASSWORD" + esh: "ES-Hadoop" + default-dist: "default distribution" + oss-dist: "OSS-only distribution" + observability: "Observability" + api-request-title: "Request" + api-prereq-title: "Prerequisites" + api-description-title: "Description" + api-path-parms-title: "Path parameters" + api-query-parms-title: "Query parameters" + api-request-body-title: "Request body" + api-response-codes-title: "Response codes" + api-response-body-title: "Response body" + api-example-title: "Example" + api-examples-title: "Examples" + api-definitions-title: "Properties" + multi-arg: "†footnoteref:[multi-arg,This parameter accepts multiple arguments.]" + multi-arg-ref: "†footnoteref:[multi-arg]" + yes-icon: "image:https://doc-icons.s3.us-east-2.amazonaws.com/icon-yes.png[Yes,20,15]" + no-icon: "image:https://doc-icons.s3.us-east-2.amazonaws.com/icon-no.png[No,20,15]" + es-repo: "https://github.com/elastic/elasticsearch/" + es-issue: "https://github.com/elastic/elasticsearch/issues/" + es-pull: "https://github.com/elastic/elasticsearch/pull/" + es-commit: "https://github.com/elastic/elasticsearch/commit/" + kib-repo: "https://github.com/elastic/kibana/" + kib-issue: "https://github.com/elastic/kibana/issues/" + kibana-issue: "'{kib-repo}issues/'" + kib-pull: "https://github.com/elastic/kibana/pull/" + kibana-pull: "'{kib-repo}pull/'" + kib-commit: "https://github.com/elastic/kibana/commit/" + ml-repo: "https://github.com/elastic/ml-cpp/" + ml-issue: "https://github.com/elastic/ml-cpp/issues/" + ml-pull: "https://github.com/elastic/ml-cpp/pull/" + ml-commit: "https://github.com/elastic/ml-cpp/commit/" + apm-repo: "https://github.com/elastic/apm-server/" + apm-issue: "https://github.com/elastic/apm-server/issues/" + apm-pull: "https://github.com/elastic/apm-server/pull/" + kibana-blob: "https://github.com/elastic/kibana/blob/current/" + apm-get-started-ref: "https://www.elastic.co/guide/en/apm/get-started/current" + apm-server-ref: "https://www.elastic.co/guide/en/apm/server/current" + apm-server-ref-v: "https://www.elastic.co/guide/en/apm/server/current" + apm-server-ref-m: "https://www.elastic.co/guide/en/apm/server/master" + apm-server-ref-62: "https://www.elastic.co/guide/en/apm/server/6.2" + apm-server-ref-64: "https://www.elastic.co/guide/en/apm/server/6.4" + apm-server-ref-70: "https://www.elastic.co/guide/en/apm/server/7.0" + apm-overview-ref-v: "https://www.elastic.co/guide/en/apm/get-started/current" + apm-overview-ref-70: "https://www.elastic.co/guide/en/apm/get-started/7.0" + apm-overview-ref-m: "https://www.elastic.co/guide/en/apm/get-started/master" + infra-guide: "https://www.elastic.co/guide/en/infrastructure/guide/current" + a-data-source: "a data view" + icon-bug: "pass:[]" + icon-checkInCircleFilled: "pass:[]" + icon-warningFilled: "pass:[]" diff --git a/docs/fields/field-details.asciidoc b/docs/fields/field-details.asciidoc deleted file mode 100644 index 1066096481..0000000000 --- a/docs/fields/field-details.asciidoc +++ /dev/null @@ -1,14392 +0,0 @@ -[[ecs-base]] -=== Base Fields - -The `base` field set contains all fields which are at the root of the events. These fields are common across all types of events. - -[discrete] -==== Base Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-timestamp]] -<> - -a| Date/time when the event originated. - -This is the date/time extracted from the event, typically representing when the event was generated by the source. - -If the event source has no original timestamp, this value is typically populated by the first time the event was received by the pipeline. - -Required field for all events. - -type: date - - - -example: `2016-05-23T08:05:34.853Z` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/OTLP-ffdcb2?style=flat[relation,title=otlp,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://github.com/search?q=repo%3Aopen-telemetry%2Fopentelemetry-proto+%22\+time_unix_nano+%22&type=code[time_unix_nano] - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/OTLP-ffdcb2?style=flat[relation,title=otlp,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://github.com/search?q=repo%3Aopen-telemetry%2Fopentelemetry-proto+%22\+observed_time_unix_nano+%22&type=code[observed_time_unix_nano] - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/OTLP-ffdcb2?style=flat[relation,title=otlp,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://github.com/search?q=repo%3Aopen-telemetry%2Fopentelemetry-proto+%22\+start_time_unix_nano+%22&type=code[start_time_unix_nano] - -Note: On logs, events and metrics, `time_unix_nano` defines the time of the event. For logs, `observed_time_unix_nano` defines the time when the event was observed by the collection system. On spans, `start_time_unix_nano` represents the start time of the span. - - -| core - -// =============================================================== - -| -[[field-labels]] -<> - -a| Custom key/value pairs. - -Can be used to add meta information to events. Should not contain nested objects. All values are stored as keyword. - -Example: `docker` and `k8s` labels. - -type: object - - - -example: `{"application": "foo-bar", "env": "production"}` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/related-efc20d?style=flat[relation,title=related,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/k8s/#k8s-pod-label[k8s.pod.label] - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/related-efc20d?style=flat[relation,title=related,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/container/#container-label[container.label] - -| core - -// =============================================================== - -| -[[field-message]] -<> - -a| For log events the message field contains the log message, optimized for viewing in a log viewer. - -For structured logs without an original message field, other fields can be concatenated to form a human-readable summary of the event. - -If multiple messages exist, they can be combined into one message. - -type: match_only_text - - - -example: `Hello World` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/OTLP-ffdcb2?style=flat[relation,title=otlp,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://github.com/search?q=repo%3Aopen-telemetry%2Fopentelemetry-proto+%22\+body+%22&type=code[body] - -Note: The `body` in OTLP is of type `Any` and can be either an unstructured log message or a structured event. - -| core - -// =============================================================== - -| -[[field-tags]] -<> - -a| List of keywords used to tag each event. - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `["production", "env2"]` - -| core - -// =============================================================== - -|===== - - -[[ecs-agent]] -=== Agent Fields - -The agent fields contain the data about the software entity, if any, that collects, detects, or observes events on a host, or takes measurements on a host. - -Examples include Beats. Agents may also run on observers. ECS agent.* fields shall be populated with details of the agent running on the host or observer where the event happened or the measurement was taken. - -[discrete] -==== Agent Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-agent-build-original]] -<> - -a| Extended build information for the agent. - -This field is intended to contain any build information that a data source may provide, no specific formatting is required. - -type: keyword - - - -example: `metricbeat version 7.6.0 (amd64), libbeat 7.6.0 [6a23e8f8f30f5001ba344e4e54d8d9cb82cb107c built 2020-02-05 23:10:10 +0000 UTC]` - -| core - -// =============================================================== - -| -[[field-agent-ephemeral-id]] -<> - -a| Ephemeral identifier of this agent (if one exists). - -This id normally changes across restarts, but `agent.id` does not. - -type: keyword - - - -example: `8a4f500f` - -| extended - -// =============================================================== - -| -[[field-agent-id]] -<> - -a| Unique identifier of this agent (if one exists). - -Example: For Beats this would be beat.id. - -type: keyword - - - -example: `8a4f500d` - -| core - -// =============================================================== - -| -[[field-agent-name]] -<> - -a| Custom name of the agent. - -This is a name that can be given to an agent. This can be helpful if for example two Filebeat instances are running on the same host but a human readable separation is needed on which Filebeat instance data is coming from. - -type: keyword - - - -example: `foo` - -| core - -// =============================================================== - -| -[[field-agent-type]] -<> - -a| Type of the agent. - -The agent type always stays the same and should be given by the agent used. In case of Filebeat the agent would always be Filebeat also if two Filebeat instances are run on the same machine. - -type: keyword - - - -example: `filebeat` - -| core - -// =============================================================== - -| -[[field-agent-version]] -<> - -a| Version of the agent. - -type: keyword - - - -example: `6.0.0-rc2` - -| core - -// =============================================================== - -|===== - - -[[ecs-as]] -=== Autonomous System Fields - -An autonomous system (AS) is a collection of connected Internet Protocol (IP) routing prefixes under the control of one or more network operators on behalf of a single administrative entity or domain that presents a common, clearly defined routing policy to the internet. - -[discrete] -==== Autonomous System Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-as-number]] -<> - -a| Unique number allocated to the autonomous system. The autonomous system number (ASN) uniquely identifies each network on the Internet. - -type: long - - - -example: `15169` - -| extended - -// =============================================================== - -| -[[field-as-organization-name]] -<> - -a| Organization name. - -type: keyword - -Multi-fields: - -* as.organization.name.text (type: match_only_text) - - - - - -example: `Google LLC` - -| extended - -// =============================================================== - -|===== - -[discrete] -==== Field Reuse - -The `as` fields are expected to be nested at: - - -* `client.as` - -* `destination.as` - -* `server.as` - -* `source.as` - -* `threat.enrichments.indicator.as` - -* `threat.indicator.as` - - -Note also that the `as` fields are not expected to be used directly at the root of the events. -[[ecs-client]] -=== Client Fields - -A client is defined as the initiator of a network connection for events regarding sessions, connections, or bidirectional flow records. - -For TCP events, the client is the initiator of the TCP connection that sends the SYN packet(s). For other protocols, the client is generally the initiator or requestor in the network transaction. Some systems use the term "originator" to refer the client in TCP connections. The client fields describe details about the system acting as the client in the network event. Client fields are usually populated in conjunction with server fields. Client fields are generally not populated for packet-level events. - -Client / server representations can add semantic context to an exchange, which is helpful to visualize the data in certain situations. If your context falls in that category, you should still ensure that source and destination are filled appropriately. - -[discrete] -==== Client Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-client-address]] -<> - -a| Some event client addresses are defined ambiguously. The event will sometimes list an IP, a domain or a unix socket. You should always store the raw address in the `.address` field. - -Then it should be duplicated to `.ip` or `.domain`, depending on which one it is. - -type: keyword - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/client/#client-address[client.address] - -| extended - -// =============================================================== - -| -[[field-client-bytes]] -<> - -a| Bytes sent from the client to the server. - -type: long - - - -example: `184` - -| core - -// =============================================================== - -| -[[field-client-domain]] -<> - -a| The domain name of the client system. - -This value may be a host name, a fully qualified domain name, or another host naming format. The value may derive from the original event or be added from enrichment. - -type: keyword - - - -example: `foo.example.com` - -| core - -// =============================================================== - -| -[[field-client-ip]] -<> - -a| IP address of the client (IPv4 or IPv6). - -type: ip - - - - - -| core - -// =============================================================== - -| -[[field-client-mac]] -<> - -a| MAC address of the client. - -The notation format from RFC 7042 is suggested: Each octet (that is, 8-bit byte) is represented by two [uppercase] hexadecimal digits giving the value of the octet as an unsigned integer. Successive octets are separated by a hyphen. - -type: keyword - - - -example: `00-00-5E-00-53-23` - -| core - -// =============================================================== - -| -[[field-client-nat-ip]] -<> - -a| Translated IP of source based NAT sessions (e.g. internal client to internet). - -Typically connections traversing load balancers, firewalls, or routers. - -type: ip - - - - - -| extended - -// =============================================================== - -| -[[field-client-nat-port]] -<> - -a| Translated port of source based NAT sessions (e.g. internal client to internet). - -Typically connections traversing load balancers, firewalls, or routers. - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-client-packets]] -<> - -a| Packets sent from the client to the server. - -type: long - - - -example: `12` - -| core - -// =============================================================== - -| -[[field-client-port]] -<> - -a| Port of the client. - -type: long - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/client/#client-port[client.port] - -| core - -// =============================================================== - -| -[[field-client-registered-domain]] -<> - -a| The highest registered client domain, stripped of the subdomain. - -For example, the registered domain for "foo.example.com" is "example.com". - -This value can be determined precisely with a list like the public suffix list (https://publicsuffix.org). Trying to approximate this by simply taking the last two labels will not work well for TLDs such as "co.uk". - -type: keyword - - - -example: `example.com` - -| extended - -// =============================================================== - -| -[[field-client-subdomain]] -<> - -a| The subdomain portion of a fully qualified domain name includes all of the names except the host name under the registered_domain. In a partially qualified domain, or if the the qualification level of the full name cannot be determined, subdomain contains all of the names below the registered domain. - -For example the subdomain portion of "www.east.mydomain.co.uk" is "east". If the domain has multiple levels of subdomain, such as "sub2.sub1.example.com", the subdomain field should contain "sub2.sub1", with no trailing period. - -type: keyword - - - -example: `east` - -| extended - -// =============================================================== - -| -[[field-client-top-level-domain]] -<> - -a| The effective top level domain (eTLD), also known as the domain suffix, is the last part of the domain name. For example, the top level domain for example.com is "com". - -This value can be determined precisely with a list like the public suffix list (https://publicsuffix.org). Trying to approximate this by simply taking the last label will not work well for effective TLDs such as "co.uk". - -type: keyword - - - -example: `co.uk` - -| extended - -// =============================================================== - -|===== - -[discrete] -==== Field Reuse - -[[ecs-client-nestings]] -[discrete] -===== Field sets that can be nested under Client - -[options="header"] -|===== -| Location | Field Set | Description - -// =============================================================== - - -| `client.as.*` -| <> -| Fields describing an Autonomous System (Internet routing prefix). - -// =============================================================== - - -| `client.geo.*` -| <> -| Fields describing a location. - -// =============================================================== - - -| `client.user.*` -| <> -| Fields to describe the user relevant to the event. - -// =============================================================== - - -|===== - - -[[ecs-cloud]] -=== Cloud Fields - -Fields related to the cloud or infrastructure the events are coming from. - -[discrete] -==== Cloud Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-cloud-account-id]] -<> - -a| The cloud account or organization id used to identify different entities in a multi-tenant environment. - -Examples: AWS account id, Google Cloud ORG Id, or other unique identifier. - -type: keyword - - - -example: `666777888999` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/cloud/#cloud-account-id[cloud.account.id] - -| extended - -// =============================================================== - -| -[[field-cloud-account-name]] -<> - -a| The cloud account name or alias used to identify different entities in a multi-tenant environment. - -Examples: AWS account name, Google Cloud ORG display name. - -type: keyword - - - -example: `elastic-dev` - -| extended - -// =============================================================== - -| -[[field-cloud-availability-zone]] -<> - -a| Availability zone in which this host, resource, or service is located. - -type: keyword - - - -example: `us-east-1c` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/cloud/#cloud-availability-zone[cloud.availability_zone] - -| extended - -// =============================================================== - -| -[[field-cloud-instance-id]] -<> - -a| Instance ID of the host machine. - -type: keyword - - - -example: `i-1234567890abcdef0` - -| extended - -// =============================================================== - -| -[[field-cloud-instance-name]] -<> - -a| Instance name of the host machine. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-cloud-machine-type]] -<> - -a| Machine type of the host machine. - -type: keyword - - - -example: `t2.medium` - -| extended - -// =============================================================== - -| -[[field-cloud-project-id]] -<> - -a| The cloud project identifier. - -Examples: Google Cloud Project id, Azure Project id. - -type: keyword - - - -example: `my-project` - -| extended - -// =============================================================== - -| -[[field-cloud-project-name]] -<> - -a| The cloud project name. - -Examples: Google Cloud Project name, Azure Project name. - -type: keyword - - - -example: `my project` - -| extended - -// =============================================================== - -| -[[field-cloud-provider]] -<> - -a| Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean. - -type: keyword - - - -example: `aws` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/cloud/#cloud-provider[cloud.provider] - -| extended - -// =============================================================== - -| -[[field-cloud-region]] -<> - -a| Region in which this host, resource, or service is located. - -type: keyword - - - -example: `us-east-1` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/cloud/#cloud-region[cloud.region] - -| extended - -// =============================================================== - -| -[[field-cloud-service-name]] -<> - -a| The cloud service name is intended to distinguish services running on different platforms within a provider, eg AWS EC2 vs Lambda, GCP GCE vs App Engine, Azure VM vs App Server. - -Examples: app engine, app service, cloud run, fargate, lambda. - -type: keyword - - - -example: `lambda` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/cloud/#cloud-platform[cloud.platform] - -| extended - -// =============================================================== - -|===== - -[discrete] -==== Field Reuse - -The `cloud` fields are expected to be nested at: - - -* `cloud.origin` - -* `cloud.target` - - -Note also that the `cloud` fields may be used directly at the root of the events. - -[[ecs-cloud-nestings]] -[discrete] -===== Field sets that can be nested under Cloud - -[options="header"] -|===== -| Location | Field Set | Description - -// =============================================================== - - -| `cloud.origin.*` -| <> -| Provides the cloud information of the origin entity in case of an incoming request or event. - -// =============================================================== - - -| `cloud.target.*` -| <> -| Provides the cloud information of the target entity in case of an outgoing request or event. - -// =============================================================== - - -|===== - - - -[discrete] -==== Cloud Field Usage - -For usage and examples of the cloud fields, please see the <> section. - -include::usage/cloud.asciidoc[] - - -[[ecs-code_signature]] -=== Code Signature Fields - -These fields contain information about binary code signatures. - -[discrete] -==== Code Signature Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-code-signature-digest-algorithm]] -<> - -a| The hashing algorithm used to sign the process. - -This value can distinguish signatures when a file is signed multiple times by the same signer but with a different digest algorithm. - -type: keyword - - - -example: `sha256` - -| extended - -// =============================================================== - -| -[[field-code-signature-exists]] -<> - -a| Boolean to capture if a signature is present. - -type: boolean - - - -example: `true` - -| core - -// =============================================================== - -| -[[field-code-signature-flags]] -<> - -a| beta:[ This field is beta and subject to change. ] - -The flags used to sign the process. - -type: keyword - - - -example: `570522385` - -| extended - -// =============================================================== - -| -[[field-code-signature-signing-id]] -<> - -a| The identifier used to sign the process. - -This is used to identify the application manufactured by a software vendor. The field is relevant to Apple *OS only. - -type: keyword - - - -example: `com.apple.xpc.proxy` - -| extended - -// =============================================================== - -| -[[field-code-signature-status]] -<> - -a| Additional information about the certificate status. - -This is useful for logging cryptographic errors with the certificate validity or trust status. Leave unpopulated if the validity or trust of the certificate was unchecked. - -type: keyword - - - -example: `ERROR_UNTRUSTED_ROOT` - -| extended - -// =============================================================== - -| -[[field-code-signature-subject-name]] -<> - -a| Subject name of the code signer - -type: keyword - - - -example: `Microsoft Corporation` - -| core - -// =============================================================== - -| -[[field-code-signature-team-id]] -<> - -a| The team identifier used to sign the process. - -This is used to identify the team or vendor of a software product. The field is relevant to Apple *OS only. - -type: keyword - - - -example: `EQHXZ8M8AV` - -| extended - -// =============================================================== - -| -[[field-code-signature-timestamp]] -<> - -a| Date and time when the code signature was generated and signed. - -type: date - - - -example: `2021-01-01T12:10:30Z` - -| extended - -// =============================================================== - -| -[[field-code-signature-trusted]] -<> - -a| Stores the trust status of the certificate chain. - -Validating the trust of the certificate chain may be complicated, and this field should only be populated by tools that actively check the status. - -type: boolean - - - -example: `true` - -| extended - -// =============================================================== - -| -[[field-code-signature-valid]] -<> - -a| Boolean to capture if the digital signature is verified against the binary content. - -Leave unpopulated if a certificate was unchecked. - -type: boolean - - - -example: `true` - -| extended - -// =============================================================== - -|===== - -[discrete] -==== Field Reuse - -The `code_signature` fields are expected to be nested at: - - -* `dll.code_signature` - -* `file.code_signature` - -* `process.code_signature` - - -Note also that the `code_signature` fields are not expected to be used directly at the root of the events. -[[ecs-container]] -=== Container Fields - -Container fields are used for meta information about the specific container that is the source of information. - -These fields help correlate data based containers from any runtime. - -[discrete] -==== Container Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-container-cpu-usage]] -<> - -a| Percent CPU used which is normalized by the number of CPU cores and it ranges from 0 to 1. Scaling factor: 1000. - -type: scaled_float - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/metric-cb00cb?style=flat[relation,title=metric,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--\+semconv+metric.container.cpu.usage+--%3E%22&type=code[container.cpu.usage] - -| extended - -// =============================================================== - -| -[[field-container-disk-read-bytes]] -<> - -a| The total number of bytes (gauge) read successfully (aggregated from all disks) since the last metric collection. - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-container-disk-write-bytes]] -<> - -a| The total number of bytes (gauge) written successfully (aggregated from all disks) since the last metric collection. - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-container-id]] -<> - -a| Unique container id. - -type: keyword - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/container/#container-id[container.id] - -| core - -// =============================================================== - -| -[[field-container-image-hash-all]] -<> - -a| An array of digests of the image the container was built on. Each digest consists of the hash algorithm and value in this format: `algorithm:value`. Algorithm names should align with the field names in the ECS hash field set. - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `[sha256:f8fefc80e3273dc756f288a63945820d6476ad64883892c771b5e2ece6bf1b26]` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/container/#container-image-repo-digests[container.image.repo_digests] - -| extended - -// =============================================================== - -| -[[field-container-image-name]] -<> - -a| Name of the image the container was built on. - -type: keyword - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/container/#container-image-name[container.image.name] - -| extended - -// =============================================================== - -| -[[field-container-image-tag]] -<> - -a| Container image tags. - -type: keyword - - -Note: this field should contain an array of values. - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/container/#container-image-tags[container.image.tags] - -| extended - -// =============================================================== - -| -[[field-container-labels]] -<> - -a| Image labels. - -type: object - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/related-efc20d?style=flat[relation,title=related,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/container/#container-label[container.label] - -| extended - -// =============================================================== - -| -[[field-container-memory-usage]] -<> - -a| Memory usage percentage and it ranges from 0 to 1. Scaling factor: 1000. - -type: scaled_float - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/metric-cb00cb?style=flat[relation,title=metric,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--\+semconv+metric.container.memory.usage+--%3E%22&type=code[container.memory.usage] - -| extended - -// =============================================================== - -| -[[field-container-name]] -<> - -a| Container name. - -type: keyword - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/container/#container-name[container.name] - -| extended - -// =============================================================== - -| -[[field-container-network-egress-bytes]] -<> - -a| The number of bytes (gauge) sent out on all network interfaces by the container since the last metric collection. - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-container-network-ingress-bytes]] -<> - -a| The number of bytes received (gauge) on all network interfaces by the container since the last metric collection. - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-container-runtime]] -<> - -a| Runtime managing this container. - -type: keyword - - - -example: `docker` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/container/#container-runtime[container.runtime] - -| extended - -// =============================================================== - -| -[[field-container-security-context-privileged]] -<> - -a| Indicates whether the container is running in privileged mode. - -type: boolean - - - - - -| extended - -// =============================================================== - -|===== - - -[[ecs-data_stream]] -=== Data Stream Fields - -The data_stream fields take part in defining the new data stream naming scheme. - -In the new data stream naming scheme the value of the data stream fields combine to the name of the actual data stream in the following manner: `{data_stream.type}-{data_stream.dataset}-{data_stream.namespace}`. This means the fields can only contain characters that are valid as part of names of data streams. More details about this can be found in this https://www.elastic.co/blog/an-introduction-to-the-elastic-data-stream-naming-scheme[blog post]. - -An Elasticsearch data stream consists of one or more backing indices, and a data stream name forms part of the backing indices names. Due to this convention, data streams must also follow index naming restrictions. For example, data stream names cannot include `\`, `/`, `*`, `?`, `"`, `<`, `>`, `|`, ` ` (space character), `,`, or `#`. Please see the Elasticsearch reference for additional https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-create-index.html#indices-create-api-path-params[restrictions]. - -beta::[ These fields are in beta and are subject to change.] - -[discrete] -==== Data Stream Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-data-stream-dataset]] -<> - -a| The field can contain anything that makes sense to signify the source of the data. - -Examples include `nginx.access`, `prometheus`, `endpoint` etc. For data streams that otherwise fit, but that do not have dataset set we use the value "generic" for the dataset value. `event.dataset` should have the same value as `data_stream.dataset`. - -Beyond the Elasticsearch data stream naming criteria noted above, the `dataset` value has additional restrictions: - - * Must not contain `-` - - * No longer than 100 characters - -type: constant_keyword - - - -example: `nginx.access` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat[relation,title=not-applicable,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] Not applicable. - -| extended - -// =============================================================== - -| -[[field-data-stream-namespace]] -<> - -a| A user defined namespace. Namespaces are useful to allow grouping of data. - -Many users already organize their indices this way, and the data stream naming scheme now provides this best practice as a default. Many users will populate this field with `default`. If no value is used, it falls back to `default`. - -Beyond the Elasticsearch index naming criteria noted above, `namespace` value has the additional restrictions: - - * Must not contain `-` - - * No longer than 100 characters - -type: constant_keyword - - - -example: `production` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat[relation,title=not-applicable,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] Not applicable. - -| extended - -// =============================================================== - -| -[[field-data-stream-type]] -<> - -a| An overarching type for the data stream. - -Currently allowed values are "logs" and "metrics". We expect to also add "traces" and "synthetics" in the near future. - -type: constant_keyword - - - -example: `logs` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat[relation,title=not-applicable,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] Not applicable. - -| extended - -// =============================================================== - -|===== - - -[[ecs-destination]] -=== Destination Fields - -Destination fields capture details about the receiver of a network exchange/packet. These fields are populated from a network event, packet, or other event containing details of a network transaction. - -Destination fields are usually populated in conjunction with source fields. The source and destination fields are considered the baseline and should always be filled if an event contains source and destination details from a network transaction. If the event also contains identification of the client and server roles, then the client and server fields should also be populated. - -[discrete] -==== Destination Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-destination-address]] -<> - -a| Some event destination addresses are defined ambiguously. The event will sometimes list an IP, a domain or a unix socket. You should always store the raw address in the `.address` field. - -Then it should be duplicated to `.ip` or `.domain`, depending on which one it is. - -type: keyword - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/destination/#destination-address[destination.address] - -| extended - -// =============================================================== - -| -[[field-destination-bytes]] -<> - -a| Bytes sent from the destination to the source. - -type: long - - - -example: `184` - -| core - -// =============================================================== - -| -[[field-destination-domain]] -<> - -a| The domain name of the destination system. - -This value may be a host name, a fully qualified domain name, or another host naming format. The value may derive from the original event or be added from enrichment. - -type: keyword - - - -example: `foo.example.com` - -| core - -// =============================================================== - -| -[[field-destination-ip]] -<> - -a| IP address of the destination (IPv4 or IPv6). - -type: ip - - - - - -| core - -// =============================================================== - -| -[[field-destination-mac]] -<> - -a| MAC address of the destination. - -The notation format from RFC 7042 is suggested: Each octet (that is, 8-bit byte) is represented by two [uppercase] hexadecimal digits giving the value of the octet as an unsigned integer. Successive octets are separated by a hyphen. - -type: keyword - - - -example: `00-00-5E-00-53-23` - -| core - -// =============================================================== - -| -[[field-destination-nat-ip]] -<> - -a| Translated ip of destination based NAT sessions (e.g. internet to private DMZ) - -Typically used with load balancers, firewalls, or routers. - -type: ip - - - - - -| extended - -// =============================================================== - -| -[[field-destination-nat-port]] -<> - -a| Port the source session is translated to by NAT Device. - -Typically used with load balancers, firewalls, or routers. - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-destination-packets]] -<> - -a| Packets sent from the destination to the source. - -type: long - - - -example: `12` - -| core - -// =============================================================== - -| -[[field-destination-port]] -<> - -a| Port of the destination. - -type: long - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/destination/#destination-port[destination.port] - -| core - -// =============================================================== - -| -[[field-destination-registered-domain]] -<> - -a| The highest registered destination domain, stripped of the subdomain. - -For example, the registered domain for "foo.example.com" is "example.com". - -This value can be determined precisely with a list like the public suffix list (https://publicsuffix.org). Trying to approximate this by simply taking the last two labels will not work well for TLDs such as "co.uk". - -type: keyword - - - -example: `example.com` - -| extended - -// =============================================================== - -| -[[field-destination-subdomain]] -<> - -a| The subdomain portion of a fully qualified domain name includes all of the names except the host name under the registered_domain. In a partially qualified domain, or if the the qualification level of the full name cannot be determined, subdomain contains all of the names below the registered domain. - -For example the subdomain portion of "www.east.mydomain.co.uk" is "east". If the domain has multiple levels of subdomain, such as "sub2.sub1.example.com", the subdomain field should contain "sub2.sub1", with no trailing period. - -type: keyword - - - -example: `east` - -| extended - -// =============================================================== - -| -[[field-destination-top-level-domain]] -<> - -a| The effective top level domain (eTLD), also known as the domain suffix, is the last part of the domain name. For example, the top level domain for example.com is "com". - -This value can be determined precisely with a list like the public suffix list (https://publicsuffix.org). Trying to approximate this by simply taking the last label will not work well for effective TLDs such as "co.uk". - -type: keyword - - - -example: `co.uk` - -| extended - -// =============================================================== - -|===== - -[discrete] -==== Field Reuse - -[[ecs-destination-nestings]] -[discrete] -===== Field sets that can be nested under Destination - -[options="header"] -|===== -| Location | Field Set | Description - -// =============================================================== - - -| `destination.as.*` -| <> -| Fields describing an Autonomous System (Internet routing prefix). - -// =============================================================== - - -| `destination.geo.*` -| <> -| Fields describing a location. - -// =============================================================== - - -| `destination.user.*` -| <> -| Fields to describe the user relevant to the event. - -// =============================================================== - - -|===== - - -[[ecs-device]] -=== Device Fields - -Fields that describe a device instance and its characteristics. Data collected for applications and processes running on a (mobile) device can be enriched with these fields to describe the identity, type and other characteristics of the device. - -This field group definition is based on the Device namespace of the OpenTelemetry Semantic Conventions (https://opentelemetry.io/docs/reference/specification/resource/semantic_conventions/device/). - -beta::[ These fields are in beta and are subject to change.] - -[discrete] -==== Device Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-device-id]] -<> - -a| The unique identifier of a device. The identifier must not change across application sessions but stay fixed for an instance of a (mobile) device. - -On iOS, this value must be equal to the vendor identifier (https://developer.apple.com/documentation/uikit/uidevice/1620059-identifierforvendor). On Android, this value must be equal to the Firebase Installation ID or a globally unique UUID which is persisted across sessions in your application. - -For GDPR and data protection law reasons this identifier should not carry information that would allow to identify a user. - -type: keyword - - - -example: `00000000-54b3-e7c7-0000-000046bffd97` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/device/#device-id[device.id] - -| extended - -// =============================================================== - -| -[[field-device-manufacturer]] -<> - -a| The vendor name of the device manufacturer. - -type: keyword - - - -example: `Samsung` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/device/#device-manufacturer[device.manufacturer] - -| extended - -// =============================================================== - -| -[[field-device-model-identifier]] -<> - -a| The machine readable identifier of the device model. - -type: keyword - - - -example: `SM-G920F` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/device/#device-model-identifier[device.model.identifier] - -| extended - -// =============================================================== - -| -[[field-device-model-name]] -<> - -a| The human readable marketing name of the device model. - -type: keyword - - - -example: `Samsung Galaxy S6` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/device/#device-model-name[device.model.name] - -| extended - -// =============================================================== - -| -[[field-device-serial-number]] -<> - -a| beta:[ This field is beta and subject to change. ] - -The unique serial number serves as a distinct identifier for each device, aiding in inventory management and device authentication. - -type: keyword - - - -example: `DJGAQS4CW5` - -| core - -// =============================================================== - -|===== - - -[[ecs-dll]] -=== DLL Fields - -These fields contain information about code libraries dynamically loaded into processes. - - - -Many operating systems refer to "shared code libraries" with different names, but this field set refers to all of the following: - -* Dynamic-link library (`.dll`) commonly used on Windows - -* Shared Object (`.so`) commonly used on Unix-like operating systems - -* Dynamic library (`.dylib`) commonly used on macOS - -[discrete] -==== DLL Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-dll-name]] -<> - -a| Name of the library. - -This generally maps to the name of the file on disk. - -type: keyword - - - -example: `kernel32.dll` - -| core - -// =============================================================== - -| -[[field-dll-origin-referrer-url]] -<> - -a| beta:[ This field is beta and subject to change. ] - -The URL of the webpage that linked to the dll file. - -type: keyword - - - -example: `http://example.com/article1.html` - -| extended - -// =============================================================== - -| -[[field-dll-origin-url]] -<> - -a| beta:[ This field is beta and subject to change. ] - -The URL where the dll file is hosted. - -type: keyword - - - -example: `http://example.com/files/example.dll` - -| extended - -// =============================================================== - -| -[[field-dll-path]] -<> - -a| Full file path of the library. - -type: keyword - - - -example: `C:\Windows\System32\kernel32.dll` - -| extended - -// =============================================================== - -|===== - -[discrete] -==== Field Reuse - -[[ecs-dll-nestings]] -[discrete] -===== Field sets that can be nested under DLL - -[options="header"] -|===== -| Location | Field Set | Description - -// =============================================================== - - -| `dll.code_signature.*` -| <> -| These fields contain information about binary code signatures. - -// =============================================================== - - -| `dll.hash.*` -| <> -| Hashes, usually file hashes. - -// =============================================================== - - -| `dll.pe.*` -| <> -| These fields contain Windows Portable Executable (PE) metadata. - -// =============================================================== - - -|===== - - -[[ecs-dns]] -=== DNS Fields - -Fields describing DNS queries and answers. - -DNS events should either represent a single DNS query prior to getting answers (`dns.type:query`) or they should represent a full exchange and contain the query details as well as all of the answers that were provided for this query (`dns.type:answer`). - -[discrete] -==== DNS Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-dns-answers]] -<> - -a| An array containing an object for each answer section returned by the server. - -The main keys that should be present in these objects are defined by ECS. Records that have more information may contain more keys than what ECS defines. - -Not all DNS data sources give all details about DNS answers. At minimum, answer objects must contain the `data` key. If more information is available, map as much of it to ECS as possible, and add any additional fields to the answer objects as custom fields. - -type: object - - -Note: this field should contain an array of values. - - - - - -| extended - -// =============================================================== - -| -[[field-dns-answers-class]] -<> - -a| The class of DNS data contained in this resource record. - -type: keyword - - - -example: `IN` - -| extended - -// =============================================================== - -| -[[field-dns-answers-data]] -<> - -a| The data describing the resource. - -The meaning of this data depends on the type and class of the resource record. - -type: keyword - - - -example: `10.10.10.10` - -| extended - -// =============================================================== - -| -[[field-dns-answers-name]] -<> - -a| The domain name to which this resource record pertains. - -If a chain of CNAME is being resolved, each answer's `name` should be the one that corresponds with the answer's `data`. It should not simply be the original `question.name` repeated. - -type: keyword - - - -example: `www.example.com` - -| extended - -// =============================================================== - -| -[[field-dns-answers-ttl]] -<> - -a| The time interval in seconds that this resource record may be cached before it should be discarded. Zero values mean that the data should not be cached. - -type: long - - - -example: `180` - -| extended - -// =============================================================== - -| -[[field-dns-answers-type]] -<> - -a| The type of data contained in this resource record. - -type: keyword - - - -example: `CNAME` - -| extended - -// =============================================================== - -| -[[field-dns-header-flags]] -<> - -a| Array of 2 letter DNS header flags. - -Expected values for this field: - -* `AA` -* `TC` -* `RD` -* `RA` -* `AD` -* `CD` -* `DO` - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `["RD", "RA"]` - -| extended - -// =============================================================== - -| -[[field-dns-id]] -<> - -a| The DNS packet identifier assigned by the program that generated the query. The identifier is copied to the response. - -type: keyword - - - -example: `62111` - -| extended - -// =============================================================== - -| -[[field-dns-op-code]] -<> - -a| The DNS operation code that specifies the kind of query in the message. This value is set by the originator of a query and copied into the response. - -type: keyword - - - -example: `QUERY` - -| extended - -// =============================================================== - -| -[[field-dns-question-class]] -<> - -a| The class of records being queried. - -type: keyword - - - -example: `IN` - -| extended - -// =============================================================== - -| -[[field-dns-question-name]] -<> - -a| The name being queried. - -If the name field contains non-printable characters (below 32 or above 126), those characters should be represented as escaped base 10 integers (\DDD). Back slashes and quotes should be escaped. Tabs, carriage returns, and line feeds should be converted to \t, \r, and \n respectively. - -type: keyword - - - -example: `www.example.com` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/dns/#dns-question-name[dns.question.name] - -| extended - -// =============================================================== - -| -[[field-dns-question-registered-domain]] -<> - -a| The highest registered domain, stripped of the subdomain. - -For example, the registered domain for "foo.example.com" is "example.com". - -This value can be determined precisely with a list like the public suffix list (https://publicsuffix.org). Trying to approximate this by simply taking the last two labels will not work well for TLDs such as "co.uk". - -type: keyword - - - -example: `example.com` - -| extended - -// =============================================================== - -| -[[field-dns-question-subdomain]] -<> - -a| The subdomain is all of the labels under the registered_domain. - -If the domain has multiple levels of subdomain, such as "sub2.sub1.example.com", the subdomain field should contain "sub2.sub1", with no trailing period. - -type: keyword - - - -example: `www` - -| extended - -// =============================================================== - -| -[[field-dns-question-top-level-domain]] -<> - -a| The effective top level domain (eTLD), also known as the domain suffix, is the last part of the domain name. For example, the top level domain for example.com is "com". - -This value can be determined precisely with a list like the public suffix list (https://publicsuffix.org). Trying to approximate this by simply taking the last label will not work well for effective TLDs such as "co.uk". - -type: keyword - - - -example: `co.uk` - -| extended - -// =============================================================== - -| -[[field-dns-question-type]] -<> - -a| The type of record being queried. - -type: keyword - - - -example: `AAAA` - -| extended - -// =============================================================== - -| -[[field-dns-resolved-ip]] -<> - -a| Array containing all IPs seen in `answers.data`. - -The `answers` array can be difficult to use, because of the variety of data formats it can contain. Extracting all IP addresses seen in there to `dns.resolved_ip` makes it possible to index them as IP addresses, and makes them easier to visualize and query for. - -type: ip - - -Note: this field should contain an array of values. - - - -example: `["10.10.10.10", "10.10.10.11"]` - -| extended - -// =============================================================== - -| -[[field-dns-response-code]] -<> - -a| The DNS response code. - -type: keyword - - - -example: `NOERROR` - -| extended - -// =============================================================== - -| -[[field-dns-type]] -<> - -a| The type of DNS event captured, query or answer. - -If your source of DNS events only gives you DNS queries, you should only create dns events of type `dns.type:query`. - -If your source of DNS events gives you answers as well, you should create one event per query (optionally as soon as the query is seen). And a second event containing all query details as well as an array of answers. - -type: keyword - - - -example: `answer` - -| extended - -// =============================================================== - -|===== - - -[[ecs-ecs]] -=== ECS Fields - -Meta-information specific to ECS. - -[discrete] -==== ECS Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-ecs-version]] -<> - -a| ECS version this event conforms to. `ecs.version` is a required field and must exist in all events. - -When querying across multiple indices -- which may conform to slightly different ECS versions -- this field lets integrations adjust to the schema version of the events. - -type: keyword - - - -example: `1.0.0` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat[relation,title=not-applicable,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] Not applicable. - -| core - -// =============================================================== - -|===== - - -[[ecs-elf]] -=== ELF Header Fields - -These fields contain Linux Executable Linkable Format (ELF) metadata. - -[discrete] -==== ELF Header Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-elf-architecture]] -<> - -a| Machine architecture of the ELF file. - -type: keyword - - - -example: `x86-64` - -| extended - -// =============================================================== - -| -[[field-elf-byte-order]] -<> - -a| Byte sequence of ELF file. - -type: keyword - - - -example: `Little Endian` - -| extended - -// =============================================================== - -| -[[field-elf-cpu-type]] -<> - -a| CPU type of the ELF file. - -type: keyword - - - -example: `Intel` - -| extended - -// =============================================================== - -| -[[field-elf-creation-date]] -<> - -a| Extracted when possible from the file's metadata. Indicates when it was built or compiled. It can also be faked by malware creators. - -type: date - - - - - -| extended - -// =============================================================== - -| -[[field-elf-exports]] -<> - -a| List of exported element names and types. - -type: flattened - - -Note: this field should contain an array of values. - - - - - -| extended - -// =============================================================== - -| -[[field-elf-go-import-hash]] -<> - -a| A hash of the Go language imports in an ELF file excluding standard library imports. An import hash can be used to fingerprint binaries even after recompilation or other code-level transformations have occurred, which would change more traditional hash values. - -The algorithm used to calculate the Go symbol hash and a reference implementation are available here: https://github.com/elastic/toutoumomoma - -type: keyword - - - -example: `10bddcb4cee42080f76c88d9ff964491` - -| extended - -// =============================================================== - -| -[[field-elf-go-imports]] -<> - -a| List of imported Go language element names and types. - -type: flattened - - - - - -| extended - -// =============================================================== - -| -[[field-elf-go-imports-names-entropy]] -<> - -a| Shannon entropy calculation from the list of Go imports. - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-elf-go-imports-names-var-entropy]] -<> - -a| Variance for Shannon entropy calculation from the list of Go imports. - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-elf-go-stripped]] -<> - -a| Set to true if the file is a Go executable that has had its symbols stripped or obfuscated and false if an unobfuscated Go executable. - -type: boolean - - - - - -| extended - -// =============================================================== - -| -[[field-elf-header-abi-version]] -<> - -a| Version of the ELF Application Binary Interface (ABI). - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-elf-header-class]] -<> - -a| Header class of the ELF file. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-elf-header-data]] -<> - -a| Data table of the ELF header. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-elf-header-entrypoint]] -<> - -a| Header entrypoint of the ELF file. - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-elf-header-object-version]] -<> - -a| "0x1" for original ELF files. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-elf-header-os-abi]] -<> - -a| Application Binary Interface (ABI) of the Linux OS. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-elf-header-type]] -<> - -a| Header type of the ELF file. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-elf-header-version]] -<> - -a| Version of the ELF header. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-elf-import-hash]] -<> - -a| A hash of the imports in an ELF file. An import hash can be used to fingerprint binaries even after recompilation or other code-level transformations have occurred, which would change more traditional hash values. - -This is an ELF implementation of the Windows PE imphash. - -type: keyword - - - -example: `d41d8cd98f00b204e9800998ecf8427e` - -| extended - -// =============================================================== - -| -[[field-elf-imports]] -<> - -a| List of imported element names and types. - -type: flattened - - -Note: this field should contain an array of values. - - - - - -| extended - -// =============================================================== - -| -[[field-elf-imports-names-entropy]] -<> - -a| Shannon entropy calculation from the list of imported element names and types. - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-elf-imports-names-var-entropy]] -<> - -a| Variance for Shannon entropy calculation from the list of imported element names and types. - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-elf-sections]] -<> - -a| An array containing an object for each section of the ELF file. - -The keys that should be present in these objects are defined by sub-fields underneath `elf.sections.*`. - -type: nested - - -Note: this field should contain an array of values. - - - - - -| extended - -// =============================================================== - -| -[[field-elf-sections-chi2]] -<> - -a| Chi-square probability distribution of the section. - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-elf-sections-entropy]] -<> - -a| Shannon entropy calculation from the section. - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-elf-sections-flags]] -<> - -a| ELF Section List flags. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-elf-sections-name]] -<> - -a| ELF Section List name. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-elf-sections-physical-offset]] -<> - -a| ELF Section List offset. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-elf-sections-physical-size]] -<> - -a| ELF Section List physical size. - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-elf-sections-type]] -<> - -a| ELF Section List type. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-elf-sections-var-entropy]] -<> - -a| Variance for Shannon entropy calculation from the section. - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-elf-sections-virtual-address]] -<> - -a| ELF Section List virtual address. - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-elf-sections-virtual-size]] -<> - -a| ELF Section List virtual size. - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-elf-segments]] -<> - -a| An array containing an object for each segment of the ELF file. - -The keys that should be present in these objects are defined by sub-fields underneath `elf.segments.*`. - -type: nested - - -Note: this field should contain an array of values. - - - - - -| extended - -// =============================================================== - -| -[[field-elf-segments-sections]] -<> - -a| ELF object segment sections. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-elf-segments-type]] -<> - -a| ELF object segment type. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-elf-shared-libraries]] -<> - -a| List of shared libraries used by this ELF object. - -type: keyword - - -Note: this field should contain an array of values. - - - - - -| extended - -// =============================================================== - -| -[[field-elf-telfhash]] -<> - -a| telfhash symbol hash for ELF file. - -type: keyword - - - - - -| extended - -// =============================================================== - -|===== - -[discrete] -==== Field Reuse - -The `elf` fields are expected to be nested at: - - -* `file.elf` - -* `process.elf` - - -Note also that the `elf` fields are not expected to be used directly at the root of the events. -[[ecs-email]] -=== Email Fields - -Event details relating to an email transaction. - -This field set focuses on the email message header, body, and attachments. Network protocols that send and receive email messages such as SMTP are outside the scope of the `email.*` fields. - -[discrete] -==== Email Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-email-attachments]] -<> - -a| A list of objects describing the attachment files sent along with an email message. - -type: nested - - -Note: this field should contain an array of values. - - - - - -| extended - -// =============================================================== - -| -[[field-email-attachments-file-extension]] -<> - -a| Attachment file extension, excluding the leading dot. - -type: keyword - - - -example: `txt` - -| extended - -// =============================================================== - -| -[[field-email-attachments-file-mime-type]] -<> - -a| The MIME media type of the attachment. - -This value will typically be extracted from the `Content-Type` MIME header field. - -type: keyword - - - -example: `text/plain` - -| extended - -// =============================================================== - -| -[[field-email-attachments-file-name]] -<> - -a| Name of the attachment file including the file extension. - -type: keyword - - - -example: `attachment.txt` - -| extended - -// =============================================================== - -| -[[field-email-attachments-file-size]] -<> - -a| Attachment file size in bytes. - -type: long - - - -example: `64329` - -| extended - -// =============================================================== - -| -[[field-email-bcc-address]] -<> - -a| The email address of BCC recipient - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `bcc.user1@example.com` - -| extended - -// =============================================================== - -| -[[field-email-cc-address]] -<> - -a| The email address of CC recipient - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `cc.user1@example.com` - -| extended - -// =============================================================== - -| -[[field-email-content-type]] -<> - -a| Information about how the message is to be displayed. - -Typically a MIME type. - -type: keyword - - - -example: `text/plain` - -| extended - -// =============================================================== - -| -[[field-email-delivery-timestamp]] -<> - -a| The date and time when the email message was received by the service or client. - -type: date - - - -example: `2020-11-10T22:12:34.8196921Z` - -| extended - -// =============================================================== - -| -[[field-email-direction]] -<> - -a| The direction of the message based on the sending and receiving domains. - -type: keyword - - - -example: `inbound` - -| extended - -// =============================================================== - -| -[[field-email-from-address]] -<> - -a| The email address of the sender, typically from the RFC 5322 `From:` header field. - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `sender@example.com` - -| extended - -// =============================================================== - -| -[[field-email-local-id]] -<> - -a| Unique identifier given to the email by the source that created the event. - -Identifier is not persistent across hops. - -type: keyword - - - -example: `c26dbea0-80d5-463b-b93c-4e8b708219ce` - -| extended - -// =============================================================== - -| -[[field-email-message-id]] -<> - -a| Identifier from the RFC 5322 `Message-ID:` email header that refers to a particular email message. - -type: wildcard - - - -example: `81ce15$8r2j59@mail01.example.com` - -| extended - -// =============================================================== - -| -[[field-email-origination-timestamp]] -<> - -a| The date and time the email message was composed. Many email clients will fill in this value automatically when the message is sent by a user. - -type: date - - - -example: `2020-11-10T22:12:34.8196921Z` - -| extended - -// =============================================================== - -| -[[field-email-reply-to-address]] -<> - -a| The address that replies should be delivered to based on the value in the RFC 5322 `Reply-To:` header. - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `reply.here@example.com` - -| extended - -// =============================================================== - -| -[[field-email-sender-address]] -<> - -a| Per RFC 5322, specifies the address responsible for the actual transmission of the message. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-email-subject]] -<> - -a| A brief summary of the topic of the message. - -type: keyword - -Multi-fields: - -* email.subject.text (type: match_only_text) - - - - - -example: `Please see this important message.` - -| extended - -// =============================================================== - -| -[[field-email-to-address]] -<> - -a| The email address of recipient - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `user1@example.com` - -| extended - -// =============================================================== - -| -[[field-email-x-mailer]] -<> - -a| The name of the application that was used to draft and send the original email message. - -type: keyword - - - -example: `Spambot v2.5` - -| extended - -// =============================================================== - -|===== - -[discrete] -==== Field Reuse - -[[ecs-email-nestings]] -[discrete] -===== Field sets that can be nested under Email - -[options="header"] -|===== -| Location | Field Set | Description - -// =============================================================== - - -| `email.attachments.file.hash.*` -| <> -| Hashes, usually file hashes. - -// =============================================================== - - -|===== - - -[[ecs-error]] -=== Error Fields - -These fields can represent errors of any kind. - -Use them for errors that happen while fetching events or in cases where the event itself contains an error. - -[discrete] -==== Error Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-error-code]] -<> - -a| Error code describing the error. - -type: keyword - - - - - -| core - -// =============================================================== - -| -[[field-error-id]] -<> - -a| Unique identifier for the error. - -type: keyword - - - - - -| core - -// =============================================================== - -| -[[field-error-message]] -<> - -a| Error message. - -type: match_only_text - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/exception/#exception-message[exception.message] - -| core - -// =============================================================== - -| -[[field-error-stack-trace]] -<> - -a| The stack trace of this error in plain text. - -type: wildcard - -Multi-fields: - -* error.stack_trace.text (type: match_only_text) - - - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/exception/#exception-stacktrace[exception.stacktrace] - -| extended - -// =============================================================== - -| -[[field-error-type]] -<> - -a| The type of the error, for example the class name of the exception. - -type: keyword - - - -example: `java.lang.NullPointerException` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/error/#error-type[error.type] - -| extended - -// =============================================================== - -|===== - - -[[ecs-event]] -=== Event Fields - -The event fields are used for context information about the log or metric event itself. - -A log is defined as an event containing details of something that happened. Log events must include the time at which the thing happened. Examples of log events include a process starting on a host, a network packet being sent from a source to a destination, or a network connection between a client and a server being initiated or closed. A metric is defined as an event containing one or more numerical measurements and the time at which the measurement was taken. Examples of metric events include memory pressure measured on a host and device temperature. See the `event.kind` definition in this section for additional details about metric and state events. - -[discrete] -==== Event Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-event-action]] -<> - -a| The action captured by the event. - -This describes the information in the event. It is more specific than `event.category`. Examples are `group-add`, `process-started`, `file-created`. The value is normally defined by the implementer. - -type: keyword - - - -example: `user-password-change` - -| core - -// =============================================================== - -| -[[field-event-agent-id-status]] -<> - -a| Agents are normally responsible for populating the `agent.id` field value. If the system receiving events is capable of validating the value based on authentication information for the client then this field can be used to reflect the outcome of that validation. - -For example if the agent's connection is authenticated with mTLS and the client cert contains the ID of the agent to which the cert was issued then the `agent.id` value in events can be checked against the certificate. If the values match then `event.agent_id_status: verified` is added to the event, otherwise one of the other allowed values should be used. - -If no validation is performed then the field should be omitted. - -The allowed values are: - -`verified` - The `agent.id` field value matches expected value obtained from auth metadata. - -`mismatch` - The `agent.id` field value does not match the expected value obtained from auth metadata. - -`missing` - There was no `agent.id` field in the event to validate. - -`auth_metadata_missing` - There was no auth metadata or it was missing information about the agent ID. - -type: keyword - - - -example: `verified` - -| extended - -// =============================================================== - -| -[[field-event-category]] -<> - -a| This is one of four ECS Categorization Fields, and indicates the second level in the ECS category hierarchy. - -`event.category` represents the "big buckets" of ECS categories. For example, filtering on `event.category:process` yields all events relating to process activity. This field is closely related to `event.type`, which is used as a subcategory. - -This field is an array. This will allow proper categorization of some events that fall in multiple categories. - -type: keyword - - -Note: this field should contain an array of values. - - - - -*Important*: The field value must be one of the following: - -api, authentication, configuration, database, driver, email, file, host, iam, intrusion_detection, library, malware, network, package, process, registry, session, threat, vulnerability, web - -To learn more about when to use which value, visit the page -<> - - -| core - -// =============================================================== - -| -[[field-event-code]] -<> - -a| Identification code for this event, if one exists. - -Some event sources use event codes to identify messages unambiguously, regardless of message language or wording adjustments over time. An example of this is the Windows Event ID. - -type: keyword - - - -example: `4648` - -| extended - -// =============================================================== - -| -[[field-event-created]] -<> - -a| `event.created` contains the date/time when the event was first read by an agent, or by your pipeline. - -This field is distinct from `@timestamp` in that `@timestamp` typically contain the time extracted from the original event. - -In most situations, these two timestamps will be slightly different. The difference can be used to calculate the delay between your source generating an event, and the time when your agent first processed it. This can be used to monitor your agent's or pipeline's ability to keep up with your event source. - -In case the two timestamps are identical, `@timestamp` should be used. - -type: date - - - -example: `2016-05-23T08:05:34.857Z` - -| core - -// =============================================================== - -| -[[field-event-dataset]] -<> - -a| Name of the dataset. - -If an event source publishes more than one type of log or events (e.g. access log, error log), the dataset is used to specify which one the event comes from. - -It's recommended but not required to start the dataset name with the module name, followed by a dot, then the dataset name. - -type: keyword - - - -example: `apache.access` - -| core - -// =============================================================== - -| -[[field-event-duration]] -<> - -a| Duration of the event in nanoseconds. - -If `event.start` and `event.end` are known this value should be the difference between the end and start time. - -type: long - - - - - -| core - -// =============================================================== - -| -[[field-event-end]] -<> - -a| `event.end` contains the date when the event ended or when the activity was last observed. - -type: date - - - - - -| extended - -// =============================================================== - -| -[[field-event-hash]] -<> - -a| Hash (perhaps logstash fingerprint) of raw field to be able to demonstrate log integrity. - -type: keyword - - - -example: `123456789012345678901234567890ABCD` - -| extended - -// =============================================================== - -| -[[field-event-id]] -<> - -a| Unique ID to describe the event. - -type: keyword - - - -example: `8a4f500d` - -| core - -// =============================================================== - -| -[[field-event-ingested]] -<> - -a| Timestamp when an event arrived in the central data store. - -This is different from `@timestamp`, which is when the event originally occurred. It's also different from `event.created`, which is meant to capture the first time an agent saw the event. - -In normal conditions, assuming no tampering, the timestamps should chronologically look like this: `@timestamp` < `event.created` < `event.ingested`. - -type: date - - - -example: `2016-05-23T08:05:35.101Z` - -| core - -// =============================================================== - -| -[[field-event-kind]] -<> - -a| This is one of four ECS Categorization Fields, and indicates the highest level in the ECS category hierarchy. - -`event.kind` gives high-level information about what type of information the event contains, without being specific to the contents of the event. For example, values of this field distinguish alert events from metric events. - -The value of this field can be used to inform how these kinds of events should be handled. They may warrant different retention, different access control, it may also help understand whether the data is coming in at a regular interval or not. - -type: keyword - - - - -*Important*: The field value must be one of the following: - -alert, asset, enrichment, event, metric, state, pipeline_error, signal - -To learn more about when to use which value, visit the page -<> - - -| core - -// =============================================================== - -| -[[field-event-module]] -<> - -a| Name of the module this data is coming from. - -If your monitoring agent supports the concept of modules or plugins to process events of a given source (e.g. Apache logs), `event.module` should contain the name of this module. - -type: keyword - - - -example: `apache` - -| core - -// =============================================================== - -| -[[field-event-original]] -<> - -a| Raw text message of entire event. Used to demonstrate log integrity or where the full log message (before splitting it up in multiple parts) may be required, e.g. for reindex. - -This field is not indexed and doc_values are disabled. It cannot be searched, but it can be retrieved from `_source`. If users wish to override this and index this field, please see `Field data types` in the `Elasticsearch Reference`. - -type: keyword - - - -example: `Sep 19 08:26:10 host CEF:0|Security| threatmanager|1.0|100| worm successfully stopped|10|src=10.0.0.1 dst=2.1.2.2spt=1232` - -| core - -// =============================================================== - -| -[[field-event-outcome]] -<> - -a| This is one of four ECS Categorization Fields, and indicates the lowest level in the ECS category hierarchy. - -`event.outcome` simply denotes whether the event represents a success or a failure from the perspective of the entity that produced the event. - -Note that when a single transaction is described in multiple events, each event may populate different values of `event.outcome`, according to their perspective. - -Also note that in the case of a compound event (a single event that contains multiple logical events), this field should be populated with the value that best captures the overall success or failure from the perspective of the event producer. - -Further note that not all events will have an associated outcome. For example, this field is generally not populated for metric events, events with `event.type:info`, or any events for which an outcome does not make logical sense. - -type: keyword - - - - -*Important*: The field value must be one of the following: - -failure, success, unknown - -To learn more about when to use which value, visit the page -<> - - -| core - -// =============================================================== - -| -[[field-event-provider]] -<> - -a| Source of the event. - -Event transports such as Syslog or the Windows Event Log typically mention the source of an event. It can be the name of the software that generated the event (e.g. Sysmon, httpd), or of a subsystem of the operating system (kernel, Microsoft-Windows-Security-Auditing). - -type: keyword - - - -example: `kernel` - -| extended - -// =============================================================== - -| -[[field-event-reason]] -<> - -a| Reason why this event happened, according to the source. - -This describes the why of a particular action or outcome captured in the event. Where `event.action` captures the action from the event, `event.reason` describes why that action was taken. For example, a web proxy with an `event.action` which denied the request may also populate `event.reason` with the reason why (e.g. `blocked site`). - -type: keyword - - - -example: `Terminated an unexpected process` - -| extended - -// =============================================================== - -| -[[field-event-reference]] -<> - -a| Reference URL linking to additional information about this event. - -This URL links to a static definition of this event. Alert events, indicated by `event.kind:alert`, are a common use case for this field. - -type: keyword - - - -example: `https://system.example.com/event/#0001234` - -| extended - -// =============================================================== - -| -[[field-event-risk-score]] -<> - -a| Risk score or priority of the event (e.g. security solutions). Use your system's original value here. - -type: float - - - - - -| core - -// =============================================================== - -| -[[field-event-risk-score-norm]] -<> - -a| Normalized risk score or priority of the event, on a scale of 0 to 100. - -This is mainly useful if you use more than one system that assigns risk scores, and you want to see a normalized value across all systems. - -type: float - - - - - -| extended - -// =============================================================== - -| -[[field-event-sequence]] -<> - -a| Sequence number of the event. - -The sequence number is a value published by some event sources, to make the exact ordering of events unambiguous, regardless of the timestamp precision. - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-event-severity]] -<> - -a| The numeric severity of the event according to your event source. - -What the different severity values mean can be different between sources and use cases. It's up to the implementer to make sure severities are consistent across events from the same source. - -The Syslog severity belongs in `log.syslog.severity.code`. `event.severity` is meant to represent the severity according to the event source (e.g. firewall, IDS). If the event source does not publish its own severity, you may optionally copy the `log.syslog.severity.code` to `event.severity`. - -type: long - - - -example: `7` - -| core - -// =============================================================== - -| -[[field-event-start]] -<> - -a| `event.start` contains the date when the event started or when the activity was first observed. - -type: date - - - - - -| extended - -// =============================================================== - -| -[[field-event-timezone]] -<> - -a| This field should be populated when the event's timestamp does not include timezone information already (e.g. default Syslog timestamps). It's optional otherwise. - -Acceptable timezone formats are: a canonical ID (e.g. "Europe/Amsterdam"), abbreviated (e.g. "EST") or an HH:mm differential (e.g. "-05:00"). - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-event-type]] -<> - -a| This is one of four ECS Categorization Fields, and indicates the third level in the ECS category hierarchy. - -`event.type` represents a categorization "sub-bucket" that, when used along with the `event.category` field values, enables filtering events down to a level appropriate for single visualization. - -This field is an array. This will allow proper categorization of some events that fall in multiple event types. - -type: keyword - - -Note: this field should contain an array of values. - - - - -*Important*: The field value must be one of the following: - -access, admin, allowed, change, connection, creation, deletion, denied, end, error, group, indicator, info, installation, protocol, start, user - -To learn more about when to use which value, visit the page -<> - - -| core - -// =============================================================== - -| -[[field-event-url]] -<> - -a| URL linking to an external system to continue investigation of this event. - -This URL links to another system where in-depth investigation of the specific occurrence of this event can take place. Alert events, indicated by `event.kind:alert`, are a common use case for this field. - -type: keyword - - - -example: `https://mysystem.example.com/alert/5271dedb-f5b0-4218-87f0-4ac4870a38fe` - -| extended - -// =============================================================== - -|===== - - -[[ecs-faas]] -=== FaaS Fields - -The user fields describe information about the function as a service (FaaS) that is relevant to the event. - -[discrete] -==== FaaS Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-faas-coldstart]] -<> - -a| Boolean value indicating a cold start of a function. - -type: boolean - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/faas/#faas-coldstart[faas.coldstart] - -| extended - -// =============================================================== - -| -[[field-faas-execution]] -<> - -a| The execution ID of the current function execution. - -type: keyword - - - -example: `af9d5aa4-a685-4c5f-a22b-444f80b3cc28` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/faas/#faas-invocation-id[faas.invocation_id] - -| extended - -// =============================================================== - -| -[[field-faas-id]] -<> - -a| The unique identifier of a serverless function. - -For AWS Lambda it's the function ARN (Amazon Resource Name) without a version or alias suffix. - -type: keyword - - - -example: `arn:aws:lambda:us-west-2:123456789012:function:my-function` - -| extended - -// =============================================================== - -| -[[field-faas-name]] -<> - -a| The name of a serverless function. - -type: keyword - - - -example: `my-function` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/faas/#faas-name[faas.name] - -| extended - -// =============================================================== - -| -[[field-faas-trigger-request-id]] -<> - -a| The ID of the trigger request , message, event, etc. - -type: keyword - - - -example: `123456789` - -| extended - -// =============================================================== - -| -[[field-faas-trigger-type]] -<> - -a| The trigger for the function execution. - -Expected values for this field: - -* `http` -* `pubsub` -* `datasource` -* `timer` -* `other` - -type: keyword - - - -example: `http` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/faas/#faas-trigger[faas.trigger] - -| extended - -// =============================================================== - -| -[[field-faas-version]] -<> - -a| The version of a serverless function. - -type: keyword - - - -example: `123` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/faas/#faas-version[faas.version] - -| extended - -// =============================================================== - -|===== - - -[[ecs-file]] -=== File Fields - -A file is defined as a set of information that has been created on, or has existed on a filesystem. - -File objects can be associated with host events, network events, and/or file events (e.g., those produced by File Integrity Monitoring [FIM] products or services). File fields provide details about the affected file associated with the event or metric. - -[discrete] -==== File Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-file-accessed]] -<> - -a| Last time the file was accessed. - -Note that not all filesystems keep track of access time. - -type: date - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-accessed[file.accessed] - -| extended - -// =============================================================== - -| -[[field-file-attributes]] -<> - -a| Array of file attributes. - -Attributes names will vary by platform. Here's a non-exhaustive list of values that are expected in this field: archive, compressed, directory, encrypted, execute, hidden, read, readonly, system, write. - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `["readonly", "system"]` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-attributes[file.attributes] - -| extended - -// =============================================================== - -| -[[field-file-created]] -<> - -a| File creation time. - -Note that not all filesystems store the creation time. - -type: date - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-created[file.created] - -| extended - -// =============================================================== - -| -[[field-file-ctime]] -<> - -a| Last time the file attributes or metadata changed. - -Note that changes to the file content will update `mtime`. This implies `ctime` will be adjusted at the same time, since `mtime` is an attribute of the file. - -type: date - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-changed[file.changed] - -| extended - -// =============================================================== - -| -[[field-file-device]] -<> - -a| Device that is the source of the file. - -type: keyword - - - -example: `sda` - -| extended - -// =============================================================== - -| -[[field-file-directory]] -<> - -a| Directory where the file is located. It should include the drive letter, when appropriate. - -type: keyword - - - -example: `/home/alice` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-directory[file.directory] - -| extended - -// =============================================================== - -| -[[field-file-drive-letter]] -<> - -a| Drive letter where the file is located. This field is only relevant on Windows. - -The value should be uppercase, and not include the colon. - -type: keyword - - - -example: `C` - -| extended - -// =============================================================== - -| -[[field-file-extension]] -<> - -a| File extension, excluding the leading dot. - -Note that when the file name has multiple extensions (example.tar.gz), only the last one should be captured ("gz", not "tar.gz"). - -type: keyword - - - -example: `png` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-extension[file.extension] - -| extended - -// =============================================================== - -| -[[field-file-fork-name]] -<> - -a| A fork is additional data associated with a filesystem object. - -On Linux, a resource fork is used to store additional data with a filesystem object. A file always has at least one fork for the data portion, and additional forks may exist. - -On NTFS, this is analogous to an Alternate Data Stream (ADS), and the default data stream for a file is just called $DATA. Zone.Identifier is commonly used by Windows to track contents downloaded from the Internet. An ADS is typically of the form: `C:\path\to\filename.extension:some_fork_name`, and `some_fork_name` is the value that should populate `fork_name`. `filename.extension` should populate `file.name`, and `extension` should populate `file.extension`. The full path, `file.path`, will include the fork name. - -type: keyword - - - -example: `Zone.Identifer` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-fork-name[file.fork_name] - -| extended - -// =============================================================== - -| -[[field-file-gid]] -<> - -a| Primary group ID (GID) of the file. - -type: keyword - - - -example: `1001` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-group-id[file.group.id] - -| extended - -// =============================================================== - -| -[[field-file-group]] -<> - -a| Primary group name of the file. - -type: keyword - - - -example: `alice` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-group-name[file.group.name] - -| extended - -// =============================================================== - -| -[[field-file-inode]] -<> - -a| Inode representing the file in the filesystem. - -type: keyword - - - -example: `256383` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-inode[file.inode] - -| extended - -// =============================================================== - -| -[[field-file-mime-type]] -<> - -a| MIME type should identify the format of the file or stream of bytes using https://www.iana.org/assignments/media-types/media-types.xhtml[IANA official types], where possible. When more than one type is applicable, the most specific type should be used. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-file-mode]] -<> - -a| Mode of the file in octal representation. - -type: keyword - - - -example: `0640` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-mode[file.mode] - -| extended - -// =============================================================== - -| -[[field-file-mtime]] -<> - -a| Last time the file content was modified. - -type: date - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-modified[file.modified] - -| extended - -// =============================================================== - -| -[[field-file-name]] -<> - -a| Name of the file including the extension, without the directory. - -type: keyword - - - -example: `example.png` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-name[file.name] - -| extended - -// =============================================================== - -| -[[field-file-origin-referrer-url]] -<> - -a| beta:[ This field is beta and subject to change. ] - -The URL of the webpage that linked to the file. - -type: keyword - - - -example: `http://example.com/article1.html` - -| extended - -// =============================================================== - -| -[[field-file-origin-url]] -<> - -a| beta:[ This field is beta and subject to change. ] - -The URL where the file is hosted. - -type: keyword - - - -example: `http://example.com/imgs/article1_img1.jpg` - -| extended - -// =============================================================== - -| -[[field-file-owner]] -<> - -a| File owner's username. - -type: keyword - - - -example: `alice` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-owner-name[file.owner.name] - -| extended - -// =============================================================== - -| -[[field-file-path]] -<> - -a| Full path to the file, including the file name. It should include the drive letter, when appropriate. - -type: keyword - -Multi-fields: - -* file.path.text (type: match_only_text) - - - - - -example: `/home/alice/example.png` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-path[file.path] - -| extended - -// =============================================================== - -| -[[field-file-size]] -<> - -a| File size in bytes. - -Only relevant when `file.type` is "file". - -type: long - - - -example: `16384` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-size[file.size] - -| extended - -// =============================================================== - -| -[[field-file-target-path]] -<> - -a| Target path for symlinks. - -type: keyword - -Multi-fields: - -* file.target_path.text (type: match_only_text) - - - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-symbolic-link-target-path[file.symbolic_link.target_path] - -| extended - -// =============================================================== - -| -[[field-file-type]] -<> - -a| File type (file, dir, or symlink). - -type: keyword - - - -example: `file` - -| extended - -// =============================================================== - -| -[[field-file-uid]] -<> - -a| The user ID (UID) or security identifier (SID) of the file owner. - -type: keyword - - - -example: `1001` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-owner-id[file.owner.id] - -| extended - -// =============================================================== - -|===== - -[discrete] -==== Field Reuse - -The `file` fields are expected to be nested at: - - -* `threat.enrichments.indicator.file` - -* `threat.indicator.file` - - -Note also that the `file` fields may be used directly at the root of the events. - -[[ecs-file-nestings]] -[discrete] -===== Field sets that can be nested under File - -[options="header"] -|===== -| Location | Field Set | Description - -// =============================================================== - - -| `file.code_signature.*` -| <> -| These fields contain information about binary code signatures. - -// =============================================================== - - -| `file.elf.*` -| <> -| These fields contain Linux Executable Linkable Format (ELF) metadata. - -// =============================================================== - - -| `file.hash.*` -| <> -| Hashes, usually file hashes. - -// =============================================================== - - -| `file.macho.*` -| <>| beta:[ This field reuse is beta and subject to change.] - -These fields contain Mac OS Mach Object file format (Mach-O) metadata. - -// =============================================================== - - -| `file.pe.*` -| <> -| These fields contain Windows Portable Executable (PE) metadata. - -// =============================================================== - - -| `file.x509.*` -| <> -| These fields contain x509 certificate metadata. - -// =============================================================== - - -|===== - - -[[ecs-geo]] -=== Geo Fields - -Geo fields can carry data about a specific location related to an event. - -This geolocation information can be derived from techniques such as Geo IP, or be user-supplied. - -[discrete] -==== Geo Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-geo-city-name]] -<> - -a| City name. - -type: keyword - - - -example: `Montreal` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/geo/#geo-locality-name[geo.locality.name] - -| core - -// =============================================================== - -| -[[field-geo-continent-code]] -<> - -a| Two-letter code representing continent's name. - -type: keyword - - - -example: `NA` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/geo/#geo-continent-code[geo.continent.code] - -| core - -// =============================================================== - -| -[[field-geo-continent-name]] -<> - -a| Name of the continent. - -type: keyword - - - -example: `North America` - -| core - -// =============================================================== - -| -[[field-geo-country-iso-code]] -<> - -a| Country ISO code. - -type: keyword - - - -example: `CA` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/geo/#geo-country-iso-code[geo.country.iso_code] - -| core - -// =============================================================== - -| -[[field-geo-country-name]] -<> - -a| Country name. - -type: keyword - - - -example: `Canada` - -| core - -// =============================================================== - -| -[[field-geo-location]] -<> - -a| Longitude and latitude. - -type: geo_point - - - -example: `{ "lon": -73.614830, "lat": 45.505918 }` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/related-efc20d?style=flat[relation,title=related,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/geo/#geo-location-lat[geo.location.lat] - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/related-efc20d?style=flat[relation,title=related,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/geo/#geo-location-lon[geo.location.lon] - -| core - -// =============================================================== - -| -[[field-geo-name]] -<> - -a| User-defined description of a location, at the level of granularity they care about. - -Could be the name of their data centers, the floor number, if this describes a local physical entity, city names. - -Not typically used in automated geolocation. - -type: keyword - - - -example: `boston-dc` - -| extended - -// =============================================================== - -| -[[field-geo-postal-code]] -<> - -a| Postal code associated with the location. - -Values appropriate for this field may also be known as a postcode or ZIP code and will vary widely from country to country. - -type: keyword - - - -example: `94040` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/geo/#geo-postal-code[geo.postal_code] - -| core - -// =============================================================== - -| -[[field-geo-region-iso-code]] -<> - -a| Region ISO code. - -type: keyword - - - -example: `CA-QC` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/geo/#geo-region-iso-code[geo.region.iso_code] - -| core - -// =============================================================== - -| -[[field-geo-region-name]] -<> - -a| Region name. - -type: keyword - - - -example: `Quebec` - -| core - -// =============================================================== - -| -[[field-geo-timezone]] -<> - -a| The time zone of the location, such as IANA time zone name. - -type: keyword - - - -example: `America/Argentina/Buenos_Aires` - -| core - -// =============================================================== - -|===== - -[discrete] -==== Field Reuse - -The `geo` fields are expected to be nested at: - - -* `client.geo` - -* `destination.geo` - -* `host.geo` - -* `observer.geo` - -* `server.geo` - -* `source.geo` - -* `threat.enrichments.indicator.geo` - -* `threat.indicator.geo` - - -Note also that the `geo` fields are not expected to be used directly at the root of the events. -[[ecs-group]] -=== Group Fields - -The group fields are meant to represent groups that are relevant to the event. - -[discrete] -==== Group Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-group-domain]] -<> - -a| Name of the directory the group is a member of. - -For example, an LDAP or Active Directory domain name. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-group-id]] -<> - -a| Unique identifier for the group on the system/platform. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-group-name]] -<> - -a| Name of the group. - -type: keyword - - - - - -| extended - -// =============================================================== - -|===== - -[discrete] -==== Field Reuse - -The `group` fields are expected to be nested at: - - -* `process.attested_groups` - -* `process.group` - -* `process.real_group` - -* `process.saved_group` - -* `process.supplemental_groups` - -* `user.group` - - -Note also that the `group` fields may be used directly at the root of the events. - - -[[ecs-hash]] -=== Hash Fields - -The hash fields represent different bitwise hash algorithms and their values. - -Field names for common hashes (e.g. MD5, SHA1) are predefined. Add fields for other hashes by lowercasing the hash algorithm name and using underscore separators as appropriate (snake case, e.g. sha3_512). - -Note that this fieldset is used for common hashes that may be computed over a range of generic bytes. Entity-specific hashes such as ja3 or imphash are placed in the fieldsets to which they relate (tls and pe, respectively). - -[discrete] -==== Hash Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-hash-cdhash]] -<> - -a| beta:[ This field is beta and subject to change. ] - -Code directory hash, utilized to uniquely identify and authenticate the integrity of the executable code. - -type: keyword - - - -example: `3783b4052fd474dbe30676b45c329e7a6d44acd9` - -| extended - -// =============================================================== - -| -[[field-hash-md5]] -<> - -a| MD5 hash. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-hash-sha1]] -<> - -a| SHA1 hash. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-hash-sha256]] -<> - -a| SHA256 hash. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-hash-sha384]] -<> - -a| SHA384 hash. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-hash-sha512]] -<> - -a| SHA512 hash. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-hash-ssdeep]] -<> - -a| SSDEEP hash. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-hash-tlsh]] -<> - -a| TLSH hash. - -type: keyword - - - - - -| extended - -// =============================================================== - -|===== - -[discrete] -==== Field Reuse - -The `hash` fields are expected to be nested at: - - -* `dll.hash` - -* `email.attachments.file.hash` - -* `file.hash` - -* `process.hash` - - -Note also that the `hash` fields are not expected to be used directly at the root of the events. -[[ecs-host]] -=== Host Fields - -A host is defined as a general computing instance. - -ECS host.* fields should be populated with details about the host on which the event happened, or from which the measurement was taken. Host types include hardware, virtual machines, Docker containers, and Kubernetes nodes. - -[discrete] -==== Host Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-host-architecture]] -<> - -a| Operating system architecture. - -type: keyword - - - -example: `x86_64` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/host/#host-arch[host.arch] - -| core - -// =============================================================== - -| -[[field-host-boot-id]] -<> - -a| Linux boot uuid taken from /proc/sys/kernel/random/boot_id. Note the boot_id value from /proc may or may not be the same in containers as on the host. Some container runtimes will bind mount a new boot_id value onto the proc file in each container. - -type: keyword - - - -example: `88a1f0ed-5ae5-41ee-af6b-41921c311872` - -| extended - -// =============================================================== - -| -[[field-host-cpu-usage]] -<> - -a| Percent CPU used which is normalized by the number of CPU cores and it ranges from 0 to 1. - -Scaling factor: 1000. - -For example: For a two core host, this value should be the average of the two cores, between 0 and 1. - -type: scaled_float - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/metric-cb00cb?style=flat[relation,title=metric,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--\+semconv+metric.system.cpu.utilization+--%3E%22&type=code[system.cpu.utilization] - -| extended - -// =============================================================== - -| -[[field-host-disk-read-bytes]] -<> - -a| The total number of bytes (gauge) read successfully (aggregated from all disks) since the last metric collection. - -type: long - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/metric-cb00cb?style=flat[relation,title=metric,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--\+semconv+metric.system.disk.io+--%3E%22&type=code[system.disk.io] - -| extended - -// =============================================================== - -| -[[field-host-disk-write-bytes]] -<> - -a| The total number of bytes (gauge) written successfully (aggregated from all disks) since the last metric collection. - -type: long - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/metric-cb00cb?style=flat[relation,title=metric,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--\+semconv+metric.system.disk.io+--%3E%22&type=code[system.disk.io] - -| extended - -// =============================================================== - -| -[[field-host-domain]] -<> - -a| Name of the domain of which the host is a member. - -For example, on Windows this could be the host's Active Directory domain or NetBIOS domain name. For Linux this could be the domain of the host's LDAP provider. - -type: keyword - - - -example: `CONTOSO` - -| extended - -// =============================================================== - -| -[[field-host-hostname]] -<> - -a| Hostname of the host. - -It normally contains what the `hostname` command returns on the host machine. - -type: keyword - - - - - -| core - -// =============================================================== - -| -[[field-host-id]] -<> - -a| Unique host id. - -As hostname is not always unique, use values that are meaningful in your environment. - -Example: The current usage of `beat.name`. - -type: keyword - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/host/#host-id[host.id] - -| core - -// =============================================================== - -| -[[field-host-ip]] -<> - -a| Host ip addresses. - -type: ip - - -Note: this field should contain an array of values. - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/host/#host-ip[host.ip] - -| core - -// =============================================================== - -| -[[field-host-mac]] -<> - -a| Host MAC addresses. - -The notation format from RFC 7042 is suggested: Each octet (that is, 8-bit byte) is represented by two [uppercase] hexadecimal digits giving the value of the octet as an unsigned integer. Successive octets are separated by a hyphen. - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `["00-00-5E-00-53-23", "00-00-5E-00-53-24"]` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/host/#host-mac[host.mac] - -| core - -// =============================================================== - -| -[[field-host-name]] -<> - -a| Name of the host. - -It can contain what hostname returns on Unix systems, the fully qualified domain name (FQDN), or a name specified by the user. The recommended value is the lowercase FQDN of the host. - -type: keyword - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/host/#host-name[host.name] - -| core - -// =============================================================== - -| -[[field-host-network-egress-bytes]] -<> - -a| The number of bytes (gauge) sent out on all network interfaces by the host since the last metric collection. - -type: long - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/metric-cb00cb?style=flat[relation,title=metric,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--\+semconv+metric.system.network.io+--%3E%22&type=code[system.network.io] - -| extended - -// =============================================================== - -| -[[field-host-network-egress-packets]] -<> - -a| The number of packets (gauge) sent out on all network interfaces by the host since the last metric collection. - -type: long - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/metric-cb00cb?style=flat[relation,title=metric,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--\+semconv+metric.system.network.packets+--%3E%22&type=code[system.network.packets] - -| extended - -// =============================================================== - -| -[[field-host-network-ingress-bytes]] -<> - -a| The number of bytes received (gauge) on all network interfaces by the host since the last metric collection. - -type: long - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/metric-cb00cb?style=flat[relation,title=metric,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--\+semconv+metric.system.network.io+--%3E%22&type=code[system.network.io] - -| extended - -// =============================================================== - -| -[[field-host-network-ingress-packets]] -<> - -a| The number of packets (gauge) received on all network interfaces by the host since the last metric collection. - -type: long - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/metric-cb00cb?style=flat[relation,title=metric,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--\+semconv+metric.system.network.packets+--%3E%22&type=code[system.network.packets] - -| extended - -// =============================================================== - -| -[[field-host-pid-ns-ino]] -<> - -a| This is the inode number of the namespace in the namespace file system (nsfs). Unsigned int inum in include/linux/ns_common.h. - -type: keyword - - - -example: `256383` - -| extended - -// =============================================================== - -| -[[field-host-type]] -<> - -a| Type of host. - -For Cloud providers this can be the machine type like `t2.medium`. If vm, this could be the container, for example, or other information meaningful in your environment. - -type: keyword - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/host/#host-type[host.type] - -| core - -// =============================================================== - -| -[[field-host-uptime]] -<> - -a| Seconds the host has been up. - -type: long - - - -example: `1325` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/metric-cb00cb?style=flat[relation,title=metric,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--\+semconv+metric.system.uptime+--%3E%22&type=code[system.uptime] - -| extended - -// =============================================================== - -|===== - -[discrete] -==== Field Reuse - -[[ecs-host-nestings]] -[discrete] -===== Field sets that can be nested under Host - -[options="header"] -|===== -| Location | Field Set | Description - -// =============================================================== - - -| `host.geo.*` -| <> -| Fields describing a location. - -// =============================================================== - - -| `host.os.*` -| <> -| OS fields contain information about the operating system. - -// =============================================================== - - -| `host.risk.*` -| <> -| Fields for describing risk score and level. - -// =============================================================== - - -|===== - - -[[ecs-http]] -=== HTTP Fields - -Fields related to HTTP activity. Use the `url` field set to store the url of the request. - -[discrete] -==== HTTP Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-http-request-body-bytes]] -<> - -a| Size in bytes of the request body. - -type: long - - - -example: `887` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/http/#http-request-body-size[http.request.body.size] - -| extended - -// =============================================================== - -| -[[field-http-request-body-content]] -<> - -a| The full HTTP request body. - -type: wildcard - -Multi-fields: - -* http.request.body.content.text (type: match_only_text) - - - - - -example: `Hello world` - -| extended - -// =============================================================== - -| -[[field-http-request-bytes]] -<> - -a| Total size in bytes of the request (body and headers). - -type: long - - - -example: `1437` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/http/#http-request-size[http.request.size] - -| extended - -// =============================================================== - -| -[[field-http-request-id]] -<> - -a| A unique identifier for each HTTP request to correlate logs between clients and servers in transactions. - -The id may be contained in a non-standard HTTP header, such as `X-Request-ID` or `X-Correlation-ID`. - -type: keyword - - - -example: `123e4567-e89b-12d3-a456-426614174000` - -| extended - -// =============================================================== - -| -[[field-http-request-method]] -<> - -a| HTTP request method. - -The value should retain its casing from the original event. For example, `GET`, `get`, and `GeT` are all considered valid values for this field. - -type: keyword - - - -example: `POST` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/http/#http-request-method-original[http.request.method_original] - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/conflict-910000?style=flat[relation,title=conflict,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/http/#http-request-method[http.request.method] - -Note: `http.request.method` in SemConv is the known, normalized, upper case value of the request method, other than the ECS' `http.request.method` that retains casing from the original event. - - -| extended - -// =============================================================== - -| -[[field-http-request-mime-type]] -<> - -a| Mime type of the body of the request. - -This value must only be populated based on the content of the request body, not on the `Content-Type` header. Comparing the mime type of a request with the request's Content-Type header can be helpful in detecting threats or misconfigured clients. - -type: keyword - - - -example: `image/gif` - -| extended - -// =============================================================== - -| -[[field-http-request-referrer]] -<> - -a| Referrer for this HTTP request. - -type: keyword - - - -example: `https://blog.example.com/` - -| extended - -// =============================================================== - -| -[[field-http-response-body-bytes]] -<> - -a| Size in bytes of the response body. - -type: long - - - -example: `887` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/http/#http-response-body-size[http.response.body.size] - -| extended - -// =============================================================== - -| -[[field-http-response-body-content]] -<> - -a| The full HTTP response body. - -type: wildcard - -Multi-fields: - -* http.response.body.content.text (type: match_only_text) - - - - - -example: `Hello world` - -| extended - -// =============================================================== - -| -[[field-http-response-bytes]] -<> - -a| Total size in bytes of the response (body and headers). - -type: long - - - -example: `1437` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/http/#http-response-size[http.response.size] - -| extended - -// =============================================================== - -| -[[field-http-response-mime-type]] -<> - -a| Mime type of the body of the response. - -This value must only be populated based on the content of the response body, not on the `Content-Type` header. Comparing the mime type of a response with the response's Content-Type header can be helpful in detecting misconfigured servers. - -type: keyword - - - -example: `image/gif` - -| extended - -// =============================================================== - -| -[[field-http-response-status-code]] -<> - -a| HTTP response status code. - -type: long - - - -example: `404` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/http/#http-response-status-code[http.response.status_code] - -| extended - -// =============================================================== - -| -[[field-http-version]] -<> - -a| HTTP version. - -type: keyword - - - -example: `1.1` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/related-efc20d?style=flat[relation,title=related,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/network/#network-protocol-name[network.protocol.name] - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/related-efc20d?style=flat[relation,title=related,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/network/#network-protocol-version[network.protocol.version] - -Note: In OTel SemConv, `network.protocol.version` specifies the HTTP version if the value of `network.protocol.name` is `http`. - - -| extended - -// =============================================================== - -|===== - - -[[ecs-interface]] -=== Interface Fields - -The interface fields are used to record ingress and egress interface information when reported by an observer (e.g. firewall, router, load balancer) in the context of the observer handling a network connection. In the case of a single observer interface (e.g. network sensor on a span port) only the observer.ingress information should be populated. - -[discrete] -==== Interface Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-interface-alias]] -<> - -a| Interface alias as reported by the system, typically used in firewall implementations for e.g. inside, outside, or dmz logical interface naming. - -type: keyword - - - -example: `outside` - -| extended - -// =============================================================== - -| -[[field-interface-id]] -<> - -a| Interface ID as reported by an observer (typically SNMP interface ID). - -type: keyword - - - -example: `10` - -| extended - -// =============================================================== - -| -[[field-interface-name]] -<> - -a| Interface name as reported by the system. - -type: keyword - - - -example: `eth0` - -| extended - -// =============================================================== - -|===== - -[discrete] -==== Field Reuse - -The `interface` fields are expected to be nested at: - - -* `observer.egress.interface` - -* `observer.ingress.interface` - - -Note also that the `interface` fields are not expected to be used directly at the root of the events. -[[ecs-log]] -=== Log Fields - -Details about the event's logging mechanism or logging transport. - -The log.* fields are typically populated with details about the logging mechanism used to create and/or transport the event. For example, syslog details belong under `log.syslog.*`. - -The details specific to your event source are typically not logged under `log.*`, but rather in `event.*` or in other ECS fields. - -[discrete] -==== Log Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-log-file-path]] -<> - -a| Full path to the log file this event came from, including the file name. It should include the drive letter, when appropriate. - -If the event wasn't read from a log file, do not populate this field. - -type: keyword - - - -example: `/var/log/fun-times.log` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/log/#log-file-path[log.file.path] - -| extended - -// =============================================================== - -| -[[field-log-level]] -<> - -a| Original log level of the log event. - -If the source of the event provides a log level or textual severity, this is the one that goes in `log.level`. If your source doesn't specify one, you may put your event transport's severity here (e.g. Syslog severity). - -Some examples are `warn`, `err`, `i`, `informational`. - -type: keyword - - - -example: `error` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/OTLP-ffdcb2?style=flat[relation,title=otlp,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://github.com/search?q=repo%3Aopen-telemetry%2Fopentelemetry-proto+%22\+severity_text+%22&type=code[severity_text] - -| core - -// =============================================================== - -| -[[field-log-logger]] -<> - -a| The name of the logger inside an application. This is usually the name of the class which initialized the logger, or can be a custom name. - -type: keyword - - - -example: `org.elasticsearch.bootstrap.Bootstrap` - -| core - -// =============================================================== - -| -[[field-log-origin-file-line]] -<> - -a| The line number of the file containing the source code which originated the log event. - -type: long - - - -example: `42` - -| extended - -// =============================================================== - -| -[[field-log-origin-file-name]] -<> - -a| The name of the file containing the source code which originated the log event. - -Note that this field is not meant to capture the log file. The correct field to capture the log file is `log.file.path`. - -type: keyword - - - -example: `Bootstrap.java` - -| extended - -// =============================================================== - -| -[[field-log-origin-function]] -<> - -a| The name of the function or method which originated the log event. - -type: keyword - - - -example: `init` - -| extended - -// =============================================================== - -| -[[field-log-syslog]] -<> - -a| The Syslog metadata of the event, if the event was transmitted via Syslog. Please see RFCs 5424 or 3164. - -type: object - - - - - -| extended - -// =============================================================== - -| -[[field-log-syslog-appname]] -<> - -a| The device or application that originated the Syslog message, if available. - -type: keyword - - - -example: `sshd` - -| extended - -// =============================================================== - -| -[[field-log-syslog-facility-code]] -<> - -a| The Syslog numeric facility of the log event, if available. - -According to RFCs 5424 and 3164, this value should be an integer between 0 and 23. - -type: long - - - -example: `23` - -| extended - -// =============================================================== - -| -[[field-log-syslog-facility-name]] -<> - -a| The Syslog text-based facility of the log event, if available. - -type: keyword - - - -example: `local7` - -| extended - -// =============================================================== - -| -[[field-log-syslog-hostname]] -<> - -a| The hostname, FQDN, or IP of the machine that originally sent the Syslog message. This is sourced from the hostname field of the syslog header. Depending on the environment, this value may be different from the host that handled the event, especially if the host handling the events is acting as a collector. - -type: keyword - - - -example: `example-host` - -| extended - -// =============================================================== - -| -[[field-log-syslog-msgid]] -<> - -a| An identifier for the type of Syslog message, if available. Only applicable for RFC 5424 messages. - -type: keyword - - - -example: `ID47` - -| extended - -// =============================================================== - -| -[[field-log-syslog-priority]] -<> - -a| Syslog numeric priority of the event, if available. - -According to RFCs 5424 and 3164, the priority is 8 * facility + severity. This number is therefore expected to contain a value between 0 and 191. - -type: long - - - -example: `135` - -| extended - -// =============================================================== - -| -[[field-log-syslog-procid]] -<> - -a| The process name or ID that originated the Syslog message, if available. - -type: keyword - - - -example: `12345` - -| extended - -// =============================================================== - -| -[[field-log-syslog-severity-code]] -<> - -a| The Syslog numeric severity of the log event, if available. - -If the event source publishing via Syslog provides a different numeric severity value (e.g. firewall, IDS), your source's numeric severity should go to `event.severity`. If the event source does not specify a distinct severity, you can optionally copy the Syslog severity to `event.severity`. - -type: long - - - -example: `3` - -| extended - -// =============================================================== - -| -[[field-log-syslog-severity-name]] -<> - -a| The Syslog numeric severity of the log event, if available. - -If the event source publishing via Syslog provides a different severity value (e.g. firewall, IDS), your source's text severity should go to `log.level`. If the event source does not specify a distinct severity, you can optionally copy the Syslog severity to `log.level`. - -type: keyword - - - -example: `Error` - -| extended - -// =============================================================== - -| -[[field-log-syslog-structured-data]] -<> - -a| Structured data expressed in RFC 5424 messages, if available. These are key-value pairs formed from the structured data portion of the syslog message, as defined in RFC 5424 Section 6.3. - -type: flattened - - - - - -| extended - -// =============================================================== - -| -[[field-log-syslog-version]] -<> - -a| The version of the Syslog protocol specification. Only applicable for RFC 5424 messages. - -type: keyword - - - -example: `1` - -| extended - -// =============================================================== - -|===== - - -[[ecs-macho]] -=== Mach-O Header Fields - -These fields contain Mac OS Mach Object file format (Mach-O) metadata. - -beta::[ These fields are in beta and are subject to change.] - -[discrete] -==== Mach-O Header Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-macho-go-import-hash]] -<> - -a| A hash of the Go language imports in a Mach-O file excluding standard library imports. An import hash can be used to fingerprint binaries even after recompilation or other code-level transformations have occurred, which would change more traditional hash values. - -The algorithm used to calculate the Go symbol hash and a reference implementation are available here: https://github.com/elastic/toutoumomoma - -type: keyword - - - -example: `10bddcb4cee42080f76c88d9ff964491` - -| extended - -// =============================================================== - -| -[[field-macho-go-imports]] -<> - -a| List of imported Go language element names and types. - -type: flattened - - - - - -| extended - -// =============================================================== - -| -[[field-macho-go-imports-names-entropy]] -<> - -a| Shannon entropy calculation from the list of Go imports. - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-macho-go-imports-names-var-entropy]] -<> - -a| Variance for Shannon entropy calculation from the list of Go imports. - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-macho-go-stripped]] -<> - -a| Set to true if the file is a Go executable that has had its symbols stripped or obfuscated and false if an unobfuscated Go executable. - -type: boolean - - - - - -| extended - -// =============================================================== - -| -[[field-macho-import-hash]] -<> - -a| A hash of the imports in a Mach-O file. An import hash can be used to fingerprint binaries even after recompilation or other code-level transformations have occurred, which would change more traditional hash values. - -This is a synonym for symhash. - -type: keyword - - - -example: `d41d8cd98f00b204e9800998ecf8427e` - -| extended - -// =============================================================== - -| -[[field-macho-imports]] -<> - -a| List of imported element names and types. - -type: flattened - - -Note: this field should contain an array of values. - - - - - -| extended - -// =============================================================== - -| -[[field-macho-imports-names-entropy]] -<> - -a| Shannon entropy calculation from the list of imported element names and types. - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-macho-imports-names-var-entropy]] -<> - -a| Variance for Shannon entropy calculation from the list of imported element names and types. - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-macho-sections]] -<> - -a| An array containing an object for each section of the Mach-O file. - -The keys that should be present in these objects are defined by sub-fields underneath `macho.sections.*`. - -type: nested - - -Note: this field should contain an array of values. - - - - - -| extended - -// =============================================================== - -| -[[field-macho-sections-entropy]] -<> - -a| Shannon entropy calculation from the section. - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-macho-sections-name]] -<> - -a| Mach-O Section List name. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-macho-sections-physical-size]] -<> - -a| Mach-O Section List physical size. - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-macho-sections-var-entropy]] -<> - -a| Variance for Shannon entropy calculation from the section. - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-macho-sections-virtual-size]] -<> - -a| Mach-O Section List virtual size. This is always the same as `physical_size`. - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-macho-symhash]] -<> - -a| A hash of the imports in a Mach-O file. An import hash can be used to fingerprint binaries even after recompilation or other code-level transformations have occurred, which would change more traditional hash values. - -This is a Mach-O implementation of the Windows PE imphash - -type: keyword - - - -example: `d3ccf195b62a9279c3c19af1080497ec` - -| extended - -// =============================================================== - -|===== - -[discrete] -==== Field Reuse - -The `macho` fields are expected to be nested at: - - -* `file.macho` - -* `process.macho` - - -Note also that the `macho` fields are not expected to be used directly at the root of the events. -[[ecs-network]] -=== Network Fields - -The network is defined as the communication path over which a host or network event happens. - -The network.* fields should be populated with details about the network activity associated with an event. - -[discrete] -==== Network Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-network-application]] -<> - -a| When a specific application or service is identified from network connection details (source/dest IPs, ports, certificates, or wire format), this field captures the application's or service's name. - -For example, the original event identifies the network connection being from a specific web service in a `https` network connection, like `facebook` or `twitter`. - -The field value must be normalized to lowercase for querying. - -type: keyword - - - -example: `aim` - -| extended - -// =============================================================== - -| -[[field-network-bytes]] -<> - -a| Total bytes transferred in both directions. - -If `source.bytes` and `destination.bytes` are known, `network.bytes` is their sum. - -type: long - - - -example: `368` - -| core - -// =============================================================== - -| -[[field-network-community-id]] -<> - -a| A hash of source and destination IPs and ports, as well as the protocol used in a communication. This is a tool-agnostic standard to identify flows. - -Learn more at https://github.com/corelight/community-id-spec. - -type: keyword - - - -example: `1:hO+sN4H+MG5MY/8hIrXPqc4ZQz0=` - -| extended - -// =============================================================== - -| -[[field-network-direction]] -<> - -a| Direction of the network traffic. - -When mapping events from a host-based monitoring context, populate this field from the host's point of view, using the values "ingress" or "egress". - -When mapping events from a network or perimeter-based monitoring context, populate this field from the point of view of the network perimeter, using the values "inbound", "outbound", "internal" or "external". - -Note that "internal" is not crossing perimeter boundaries, and is meant to describe communication between two hosts within the perimeter. Note also that "external" is meant to describe traffic between two hosts that are external to the perimeter. This could for example be useful for ISPs or VPN service providers. - -Expected values for this field: - -* `ingress` -* `egress` -* `inbound` -* `outbound` -* `internal` -* `external` -* `unknown` - -type: keyword - - - -example: `inbound` - -| core - -// =============================================================== - -| -[[field-network-forwarded-ip]] -<> - -a| Host IP address when the source IP address is the proxy. - -type: ip - - - -example: `192.1.1.2` - -| core - -// =============================================================== - -| -[[field-network-iana-number]] -<> - -a| IANA Protocol Number (https://www.iana.org/assignments/protocol-numbers/protocol-numbers.xhtml). Standardized list of protocols. This aligns well with NetFlow and sFlow related logs which use the IANA Protocol Number. - -type: keyword - - - -example: `6` - -| extended - -// =============================================================== - -| -[[field-network-inner]] -<> - -a| Network.inner fields are added in addition to network.vlan fields to describe the innermost VLAN when q-in-q VLAN tagging is present. Allowed fields include vlan.id and vlan.name. Inner vlan fields are typically used when sending traffic with multiple 802.1q encapsulations to a network sensor (e.g. Zeek, Wireshark.) - -type: object - - - - - -| extended - -// =============================================================== - -| -[[field-network-name]] -<> - -a| Name given by operators to sections of their network. - -type: keyword - - - -example: `Guest Wifi` - -| extended - -// =============================================================== - -| -[[field-network-packets]] -<> - -a| Total packets transferred in both directions. - -If `source.packets` and `destination.packets` are known, `network.packets` is their sum. - -type: long - - - -example: `24` - -| core - -// =============================================================== - -| -[[field-network-protocol]] -<> - -a| In the OSI Model this would be the Application Layer protocol. For example, `http`, `dns`, or `ssh`. - -The field value must be normalized to lowercase for querying. - -type: keyword - - - -example: `http` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/network/#network-protocol-name[network.protocol.name] - -| core - -// =============================================================== - -| -[[field-network-transport]] -<> - -a| Same as network.iana_number, but instead using the Keyword name of the transport layer (udp, tcp, ipv6-icmp, etc.) - -The field value must be normalized to lowercase for querying. - -type: keyword - - - -example: `tcp` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/network/#network-transport[network.transport] - -| core - -// =============================================================== - -| -[[field-network-type]] -<> - -a| In the OSI Model this would be the Network Layer. ipv4, ipv6, ipsec, pim, etc - -The field value must be normalized to lowercase for querying. - -type: keyword - - - -example: `ipv4` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/network/#network-type[network.type] - -| core - -// =============================================================== - -|===== - -[discrete] -==== Field Reuse - -[[ecs-network-nestings]] -[discrete] -===== Field sets that can be nested under Network - -[options="header"] -|===== -| Location | Field Set | Description - -// =============================================================== - - -| `network.inner.vlan.*` -| <> -| Fields to describe observed VLAN information. - -// =============================================================== - - -| `network.vlan.*` -| <> -| Fields to describe observed VLAN information. - -// =============================================================== - - -|===== - - -[[ecs-observer]] -=== Observer Fields - -An observer is defined as a special network, security, or application device used to detect, observe, or create network, security, or application-related events and metrics. - -This could be a custom hardware appliance or a server that has been configured to run special network, security, or application software. Examples include firewalls, web proxies, intrusion detection/prevention systems, network monitoring sensors, web application firewalls, data loss prevention systems, and APM servers. The observer.* fields shall be populated with details of the system, if any, that detects, observes and/or creates a network, security, or application event or metric. Message queues and ETL components used in processing events or metrics are not considered observers in ECS. - -[discrete] -==== Observer Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-observer-egress]] -<> - -a| Observer.egress holds information like interface number and name, vlan, and zone information to classify egress traffic. Single armed monitoring such as a network sensor on a span port should only use observer.ingress to categorize traffic. - -type: object - - - - - -| extended - -// =============================================================== - -| -[[field-observer-egress-zone]] -<> - -a| Network zone of outbound traffic as reported by the observer to categorize the destination area of egress traffic, e.g. Internal, External, DMZ, HR, Legal, etc. - -type: keyword - - - -example: `Public_Internet` - -| extended - -// =============================================================== - -| -[[field-observer-hostname]] -<> - -a| Hostname of the observer. - -type: keyword - - - - - -| core - -// =============================================================== - -| -[[field-observer-ingress]] -<> - -a| Observer.ingress holds information like interface number and name, vlan, and zone information to classify ingress traffic. Single armed monitoring such as a network sensor on a span port should only use observer.ingress to categorize traffic. - -type: object - - - - - -| extended - -// =============================================================== - -| -[[field-observer-ingress-zone]] -<> - -a| Network zone of incoming traffic as reported by the observer to categorize the source area of ingress traffic. e.g. internal, External, DMZ, HR, Legal, etc. - -type: keyword - - - -example: `DMZ` - -| extended - -// =============================================================== - -| -[[field-observer-ip]] -<> - -a| IP addresses of the observer. - -type: ip - - -Note: this field should contain an array of values. - - - - - -| core - -// =============================================================== - -| -[[field-observer-mac]] -<> - -a| MAC addresses of the observer. - -The notation format from RFC 7042 is suggested: Each octet (that is, 8-bit byte) is represented by two [uppercase] hexadecimal digits giving the value of the octet as an unsigned integer. Successive octets are separated by a hyphen. - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `["00-00-5E-00-53-23", "00-00-5E-00-53-24"]` - -| core - -// =============================================================== - -| -[[field-observer-name]] -<> - -a| Custom name of the observer. - -This is a name that can be given to an observer. This can be helpful for example if multiple firewalls of the same model are used in an organization. - -If no custom name is needed, the field can be left empty. - -type: keyword - - - -example: `1_proxySG` - -| extended - -// =============================================================== - -| -[[field-observer-product]] -<> - -a| The product name of the observer. - -type: keyword - - - -example: `s200` - -| extended - -// =============================================================== - -| -[[field-observer-serial-number]] -<> - -a| Observer serial number. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-observer-type]] -<> - -a| The type of the observer the data is coming from. - -There is no predefined list of observer types. Some examples are `forwarder`, `firewall`, `ids`, `ips`, `proxy`, `poller`, `sensor`, `APM server`. - -type: keyword - - - -example: `firewall` - -| core - -// =============================================================== - -| -[[field-observer-vendor]] -<> - -a| Vendor name of the observer. - -type: keyword - - - -example: `Symantec` - -| core - -// =============================================================== - -| -[[field-observer-version]] -<> - -a| Observer version. - -type: keyword - - - - - -| core - -// =============================================================== - -|===== - -[discrete] -==== Field Reuse - -[[ecs-observer-nestings]] -[discrete] -===== Field sets that can be nested under Observer - -[options="header"] -|===== -| Location | Field Set | Description - -// =============================================================== - - -| `observer.egress.interface.*` -| <> -| Fields to describe observer interface information. - -// =============================================================== - - -| `observer.egress.vlan.*` -| <> -| Fields to describe observed VLAN information. - -// =============================================================== - - -| `observer.geo.*` -| <> -| Fields describing a location. - -// =============================================================== - - -| `observer.ingress.interface.*` -| <> -| Fields to describe observer interface information. - -// =============================================================== - - -| `observer.ingress.vlan.*` -| <> -| Fields to describe observed VLAN information. - -// =============================================================== - - -| `observer.os.*` -| <> -| OS fields contain information about the operating system. - -// =============================================================== - - -|===== - - -[[ecs-orchestrator]] -=== Orchestrator Fields - -Fields that describe the resources which container orchestrators manage or act upon. - -[discrete] -==== Orchestrator Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-orchestrator-api-version]] -<> - -a| API version being used to carry out the action - -type: keyword - - - -example: `v1beta1` - -| extended - -// =============================================================== - -| -[[field-orchestrator-cluster-id]] -<> - -a| Unique ID of the cluster. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-orchestrator-cluster-name]] -<> - -a| Name of the cluster. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-orchestrator-cluster-url]] -<> - -a| URL of the API used to manage the cluster. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-orchestrator-cluster-version]] -<> - -a| The version of the cluster. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-orchestrator-namespace]] -<> - -a| Namespace in which the action is taking place. - -type: keyword - - - -example: `kube-system` - -| extended - -// =============================================================== - -| -[[field-orchestrator-organization]] -<> - -a| Organization affected by the event (for multi-tenant orchestrator setups). - -type: keyword - - - -example: `elastic` - -| extended - -// =============================================================== - -| -[[field-orchestrator-resource-annotation]] -<> - -a| The list of annotations added to the resource. - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `['key1:value1', 'key2:value2', 'key3:value3']` - -| extended - -// =============================================================== - -| -[[field-orchestrator-resource-id]] -<> - -a| Unique ID of the resource being acted upon. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-orchestrator-resource-ip]] -<> - -a| IP address assigned to the resource associated with the event being observed. In the case of a Kubernetes Pod, this array would contain only one element: the IP of the Pod (as opposed to the Node on which the Pod is running). - -type: ip - - -Note: this field should contain an array of values. - - - - - -| extended - -// =============================================================== - -| -[[field-orchestrator-resource-label]] -<> - -a| The list of labels added to the resource. - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `['key1:value1', 'key2:value2', 'key3:value3']` - -| extended - -// =============================================================== - -| -[[field-orchestrator-resource-name]] -<> - -a| Name of the resource being acted upon. - -type: keyword - - - -example: `test-pod-cdcws` - -| extended - -// =============================================================== - -| -[[field-orchestrator-resource-parent-type]] -<> - -a| Type or kind of the parent resource associated with the event being observed. In Kubernetes, this will be the name of a built-in workload resource (e.g., Deployment, StatefulSet, DaemonSet). - -type: keyword - - - -example: `DaemonSet` - -| extended - -// =============================================================== - -| -[[field-orchestrator-resource-type]] -<> - -a| Type of resource being acted upon. - -type: keyword - - - -example: `service` - -| extended - -// =============================================================== - -| -[[field-orchestrator-type]] -<> - -a| Orchestrator cluster type (e.g. kubernetes, nomad or cloudfoundry). - -type: keyword - - - -example: `kubernetes` - -| extended - -// =============================================================== - -|===== - - -[[ecs-organization]] -=== Organization Fields - -The organization fields enrich data with information about the company or entity the data is associated with. - -These fields help you arrange or filter data stored in an index by one or multiple organizations. - -[discrete] -==== Organization Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-organization-id]] -<> - -a| Unique identifier for the organization. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-organization-name]] -<> - -a| Organization name. - -type: keyword - -Multi-fields: - -* organization.name.text (type: match_only_text) - - - - - - - -| extended - -// =============================================================== - -|===== - - -[[ecs-os]] -=== Operating System Fields - -The OS fields contain information about the operating system. - -[discrete] -==== Operating System Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-os-family]] -<> - -a| OS family (such as redhat, debian, freebsd, windows). - -type: keyword - - - -example: `debian` - -| extended - -// =============================================================== - -| -[[field-os-full]] -<> - -a| Operating system name, including the version or code name. - -type: keyword - -Multi-fields: - -* os.full.text (type: match_only_text) - - - - - -example: `Mac OS Mojave` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/os/#os-description[os.description] - -| extended - -// =============================================================== - -| -[[field-os-kernel]] -<> - -a| Operating system kernel version as a raw string. - -type: keyword - - - -example: `4.4.0-112-generic` - -| extended - -// =============================================================== - -| -[[field-os-name]] -<> - -a| Operating system name, without the version. - -type: keyword - -Multi-fields: - -* os.name.text (type: match_only_text) - - - - - -example: `Mac OS X` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/os/#os-name[os.name] - -| extended - -// =============================================================== - -| -[[field-os-platform]] -<> - -a| Operating system platform (such centos, ubuntu, windows). - -type: keyword - - - -example: `darwin` - -| extended - -// =============================================================== - -| -[[field-os-type]] -<> - -a| Use the `os.type` field to categorize the operating system into one of the broad commercial families. - -If the OS you're dealing with is not listed as an expected value, the field should not be populated. Please let us know by opening an issue with ECS, to propose its addition. - -Expected values for this field: - -* `linux` -* `macos` -* `unix` -* `windows` -* `ios` -* `android` - -type: keyword - - - -example: `macos` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/conflict-910000?style=flat[relation,title=conflict,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/os/#os-type[os.type] - -Note: The expected values in ECS for `os.type` do not correspond with the values defined in semantic conventions! - -| extended - -// =============================================================== - -| -[[field-os-version]] -<> - -a| Operating system version as a raw string. - -type: keyword - - - -example: `10.14.1` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/os/#os-version[os.version] - -| extended - -// =============================================================== - -|===== - -[discrete] -==== Field Reuse - -The `os` fields are expected to be nested at: - - -* `host.os` - -* `observer.os` - -* `user_agent.os` - - -Note also that the `os` fields are not expected to be used directly at the root of the events. -[[ecs-package]] -=== Package Fields - -These fields contain information about an installed software package. It contains general information about a package, such as name, version or size. It also contains installation details, such as time or location. - -[discrete] -==== Package Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-package-architecture]] -<> - -a| Package architecture. - -type: keyword - - - -example: `x86_64` - -| extended - -// =============================================================== - -| -[[field-package-build-version]] -<> - -a| Additional information about the build version of the installed package. - -For example use the commit SHA of a non-released package. - -type: keyword - - - -example: `36f4f7e89dd61b0988b12ee000b98966867710cd` - -| extended - -// =============================================================== - -| -[[field-package-checksum]] -<> - -a| Checksum of the installed package for verification. - -type: keyword - - - -example: `68b329da9893e34099c7d8ad5cb9c940` - -| extended - -// =============================================================== - -| -[[field-package-description]] -<> - -a| Description of the package. - -type: keyword - - - -example: `Open source programming language to build simple/reliable/efficient software.` - -| extended - -// =============================================================== - -| -[[field-package-install-scope]] -<> - -a| Indicating how the package was installed, e.g. user-local, global. - -type: keyword - - - -example: `global` - -| extended - -// =============================================================== - -| -[[field-package-installed]] -<> - -a| Time when package was installed. - -type: date - - - - - -| extended - -// =============================================================== - -| -[[field-package-license]] -<> - -a| License under which the package was released. - -Use a short name, e.g. the license identifier from SPDX License List where possible (https://spdx.org/licenses/). - -type: keyword - - - -example: `Apache License 2.0` - -| extended - -// =============================================================== - -| -[[field-package-name]] -<> - -a| Package name - -type: keyword - - - -example: `go` - -| extended - -// =============================================================== - -| -[[field-package-path]] -<> - -a| Path where the package is installed. - -type: keyword - - - -example: `/usr/local/Cellar/go/1.12.9/` - -| extended - -// =============================================================== - -| -[[field-package-reference]] -<> - -a| Home page or reference URL of the software in this package, if available. - -type: keyword - - - -example: `https://golang.org` - -| extended - -// =============================================================== - -| -[[field-package-size]] -<> - -a| Package size in bytes. - -type: long - - - -example: `62231` - -| extended - -// =============================================================== - -| -[[field-package-type]] -<> - -a| Type of package. - -This should contain the package file type, rather than the package manager name. Examples: rpm, dpkg, brew, npm, gem, nupkg, jar. - -type: keyword - - - -example: `rpm` - -| extended - -// =============================================================== - -| -[[field-package-version]] -<> - -a| Package version - -type: keyword - - - -example: `1.12.9` - -| extended - -// =============================================================== - -|===== - - -[[ecs-pe]] -=== PE Header Fields - -These fields contain Windows Portable Executable (PE) metadata. - -[discrete] -==== PE Header Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-pe-architecture]] -<> - -a| CPU architecture target for the file. - -type: keyword - - - -example: `x64` - -| extended - -// =============================================================== - -| -[[field-pe-company]] -<> - -a| Internal company name of the file, provided at compile-time. - -type: keyword - - - -example: `Microsoft Corporation` - -| extended - -// =============================================================== - -| -[[field-pe-description]] -<> - -a| Internal description of the file, provided at compile-time. - -type: keyword - - - -example: `Paint` - -| extended - -// =============================================================== - -| -[[field-pe-file-version]] -<> - -a| Internal version of the file, provided at compile-time. - -type: keyword - - - -example: `6.3.9600.17415` - -| extended - -// =============================================================== - -| -[[field-pe-go-import-hash]] -<> - -a| A hash of the Go language imports in a PE file excluding standard library imports. An import hash can be used to fingerprint binaries even after recompilation or other code-level transformations have occurred, which would change more traditional hash values. - -The algorithm used to calculate the Go symbol hash and a reference implementation are available here: https://github.com/elastic/toutoumomoma - -type: keyword - - - -example: `10bddcb4cee42080f76c88d9ff964491` - -| extended - -// =============================================================== - -| -[[field-pe-go-imports]] -<> - -a| List of imported Go language element names and types. - -type: flattened - - - - - -| extended - -// =============================================================== - -| -[[field-pe-go-imports-names-entropy]] -<> - -a| Shannon entropy calculation from the list of Go imports. - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-pe-go-imports-names-var-entropy]] -<> - -a| Variance for Shannon entropy calculation from the list of Go imports. - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-pe-go-stripped]] -<> - -a| Set to true if the file is a Go executable that has had its symbols stripped or obfuscated and false if an unobfuscated Go executable. - -type: boolean - - - - - -| extended - -// =============================================================== - -| -[[field-pe-imphash]] -<> - -a| A hash of the imports in a PE file. An imphash -- or import hash -- can be used to fingerprint binaries even after recompilation or other code-level transformations have occurred, which would change more traditional hash values. - -Learn more at https://www.fireeye.com/blog/threat-research/2014/01/tracking-malware-import-hashing.html. - -type: keyword - - - -example: `0c6803c4e922103c4dca5963aad36ddf` - -| extended - -// =============================================================== - -| -[[field-pe-import-hash]] -<> - -a| A hash of the imports in a PE file. An import hash can be used to fingerprint binaries even after recompilation or other code-level transformations have occurred, which would change more traditional hash values. - -This is a synonym for imphash. - -type: keyword - - - -example: `d41d8cd98f00b204e9800998ecf8427e` - -| extended - -// =============================================================== - -| -[[field-pe-imports]] -<> - -a| List of imported element names and types. - -type: flattened - - -Note: this field should contain an array of values. - - - - - -| extended - -// =============================================================== - -| -[[field-pe-imports-names-entropy]] -<> - -a| Shannon entropy calculation from the list of imported element names and types. - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-pe-imports-names-var-entropy]] -<> - -a| Variance for Shannon entropy calculation from the list of imported element names and types. - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-pe-original-file-name]] -<> - -a| Internal name of the file, provided at compile-time. - -type: keyword - - - -example: `MSPAINT.EXE` - -| extended - -// =============================================================== - -| -[[field-pe-pehash]] -<> - -a| A hash of the PE header and data from one or more PE sections. An pehash can be used to cluster files by transforming structural information about a file into a hash value. - -Learn more at https://www.usenix.org/legacy/events/leet09/tech/full_papers/wicherski/wicherski_html/index.html. - -type: keyword - - - -example: `73ff189b63cd6be375a7ff25179a38d347651975` - -| extended - -// =============================================================== - -| -[[field-pe-product]] -<> - -a| Internal product name of the file, provided at compile-time. - -type: keyword - - - -example: `Microsoft® Windows® Operating System` - -| extended - -// =============================================================== - -| -[[field-pe-sections]] -<> - -a| An array containing an object for each section of the PE file. - -The keys that should be present in these objects are defined by sub-fields underneath `pe.sections.*`. - -type: nested - - -Note: this field should contain an array of values. - - - - - -| extended - -// =============================================================== - -| -[[field-pe-sections-entropy]] -<> - -a| Shannon entropy calculation from the section. - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-pe-sections-name]] -<> - -a| PE Section List name. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-pe-sections-physical-size]] -<> - -a| PE Section List physical size. - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-pe-sections-var-entropy]] -<> - -a| Variance for Shannon entropy calculation from the section. - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-pe-sections-virtual-size]] -<> - -a| PE Section List virtual size. This is always the same as `physical_size`. - -type: long - - - - - -| extended - -// =============================================================== - -|===== - -[discrete] -==== Field Reuse - -The `pe` fields are expected to be nested at: - - -* `dll.pe` - -* `file.pe` - -* `process.pe` - - -Note also that the `pe` fields are not expected to be used directly at the root of the events. -[[ecs-process]] -=== Process Fields - -These fields contain information about a process. - -These fields can help you correlate metrics information with a process id/name from a log message. The `process.pid` often stays in the metric itself and is copied to the global field for correlation. - -[discrete] -==== Process Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-process-args]] -<> - -a| Array of process arguments, starting with the absolute path to the executable. - -May be filtered to protect sensitive information. - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `["/usr/bin/ssh", "-l", "user", "10.0.0.16"]` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-command-args[process.command_args] - -| extended - -// =============================================================== - -| -[[field-process-args-count]] -<> - -a| Length of the process.args array. - -This field can be useful for querying or performing bucket analysis on how many arguments were provided to start a process. More arguments may be an indication of suspicious activity. - -type: long - - - -example: `4` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-args-count[process.args_count] - -| extended - -// =============================================================== - -| -[[field-process-command-line]] -<> - -a| Full command line that started the process, including the absolute path to the executable, and all arguments. - -Some arguments may be filtered to protect sensitive information. - -type: wildcard - -Multi-fields: - -* process.command_line.text (type: match_only_text) - - - - - -example: `/usr/bin/ssh -l user 10.0.0.16` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-command-line[process.command_line] - -| extended - -// =============================================================== - -| -[[field-process-end]] -<> - -a| The time the process ended. - -type: date - - - -example: `2016-05-23T08:05:34.853Z` - -| extended - -// =============================================================== - -| -[[field-process-entity-id]] -<> - -a| Unique identifier for the process. - -The implementation of this is specified by the data source, but some examples of what could be used here are a process-generated UUID, Sysmon Process GUIDs, or a hash of some uniquely identifying components of a process. - -Constructing a globally unique identifier is a common practice to mitigate PID reuse as well as to identify a specific process over time, across multiple monitored hosts. - -type: keyword - - - -example: `c2c455d9f99375d` - -| extended - -// =============================================================== - -| -[[field-process-entry-meta-type]] -<> - -a| The entry type for the entry session leader. Values include: init(e.g systemd), sshd, ssm, kubelet, teleport, terminal, console - -Note: This field is only set on process.session_leader. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-process-env-vars]] -<> - -a| Array of environment variable bindings. Captured from a snapshot of the environment at the time of execution. - -May be filtered to protect sensitive information. - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `["PATH=/usr/local/bin:/usr/bin", "USER=ubuntu"]` - -| extended - -// =============================================================== - -| -[[field-process-executable]] -<> - -a| Absolute path to the process executable. - -type: keyword - -Multi-fields: - -* process.executable.text (type: match_only_text) - - - - - -example: `/usr/bin/ssh` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-executable-path[process.executable.path] - -| extended - -// =============================================================== - -| -[[field-process-exit-code]] -<> - -a| The exit code of the process, if this is a termination event. - -The field should be absent if there is no exit code for the event (e.g. process start). - -type: long - - - -example: `137` - -| extended - -// =============================================================== - -| -[[field-process-interactive]] -<> - -a| Whether the process is connected to an interactive shell. - -Process interactivity is inferred from the processes file descriptors. If the character device for the controlling tty is the same as stdin and stderr for the process, the process is considered interactive. - -Note: A non-interactive process can belong to an interactive session and is simply one that does not have open file descriptors reading the controlling TTY on FD 0 (stdin) or writing to the controlling TTY on FD 2 (stderr). A backgrounded process is still considered interactive if stdin and stderr are connected to the controlling TTY. - -type: boolean - - - -example: `True` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-interactive[process.interactive] - -| extended - -// =============================================================== - -| -[[field-process-io]] -<> - -a| A chunk of input or output (IO) from a single process. - -This field only appears on the top level process object, which is the process that wrote the output or read the input. - -type: object - - - - - -| extended - -// =============================================================== - -| -[[field-process-io-bytes-skipped]] -<> - -a| An array of byte offsets and lengths denoting where IO data has been skipped. - -type: object - - -Note: this field should contain an array of values. - - - - - -| extended - -// =============================================================== - -| -[[field-process-io-bytes-skipped-length]] -<> - -a| The length of bytes skipped. - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-process-io-bytes-skipped-offset]] -<> - -a| The byte offset into this event's io.text (or io.bytes in the future) where length bytes were skipped. - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-process-io-max-bytes-per-process-exceeded]] -<> - -a| If true, the process producing the output has exceeded the max_kilobytes_per_process configuration setting. - -type: boolean - - - - - -| extended - -// =============================================================== - -| -[[field-process-io-text]] -<> - -a| A chunk of output or input sanitized to UTF-8. - -Best efforts are made to ensure complete lines are captured in these events. Assumptions should NOT be made that multiple lines will appear in the same event. TTY output may contain terminal control codes such as for cursor movement, so some string queries may not match due to terminal codes inserted between characters of a word. - -type: wildcard - - - - - -| extended - -// =============================================================== - -| -[[field-process-io-total-bytes-captured]] -<> - -a| The total number of bytes captured in this event. - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-process-io-total-bytes-skipped]] -<> - -a| The total number of bytes that were not captured due to implementation restrictions such as buffer size limits. Implementors should strive to ensure this value is always zero - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-process-io-type]] -<> - -a| The type of object on which the IO action (read or write) was taken. - -Currently only 'tty' is supported. Other types may be added in the future for 'file' and 'socket' support. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-process-name]] -<> - -a| Process name. - -Sometimes called program name or similar. - -type: keyword - -Multi-fields: - -* process.name.text (type: match_only_text) - - - - - -example: `ssh` - -| extended - -// =============================================================== - -| -[[field-process-pid]] -<> - -a| Process id. - -type: long - - - -example: `4242` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-pid[process.pid] - -| core - -// =============================================================== - -| -[[field-process-same-as-process]] -<> - -a| This boolean is used to identify if a leader process is the same as the top level process. - -For example, if `process.group_leader.same_as_process = true`, it means the process event in question is the leader of its process group. Details under `process.*` like `pid` would be the same under `process.group_leader.*` The same applies for both `process.session_leader` and `process.entry_leader`. - -This field exists to the benefit of EQL and other rule engines since it's not possible to compare equality between two fields in a single document. e.g `process.entity_id` = `process.group_leader.entity_id` (top level process is the process group leader) OR `process.entity_id` = `process.entry_leader.entity_id` (top level process is the entry session leader) - -Instead these rules could be written like: `process.group_leader.same_as_process: true` OR `process.entry_leader.same_as_process: true` - -Note: This field is only set on `process.entry_leader`, `process.session_leader` and `process.group_leader`. - -type: boolean - - - -example: `True` - -| extended - -// =============================================================== - -| -[[field-process-start]] -<> - -a| The time the process started. - -type: date - - - -example: `2016-05-23T08:05:34.853Z` - -| extended - -// =============================================================== - -| -[[field-process-thread-capabilities-effective]] -<> - -a| This is the set of capabilities used by the kernel to perform permission checks for the thread. - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `["CAP_BPF", "CAP_SYS_ADMIN"]` - -| extended - -// =============================================================== - -| -[[field-process-thread-capabilities-permitted]] -<> - -a| This is a limiting superset for the effective capabilities that the thread may assume. - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `["CAP_BPF", "CAP_SYS_ADMIN"]` - -| extended - -// =============================================================== - -| -[[field-process-thread-id]] -<> - -a| Thread ID. - -type: long - - - -example: `4242` - -| extended - -// =============================================================== - -| -[[field-process-thread-name]] -<> - -a| Thread name. - -type: keyword - - - -example: `thread-0` - -| extended - -// =============================================================== - -| -[[field-process-title]] -<> - -a| Process title. - -The proctitle, some times the same as process name. Can also be different: for example a browser setting its title to the web page currently opened. - -type: keyword - -Multi-fields: - -* process.title.text (type: match_only_text) - - - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-title[process.title] - -| extended - -// =============================================================== - -| -[[field-process-tty]] -<> - -a| Information about the controlling TTY device. If set, the process belongs to an interactive session. - -type: object - - - - - -| extended - -// =============================================================== - -| -[[field-process-tty-char-device-major]] -<> - -a| The major number identifies the driver associated with the device. The character device's major and minor numbers can be algorithmically combined to produce the more familiar terminal identifiers such as "ttyS0" and "pts/0". For more details, please refer to the Linux kernel documentation. - -type: long - - - -example: `4` - -| extended - -// =============================================================== - -| -[[field-process-tty-char-device-minor]] -<> - -a| The minor number is used only by the driver specified by the major number; other parts of the kernel don’t use it, and merely pass it along to the driver. It is common for a driver to control several devices; the minor number provides a way for the driver to differentiate among them. - -type: long - - - -example: `1` - -| extended - -// =============================================================== - -| -[[field-process-tty-columns]] -<> - -a| The number of character columns per line. e.g terminal width - -Terminal sizes can change, so this value reflects the maximum value for a given IO event. i.e. where event.action = 'text_output' - -type: long - - - -example: `80` - -| extended - -// =============================================================== - -| -[[field-process-tty-rows]] -<> - -a| The number of character rows in the terminal. e.g terminal height - -Terminal sizes can change, so this value reflects the maximum value for a given IO event. i.e. where event.action = 'text_output' - -type: long - - - -example: `24` - -| extended - -// =============================================================== - -| -[[field-process-uptime]] -<> - -a| Seconds the process has been up. - -type: long - - - -example: `1325` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/metric-cb00cb?style=flat[relation,title=metric,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--\+semconv+metric.process.uptime+--%3E%22&type=code[process.uptime] - -| extended - -// =============================================================== - -| -[[field-process-vpid]] -<> - -a| Virtual process id. - -The process id within a pid namespace. This is not necessarily unique across all processes on the host but it is unique within the process namespace that the process exists within. - -type: long - - - -example: `4242` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-vpid[process.vpid] - -| core - -// =============================================================== - -| -[[field-process-working-directory]] -<> - -a| The working directory of the process. - -type: keyword - -Multi-fields: - -* process.working_directory.text (type: match_only_text) - - - - - -example: `/home/alice` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-working-directory[process.working_directory] - -| extended - -// =============================================================== - -|===== - -[discrete] -==== Field Reuse - -The `process` fields are expected to be nested at: - - -* `process.entry_leader` - -* `process.entry_leader.parent` - -* `process.entry_leader.parent.session_leader` - -* `process.group_leader` - -* `process.parent` - -* `process.parent.group_leader` - -* `process.previous` - -* `process.responsible` - -* `process.session_leader` - -* `process.session_leader.parent` - -* `process.session_leader.parent.session_leader` - - -Note also that the `process` fields may be used directly at the root of the events. - -[[ecs-process-nestings]] -[discrete] -===== Field sets that can be nested under Process - -[options="header"] -|===== -| Location | Field Set | Description - -// =============================================================== - - -| `process.attested_groups.*` -| <>| beta:[ Reusing the `group` fields in this location is currently considered beta.] - -The externally attested groups based on an external source such as the Kube API. - -Note: this reuse should contain an array of group field set objects. - -// =============================================================== - - -| `process.attested_user.*` -| <>| beta:[ Reusing the `user` fields in this location is currently considered beta.] - -The externally attested user based on an external source such as the Kube API. - -// =============================================================== - - -| `process.code_signature.*` -| <> -| These fields contain information about binary code signatures. - -// =============================================================== - - -| `process.elf.*` -| <> -| These fields contain Linux Executable Linkable Format (ELF) metadata. - -// =============================================================== - - -| `process.entry_leader.*` -| <> -| First process from terminal or remote access via SSH, SSM, etc OR a service directly started by the init process. - -// =============================================================== - - -| `process.entry_leader.parent.*` -| <> -| Information about the entry leader's parent process. Only pid, start and entity_id fields are set. - -// =============================================================== - - -| `process.entry_leader.parent.session_leader.*` -| <> -| Information about the parent session of the entry leader. Only pid, start and entity_id fields are set. - -// =============================================================== - - -| `process.entry_meta.source.*` -| <> -| Remote client information such as ip, port and geo location. - -// =============================================================== - - -| `process.group.*` -| <> -| The effective group (egid). - -// =============================================================== - - -| `process.group_leader.*` -| <> -| Information about the process group leader. In some cases this may be the same as the top level process. - -// =============================================================== - - -| `process.hash.*` -| <> -| Hashes, usually file hashes. - -// =============================================================== - - -| `process.macho.*` -| <>| beta:[ This field reuse is beta and subject to change.] - -These fields contain Mac OS Mach Object file format (Mach-O) metadata. - -// =============================================================== - - -| `process.parent.*` -| <> -| Information about the parent process. - -// =============================================================== - - -| `process.parent.group_leader.*` -| <> -| Information about the parent's process group leader. Only pid, start and entity_id fields are set. - -// =============================================================== - - -| `process.pe.*` -| <> -| These fields contain Windows Portable Executable (PE) metadata. - -// =============================================================== - - -| `process.previous.*` -| <> -| An array of previous executions for the process, including the initial fork. Only executable and args are set. - -Note: this reuse should contain an array of process field set objects. - -// =============================================================== - - -| `process.real_group.*` -| <> -| The real group (rgid). - -// =============================================================== - - -| `process.real_user.*` -| <> -| The real user (ruid). Identifies the real owner of the process. - -// =============================================================== - - -| `process.responsible.*` -| <>| beta:[ This field is beta and subject to change.] - -Responsible process in macOS tracks the originating process of an app, key for understanding permissions and hierarchy. - -// =============================================================== - - -| `process.saved_group.*` -| <> -| The saved group (sgid). - -// =============================================================== - - -| `process.saved_user.*` -| <> -| The saved user (suid). - -// =============================================================== - - -| `process.session_leader.*` -| <> -| Often the same as entry_leader. When it differs, it represents a session started within another session. e.g. using tmux - -// =============================================================== - - -| `process.session_leader.parent.*` -| <> -| Information about the session leader's parent process. Only pid, start and entity_id fields are set. - -// =============================================================== - - -| `process.session_leader.parent.session_leader.*` -| <> -| Information about the parent session of the session leader. Only pid, start and entity_id fields are set. - -// =============================================================== - - -| `process.supplemental_groups.*` -| <> -| An array of supplemental groups. - -Note: this reuse should contain an array of group field set objects. - -// =============================================================== - - -| `process.user.*` -| <> -| The effective user (euid). - -// =============================================================== - - -|===== - - -[[ecs-registry]] -=== Registry Fields - -Fields related to Windows Registry operations. - -[discrete] -==== Registry Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-registry-data-bytes]] -<> - -a| Original bytes written with base64 encoding. - -For Windows registry operations, such as SetValueEx and RegQueryValueEx, this corresponds to the data pointed by `lp_data`. This is optional but provides better recoverability and should be populated for REG_BINARY encoded values. - -type: keyword - - - -example: `ZQBuAC0AVQBTAAAAZQBuAAAAAAA=` - -| extended - -// =============================================================== - -| -[[field-registry-data-strings]] -<> - -a| Content when writing string types. - -Populated as an array when writing string data to the registry. For single string registry types (REG_SZ, REG_EXPAND_SZ), this should be an array with one string. For sequences of string with REG_MULTI_SZ, this array will be variable length. For numeric data, such as REG_DWORD and REG_QWORD, this should be populated with the decimal representation (e.g `"1"`). - -type: wildcard - - -Note: this field should contain an array of values. - - - -example: `["C:\rta\red_ttp\bin\myapp.exe"]` - -| core - -// =============================================================== - -| -[[field-registry-data-type]] -<> - -a| Standard registry type for encoding contents - -type: keyword - - - -example: `REG_SZ` - -| core - -// =============================================================== - -| -[[field-registry-hive]] -<> - -a| Abbreviated name for the hive. - -type: keyword - - - -example: `HKLM` - -| core - -// =============================================================== - -| -[[field-registry-key]] -<> - -a| Hive-relative path of keys. - -type: keyword - - - -example: `SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\winword.exe` - -| core - -// =============================================================== - -| -[[field-registry-path]] -<> - -a| Full path, including hive, key and value - -type: keyword - - - -example: `HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\winword.exe\Debugger` - -| core - -// =============================================================== - -| -[[field-registry-value]] -<> - -a| Name of the value written. - -type: keyword - - - -example: `Debugger` - -| core - -// =============================================================== - -|===== - -[discrete] -==== Field Reuse - -The `registry` fields are expected to be nested at: - - -* `threat.enrichments.indicator.registry` - -* `threat.indicator.registry` - - -Note also that the `registry` fields may be used directly at the root of the events. - - -[[ecs-related]] -=== Related Fields - -This field set is meant to facilitate pivoting around a piece of data. - -Some pieces of information can be seen in many places in an ECS event. To facilitate searching for them, store an array of all seen values to their corresponding field in `related.`. - -A concrete example is IP addresses, which can be under host, observer, source, destination, client, server, and network.forwarded_ip. If you append all IPs to `related.ip`, you can then search for a given IP trivially, no matter where it appeared, by querying `related.ip:192.0.2.15`. - -[discrete] -==== Related Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-related-hash]] -<> - -a| All the hashes seen on your event. Populating this field, then using it to search for hashes can help in situations where you're unsure what the hash algorithm is (and therefore which key name to search). - -type: keyword - - -Note: this field should contain an array of values. - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat[relation,title=not-applicable,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] Not applicable. - -| extended - -// =============================================================== - -| -[[field-related-hosts]] -<> - -a| All hostnames or other host identifiers seen on your event. Example identifiers include FQDNs, domain names, workstation names, or aliases. - -type: keyword - - -Note: this field should contain an array of values. - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat[relation,title=not-applicable,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] Not applicable. - -| extended - -// =============================================================== - -| -[[field-related-ip]] -<> - -a| All of the IPs seen on your event. - -type: ip - - -Note: this field should contain an array of values. - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat[relation,title=not-applicable,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] Not applicable. - -| extended - -// =============================================================== - -| -[[field-related-user]] -<> - -a| All the user names or other user identifiers seen on the event. - -type: keyword - - -Note: this field should contain an array of values. - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat[relation,title=not-applicable,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] Not applicable. - -| extended - -// =============================================================== - -|===== - - -[[ecs-risk]] -=== Risk information Fields - -Fields for describing risk score and risk level of entities such as hosts and users. These fields are not allowed to be nested under `event.*`. Please continue to use `event.risk_score` and `event.risk_score_norm` for event risk. - -beta::[ These fields are in beta and are subject to change.] - -[discrete] -==== Risk information Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-risk-calculated-level]] -<> - -a| A risk classification level calculated by an internal system as part of entity analytics and entity risk scoring. - -type: keyword - - - -example: `High` - -| extended - -// =============================================================== - -| -[[field-risk-calculated-score]] -<> - -a| A risk classification score calculated by an internal system as part of entity analytics and entity risk scoring. - -type: float - - - -example: `880.73` - -| extended - -// =============================================================== - -| -[[field-risk-calculated-score-norm]] -<> - -a| A risk classification score calculated by an internal system as part of entity analytics and entity risk scoring, and normalized to a range of 0 to 100. - -type: float - - - -example: `88.73` - -| extended - -// =============================================================== - -| -[[field-risk-static-level]] -<> - -a| A risk classification level obtained from outside the system, such as from some external Threat Intelligence Platform. - -type: keyword - - - -example: `High` - -| extended - -// =============================================================== - -| -[[field-risk-static-score]] -<> - -a| A risk classification score obtained from outside the system, such as from some external Threat Intelligence Platform. - -type: float - - - -example: `830.0` - -| extended - -// =============================================================== - -| -[[field-risk-static-score-norm]] -<> - -a| A risk classification score obtained from outside the system, such as from some external Threat Intelligence Platform, and normalized to a range of 0 to 100. - -type: float - - - -example: `83.0` - -| extended - -// =============================================================== - -|===== - -[discrete] -==== Field Reuse - -The `risk` fields are expected to be nested at: - - -* `host.risk` - -* `user.risk` - - -Note also that the `risk` fields are not expected to be used directly at the root of the events. -[[ecs-rule]] -=== Rule Fields - -Rule fields are used to capture the specifics of any observer or agent rules that generate alerts or other notable events. - -Examples of data sources that would populate the rule fields include: network admission control platforms, network or host IDS/IPS, network firewalls, web application firewalls, url filters, endpoint detection and response (EDR) systems, etc. - -[discrete] -==== Rule Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-rule-author]] -<> - -a| Name, organization, or pseudonym of the author or authors who created the rule used to generate this event. - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `["Star-Lord"]` - -| extended - -// =============================================================== - -| -[[field-rule-category]] -<> - -a| A categorization value keyword used by the entity using the rule for detection of this event. - -type: keyword - - - -example: `Attempted Information Leak` - -| extended - -// =============================================================== - -| -[[field-rule-description]] -<> - -a| The description of the rule generating the event. - -type: keyword - - - -example: `Block requests to public DNS over HTTPS / TLS protocols` - -| extended - -// =============================================================== - -| -[[field-rule-id]] -<> - -a| A rule ID that is unique within the scope of an agent, observer, or other entity using the rule for detection of this event. - -type: keyword - - - -example: `101` - -| extended - -// =============================================================== - -| -[[field-rule-license]] -<> - -a| Name of the license under which the rule used to generate this event is made available. - -type: keyword - - - -example: `Apache 2.0` - -| extended - -// =============================================================== - -| -[[field-rule-name]] -<> - -a| The name of the rule or signature generating the event. - -type: keyword - - - -example: `BLOCK_DNS_over_TLS` - -| extended - -// =============================================================== - -| -[[field-rule-reference]] -<> - -a| Reference URL to additional information about the rule used to generate this event. - -The URL can point to the vendor's documentation about the rule. If that's not available, it can also be a link to a more general page describing this type of alert. - -type: keyword - - - -example: `https://en.wikipedia.org/wiki/DNS_over_TLS` - -| extended - -// =============================================================== - -| -[[field-rule-ruleset]] -<> - -a| Name of the ruleset, policy, group, or parent category in which the rule used to generate this event is a member. - -type: keyword - - - -example: `Standard_Protocol_Filters` - -| extended - -// =============================================================== - -| -[[field-rule-uuid]] -<> - -a| A rule ID that is unique within the scope of a set or group of agents, observers, or other entities using the rule for detection of this event. - -type: keyword - - - -example: `1100110011` - -| extended - -// =============================================================== - -| -[[field-rule-version]] -<> - -a| The version / revision of the rule being used for analysis. - -type: keyword - - - -example: `1.1` - -| extended - -// =============================================================== - -|===== - - -[[ecs-server]] -=== Server Fields - -A Server is defined as the responder in a network connection for events regarding sessions, connections, or bidirectional flow records. - -For TCP events, the server is the receiver of the initial SYN packet(s) of the TCP connection. For other protocols, the server is generally the responder in the network transaction. Some systems actually use the term "responder" to refer the server in TCP connections. The server fields describe details about the system acting as the server in the network event. Server fields are usually populated in conjunction with client fields. Server fields are generally not populated for packet-level events. - -Client / server representations can add semantic context to an exchange, which is helpful to visualize the data in certain situations. If your context falls in that category, you should still ensure that source and destination are filled appropriately. - -[discrete] -==== Server Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-server-address]] -<> - -a| Some event server addresses are defined ambiguously. The event will sometimes list an IP, a domain or a unix socket. You should always store the raw address in the `.address` field. - -Then it should be duplicated to `.ip` or `.domain`, depending on which one it is. - -type: keyword - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/server/#server-address[server.address] - -| extended - -// =============================================================== - -| -[[field-server-bytes]] -<> - -a| Bytes sent from the server to the client. - -type: long - - - -example: `184` - -| core - -// =============================================================== - -| -[[field-server-domain]] -<> - -a| The domain name of the server system. - -This value may be a host name, a fully qualified domain name, or another host naming format. The value may derive from the original event or be added from enrichment. - -type: keyword - - - -example: `foo.example.com` - -| core - -// =============================================================== - -| -[[field-server-ip]] -<> - -a| IP address of the server (IPv4 or IPv6). - -type: ip - - - - - -| core - -// =============================================================== - -| -[[field-server-mac]] -<> - -a| MAC address of the server. - -The notation format from RFC 7042 is suggested: Each octet (that is, 8-bit byte) is represented by two [uppercase] hexadecimal digits giving the value of the octet as an unsigned integer. Successive octets are separated by a hyphen. - -type: keyword - - - -example: `00-00-5E-00-53-23` - -| core - -// =============================================================== - -| -[[field-server-nat-ip]] -<> - -a| Translated ip of destination based NAT sessions (e.g. internet to private DMZ) - -Typically used with load balancers, firewalls, or routers. - -type: ip - - - - - -| extended - -// =============================================================== - -| -[[field-server-nat-port]] -<> - -a| Translated port of destination based NAT sessions (e.g. internet to private DMZ) - -Typically used with load balancers, firewalls, or routers. - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-server-packets]] -<> - -a| Packets sent from the server to the client. - -type: long - - - -example: `12` - -| core - -// =============================================================== - -| -[[field-server-port]] -<> - -a| Port of the server. - -type: long - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/server/#server-port[server.port] - -| core - -// =============================================================== - -| -[[field-server-registered-domain]] -<> - -a| The highest registered server domain, stripped of the subdomain. - -For example, the registered domain for "foo.example.com" is "example.com". - -This value can be determined precisely with a list like the public suffix list (https://publicsuffix.org). Trying to approximate this by simply taking the last two labels will not work well for TLDs such as "co.uk". - -type: keyword - - - -example: `example.com` - -| extended - -// =============================================================== - -| -[[field-server-subdomain]] -<> - -a| The subdomain portion of a fully qualified domain name includes all of the names except the host name under the registered_domain. In a partially qualified domain, or if the the qualification level of the full name cannot be determined, subdomain contains all of the names below the registered domain. - -For example the subdomain portion of "www.east.mydomain.co.uk" is "east". If the domain has multiple levels of subdomain, such as "sub2.sub1.example.com", the subdomain field should contain "sub2.sub1", with no trailing period. - -type: keyword - - - -example: `east` - -| extended - -// =============================================================== - -| -[[field-server-top-level-domain]] -<> - -a| The effective top level domain (eTLD), also known as the domain suffix, is the last part of the domain name. For example, the top level domain for example.com is "com". - -This value can be determined precisely with a list like the public suffix list (https://publicsuffix.org). Trying to approximate this by simply taking the last label will not work well for effective TLDs such as "co.uk". - -type: keyword - - - -example: `co.uk` - -| extended - -// =============================================================== - -|===== - -[discrete] -==== Field Reuse - -[[ecs-server-nestings]] -[discrete] -===== Field sets that can be nested under Server - -[options="header"] -|===== -| Location | Field Set | Description - -// =============================================================== - - -| `server.as.*` -| <> -| Fields describing an Autonomous System (Internet routing prefix). - -// =============================================================== - - -| `server.geo.*` -| <> -| Fields describing a location. - -// =============================================================== - - -| `server.user.*` -| <> -| Fields to describe the user relevant to the event. - -// =============================================================== - - -|===== - - -[[ecs-service]] -=== Service Fields - -The service fields describe the service for or from which the data was collected. - -These fields help you find and correlate logs for a specific service and version. - -[discrete] -==== Service Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-service-address]] -<> - -a| Address where data about this service was collected from. - -This should be a URI, network address (ipv4:port or [ipv6]:port) or a resource path (sockets). - -type: keyword - - - -example: `172.26.0.2:5432` - -| extended - -// =============================================================== - -| -[[field-service-environment]] -<> - -a| beta:[ This field is beta and subject to change. ] - -Identifies the environment where the service is running. - -If the same service runs in different environments (production, staging, QA, development, etc.), the environment can identify other instances of the same service. Can also group services and applications from the same environment. - -type: keyword - - - -example: `production` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/deployment/#deployment-environment-name[deployment.environment.name] - -| extended - -// =============================================================== - -| -[[field-service-ephemeral-id]] -<> - -a| Ephemeral identifier of this service (if one exists). - -This id normally changes across restarts, but `service.id` does not. - -type: keyword - - - -example: `8a4f500f` - -| extended - -// =============================================================== - -| -[[field-service-id]] -<> - -a| Unique identifier of the running service. If the service is comprised of many nodes, the `service.id` should be the same for all nodes. - -This id should uniquely identify the service. This makes it possible to correlate logs and metrics for one specific service, no matter which particular node emitted the event. - -Note that if you need to see the events from one specific host of the service, you should filter on that `host.name` or `host.id` instead. - -type: keyword - - - -example: `d37e5ebfe0ae6c4972dbe9f0174a1637bb8247f6` - -| core - -// =============================================================== - -| -[[field-service-name]] -<> - -a| Name of the service data is collected from. - -The name of the service is normally user given. This allows for distributed services that run on multiple hosts to correlate the related instances based on the name. - -In the case of Elasticsearch the `service.name` could contain the cluster name. For Beats the `service.name` is by default a copy of the `service.type` field if no name is specified. - -type: keyword - - - -example: `elasticsearch-metrics` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/service/#service-name[service.name] - -| core - -// =============================================================== - -| -[[field-service-node-name]] -<> - -a| Name of a service node. - -This allows for two nodes of the same service running on the same host to be differentiated. Therefore, `service.node.name` should typically be unique across nodes of a given service. - -In the case of Elasticsearch, the `service.node.name` could contain the unique node name within the Elasticsearch cluster. In cases where the service doesn't have the concept of a node name, the host name or container name can be used to distinguish running instances that make up this service. If those do not provide uniqueness (e.g. multiple instances of the service running on the same host) - the node name can be manually set. - -type: keyword - - - -example: `instance-0000000016` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/service/#service-instance-id[service.instance.id] - -| extended - -// =============================================================== - -| -[[field-service-node-role]] -<> - -a| Deprecated for removal in next major version release. This field will be superseded by `node.roles`. - -Role of a service node. - -This allows for distinction between different running roles of the same service. - -In the case of Kibana, the `service.node.role` could be `ui` or `background_tasks`. - -In the case of Elasticsearch, the `service.node.role` could be `master` or `data`. - -Other services could use this to distinguish between a `web` and `worker` role running as part of the service. - -type: keyword - - - -example: `background_tasks` - -| extended - -// =============================================================== - -| -[[field-service-node-roles]] -<> - -a| Roles of a service node. - -This allows for distinction between different running roles of the same service. - -In the case of Kibana, the `service.node.role` could be `ui` or `background_tasks` or both. - -In the case of Elasticsearch, the `service.node.role` could be `master` or `data` or both. - -Other services could use this to distinguish between a `web` and `worker` role running as part of the service. - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `["ui", "background_tasks"]` - -| extended - -// =============================================================== - -| -[[field-service-state]] -<> - -a| Current state of the service. - -type: keyword - - - - - -| core - -// =============================================================== - -| -[[field-service-type]] -<> - -a| The type of the service data is collected from. - -The type can be used to group and correlate logs and metrics from one service type. - -Example: If logs or metrics are collected from Elasticsearch, `service.type` would be `elasticsearch`. - -type: keyword - - - -example: `elasticsearch` - -| core - -// =============================================================== - -| -[[field-service-version]] -<> - -a| Version of the service the data was collected from. - -This allows to look at a data set only for a specific version of a service. - -type: keyword - - - -example: `3.2.4` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/service/#service-version[service.version] - -| core - -// =============================================================== - -|===== - -[discrete] -==== Field Reuse - -The `service` fields are expected to be nested at: - - -* `service.origin` - -* `service.target` - - -Note also that the `service` fields may be used directly at the root of the events. - -[[ecs-service-nestings]] -[discrete] -===== Field sets that can be nested under Service - -[options="header"] -|===== -| Location | Field Set | Description - -// =============================================================== - - -| `service.origin.*` -| <>| beta:[ Reusing the `service` fields in this location is currently considered beta.] - -Describes the origin service in case of an incoming request or event. - -// =============================================================== - - -| `service.target.*` -| <>| beta:[ Reusing the `service` fields in this location is currently considered beta.] - -Describes the target service in case of an outgoing request or event. - -// =============================================================== - - -|===== - - - -[discrete] -==== Service Field Usage - -For usage and examples of the service fields, please see the <> section. - -include::usage/service.asciidoc[] - - -[[ecs-source]] -=== Source Fields - -Source fields capture details about the sender of a network exchange/packet. These fields are populated from a network event, packet, or other event containing details of a network transaction. - -Source fields are usually populated in conjunction with destination fields. The source and destination fields are considered the baseline and should always be filled if an event contains source and destination details from a network transaction. If the event also contains identification of the client and server roles, then the client and server fields should also be populated. - -[discrete] -==== Source Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-source-address]] -<> - -a| Some event source addresses are defined ambiguously. The event will sometimes list an IP, a domain or a unix socket. You should always store the raw address in the `.address` field. - -Then it should be duplicated to `.ip` or `.domain`, depending on which one it is. - -type: keyword - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/source/#source-address[source.address] - -| extended - -// =============================================================== - -| -[[field-source-bytes]] -<> - -a| Bytes sent from the source to the destination. - -type: long - - - -example: `184` - -| core - -// =============================================================== - -| -[[field-source-domain]] -<> - -a| The domain name of the source system. - -This value may be a host name, a fully qualified domain name, or another host naming format. The value may derive from the original event or be added from enrichment. - -type: keyword - - - -example: `foo.example.com` - -| core - -// =============================================================== - -| -[[field-source-ip]] -<> - -a| IP address of the source (IPv4 or IPv6). - -type: ip - - - - - -| core - -// =============================================================== - -| -[[field-source-mac]] -<> - -a| MAC address of the source. - -The notation format from RFC 7042 is suggested: Each octet (that is, 8-bit byte) is represented by two [uppercase] hexadecimal digits giving the value of the octet as an unsigned integer. Successive octets are separated by a hyphen. - -type: keyword - - - -example: `00-00-5E-00-53-23` - -| core - -// =============================================================== - -| -[[field-source-nat-ip]] -<> - -a| Translated ip of source based NAT sessions (e.g. internal client to internet) - -Typically connections traversing load balancers, firewalls, or routers. - -type: ip - - - - - -| extended - -// =============================================================== - -| -[[field-source-nat-port]] -<> - -a| Translated port of source based NAT sessions. (e.g. internal client to internet) - -Typically used with load balancers, firewalls, or routers. - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-source-packets]] -<> - -a| Packets sent from the source to the destination. - -type: long - - - -example: `12` - -| core - -// =============================================================== - -| -[[field-source-port]] -<> - -a| Port of the source. - -type: long - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/source/#source-port[source.port] - -| core - -// =============================================================== - -| -[[field-source-registered-domain]] -<> - -a| The highest registered source domain, stripped of the subdomain. - -For example, the registered domain for "foo.example.com" is "example.com". - -This value can be determined precisely with a list like the public suffix list (https://publicsuffix.org). Trying to approximate this by simply taking the last two labels will not work well for TLDs such as "co.uk". - -type: keyword - - - -example: `example.com` - -| extended - -// =============================================================== - -| -[[field-source-subdomain]] -<> - -a| The subdomain portion of a fully qualified domain name includes all of the names except the host name under the registered_domain. In a partially qualified domain, or if the the qualification level of the full name cannot be determined, subdomain contains all of the names below the registered domain. - -For example the subdomain portion of "www.east.mydomain.co.uk" is "east". If the domain has multiple levels of subdomain, such as "sub2.sub1.example.com", the subdomain field should contain "sub2.sub1", with no trailing period. - -type: keyword - - - -example: `east` - -| extended - -// =============================================================== - -| -[[field-source-top-level-domain]] -<> - -a| The effective top level domain (eTLD), also known as the domain suffix, is the last part of the domain name. For example, the top level domain for example.com is "com". - -This value can be determined precisely with a list like the public suffix list (https://publicsuffix.org). Trying to approximate this by simply taking the last label will not work well for effective TLDs such as "co.uk". - -type: keyword - - - -example: `co.uk` - -| extended - -// =============================================================== - -|===== - -[discrete] -==== Field Reuse - -The `source` fields are expected to be nested at: - - -* `process.entry_meta.source` - - -Note also that the `source` fields may be used directly at the root of the events. - -[[ecs-source-nestings]] -[discrete] -===== Field sets that can be nested under Source - -[options="header"] -|===== -| Location | Field Set | Description - -// =============================================================== - - -| `source.as.*` -| <> -| Fields describing an Autonomous System (Internet routing prefix). - -// =============================================================== - - -| `source.geo.*` -| <> -| Fields describing a location. - -// =============================================================== - - -| `source.user.*` -| <> -| Fields to describe the user relevant to the event. - -// =============================================================== - - -|===== - - -[[ecs-threat]] -=== Threat Fields - -Fields to classify events and alerts according to a threat taxonomy such as the MITRE ATT&CK® framework. - -These fields are for users to classify alerts from all of their sources (e.g. IDS, NGFW, etc.) within a common taxonomy. The threat.tactic.* fields are meant to capture the high level category of the threat (e.g. "impact"). The threat.technique.* fields are meant to capture which kind of approach is used by this detected threat, to accomplish the goal (e.g. "endpoint denial of service"). - -[discrete] -==== Threat Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-threat-enrichments]] -<> - -a| A list of associated indicators objects enriching the event, and the context of that association/enrichment. - -type: nested - - -Note: this field should contain an array of values. - - - - - -| extended - -// =============================================================== - -| -[[field-threat-enrichments-indicator]] -<> - -a| Object containing associated indicators enriching the event. - -type: object - - - - - -| extended - -// =============================================================== - -| -[[field-threat-enrichments-indicator-confidence]] -<> - -a| Identifies the vendor-neutral confidence rating using the None/Low/Medium/High scale defined in Appendix A of the STIX 2.1 framework. Vendor-specific confidence scales may be added as custom fields. - -Expected values for this field: - -* `Not Specified` -* `None` -* `Low` -* `Medium` -* `High` - -type: keyword - - - -example: `Medium` - -| extended - -// =============================================================== - -| -[[field-threat-enrichments-indicator-description]] -<> - -a| Describes the type of action conducted by the threat. - -type: keyword - - - -example: `IP x.x.x.x was observed delivering the Angler EK.` - -| extended - -// =============================================================== - -| -[[field-threat-enrichments-indicator-email-address]] -<> - -a| Identifies a threat indicator as an email address (irrespective of direction). - -type: keyword - - - -example: `phish@example.com` - -| extended - -// =============================================================== - -| -[[field-threat-enrichments-indicator-first-seen]] -<> - -a| The date and time when intelligence source first reported sighting this indicator. - -type: date - - - -example: `2020-11-05T17:25:47.000Z` - -| extended - -// =============================================================== - -| -[[field-threat-enrichments-indicator-ip]] -<> - -a| Identifies a threat indicator as an IP address (irrespective of direction). - -type: ip - - - -example: `1.2.3.4` - -| extended - -// =============================================================== - -| -[[field-threat-enrichments-indicator-last-seen]] -<> - -a| The date and time when intelligence source last reported sighting this indicator. - -type: date - - - -example: `2020-11-05T17:25:47.000Z` - -| extended - -// =============================================================== - -| -[[field-threat-enrichments-indicator-marking-tlp]] -<> - -a| Traffic Light Protocol sharing markings. - -Expected values for this field: - -* `WHITE` -* `CLEAR` -* `GREEN` -* `AMBER` -* `AMBER+STRICT` -* `RED` - -type: keyword - - - -example: `CLEAR` - -| extended - -// =============================================================== - -| -[[field-threat-enrichments-indicator-marking-tlp-version]] -<> - -a| Traffic Light Protocol version. - -type: keyword - - - -example: `2.0` - -| extended - -// =============================================================== - -| -[[field-threat-enrichments-indicator-modified-at]] -<> - -a| The date and time when intelligence source last modified information for this indicator. - -type: date - - - -example: `2020-11-05T17:25:47.000Z` - -| extended - -// =============================================================== - -| -[[field-threat-enrichments-indicator-name]] -<> - -a| The display name indicator in an UI friendly format - -URL, IP address, email address, registry key, port number, hash value, or other relevant name can serve as the display name. - -type: keyword - - - -example: `5.2.75.227` - -| extended - -// =============================================================== - -| -[[field-threat-enrichments-indicator-port]] -<> - -a| Identifies a threat indicator as a port number (irrespective of direction). - -type: long - - - -example: `443` - -| extended - -// =============================================================== - -| -[[field-threat-enrichments-indicator-provider]] -<> - -a| The name of the indicator's provider. - -type: keyword - - - -example: `lrz_urlhaus` - -| extended - -// =============================================================== - -| -[[field-threat-enrichments-indicator-reference]] -<> - -a| Reference URL linking to additional information about this indicator. - -type: keyword - - - -example: `https://system.example.com/indicator/0001234` - -| extended - -// =============================================================== - -| -[[field-threat-enrichments-indicator-scanner-stats]] -<> - -a| Count of AV/EDR vendors that successfully detected malicious file or URL. - -type: long - - - -example: `4` - -| extended - -// =============================================================== - -| -[[field-threat-enrichments-indicator-sightings]] -<> - -a| Number of times this indicator was observed conducting threat activity. - -type: long - - - -example: `20` - -| extended - -// =============================================================== - -| -[[field-threat-enrichments-indicator-type]] -<> - -a| Type of indicator as represented by Cyber Observable in STIX 2.0. - -Expected values for this field: - -* `autonomous-system` -* `artifact` -* `directory` -* `domain-name` -* `email-addr` -* `file` -* `ipv4-addr` -* `ipv6-addr` -* `mac-addr` -* `mutex` -* `port` -* `process` -* `software` -* `url` -* `user-account` -* `windows-registry-key` -* `x509-certificate` - -type: keyword - - - -example: `ipv4-addr` - -| extended - -// =============================================================== - -| -[[field-threat-enrichments-matched-atomic]] -<> - -a| Identifies the atomic indicator value that matched a local environment endpoint or network event. - -type: keyword - - - -example: `bad-domain.com` - -| extended - -// =============================================================== - -| -[[field-threat-enrichments-matched-field]] -<> - -a| Identifies the field of the atomic indicator that matched a local environment endpoint or network event. - -type: keyword - - - -example: `file.hash.sha256` - -| extended - -// =============================================================== - -| -[[field-threat-enrichments-matched-id]] -<> - -a| Identifies the _id of the indicator document enriching the event. - -type: keyword - - - -example: `ff93aee5-86a1-4a61-b0e6-0cdc313d01b5` - -| extended - -// =============================================================== - -| -[[field-threat-enrichments-matched-index]] -<> - -a| Identifies the _index of the indicator document enriching the event. - -type: keyword - - - -example: `filebeat-8.0.0-2021.05.23-000011` - -| extended - -// =============================================================== - -| -[[field-threat-enrichments-matched-occurred]] -<> - -a| Indicates when the indicator match was generated - -type: date - - - -example: `2021-10-05T17:00:58.326Z` - -| extended - -// =============================================================== - -| -[[field-threat-enrichments-matched-type]] -<> - -a| Identifies the type of match that caused the event to be enriched with the given indicator - -type: keyword - - - -example: `indicator_match_rule` - -| extended - -// =============================================================== - -| -[[field-threat-feed-dashboard-id]] -<> - -a| The saved object ID of the dashboard belonging to the threat feed for displaying dashboard links to threat feeds in Kibana. - -type: keyword - - - -example: `5ba16340-72e6-11eb-a3e3-b3cc7c78a70f` - -| extended - -// =============================================================== - -| -[[field-threat-feed-description]] -<> - -a| Description of the threat feed in a UI friendly format. - -type: keyword - - - -example: `Threat feed from the AlienVault Open Threat eXchange network.` - -| extended - -// =============================================================== - -| -[[field-threat-feed-name]] -<> - -a| The name of the threat feed in UI friendly format. - -type: keyword - - - -example: `AlienVault OTX` - -| extended - -// =============================================================== - -| -[[field-threat-feed-reference]] -<> - -a| Reference information for the threat feed in a UI friendly format. - -type: keyword - - - -example: `https://otx.alienvault.com` - -| extended - -// =============================================================== - -| -[[field-threat-framework]] -<> - -a| Name of the threat framework used to further categorize and classify the tactic and technique of the reported threat. Framework classification can be provided by detecting systems, evaluated at ingest time, or retrospectively tagged to events. - -type: keyword - - - -example: `MITRE ATT&CK` - -| extended - -// =============================================================== - -| -[[field-threat-group-alias]] -<> - -a| The alias(es) of the group for a set of related intrusion activity that are tracked by a common name in the security community. - -While not required, you can use a MITRE ATT&CK® group alias(es). - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `[ "Magecart Group 6" ]` - -| extended - -// =============================================================== - -| -[[field-threat-group-id]] -<> - -a| The id of the group for a set of related intrusion activity that are tracked by a common name in the security community. - -While not required, you can use a MITRE ATT&CK® group id. - -type: keyword - - - -example: `G0037` - -| extended - -// =============================================================== - -| -[[field-threat-group-name]] -<> - -a| The name of the group for a set of related intrusion activity that are tracked by a common name in the security community. - -While not required, you can use a MITRE ATT&CK® group name. - -type: keyword - - - -example: `FIN6` - -| extended - -// =============================================================== - -| -[[field-threat-group-reference]] -<> - -a| The reference URL of the group for a set of related intrusion activity that are tracked by a common name in the security community. - -While not required, you can use a MITRE ATT&CK® group reference URL. - -type: keyword - - - -example: `https://attack.mitre.org/groups/G0037/` - -| extended - -// =============================================================== - -| -[[field-threat-indicator-confidence]] -<> - -a| Identifies the vendor-neutral confidence rating using the None/Low/Medium/High scale defined in Appendix A of the STIX 2.1 framework. Vendor-specific confidence scales may be added as custom fields. - -Expected values for this field: - -* `Not Specified` -* `None` -* `Low` -* `Medium` -* `High` - -type: keyword - - - -example: `Medium` - -| extended - -// =============================================================== - -| -[[field-threat-indicator-description]] -<> - -a| Describes the type of action conducted by the threat. - -type: keyword - - - -example: `IP x.x.x.x was observed delivering the Angler EK.` - -| extended - -// =============================================================== - -| -[[field-threat-indicator-email-address]] -<> - -a| Identifies a threat indicator as an email address (irrespective of direction). - -type: keyword - - - -example: `phish@example.com` - -| extended - -// =============================================================== - -| -[[field-threat-indicator-first-seen]] -<> - -a| The date and time when intelligence source first reported sighting this indicator. - -type: date - - - -example: `2020-11-05T17:25:47.000Z` - -| extended - -// =============================================================== - -| -[[field-threat-indicator-id]] -<> - -a| The ID of the indicator used by this threat to conduct behavior commonly modeled using MITRE ATT&CK®. This field can have multiple values to allow for the identification of the same indicator across systems that use different ID formats. - -While not required, a common approach is to use a STIX 2.x indicator ID. - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `[indicator--d7008e06-ab86-415a-9803-3c81ce2d3c37]` - -| extended - -// =============================================================== - -| -[[field-threat-indicator-ip]] -<> - -a| Identifies a threat indicator as an IP address (irrespective of direction). - -type: ip - - - -example: `1.2.3.4` - -| extended - -// =============================================================== - -| -[[field-threat-indicator-last-seen]] -<> - -a| The date and time when intelligence source last reported sighting this indicator. - -type: date - - - -example: `2020-11-05T17:25:47.000Z` - -| extended - -// =============================================================== - -| -[[field-threat-indicator-marking-tlp]] -<> - -a| Traffic Light Protocol sharing markings. - -Expected values for this field: - -* `WHITE` -* `CLEAR` -* `GREEN` -* `AMBER` -* `AMBER+STRICT` -* `RED` - -type: keyword - - - -example: `CLEAR` - -| extended - -// =============================================================== - -| -[[field-threat-indicator-marking-tlp-version]] -<> - -a| Traffic Light Protocol version. - -type: keyword - - - -example: `2.0` - -| extended - -// =============================================================== - -| -[[field-threat-indicator-modified-at]] -<> - -a| The date and time when intelligence source last modified information for this indicator. - -type: date - - - -example: `2020-11-05T17:25:47.000Z` - -| extended - -// =============================================================== - -| -[[field-threat-indicator-name]] -<> - -a| The display name indicator in an UI friendly format - -URL, IP address, email address, registry key, port number, hash value, or other relevant name can serve as the display name. - -type: keyword - - - -example: `5.2.75.227` - -| extended - -// =============================================================== - -| -[[field-threat-indicator-port]] -<> - -a| Identifies a threat indicator as a port number (irrespective of direction). - -type: long - - - -example: `443` - -| extended - -// =============================================================== - -| -[[field-threat-indicator-provider]] -<> - -a| The name of the indicator's provider. - -type: keyword - - - -example: `lrz_urlhaus` - -| extended - -// =============================================================== - -| -[[field-threat-indicator-reference]] -<> - -a| Reference URL linking to additional information about this indicator. - -type: keyword - - - -example: `https://system.example.com/indicator/0001234` - -| extended - -// =============================================================== - -| -[[field-threat-indicator-scanner-stats]] -<> - -a| Count of AV/EDR vendors that successfully detected malicious file or URL. - -type: long - - - -example: `4` - -| extended - -// =============================================================== - -| -[[field-threat-indicator-sightings]] -<> - -a| Number of times this indicator was observed conducting threat activity. - -type: long - - - -example: `20` - -| extended - -// =============================================================== - -| -[[field-threat-indicator-type]] -<> - -a| Type of indicator as represented by Cyber Observable in STIX 2.0. - -Expected values for this field: - -* `autonomous-system` -* `artifact` -* `directory` -* `domain-name` -* `email-addr` -* `file` -* `ipv4-addr` -* `ipv6-addr` -* `mac-addr` -* `mutex` -* `port` -* `process` -* `software` -* `url` -* `user-account` -* `windows-registry-key` -* `x509-certificate` - -type: keyword - - - -example: `ipv4-addr` - -| extended - -// =============================================================== - -| -[[field-threat-software-alias]] -<> - -a| The alias(es) of the software for a set of related intrusion activity that are tracked by a common name in the security community. - -While not required, you can use a MITRE ATT&CK® associated software description. - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `[ "X-Agent" ]` - -| extended - -// =============================================================== - -| -[[field-threat-software-id]] -<> - -a| The id of the software used by this threat to conduct behavior commonly modeled using MITRE ATT&CK®. - -While not required, you can use a MITRE ATT&CK® software id. - -type: keyword - - - -example: `S0552` - -| extended - -// =============================================================== - -| -[[field-threat-software-name]] -<> - -a| The name of the software used by this threat to conduct behavior commonly modeled using MITRE ATT&CK®. - -While not required, you can use a MITRE ATT&CK® software name. - -type: keyword - - - -example: `AdFind` - -| extended - -// =============================================================== - -| -[[field-threat-software-platforms]] -<> - -a| The platforms of the software used by this threat to conduct behavior commonly modeled using MITRE ATT&CK®. - -While not required, you can use MITRE ATT&CK® software platform values. - -Expected values for this field: - -* `AWS` -* `Azure` -* `Azure AD` -* `GCP` -* `Linux` -* `macOS` -* `Network` -* `Office 365` -* `SaaS` -* `Windows` - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `[ "Windows" ]` - -| extended - -// =============================================================== - -| -[[field-threat-software-reference]] -<> - -a| The reference URL of the software used by this threat to conduct behavior commonly modeled using MITRE ATT&CK®. - -While not required, you can use a MITRE ATT&CK® software reference URL. - -type: keyword - - - -example: `https://attack.mitre.org/software/S0552/` - -| extended - -// =============================================================== - -| -[[field-threat-software-type]] -<> - -a| The type of software used by this threat to conduct behavior commonly modeled using MITRE ATT&CK®. - -While not required, you can use a MITRE ATT&CK® software type. - -Expected values for this field: - -* `Malware` -* `Tool` - -type: keyword - - - -example: `Tool` - -| extended - -// =============================================================== - -| -[[field-threat-tactic-id]] -<> - -a| The id of tactic used by this threat. You can use a MITRE ATT&CK® tactic, for example. (ex. https://attack.mitre.org/tactics/TA0002/ ) - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `TA0002` - -| extended - -// =============================================================== - -| -[[field-threat-tactic-name]] -<> - -a| Name of the type of tactic used by this threat. You can use a MITRE ATT&CK® tactic, for example. (ex. https://attack.mitre.org/tactics/TA0002/) - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `Execution` - -| extended - -// =============================================================== - -| -[[field-threat-tactic-reference]] -<> - -a| The reference url of tactic used by this threat. You can use a MITRE ATT&CK® tactic, for example. (ex. https://attack.mitre.org/tactics/TA0002/ ) - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `https://attack.mitre.org/tactics/TA0002/` - -| extended - -// =============================================================== - -| -[[field-threat-technique-id]] -<> - -a| The id of technique used by this threat. You can use a MITRE ATT&CK® technique, for example. (ex. https://attack.mitre.org/techniques/T1059/) - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `T1059` - -| extended - -// =============================================================== - -| -[[field-threat-technique-name]] -<> - -a| The name of technique used by this threat. You can use a MITRE ATT&CK® technique, for example. (ex. https://attack.mitre.org/techniques/T1059/) - -type: keyword - -Multi-fields: - -* threat.technique.name.text (type: match_only_text) - - - - -Note: this field should contain an array of values. - - - -example: `Command and Scripting Interpreter` - -| extended - -// =============================================================== - -| -[[field-threat-technique-reference]] -<> - -a| The reference url of technique used by this threat. You can use a MITRE ATT&CK® technique, for example. (ex. https://attack.mitre.org/techniques/T1059/) - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `https://attack.mitre.org/techniques/T1059/` - -| extended - -// =============================================================== - -| -[[field-threat-technique-subtechnique-id]] -<> - -a| The full id of subtechnique used by this threat. You can use a MITRE ATT&CK® subtechnique, for example. (ex. https://attack.mitre.org/techniques/T1059/001/) - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `T1059.001` - -| extended - -// =============================================================== - -| -[[field-threat-technique-subtechnique-name]] -<> - -a| The name of subtechnique used by this threat. You can use a MITRE ATT&CK® subtechnique, for example. (ex. https://attack.mitre.org/techniques/T1059/001/) - -type: keyword - -Multi-fields: - -* threat.technique.subtechnique.name.text (type: match_only_text) - - - - -Note: this field should contain an array of values. - - - -example: `PowerShell` - -| extended - -// =============================================================== - -| -[[field-threat-technique-subtechnique-reference]] -<> - -a| The reference url of subtechnique used by this threat. You can use a MITRE ATT&CK® subtechnique, for example. (ex. https://attack.mitre.org/techniques/T1059/001/) - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `https://attack.mitre.org/techniques/T1059/001/` - -| extended - -// =============================================================== - -|===== - -[discrete] -==== Field Reuse - -[[ecs-threat-nestings]] -[discrete] -===== Field sets that can be nested under Threat - -[options="header"] -|===== -| Location | Field Set | Description - -// =============================================================== - - -| `threat.enrichments.indicator.as.*` -| <> -| Fields describing an Autonomous System (Internet routing prefix). - -// =============================================================== - - -| `threat.enrichments.indicator.file.*` -| <> -| Fields describing files. - -// =============================================================== - - -| `threat.enrichments.indicator.geo.*` -| <> -| Fields describing a location. - -// =============================================================== - - -| `threat.enrichments.indicator.registry.*` -| <> -| Fields related to Windows Registry operations. - -// =============================================================== - - -| `threat.enrichments.indicator.url.*` -| <> -| Fields that let you store URLs in various forms. - -// =============================================================== - - -| `threat.enrichments.indicator.x509.*` -| <> -| These fields contain x509 certificate metadata. - -// =============================================================== - - -| `threat.indicator.as.*` -| <> -| Fields describing an Autonomous System (Internet routing prefix). - -// =============================================================== - - -| `threat.indicator.file.*` -| <> -| Fields describing files. - -// =============================================================== - - -| `threat.indicator.geo.*` -| <> -| Fields describing a location. - -// =============================================================== - - -| `threat.indicator.registry.*` -| <> -| Fields related to Windows Registry operations. - -// =============================================================== - - -| `threat.indicator.url.*` -| <> -| Fields that let you store URLs in various forms. - -// =============================================================== - - -| `threat.indicator.x509.*` -| <> -| These fields contain x509 certificate metadata. - -// =============================================================== - - -|===== - - - -[discrete] -==== Threat Field Usage - -For usage and examples of the threat fields, please see the <> section. - -include::usage/threat.asciidoc[] - - -[[ecs-tls]] -=== TLS Fields - -Fields related to a TLS connection. These fields focus on the TLS protocol itself and intentionally avoids in-depth analysis of the related x.509 certificate files. - -[discrete] -==== TLS Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-tls-cipher]] -<> - -a| String indicating the cipher used during the current connection. - -type: keyword - - - -example: `TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-cipher[tls.cipher] - -| extended - -// =============================================================== - -| -[[field-tls-client-certificate]] -<> - -a| PEM-encoded stand-alone certificate offered by the client. This is usually mutually-exclusive of `client.certificate_chain` since this value also exists in that list. - -type: keyword - - - -example: `MII...` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-certificate[tls.client.certificate] - -| extended - -// =============================================================== - -| -[[field-tls-client-certificate-chain]] -<> - -a| Array of PEM-encoded certificates that make up the certificate chain offered by the client. This is usually mutually-exclusive of `client.certificate` since that value should be the first certificate in the chain. - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `["MII...", "MII..."]` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-certificate-chain[tls.client.certificate_chain] - -| extended - -// =============================================================== - -| -[[field-tls-client-hash-md5]] -<> - -a| Certificate fingerprint using the MD5 digest of DER-encoded version of certificate offered by the client. For consistency with other hash values, this value should be formatted as an uppercase hash. - -type: keyword - - - -example: `0F76C7F2C55BFD7D8E8B8F4BFBF0C9EC` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-hash-md5[tls.client.hash.md5] - -| extended - -// =============================================================== - -| -[[field-tls-client-hash-sha1]] -<> - -a| Certificate fingerprint using the SHA1 digest of DER-encoded version of certificate offered by the client. For consistency with other hash values, this value should be formatted as an uppercase hash. - -type: keyword - - - -example: `9E393D93138888D288266C2D915214D1D1CCEB2A` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-hash-sha1[tls.client.hash.sha1] - -| extended - -// =============================================================== - -| -[[field-tls-client-hash-sha256]] -<> - -a| Certificate fingerprint using the SHA256 digest of DER-encoded version of certificate offered by the client. For consistency with other hash values, this value should be formatted as an uppercase hash. - -type: keyword - - - -example: `0687F666A054EF17A08E2F2162EAB4CBC0D265E1D7875BE74BF3C712CA92DAF0` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-hash-sha256[tls.client.hash.sha256] - -| extended - -// =============================================================== - -| -[[field-tls-client-issuer]] -<> - -a| Distinguished name of subject of the issuer of the x.509 certificate presented by the client. - -type: keyword - - - -example: `CN=Example Root CA, OU=Infrastructure Team, DC=example, DC=com` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-issuer[tls.client.issuer] - -| extended - -// =============================================================== - -| -[[field-tls-client-ja3]] -<> - -a| A hash that identifies clients based on how they perform an SSL/TLS handshake. - -type: keyword - - - -example: `d4e5b18d6b55c71272893221c96ba240` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-ja3[tls.client.ja3] - -| extended - -// =============================================================== - -| -[[field-tls-client-not-after]] -<> - -a| Date/Time indicating when client certificate is no longer considered valid. - -type: date - - - -example: `2021-01-01T00:00:00.000Z` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-not-after[tls.client.not_after] - -| extended - -// =============================================================== - -| -[[field-tls-client-not-before]] -<> - -a| Date/Time indicating when client certificate is first considered valid. - -type: date - - - -example: `1970-01-01T00:00:00.000Z` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-not-before[tls.client.not_before] - -| extended - -// =============================================================== - -| -[[field-tls-client-server-name]] -<> - -a| Also called an SNI, this tells the server which hostname to which the client is attempting to connect to. When this value is available, it should get copied to `destination.domain`. - -type: keyword - - - -example: `www.elastic.co` - -| extended - -// =============================================================== - -| -[[field-tls-client-subject]] -<> - -a| Distinguished name of subject of the x.509 certificate presented by the client. - -type: keyword - - - -example: `CN=myclient, OU=Documentation Team, DC=example, DC=com` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-subject[tls.client.subject] - -| extended - -// =============================================================== - -| -[[field-tls-client-supported-ciphers]] -<> - -a| Array of ciphers offered by the client during the client hello. - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `["TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384", "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384", "..."]` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-supported-ciphers[tls.client.supported_ciphers] - -| extended - -// =============================================================== - -| -[[field-tls-curve]] -<> - -a| String indicating the curve used for the given cipher, when applicable. - -type: keyword - - - -example: `secp256r1` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-curve[tls.curve] - -| extended - -// =============================================================== - -| -[[field-tls-established]] -<> - -a| Boolean flag indicating if the TLS negotiation was successful and transitioned to an encrypted tunnel. - -type: boolean - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-established[tls.established] - -| extended - -// =============================================================== - -| -[[field-tls-next-protocol]] -<> - -a| String indicating the protocol being tunneled. Per the values in the IANA registry (https://www.iana.org/assignments/tls-extensiontype-values/tls-extensiontype-values.xhtml#alpn-protocol-ids), this string should be lower case. - -type: keyword - - - -example: `http/1.1` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-next-protocol[tls.next_protocol] - -| extended - -// =============================================================== - -| -[[field-tls-resumed]] -<> - -a| Boolean flag indicating if this TLS connection was resumed from an existing TLS negotiation. - -type: boolean - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-resumed[tls.resumed] - -| extended - -// =============================================================== - -| -[[field-tls-server-certificate]] -<> - -a| PEM-encoded stand-alone certificate offered by the server. This is usually mutually-exclusive of `server.certificate_chain` since this value also exists in that list. - -type: keyword - - - -example: `MII...` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-certificate[tls.server.certificate] - -| extended - -// =============================================================== - -| -[[field-tls-server-certificate-chain]] -<> - -a| Array of PEM-encoded certificates that make up the certificate chain offered by the server. This is usually mutually-exclusive of `server.certificate` since that value should be the first certificate in the chain. - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `["MII...", "MII..."]` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-certificate-chain[tls.server.certificate_chain] - -| extended - -// =============================================================== - -| -[[field-tls-server-hash-md5]] -<> - -a| Certificate fingerprint using the MD5 digest of DER-encoded version of certificate offered by the server. For consistency with other hash values, this value should be formatted as an uppercase hash. - -type: keyword - - - -example: `0F76C7F2C55BFD7D8E8B8F4BFBF0C9EC` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-hash-md5[tls.server.hash.md5] - -| extended - -// =============================================================== - -| -[[field-tls-server-hash-sha1]] -<> - -a| Certificate fingerprint using the SHA1 digest of DER-encoded version of certificate offered by the server. For consistency with other hash values, this value should be formatted as an uppercase hash. - -type: keyword - - - -example: `9E393D93138888D288266C2D915214D1D1CCEB2A` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-hash-sha1[tls.server.hash.sha1] - -| extended - -// =============================================================== - -| -[[field-tls-server-hash-sha256]] -<> - -a| Certificate fingerprint using the SHA256 digest of DER-encoded version of certificate offered by the server. For consistency with other hash values, this value should be formatted as an uppercase hash. - -type: keyword - - - -example: `0687F666A054EF17A08E2F2162EAB4CBC0D265E1D7875BE74BF3C712CA92DAF0` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-hash-sha256[tls.server.hash.sha256] - -| extended - -// =============================================================== - -| -[[field-tls-server-issuer]] -<> - -a| Subject of the issuer of the x.509 certificate presented by the server. - -type: keyword - - - -example: `CN=Example Root CA, OU=Infrastructure Team, DC=example, DC=com` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-issuer[tls.server.issuer] - -| extended - -// =============================================================== - -| -[[field-tls-server-ja3s]] -<> - -a| A hash that identifies servers based on how they perform an SSL/TLS handshake. - -type: keyword - - - -example: `394441ab65754e2207b1e1b457b3641d` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-ja3s[tls.server.ja3s] - -| extended - -// =============================================================== - -| -[[field-tls-server-not-after]] -<> - -a| Timestamp indicating when server certificate is no longer considered valid. - -type: date - - - -example: `2021-01-01T00:00:00.000Z` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-not-after[tls.server.not_after] - -| extended - -// =============================================================== - -| -[[field-tls-server-not-before]] -<> - -a| Timestamp indicating when server certificate is first considered valid. - -type: date - - - -example: `1970-01-01T00:00:00.000Z` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-not-before[tls.server.not_before] - -| extended - -// =============================================================== - -| -[[field-tls-server-subject]] -<> - -a| Subject of the x.509 certificate presented by the server. - -type: keyword - - - -example: `CN=www.example.com, OU=Infrastructure Team, DC=example, DC=com` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-subject[tls.server.subject] - -| extended - -// =============================================================== - -| -[[field-tls-version]] -<> - -a| Numeric part of the version parsed from the original string. - -type: keyword - - - -example: `1.2` - -| extended - -// =============================================================== - -| -[[field-tls-version-protocol]] -<> - -a| Normalized lowercase protocol name parsed from original string. - -type: keyword - - - -example: `tls` - -| extended - -// =============================================================== - -|===== - -[discrete] -==== Field Reuse - -[[ecs-tls-nestings]] -[discrete] -===== Field sets that can be nested under TLS - -[options="header"] -|===== -| Location | Field Set | Description - -// =============================================================== - - -| `tls.client.x509.*` -| <> -| These fields contain x509 certificate metadata. - -// =============================================================== - - -| `tls.server.x509.*` -| <> -| These fields contain x509 certificate metadata. - -// =============================================================== - - -|===== - - -[[ecs-tracing]] -=== Tracing Fields - -Distributed tracing makes it possible to analyze performance throughout a microservice architecture all in one view. This is accomplished by tracing all of the requests - from the initial web request in the front-end service - to queries made through multiple back-end services. - -Unlike most field sets in ECS, the tracing fields are *not* nested under the field set name. In other words, the correct field name is `trace.id`, not `tracing.trace.id`, and so on. - -[discrete] -==== Tracing Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-span-id]] -<> - -a| Unique identifier of the span within the scope of its trace. - -A span represents an operation within a transaction, such as a request to another service, or a database query. - -type: keyword - - - -example: `3ff9a8981b7ccd5a` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/OTLP-ffdcb2?style=flat[relation,title=otlp,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://github.com/search?q=repo%3Aopen-telemetry%2Fopentelemetry-proto+%22\+span_id+%22&type=code[span_id] - -| extended - -// =============================================================== - -| -[[field-trace-id]] -<> - -a| Unique identifier of the trace. - -A trace groups multiple events like transactions that belong together. For example, a user request handled by multiple inter-connected services. - -type: keyword - - - -example: `4bf92f3577b34da6a3ce929d0e0e4736` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/OTLP-ffdcb2?style=flat[relation,title=otlp,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://github.com/search?q=repo%3Aopen-telemetry%2Fopentelemetry-proto+%22\+trace_id+%22&type=code[trace_id] - -| extended - -// =============================================================== - -| -[[field-transaction-id]] -<> - -a| Unique identifier of the transaction within the scope of its trace. - -A transaction is the highest level of work measured within a service, such as a request to a server. - -type: keyword - - - -example: `00f067aa0ba902b7` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat[relation,title=not-applicable,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] Not applicable. - -| extended - -// =============================================================== - -|===== - - -[[ecs-url]] -=== URL Fields - -URL fields provide support for complete or partial URLs, and supports the breaking down into scheme, domain, path, and so on. - -[discrete] -==== URL Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-url-domain]] -<> - -a| Domain of the url, such as "www.elastic.co". - -In some cases a URL may refer to an IP and/or port directly, without a domain name. In this case, the IP address would go to the `domain` field. - -If the URL contains a literal IPv6 address enclosed by `[` and `]` (IETF RFC 2732), the `[` and `]` characters should also be captured in the `domain` field. - -type: keyword - - - -example: `www.elastic.co` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-domain[url.domain] - -| extended - -// =============================================================== - -| -[[field-url-extension]] -<> - -a| The field contains the file extension from the original request url, excluding the leading dot. - -The file extension is only set if it exists, as not every url has a file extension. - -The leading period must not be included. For example, the value must be "png", not ".png". - -Note that when the file name has multiple extensions (example.tar.gz), only the last one should be captured ("gz", not "tar.gz"). - -type: keyword - - - -example: `png` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-extension[url.extension] - -| extended - -// =============================================================== - -| -[[field-url-fragment]] -<> - -a| Portion of the url after the `#`, such as "top". - -The `#` is not part of the fragment. - -type: keyword - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-fragment[url.fragment] - -| extended - -// =============================================================== - -| -[[field-url-full]] -<> - -a| If full URLs are important to your use case, they should be stored in `url.full`, whether this field is reconstructed or present in the event source. - -type: wildcard - -Multi-fields: - -* url.full.text (type: match_only_text) - - - - - -example: `https://www.elastic.co:443/search?q=elasticsearch#top` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-full[url.full] - -| extended - -// =============================================================== - -| -[[field-url-original]] -<> - -a| Unmodified original url as seen in the event source. - -Note that in network monitoring, the observed URL may be a full URL, whereas in access logs, the URL is often just represented as a path. - -This field is meant to represent the URL as it was observed, complete or not. - -type: wildcard - -Multi-fields: - -* url.original.text (type: match_only_text) - - - - - -example: `https://www.elastic.co:443/search?q=elasticsearch#top or /search?q=elasticsearch` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-original[url.original] - -| extended - -// =============================================================== - -| -[[field-url-password]] -<> - -a| Password of the request. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-url-path]] -<> - -a| Path of the request, such as "/search". - -type: wildcard - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-path[url.path] - -| extended - -// =============================================================== - -| -[[field-url-port]] -<> - -a| Port of the request, such as 443. - -type: long - - - -example: `443` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-port[url.port] - -| extended - -// =============================================================== - -| -[[field-url-query]] -<> - -a| The query field describes the query string of the request, such as "q=elasticsearch". - -The `?` is excluded from the query string. If a URL contains no `?`, there is no query field. If there is a `?` but no query, the query field exists with an empty string. The `exists` query can be used to differentiate between the two cases. - -type: keyword - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-query[url.query] - -| extended - -// =============================================================== - -| -[[field-url-registered-domain]] -<> - -a| The highest registered url domain, stripped of the subdomain. - -For example, the registered domain for "foo.example.com" is "example.com". - -This value can be determined precisely with a list like the public suffix list (https://publicsuffix.org). Trying to approximate this by simply taking the last two labels will not work well for TLDs such as "co.uk". - -type: keyword - - - -example: `example.com` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-registered-domain[url.registered_domain] - -| extended - -// =============================================================== - -| -[[field-url-scheme]] -<> - -a| Scheme of the request, such as "https". - -Note: The `:` is not part of the scheme. - -type: keyword - - - -example: `https` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-scheme[url.scheme] - -| extended - -// =============================================================== - -| -[[field-url-subdomain]] -<> - -a| The subdomain portion of a fully qualified domain name includes all of the names except the host name under the registered_domain. In a partially qualified domain, or if the the qualification level of the full name cannot be determined, subdomain contains all of the names below the registered domain. - -For example the subdomain portion of "www.east.mydomain.co.uk" is "east". If the domain has multiple levels of subdomain, such as "sub2.sub1.example.com", the subdomain field should contain "sub2.sub1", with no trailing period. - -type: keyword - - - -example: `east` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-subdomain[url.subdomain] - -| extended - -// =============================================================== - -| -[[field-url-top-level-domain]] -<> - -a| The effective top level domain (eTLD), also known as the domain suffix, is the last part of the domain name. For example, the top level domain for example.com is "com". - -This value can be determined precisely with a list like the public suffix list (https://publicsuffix.org). Trying to approximate this by simply taking the last label will not work well for effective TLDs such as "co.uk". - -type: keyword - - - -example: `co.uk` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-top-level-domain[url.top_level_domain] - -| extended - -// =============================================================== - -| -[[field-url-username]] -<> - -a| Username of the request. - -type: keyword - - - - - -| extended - -// =============================================================== - -|===== - -[discrete] -==== Field Reuse - -The `url` fields are expected to be nested at: - - -* `threat.enrichments.indicator.url` - -* `threat.indicator.url` - - -Note also that the `url` fields may be used directly at the root of the events. - - -[[ecs-user]] -=== User Fields - -The user fields describe information about the user that is relevant to the event. - -Fields can have one entry or multiple entries. If a user has more than one id, provide an array that includes all of them. - -[discrete] -==== User Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-user-domain]] -<> - -a| Name of the directory the user is a member of. - -For example, an LDAP or Active Directory domain name. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-user-email]] -<> - -a| User email address. - -type: keyword - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/user/#user-email[user.email] - -| extended - -// =============================================================== - -| -[[field-user-full-name]] -<> - -a| User's full name, if available. - -type: keyword - -Multi-fields: - -* user.full_name.text (type: match_only_text) - - - - - -example: `Albert Einstein` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/user/#user-full-name[user.full_name] - -| extended - -// =============================================================== - -| -[[field-user-hash]] -<> - -a| Unique user hash to correlate information for a user in anonymized form. - -Useful if `user.id` or `user.name` contain confidential information and cannot be used. - -type: keyword - - - - - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/user/#user-hash[user.hash] - -| extended - -// =============================================================== - -| -[[field-user-id]] -<> - -a| Unique identifier of the user. - -type: keyword - - - -example: `S-1-5-21-202424912787-2692429404-2351956786-1000` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/user/#user-id[user.id] - -| core - -// =============================================================== - -| -[[field-user-name]] -<> - -a| Short name or login of the user. - -type: keyword - -Multi-fields: - -* user.name.text (type: match_only_text) - - - - - -example: `a.einstein` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/user/#user-name[user.name] - -| core - -// =============================================================== - -| -[[field-user-roles]] -<> - -a| Array of user roles at the time of the event. - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `["kibana_admin", "reporting_user"]` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/user/#user-roles[user.roles] - -| extended - -// =============================================================== - -|===== - -[discrete] -==== Field Reuse - -The `user` fields are expected to be nested at: - - -* `client.user` - -* `destination.user` - -* `process.attested_user` - -* `process.real_user` - -* `process.saved_user` - -* `process.user` - -* `server.user` - -* `source.user` - -* `user.changes` - -* `user.effective` - -* `user.target` - - -Note also that the `user` fields may be used directly at the root of the events. - -[[ecs-user-nestings]] -[discrete] -===== Field sets that can be nested under User - -[options="header"] -|===== -| Location | Field Set | Description - -// =============================================================== - - -| `user.changes.*` -| <> -| Captures changes made to a user. - -// =============================================================== - - -| `user.effective.*` -| <> -| User whose privileges were assumed. - -// =============================================================== - - -| `user.group.*` -| <> -| User's group relevant to the event. - -// =============================================================== - - -| `user.risk.*` -| <> -| Fields for describing risk score and level. - -// =============================================================== - - -| `user.target.*` -| <> -| Targeted user of action taken. - -// =============================================================== - - -|===== - - - -[discrete] -==== User Field Usage - -For usage and examples of the user fields, please see the <> section. - -include::usage/user.asciidoc[] - - -[[ecs-user_agent]] -=== User agent Fields - -The user_agent fields normally come from a browser request. - -They often show up in web service logs coming from the parsed user agent string. - -[discrete] -==== User agent Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-user-agent-device-name]] -<> - -a| Name of the device. - -type: keyword - - - -example: `iPhone` - -| extended - -// =============================================================== - -| -[[field-user-agent-name]] -<> - -a| Name of the user agent. - -type: keyword - - - -example: `Safari` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/user_agent/#user-agent-name[user_agent.name] - -| extended - -// =============================================================== - -| -[[field-user-agent-original]] -<> - -a| Unparsed user_agent string. - -type: keyword - -Multi-fields: - -* user_agent.original.text (type: match_only_text) - - - - - -example: `Mozilla/5.0 (iPhone; CPU iPhone OS 12_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.0 Mobile/15E148 Safari/604.1` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/user_agent/#user-agent-original[user_agent.original] - -| extended - -// =============================================================== - -| -[[field-user-agent-version]] -<> - -a| Version of the user agent. - -type: keyword - - - -example: `12.0` - -image:https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry[OTel Badge] image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] https://opentelemetry.io/docs/specs/semconv/attributes-registry/user_agent/#user-agent-version[user_agent.version] - -| extended - -// =============================================================== - -|===== - -[discrete] -==== Field Reuse - -[[ecs-user_agent-nestings]] -[discrete] -===== Field sets that can be nested under User agent - -[options="header"] -|===== -| Location | Field Set | Description - -// =============================================================== - - -| `user_agent.os.*` -| <> -| OS fields contain information about the operating system. - -// =============================================================== - - -|===== - - -[[ecs-vlan]] -=== VLAN Fields - -The VLAN fields are used to identify 802.1q tag(s) of a packet, as well as ingress and egress VLAN associations of an observer in relation to a specific packet or connection. - -Network.vlan fields are used to record a single VLAN tag, or the outer tag in the case of q-in-q encapsulations, for a packet or connection as observed, typically provided by a network sensor (e.g. Zeek, Wireshark) passively reporting on traffic. - -Network.inner VLAN fields are used to report inner q-in-q 802.1q tags (multiple 802.1q encapsulations) as observed, typically provided by a network sensor (e.g. Zeek, Wireshark) passively reporting on traffic. Network.inner VLAN fields should only be used in addition to network.vlan fields to indicate q-in-q tagging. - -Observer.ingress and observer.egress VLAN values are used to record observer specific information when observer events contain discrete ingress and egress VLAN information, typically provided by firewalls, routers, or load balancers. - -[discrete] -==== VLAN Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-vlan-id]] -<> - -a| VLAN ID as reported by the observer. - -type: keyword - - - -example: `10` - -| extended - -// =============================================================== - -| -[[field-vlan-name]] -<> - -a| Optional VLAN name as reported by the observer. - -type: keyword - - - -example: `outside` - -| extended - -// =============================================================== - -|===== - -[discrete] -==== Field Reuse - -The `vlan` fields are expected to be nested at: - - -* `network.inner.vlan` - -* `network.vlan` - -* `observer.egress.vlan` - -* `observer.ingress.vlan` - - -Note also that the `vlan` fields are not expected to be used directly at the root of the events. -[[ecs-volume]] -=== Volume Fields - -Fields related to storage volume details. - -[discrete] -==== Volume Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-volume-bus-type]] -<> - -a| Bus type of the device, such as `Nvme`, `Usb`, or `FileBackedVirtual`. - -type: keyword - - - -example: `FileBackedVirtual` - -| extended - -// =============================================================== - -| -[[field-volume-default-access]] -<> - -a| Describes the default access(es) of the volume. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-volume-device-name]] -<> - -a| Full path of the volume device. - -Only populate this field for POSIX system volumes. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-volume-device-type]] -<> - -a| Volume device type. - -The most frequently seen volume device types are `Disk File System` and `CD-ROM File System`. - -type: keyword - - - -example: `CD-ROM File System` - -| extended - -// =============================================================== - -| -[[field-volume-dos-name]] -<> - -a| The MS-DOS name of a device. - -DOS device name is in the format of driver letters, such as `C:`. The field is relevant to Windows systems only. - -type: keyword - - - -example: `E:` - -| extended - -// =============================================================== - -| -[[field-volume-file-system-type]] -<> - -a| Volume device file system type. - -The most common volume file system types are `NTFS` and `UDF`. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-volume-mount-name]] -<> - -a| Mount name of the volume device. - -Only populate this field for POSIX system volumes. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-volume-nt-name]] -<> - -a| The NT device name. - -NT device name uses a format of `\Device\HarddiskVolume2`. The field is relevant to Windows systems only. - -type: keyword - - - -example: `\Device\Cdrom1` - -| extended - -// =============================================================== - -| -[[field-volume-product-id]] -<> - -a| ProductID of the device. - -The vendor provides the ProductID for the volume, if any. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-volume-product-name]] -<> - -a| Product name of the volume. - -The volume device vendor provides this value. - -type: keyword - - - -example: `Virtual DVD-ROM` - -| extended - -// =============================================================== - -| -[[field-volume-removable]] -<> - -a| Indicates if the volume is removable. - -type: boolean - - - - - -| extended - -// =============================================================== - -| -[[field-volume-serial-number]] -<> - -a| Serial number identifier for the volume device. - -The serial number is provided by the vendor of the device, if any. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-volume-size]] -<> - -a| Size of the volume device in bytes. - -type: long - - - - - -| extended - -// =============================================================== - -| -[[field-volume-vendor-id]] -<> - -a| VendorID of the volume device. - -The volume device vendor provides this value. - -type: keyword - - - - - -| extended - -// =============================================================== - -| -[[field-volume-vendor-name]] -<> - -a| Vendor name of the volume device. - -The value is provided by the vendor of the device. - -type: keyword - - - -example: `Msft` - -| extended - -// =============================================================== - -| -[[field-volume-writable]] -<> - -a| Indicates if the volume is writable. - -type: boolean - - - - - -| extended - -// =============================================================== - -|===== - - -[[ecs-vulnerability]] -=== Vulnerability Fields - -The vulnerability fields describe information about a vulnerability that is relevant to an event. - -[discrete] -==== Vulnerability Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-vulnerability-category]] -<> - -a| The type of system or architecture that the vulnerability affects. These may be platform-specific (for example, Debian or SUSE) or general (for example, Database or Firewall). For example (https://qualysguard.qualys.com/qwebhelp/fo_portal/knowledgebase/vulnerability_categories.htm) - -This field must be an array. - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `["Firewall"]` - -| extended - -// =============================================================== - -| -[[field-vulnerability-classification]] -<> - -a| The classification of the vulnerability scoring system. For example (https://www.first.org/cvss/) - -type: keyword - - - -example: `CVSS` - -| extended - -// =============================================================== - -| -[[field-vulnerability-description]] -<> - -a| The description of the vulnerability that provides additional context of the vulnerability. For example (https://cve.mitre.org/about/faqs.html#cve_entry_descriptions_created) - -type: keyword - -Multi-fields: - -* vulnerability.description.text (type: match_only_text) - - - - - -example: `In macOS before 2.12.6, there is a vulnerability in the RPC...` - -| extended - -// =============================================================== - -| -[[field-vulnerability-enumeration]] -<> - -a| The type of identifier used for this vulnerability. For example (https://cve.mitre.org/about/) - -type: keyword - - - -example: `CVE` - -| extended - -// =============================================================== - -| -[[field-vulnerability-id]] -<> - -a| The identification (ID) is the number portion of a vulnerability entry. It includes a unique identification number for the vulnerability. For example (https://cve.mitre.org/about/faqs.html#what_is_cve_id) - -type: keyword - - - -example: `CVE-2019-00001` - -| extended - -// =============================================================== - -| -[[field-vulnerability-reference]] -<> - -a| A resource that provides additional information, context, and mitigations for the identified vulnerability. - -type: keyword - - - -example: `https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-6111` - -| extended - -// =============================================================== - -| -[[field-vulnerability-report-id]] -<> - -a| The report or scan identification number. - -type: keyword - - - -example: `20191018.0001` - -| extended - -// =============================================================== - -| -[[field-vulnerability-scanner-vendor]] -<> - -a| The name of the vulnerability scanner vendor. - -type: keyword - - - -example: `Tenable` - -| extended - -// =============================================================== - -| -[[field-vulnerability-score-base]] -<> - -a| Scores can range from 0.0 to 10.0, with 10.0 being the most severe. - -Base scores cover an assessment for exploitability metrics (attack vector, complexity, privileges, and user interaction), impact metrics (confidentiality, integrity, and availability), and scope. For example (https://www.first.org/cvss/specification-document) - -type: float - - - -example: `5.5` - -| extended - -// =============================================================== - -| -[[field-vulnerability-score-environmental]] -<> - -a| Scores can range from 0.0 to 10.0, with 10.0 being the most severe. - -Environmental scores cover an assessment for any modified Base metrics, confidentiality, integrity, and availability requirements. For example (https://www.first.org/cvss/specification-document) - -type: float - - - -example: `5.5` - -| extended - -// =============================================================== - -| -[[field-vulnerability-score-temporal]] -<> - -a| Scores can range from 0.0 to 10.0, with 10.0 being the most severe. - -Temporal scores cover an assessment for code maturity, remediation level, and confidence. For example (https://www.first.org/cvss/specification-document) - -type: float - - - - - -| extended - -// =============================================================== - -| -[[field-vulnerability-score-version]] -<> - -a| The National Vulnerability Database (NVD) provides qualitative severity rankings of "Low", "Medium", and "High" for CVSS v2.0 base score ranges in addition to the severity ratings for CVSS v3.0 as they are defined in the CVSS v3.0 specification. - -CVSS is owned and managed by FIRST.Org, Inc. (FIRST), a US-based non-profit organization, whose mission is to help computer security incident response teams across the world. For example (https://nvd.nist.gov/vuln-metrics/cvss) - -type: keyword - - - -example: `2.0` - -| extended - -// =============================================================== - -| -[[field-vulnerability-severity]] -<> - -a| The severity of the vulnerability can help with metrics and internal prioritization regarding remediation. For example (https://nvd.nist.gov/vuln-metrics/cvss) - -type: keyword - - - -example: `Critical` - -| extended - -// =============================================================== - -|===== - - -[[ecs-x509]] -=== x509 Certificate Fields - -This implements the common core fields for x509 certificates. This information is likely logged with TLS sessions, digital signatures found in executable binaries, S/MIME information in email bodies, or analysis of files on disk. - -When the certificate relates to a file, use the fields at `file.x509`. When hashes of the DER-encoded certificate are available, the `hash` data set should be populated as well (e.g. `file.hash.sha256`). - -Events that contain certificate information about network connections, should use the x509 fields under the relevant TLS fields: `tls.server.x509` and/or `tls.client.x509`. - -[discrete] -==== x509 Certificate Field Details - -[options="header"] -|===== -| Field | Description | Level - -// =============================================================== - -| -[[field-x509-alternative-names]] -<> - -a| List of subject alternative names (SAN). Name types vary by certificate authority and certificate type but commonly contain IP addresses, DNS names (and wildcards), and email addresses. - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `*.elastic.co` - -| extended - -// =============================================================== - -| -[[field-x509-issuer-common-name]] -<> - -a| List of common name (CN) of issuing certificate authority. - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `Example SHA2 High Assurance Server CA` - -| extended - -// =============================================================== - -| -[[field-x509-issuer-country]] -<> - -a| List of country \(C) codes - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `US` - -| extended - -// =============================================================== - -| -[[field-x509-issuer-distinguished-name]] -<> - -a| Distinguished name (DN) of issuing certificate authority. - -type: keyword - - - -example: `C=US, O=Example Inc, OU=www.example.com, CN=Example SHA2 High Assurance Server CA` - -| extended - -// =============================================================== - -| -[[field-x509-issuer-locality]] -<> - -a| List of locality names (L) - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `Mountain View` - -| extended - -// =============================================================== - -| -[[field-x509-issuer-organization]] -<> - -a| List of organizations (O) of issuing certificate authority. - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `Example Inc` - -| extended - -// =============================================================== - -| -[[field-x509-issuer-organizational-unit]] -<> - -a| List of organizational units (OU) of issuing certificate authority. - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `www.example.com` - -| extended - -// =============================================================== - -| -[[field-x509-issuer-state-or-province]] -<> - -a| List of state or province names (ST, S, or P) - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `California` - -| extended - -// =============================================================== - -| -[[field-x509-not-after]] -<> - -a| Time at which the certificate is no longer considered valid. - -type: date - - - -example: `2020-07-16T03:15:39Z` - -| extended - -// =============================================================== - -| -[[field-x509-not-before]] -<> - -a| Time at which the certificate is first considered valid. - -type: date - - - -example: `2019-08-16T01:40:25Z` - -| extended - -// =============================================================== - -| -[[field-x509-public-key-algorithm]] -<> - -a| Algorithm used to generate the public key. - -type: keyword - - - -example: `RSA` - -| extended - -// =============================================================== - -| -[[field-x509-public-key-curve]] -<> - -a| The curve used by the elliptic curve public key algorithm. This is algorithm specific. - -type: keyword - - - -example: `nistp521` - -| extended - -// =============================================================== - -| -[[field-x509-public-key-exponent]] -<> - -a| Exponent used to derive the public key. This is algorithm specific. - -type: long - - - -example: `65537` - -| extended - -// =============================================================== - -| -[[field-x509-public-key-size]] -<> - -a| The size of the public key space in bits. - -type: long - - - -example: `2048` - -| extended - -// =============================================================== - -| -[[field-x509-serial-number]] -<> - -a| Unique serial number issued by the certificate authority. For consistency, this must be encoded in base 16 and formatted without colons and uppercase characters. - -type: keyword - - - -example: `55FBB9C7DEBF09809D12CCAA` - -| extended - -// =============================================================== - -| -[[field-x509-signature-algorithm]] -<> - -a| Identifier for certificate signature algorithm. We recommend using names found in Go Lang Crypto library. See https://github.com/golang/go/blob/go1.14/src/crypto/x509/x509.go#L337-L353. - -type: keyword - - - -example: `SHA256-RSA` - -| extended - -// =============================================================== - -| -[[field-x509-subject-common-name]] -<> - -a| List of common names (CN) of subject. - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `shared.global.example.net` - -| extended - -// =============================================================== - -| -[[field-x509-subject-country]] -<> - -a| List of country \(C) code - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `US` - -| extended - -// =============================================================== - -| -[[field-x509-subject-distinguished-name]] -<> - -a| Distinguished name (DN) of the certificate subject entity. - -type: keyword - - - -example: `C=US, ST=California, L=San Francisco, O=Example, Inc., CN=shared.global.example.net` - -| extended - -// =============================================================== - -| -[[field-x509-subject-locality]] -<> - -a| List of locality names (L) - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `San Francisco` - -| extended - -// =============================================================== - -| -[[field-x509-subject-organization]] -<> - -a| List of organizations (O) of subject. - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `Example, Inc.` - -| extended - -// =============================================================== - -| -[[field-x509-subject-organizational-unit]] -<> - -a| List of organizational units (OU) of subject. - -type: keyword - - -Note: this field should contain an array of values. - - - - - -| extended - -// =============================================================== - -| -[[field-x509-subject-state-or-province]] -<> - -a| List of state or province names (ST, S, or P) - -type: keyword - - -Note: this field should contain an array of values. - - - -example: `California` - -| extended - -// =============================================================== - -| -[[field-x509-version-number]] -<> - -a| Version of x509 format. - -type: keyword - - - -example: `3` - -| extended - -// =============================================================== - -|===== - -[discrete] -==== Field Reuse - -The `x509` fields are expected to be nested at: - - -* `file.x509` - -* `threat.enrichments.indicator.x509` - -* `threat.indicator.x509` - -* `tls.client.x509` - -* `tls.server.x509` - - -Note also that the `x509` fields are not expected to be used directly at the root of the events. diff --git a/docs/fields/field-values.asciidoc b/docs/fields/field-values.asciidoc deleted file mode 100644 index c05396aa7e..0000000000 --- a/docs/fields/field-values.asciidoc +++ /dev/null @@ -1,630 +0,0 @@ -[[ecs-category-field-values-reference]] -== {ecs} Categorization Fields - -At a high level, ECS provides fields to classify events in two different ways: -"Where it's from" (e.g., `event.module`, `event.dataset`, `agent.type`, `observer.type`, etc.), -and "What it is." The categorization fields hold the "What it is" information, -independent of the source of the events. - -ECS defines four categorization fields for this purpose, each of which falls under the `event.*` field set. - -[float] -[[ecs-category-fields]] -=== Categorization Fields - -* <> -* <> -* <> -* <> - -NOTE: If your events don't match any of these categorization values, you should -leave the fields empty. This will ensure you can start populating the fields -once the appropriate categorization values are published, in a later release. - -[float] -[[ecs-category-usage]] -=== Categorization Usage - -<> contains examples combining the categorization fields to classify different types of events. - - -[[ecs-allowed-values-event-kind]] -=== ECS Categorization Field: event.kind - -This is one of four ECS Categorization Fields, and indicates the highest level in the ECS category hierarchy. - -`event.kind` gives high-level information about what type of information the event contains, without being specific to the contents of the event. For example, values of this field distinguish alert events from metric events. - -The value of this field can be used to inform how these kinds of events should be handled. They may warrant different retention, different access control, it may also help understand whether the data is coming in at a regular interval or not. - -*Allowed Values* - -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> - -[float] -[[ecs-event-kind-alert]] -==== alert - -This value indicates an event such as an alert or notable event, triggered by a detection rule executing externally to the Elastic Stack. - -`event.kind:alert` is often populated for events coming from firewalls, intrusion detection systems, endpoint detection and response systems, and so on. - -This value is not used by Elastic solutions for alert documents that are created by rules executing within the Kibana alerting framework. - - - -[float] -[[ecs-event-kind-asset]] -==== asset - -This value indicates events whose primary purpose is to store an inventory of assets/entities and their attributes. Assets/entities are objects (such as users and hosts) that are expected to be subjects of detailed analysis within the system. - -Examples include lists of user identities or accounts ingested from directory services such as Active Directory (AD), inventory of hosts pulled from configuration management databases (CMDB), and lists of cloud storage buckets pulled from cloud provider APIs. - -This value is used by Elastic Security for asset management solutions. `event.kind: asset` is not used for normal system events or logs that are coming from an asset/entity, nor is it used for system events or logs coming from a directory or CMDB system. - - - -[float] -[[ecs-event-kind-enrichment]] -==== enrichment - -The `enrichment` value indicates an event collected to provide additional context, often to other events. - -An example is collecting indicators of compromise (IOCs) from a threat intelligence provider with the intent to use those values to enrich other events. The IOC events from the intelligence provider should be categorized as `event.kind:enrichment`. - - - -[float] -[[ecs-event-kind-event]] -==== event - -This value is the most general and most common value for this field. It is used to represent events that indicate that something happened. - - - -[float] -[[ecs-event-kind-metric]] -==== metric - -This value is used to indicate that this event describes a numeric measurement taken at given point in time. - -Examples include CPU utilization, memory usage, or device temperature. - -Metric events are often collected on a predictable frequency, such as once every few seconds, or once a minute, but can also be used to describe ad-hoc numeric metric queries. - - - -[float] -[[ecs-event-kind-state]] -==== state - -The state value is similar to metric, indicating that this event describes a measurement taken at given point in time, except that the measurement does not result in a numeric value, but rather one of a fixed set of categorical values that represent conditions or states. - -Examples include periodic events reporting Elasticsearch cluster state (green/yellow/red), the state of a TCP connection (open, closed, fin_wait, etc.), the state of a host with respect to a software vulnerability (vulnerable, not vulnerable), and the state of a system regarding compliance with a regulatory standard (compliant, not compliant). - -Note that an event that describes a change of state would not use `event.kind:state`, but instead would use 'event.kind:event' since a state change fits the more general event definition of something that happened. - -State events are often collected on a predictable frequency, such as once every few seconds, once a minute, once an hour, or once a day, but can also be used to describe ad-hoc state queries. - - - -[float] -[[ecs-event-kind-pipeline_error]] -==== pipeline_error - -This value indicates that an error occurred during the ingestion of this event, and that event data may be missing, inconsistent, or incorrect. `event.kind:pipeline_error` is often associated with parsing errors. - - - -[float] -[[ecs-event-kind-signal]] -==== signal - -This value is used by Elastic solutions (e.g., Security, Observability) for alert documents that are created by rules executing within the Kibana alerting framework. - -Usage of this value is reserved, and data ingestion pipelines must not populate `event.kind` with the value "signal". - - - -[[ecs-allowed-values-event-category]] -=== ECS Categorization Field: event.category - -This is one of four ECS Categorization Fields, and indicates the second level in the ECS category hierarchy. - -`event.category` represents the "big buckets" of ECS categories. For example, filtering on `event.category:process` yields all events relating to process activity. This field is closely related to `event.type`, which is used as a subcategory. - -This field is an array. This will allow proper categorization of some events that fall in multiple categories. - -*Allowed Values* - -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> - -[float] -[[ecs-event-category-api]] -==== api - -Events in this category annotate API calls that occured on a system. Typical sources for those events could be from the Operating System level through the native libraries (for example Windows Win32, Linux libc, etc.), or managed sources of events (such as ETW, syslog), but can also include network protocols (such as SOAP, RPC, Websocket, REST, etc.) - - -*Expected event types for category api:* - -access, admin, allowed, change, creation, deletion, denied, end, info, start, user - - -[float] -[[ecs-event-category-authentication]] -==== authentication - -Events in this category are related to the challenge and response process in which credentials are supplied and verified to allow the creation of a session. Common sources for these logs are Windows event logs and ssh logs. Visualize and analyze events in this category to look for failed logins, and other authentication-related activity. - - -*Expected event types for category authentication:* - -start, end, info - - -[float] -[[ecs-event-category-configuration]] -==== configuration - -Events in the configuration category have to deal with creating, modifying, or deleting the settings or parameters of an application, process, or system. - -Example sources include security policy change logs, configuration auditing logging, and system integrity monitoring. - - -*Expected event types for category configuration:* - -access, change, creation, deletion, info - - -[float] -[[ecs-event-category-database]] -==== database - -The database category denotes events and metrics relating to a data storage and retrieval system. Note that use of this category is not limited to relational database systems. Examples include event logs from MS SQL, MySQL, Elasticsearch, MongoDB, etc. Use this category to visualize and analyze database activity such as accesses and changes. - - -*Expected event types for category database:* - -access, change, info, error - - -[float] -[[ecs-event-category-driver]] -==== driver - -Events in the driver category have to do with operating system device drivers and similar software entities such as Windows drivers, kernel extensions, kernel modules, etc. - -Use events and metrics in this category to visualize and analyze driver-related activity and status on hosts. - - -*Expected event types for category driver:* - -change, end, info, start - - -[float] -[[ecs-event-category-email]] -==== email - -This category is used for events relating to email messages, email attachments, and email network or protocol activity. - -Emails events can be produced by email security gateways, mail transfer agents, email cloud service providers, or mail server monitoring applications. - - -*Expected event types for category email:* - -info - - -[float] -[[ecs-event-category-file]] -==== file - -Relating to a set of information that has been created on, or has existed on a filesystem. Use this category of events to visualize and analyze the creation, access, and deletions of files. Events in this category can come from both host-based and network-based sources. An example source of a network-based detection of a file transfer would be the Zeek file.log. - - -*Expected event types for category file:* - -access, change, creation, deletion, info - - -[float] -[[ecs-event-category-host]] -==== host - -Use this category to visualize and analyze information such as host inventory or host lifecycle events. - -Most of the events in this category can usually be observed from the outside, such as from a hypervisor or a control plane's point of view. Some can also be seen from within, such as "start" or "end". - -Note that this category is for information about hosts themselves; it is not meant to capture activity "happening on a host". - - -*Expected event types for category host:* - -access, change, end, info, start - - -[float] -[[ecs-event-category-iam]] -==== iam - -Identity and access management (IAM) events relating to users, groups, and administration. Use this category to visualize and analyze IAM-related logs and data from active directory, LDAP, Okta, Duo, and other IAM systems. - - -*Expected event types for category iam:* - -admin, change, creation, deletion, group, info, user - - -[float] -[[ecs-event-category-intrusion_detection]] -==== intrusion_detection - -Relating to intrusion detections from IDS/IPS systems and functions, both network and host-based. Use this category to visualize and analyze intrusion detection alerts from systems such as Snort, Suricata, and Palo Alto threat detections. - - -*Expected event types for category intrusion_detection:* - -allowed, denied, info - - -[float] -[[ecs-event-category-library]] -==== library - -Events in this category refer to the loading of a library, such as (dll / so / dynlib), into a process. Use this category to visualize and analyze library loading related activity on hosts. Keep in mind that driver related activity will be captured under the "driver" category above. - - -*Expected event types for category library:* - -start - - -[float] -[[ecs-event-category-malware]] -==== malware - -Malware detection events and alerts. Use this category to visualize and analyze malware detections from EDR/EPP systems such as Elastic Endpoint Security, Symantec Endpoint Protection, Crowdstrike, and network IDS/IPS systems such as Suricata, or other sources of malware-related events such as Palo Alto Networks threat logs and Wildfire logs. - - -*Expected event types for category malware:* - -info - - -[float] -[[ecs-event-category-network]] -==== network - -Relating to all network activity, including network connection lifecycle, network traffic, and essentially any event that includes an IP address. Many events containing decoded network protocol transactions fit into this category. Use events in this category to visualize or analyze counts of network ports, protocols, addresses, geolocation information, etc. - - -*Expected event types for category network:* - -access, allowed, connection, denied, end, info, protocol, start - - -[float] -[[ecs-event-category-package]] -==== package - -Relating to software packages installed on hosts. Use this category to visualize and analyze inventory of software installed on various hosts, or to determine host vulnerability in the absence of vulnerability scan data. - - -*Expected event types for category package:* - -access, change, deletion, info, installation, start - - -[float] -[[ecs-event-category-process]] -==== process - -Use this category of events to visualize and analyze process-specific information such as lifecycle events or process ancestry. - - -*Expected event types for category process:* - -access, change, end, info, start - - -[float] -[[ecs-event-category-registry]] -==== registry - -Having to do with settings and assets stored in the Windows registry. Use this category to visualize and analyze activity such as registry access and modifications. - - -*Expected event types for category registry:* - -access, change, creation, deletion - - -[float] -[[ecs-event-category-session]] -==== session - -The session category is applied to events and metrics regarding logical persistent connections to hosts and services. Use this category to visualize and analyze interactive or automated persistent connections between assets. Data for this category may come from Windows Event logs, SSH logs, or stateless sessions such as HTTP cookie-based sessions, etc. - - -*Expected event types for category session:* - -start, end, info - - -[float] -[[ecs-event-category-threat]] -==== threat - -Use this category to visualize and analyze events describing threat actors' targets, motives, or behaviors. - - -*Expected event types for category threat:* - -indicator - - -[float] -[[ecs-event-category-vulnerability]] -==== vulnerability - -Relating to vulnerability scan results. Use this category to analyze vulnerabilities detected by Tenable, Qualys, internal scanners, and other vulnerability management sources. - - -*Expected event types for category vulnerability:* - -info - - -[float] -[[ecs-event-category-web]] -==== web - -Relating to web server access. Use this category to create a dashboard of web server/proxy activity from apache, IIS, nginx web servers, etc. Note: events from network observers such as Zeek http log may also be included in this category. - - -*Expected event types for category web:* - -access, error, info - - -[[ecs-allowed-values-event-type]] -=== ECS Categorization Field: event.type - -This is one of four ECS Categorization Fields, and indicates the third level in the ECS category hierarchy. - -`event.type` represents a categorization "sub-bucket" that, when used along with the `event.category` field values, enables filtering events down to a level appropriate for single visualization. - -This field is an array. This will allow proper categorization of some events that fall in multiple event types. - -*Allowed Values* - -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> - -[float] -[[ecs-event-type-access]] -==== access - -The access event type is used for the subset of events within a category that indicate that something was accessed. Common examples include `event.category:database AND event.type:access`, or `event.category:file AND event.type:access`. Note for file access, both directory listings and file opens should be included in this subcategory. You can further distinguish access operations using the ECS `event.action` field. - - - -[float] -[[ecs-event-type-admin]] -==== admin - -The admin event type is used for the subset of events within a category that are related to admin objects. For example, administrative changes within an IAM framework that do not specifically affect a user or group (e.g., adding new applications to a federation solution or connecting discrete forests in Active Directory) would fall into this subcategory. Common example: `event.category:iam AND event.type:change AND event.type:admin`. You can further distinguish admin operations using the ECS `event.action` field. - - - -[float] -[[ecs-event-type-allowed]] -==== allowed - -The allowed event type is used for the subset of events within a category that indicate that something was allowed. Common examples include `event.category:network AND event.type:connection AND event.type:allowed` (to indicate a network firewall event for which the firewall disposition was to allow the connection to complete) and `event.category:intrusion_detection AND event.type:allowed` (to indicate a network intrusion prevention system event for which the IPS disposition was to allow the connection to complete). You can further distinguish allowed operations using the ECS `event.action` field, populating with values of your choosing, such as "allow", "detect", or "pass". - - - -[float] -[[ecs-event-type-change]] -==== change - -The change event type is used for the subset of events within a category that indicate that something has changed. If semantics best describe an event as modified, then include them in this subcategory. Common examples include `event.category:process AND event.type:change`, and `event.category:file AND event.type:change`. You can further distinguish change operations using the ECS `event.action` field. - - - -[float] -[[ecs-event-type-connection]] -==== connection - -Used primarily with `event.category:network` this value is used for the subset of network traffic that includes sufficient information for the event to be included in flow or connection analysis. Events in this subcategory will contain at least source and destination IP addresses, source and destination TCP/UDP ports, and will usually contain counts of bytes and/or packets transferred. Events in this subcategory may contain unidirectional or bidirectional information, including summary information. Use this subcategory to visualize and analyze network connections. Flow analysis, including Netflow, IPFIX, and other flow-related events fit in this subcategory. Note that firewall events from many Next-Generation Firewall (NGFW) devices will also fit into this subcategory. A common filter for flow/connection information would be `event.category:network AND event.type:connection AND event.type:end` (to view or analyze all completed network connections, ignoring mid-flow reports). You can further distinguish connection events using the ECS `event.action` field, populating with values of your choosing, such as "timeout", or "reset". - - - -[float] -[[ecs-event-type-creation]] -==== creation - -The "creation" event type is used for the subset of events within a category that indicate that something was created. A common example is `event.category:file AND event.type:creation`. - - - -[float] -[[ecs-event-type-deletion]] -==== deletion - -The deletion event type is used for the subset of events within a category that indicate that something was deleted. A common example is `event.category:file AND event.type:deletion` to indicate that a file has been deleted. - - - -[float] -[[ecs-event-type-denied]] -==== denied - -The denied event type is used for the subset of events within a category that indicate that something was denied. Common examples include `event.category:network AND event.type:denied` (to indicate a network firewall event for which the firewall disposition was to deny the connection) and `event.category:intrusion_detection AND event.type:denied` (to indicate a network intrusion prevention system event for which the IPS disposition was to deny the connection to complete). You can further distinguish denied operations using the ECS `event.action` field, populating with values of your choosing, such as "blocked", "dropped", or "quarantined". - - - -[float] -[[ecs-event-type-end]] -==== end - -The end event type is used for the subset of events within a category that indicate something has ended. A common example is `event.category:process AND event.type:end`. - - - -[float] -[[ecs-event-type-error]] -==== error - -The error event type is used for the subset of events within a category that indicate or describe an error. A common example is `event.category:database AND event.type:error`. Note that pipeline errors that occur during the event ingestion process should not use this `event.type` value. Instead, they should use `event.kind:pipeline_error`. - - - -[float] -[[ecs-event-type-group]] -==== group - -The group event type is used for the subset of events within a category that are related to group objects. Common example: `event.category:iam AND event.type:creation AND event.type:group`. You can further distinguish group operations using the ECS `event.action` field. - - - -[float] -[[ecs-event-type-indicator]] -==== indicator - -The indicator event type is used for the subset of events within a category that contain details about indicators of compromise (IOCs). - -A common example is `event.category:threat AND event.type:indicator`. - - - -[float] -[[ecs-event-type-info]] -==== info - -The info event type is used for the subset of events within a category that indicate that they are purely informational, and don't report a state change, or any type of action. For example, an initial run of a file integrity monitoring system (FIM), where an agent reports all files under management, would fall into the "info" subcategory. Similarly, an event containing a dump of all currently running processes (as opposed to reporting that a process started/ended) would fall into the "info" subcategory. An additional common examples is `event.category:intrusion_detection AND event.type:info`. - - - -[float] -[[ecs-event-type-installation]] -==== installation - -The installation event type is used for the subset of events within a category that indicate that something was installed. A common example is `event.category:package` AND `event.type:installation`. - - - -[float] -[[ecs-event-type-protocol]] -==== protocol - -The protocol event type is used for the subset of events within a category that indicate that they contain protocol details or analysis, beyond simply identifying the protocol. Generally, network events that contain specific protocol details will fall into this subcategory. A common example is `event.category:network AND event.type:protocol AND event.type:connection AND event.type:end` (to indicate that the event is a network connection event sent at the end of a connection that also includes a protocol detail breakdown). Note that events that only indicate the name or id of the protocol should not use the protocol value. Further note that when the protocol subcategory is used, the identified protocol is populated in the ECS `network.protocol` field. - - - -[float] -[[ecs-event-type-start]] -==== start - -The start event type is used for the subset of events within a category that indicate something has started. A common example is `event.category:process AND event.type:start`. - - - -[float] -[[ecs-event-type-user]] -==== user - -The user event type is used for the subset of events within a category that are related to user objects. Common example: `event.category:iam AND event.type:deletion AND event.type:user`. You can further distinguish user operations using the ECS `event.action` field. - - - -[[ecs-allowed-values-event-outcome]] -=== ECS Categorization Field: event.outcome - -This is one of four ECS Categorization Fields, and indicates the lowest level in the ECS category hierarchy. - -`event.outcome` simply denotes whether the event represents a success or a failure from the perspective of the entity that produced the event. - -Note that when a single transaction is described in multiple events, each event may populate different values of `event.outcome`, according to their perspective. - -Also note that in the case of a compound event (a single event that contains multiple logical events), this field should be populated with the value that best captures the overall success or failure from the perspective of the event producer. - -Further note that not all events will have an associated outcome. For example, this field is generally not populated for metric events, events with `event.type:info`, or any events for which an outcome does not make logical sense. - -*Allowed Values* - -* <> -* <> -* <> - -[float] -[[ecs-event-outcome-failure]] -==== failure - -Indicates that this event describes a failed result. A common example is `event.category:file AND event.type:access AND event.outcome:failure` to indicate that a file access was attempted, but was not successful. - - - -[float] -[[ecs-event-outcome-success]] -==== success - -Indicates that this event describes a successful result. A common example is `event.category:file AND event.type:create AND event.outcome:success` to indicate that a file was successfully created. - - - -[float] -[[ecs-event-outcome-unknown]] -==== unknown - -Indicates that this event describes only an attempt for which the result is unknown from the perspective of the event producer. For example, if the event contains information only about the request side of a transaction that results in a response, populating `event.outcome:unknown` in the request event is appropriate. The unknown value should not be used when an outcome doesn't make logical sense for the event. In such cases `event.outcome` should not be populated. - - - - -include::field-values-usage.asciidoc[] diff --git a/docs/fields/fields.asciidoc b/docs/fields/fields.asciidoc deleted file mode 100644 index 4aba1029c9..0000000000 --- a/docs/fields/fields.asciidoc +++ /dev/null @@ -1,129 +0,0 @@ -[[ecs-field-reference]] -== {ecs} Field Reference - -This is the documentation of ECS version 9.1.0-dev. - -ECS defines multiple groups of related fields. They are called "field sets". -The <> field set is the only one whose fields are defined -at the root of the event. - -All other field sets are defined as objects in {es}, under which -all fields are defined. - -For a single page representation of all fields, please see the -{ecs_github_repo_link}/generated/csv/fields.csv[generated CSV of fields]. - -[float] -[[ecs-fieldsets]] -=== Field Sets -[cols="<,<",options="header",] -|===== -| Field Set | Description - -| <> | All fields defined directly at the root of the events. - -| <> | Fields about the monitoring agent. - -| <> | Fields describing an Autonomous System (Internet routing prefix). - -| <> | Fields about the client side of a network connection, used with server. - -| <> | Fields about the cloud resource. - -| <> | These fields contain information about binary code signatures. - -| <> | Fields describing the container that generated this event. - -| <> | The data_stream fields take part in defining the new data stream naming scheme. - -| <> | Fields about the destination side of a network connection, used with source. - -| <> | Fields characterizing a (mobile) device a process or application is running on. - -| <> | These fields contain information about code libraries dynamically loaded into processes. - -| <> | Fields describing DNS queries and answers. - -| <> | Meta-information specific to ECS. - -| <> | These fields contain Linux Executable Linkable Format (ELF) metadata. - -| <> | Describes an email transaction. - -| <> | Fields about errors of any kind. - -| <> | Fields breaking down the event details. - -| <> | Fields describing functions as a service. - -| <> | Fields describing files. - -| <> | Fields describing a location. - -| <> | User's group relevant to the event. - -| <> | Hashes, usually file hashes. - -| <> | Fields describing the relevant computing instance. - -| <> | Fields describing an HTTP request. - -| <> | Fields to describe observer interface information. - -| <> | Details about the event's logging mechanism. - -| <> | These fields contain Mac OS Mach Object file format (Mach-O) metadata. - -| <> | Fields describing the communication path over which the event happened. - -| <> | Fields describing an entity observing the event from outside the host. - -| <> | Fields relevant to container orchestrators. - -| <> | Fields describing the organization or company the event is associated with. - -| <> | OS fields contain information about the operating system. - -| <> | These fields contain information about an installed software package. - -| <> | These fields contain Windows Portable Executable (PE) metadata. - -| <> | These fields contain information about a process. - -| <> | Fields related to Windows Registry operations. - -| <> | Fields meant to facilitate pivoting around a piece of data. - -| <> | Fields for describing risk score and level. - -| <> | Fields to capture details about rules used to generate alerts or other notable events. - -| <> | Fields about the server side of a network connection, used with client. - -| <> | Fields describing the service for or from which the data was collected. - -| <> | Fields about the source side of a network connection, used with destination. - -| <> | Fields to classify events and alerts according to a threat taxonomy. - -| <> | Fields describing a TLS connection. - -| <> | Fields related to distributed tracing. - -| <> | Fields that let you store URLs in various forms. - -| <> | Fields to describe the user relevant to the event. - -| <> | Fields to describe a browser user_agent string. - -| <> | Fields to describe observed VLAN information. - -| <> | Fields related to storage volume details. - -| <> | Fields to describe the vulnerability relevant to an event. - -| <> | These fields contain x509 certificate metadata. - -|===== - -include::field-details.asciidoc[] diff --git a/docs/fields/usage/cloud.asciidoc b/docs/fields/usage/cloud.asciidoc deleted file mode 100644 index e0432c3afc..0000000000 --- a/docs/fields/usage/cloud.asciidoc +++ /dev/null @@ -1,78 +0,0 @@ -[[ecs-cloud-usage]] -==== Cloud Fields Usage and Examples - -Here are the subjects covered in this page. - -* <> -** <> -** <> - -[discrete] -[[ecs-cloud-usage-field-reuse]] -===== Field reuse - -The cloud fields can be used to -* <> -* or to <> that has a direct invocation relationship to the observed service or resource - -[discrete] -[[ecs-cloud-usage-cloud-at-root]] -====== Cloud fields at the Root of an Event - -Use the cloud fields at the root of an event to describe the cloud resource the event primarily relates to. -An example for this use case is a log entry being recorded for a service that is deployed in a cloud environment: - -[source,json] ------------ -{ - "cloud": { - "provider": "aws", - "region": "us-east-1", - "service": { "name": "ec2" } - } -} ------------ - -[discrete] -[[ecs-cloud-usage-origin-target]] -====== Describing external cloud resources in an invocation relationship - -Managed cloud services can be in an invocation relationship to the observed service (i.e. service for which the corresponding event is captured). -For instance, an observed service running on AWS Lambda can be invoked through an AWS API Gateway. -Another example is an observed service that invokes an external cloud service (e.g. AWS Simple Email Service). -In the context of an invocation relationship, cloud fields can be nested under `cloud.origin.*` and `cloud.target.*`, respectively, -to capture the cloud context on origin or target cloud services from the perspective of an observed service. -This concept is similar to <> under `service.origin.*` and `service.target.*`. - -Let's consider an exemplary event that represents an inbound AWS Lambda invocation coming from an AWS API Gateway. Use the following `cloud.origin.*` -nesting to describe the API Gateway service from the perspective of the AWS Lambda service: - -[source,json] ------------ -{ - "service": { <1> - "name": "MyLambdaFunction", - "version": "1.0.0", - "origin": { <2> - "name": "MyGateway", - "version" "2.0", - } - }, - "cloud": { <3> - "provider": "aws", - "region": "us-east-1", - "service": { "name": "lambda" }, - "origin": { <4> - "provider": "aws", - "region": "eu-west-1", - "service": { "name": "apigateway" } - } - } -} ------------ -<1> Describes the observed AWS Lambda function -<2> Describes the API Gateway service where the inbound request comes from -<3> Describes the cloud context of the observed AWS Lambda function -<4> Describes the cloud context of the API Gateway service where the inbound request comes from - -Note that `cloud.origin.*` and `cloud.target.*` fields should only be used on events that represent an invocation relationship. \ No newline at end of file diff --git a/docs/fields/usage/service.asciidoc b/docs/fields/usage/service.asciidoc deleted file mode 100644 index c6e217580b..0000000000 --- a/docs/fields/usage/service.asciidoc +++ /dev/null @@ -1,92 +0,0 @@ -[[ecs-service-usage]] -==== Service Fields Usage and Examples - -Here are the subjects covered in this page. - -* <> -** <> -** <> - -[discrete] -[[ecs-service-usage-field-reuse]] -===== Field reuse - -The service fields can be used to -* <> (i.e. observed service) -* or to <> to the observed service - -[discrete] -[[ecs-service-usage-service-at-root]] -====== Service fields at the Root of an Event - -Use the service fields at the root of an event to describe the service the event primarily relates to. -An example for this use case is a log entry being recorded for a particular service or appplication (e.g. `MyService`): - -[source,json] ------------ -{ - "service": { <1> - "id": "d37e5ebfe0ae6c4972dbe9f0174a1637bb8247f6", - "name": "MyService", - "version": "1.0.0" - }, - ... -} ------------ -<1> Describes the service for which the log entry is being captured - -[discrete] -[[ecs-service-usage-origin-target]] -====== Describing external services in an invocation relationship - -Multiple services can be in an invocation relationship. -Where it is possible to apply https://www.elastic.co/guide/en/apm/guide/current/apm-distributed-tracing.html[distributed tracing] on all the involved services -describe the individual services <> -and use the <> to represent the invocation relationship. - -There are situations when distributed tracing cannot be applied on some external services that are in an invocation relationship to an observed service. -Let's consider the example of a service `MyService` being deployed on a cloud provider with an upstream API gateway that passes through requests to -`MyService` (with additional context information about the API gateway itself). -To describe the API gateway as a service from the perspective of `MyService` one can self-nest the service fields under `service.origin`: - -[source,json] ------------ -{ - "service": { <1> - "id": "d37e5ebfe0ae6c4972dbe9f0174a1637bb8247f6", - "name": "MyService", - "version": "1.0.0", - "origin": { <2> - "id": "api-gateway-46372994637e2b4567", - "name": "SomeGateway", - "version" "2.5.1", - } - }, - ... -} ------------ -<1> Describes the observed service that receives the inbound request from an external service -<2> Describes the origin external service of the inbound request - -Similar to the usage of `service.origin` fields the service fields can be self-nested under `service.target.*` to describe an external target service for an outbound request: - -[source,json] ------------ -{ - "service": { <1> - "id": "d37e5ebfe0ae6c4972dbe9f0174a1637bb8247f6", - "name": "MyService", - "version": "1.0.0", - "target": { <2> - "id": "sms-service-0xe6c4272dbeAf0134", - "name": "ManagedSMSService", - "version": "1.9.0", - } - }, - ... -} ------------ -<1> Describes the observed service that emits the outbound request to an external service -<2> Describes the target external service of the outbound request - -Note that `service.origin.*` and `service.target.*` fields should only be used on events that represent an invocation relationship. diff --git a/docs/fields/usage/user.asciidoc b/docs/fields/usage/user.asciidoc deleted file mode 100644 index 6951723edb..0000000000 --- a/docs/fields/usage/user.asciidoc +++ /dev/null @@ -1,420 +0,0 @@ -[[ecs-user-usage]] -==== User Fields Usage and Examples - -Here are the subjects covered in this page. - -* <> -* <> - -* <>, or all places user fields can appear -** <> -** <> -** <> -** <> -** <> -** <> - -* <> - -* <> - -[discrete] -[[ecs-user-usage-categorization]] -===== Categorization - -User fields can be present in any kind of event, without affecting the event's -categorization. - -However when the event is about IAM (Identity and Account Management), -it should be categorized as follows. In this section we'll cover specifically -`event.category` and `event.type` with regards to IAM activity. Make sure to read -the <> to see all allowed -values, and read more about `event.kind` and `event.outcome`. - -NOTE: IAM activity is a bit particular in that events are expected to be assigned 2 event types. -One of them indicates the type of activity (creation, deletion, change, etc.), -and the other indicates whether a user or a group is the target of the management activity. - -Many sections of the examples below are elided, in order to focus on the categorization -of the events. - -Creation of group "test-group": - -```JSON -{ - "event": { - "kind": "event", - "category": ["iam"], <1> - "type": ["group", "creation"], <2> - "outcome": "success" - }, - "group": { "name": "test-group", ... }, - "user": { ... }, - "related": { "user": [ ... ] } -} -``` -<1> Category "iam" -<2> Both relevant event types to a group creation - -Adding "test-user" to "test-group": - -```JSON -{ - "event": { - "kind": "event", - "category": ["iam"], <1> - "type": ["user", "change"], <2> - "action": "user added to group", <3> - "outcome": "success" - }, - "user": { - ... - "target": { <4> - "name": "test-user", - "group": { "name": "test-group" } - } - }, - "related": { "user": [ ... ] } -} -``` -<1> Category "iam" -<2> Both relevant event types to a user modification -<3> `event.action` is not a categorization field, and has no mandated value. It can be populated based on source event details or by a pipeline, to ensure the event captures all subtleties of what's happening. -<4> How to use all possible user fields is detailed below. - -[discrete] -[[ecs-user-identifiers]] -===== User identifiers - -Different systems use different values for user identifiers. Here are a few pointers -to help normalize some simple cases. - -* When a system provides a composite value for the user name (e.g. DOMAINNAME\username), - capture the domain name in `user.domain` and the user name (without the domain) in `user.name`. -* When a system uses an email address as the main identifier, populate both - `user.id` and `user.email` with it. - -[discrete] -[[ecs-user-usage-field-reuse]] -===== Field reuse - -The user fields can be reused (or appear) in many places across ECS. This makes -it possible to capture many users relevant to a single event. - -Here's the full list of places where the user fields can appear: - -* `user.*` -* `user.effective.*` -* `user.target.*` -* `user.changes.*` -* `source.user.*` -* `destination.user.*` -* `client.user.*` -* `server.user.*` - -Let's go over the meaning of each. - -The examples below will only populate `user.name` and sometimes `user.id` inside -the various `user` nestings, for readability. -However in implementations, unless otherwise noted, all `user` fields that can -reasonably be populated in each location should be populated. - -[discrete] -[[ecs-user-usage-user-at-root]] -====== User fields at the Root of an Event - -The user fields at the root of an event are used to capture the user -performing the main action described by the event. This is especially important -when there's more than one user present on the event. `user.*` fields at the root -of the event represent the user performing the action. - -In many cases, events that only mention one user should populate the user fields -at the root of the event, even if the user is not the one performing the action. - -In cases where a purpose-specific user field such as `url.username` is populated, -`user.name` should also be populated with the same user name. - -[source,json] ------------ -{ - "url": { "username": "alice" }, <1> - "user": { "name": "alice" }, <2> - "related": { "user": ["alice"] } -} ------------ -<1> Purpose-specific username field -<2> Username copied to `user.name` to establish `user.name` as a reliable baseline. - -[discrete] -[[ecs-user-usage-remote-logons]] -====== Remote Logons - -When users are crossing host boundaries, the users are captured at -`source.user` and `destination.user`. - -Examples of data sources where this is applicable: - -* Remote logons via ssh, kerberos -* Firewalls observing network traffic - -In order to align with ECS' design of having `user` at the root of the event as the -user performing the action, all `source.user` fields should be copied to `user` at the root. - -Here's an example where user "alice" logs on to another host as user "deus": - -[source,json] ------------ -{ - "user": { - "name": "alice" - }, - "source": { - "user": { - "name": "alice" - }, - "ip": "10.42.42.42" - }, - "destination": { - "user": { - "name": "deus" - }, - "ip": "10.42.42.43" - }, - "related": { "user": ["alice", "deus"] } -} ------------ - -Whenever an event source populates the `client` and `server` fields in addition -to `source` and `destination`, the user fields should be copied accordingly as well. -You can review <> to learn more about -mapping network events. - -[discrete] -[[ecs-user-usage-privilege-changes]] -====== Privilege Changes - -The `user.effective` fields are relevant when there's a privilege escalation or demotion -and it's possible to determine the user requesting/performing the escalation. - -Use the `user` fields at the root to capture who is requesting the privilege change, -and `user.effective` to capture the requested privilege level, whether or not the -privilege change was successful. - -Here are examples where this is applicable: - -* A user changing identity on a host. -** Examples: sudo, su, Run as. -* Running a program as a different user. Examples: -** A trusted user runs a specific admin command as root via a mechanism such as the Posix setuid/setgid. -** A service manager with administrator privileges starts child processes as limited - users, for security purposes (e.g. root runs Apache HTTPD as user "apache") - -In cases where the event source only gives information about the effective user -and not who requested different privileges, the `user` fields at the root of the -event should be used instead of `user.effective`. - -Here's an example of user "alice" running a command as root via sudo: - -[source,json] ------------ -{ - "user": { - "name": "alice", - "id": "1001", - "effective": { - "name": "root", - "id": "1" - } - }, - "related": { "user": ["alice", "root"] } -} ------------ - -When it's not possible (or it's prohibitive) to determine which user is requesting -different privilege levels, it's acceptable to capture the effective user at the -root of the event. Typically a privilege change event will already have happened, -for example: bob "su" as root; and subsequent events will show the root user -performing the actions. - -[discrete] -[[ecs-user-usage-iam]] -====== Identity and Access Management - -Whenever a user is performing an action that affects another user -- typically -in IAM scenarios -- the user affected by the action is captured at -`user.target`. The user performing the IAM activity is captured at the root -of the event. - -Examples of IAM activity include: - -* user-a creates or deletes user-b -* user-a modifies user-b - -In the create/delete scenarios, there's either no prior state (user creation) -or no post state (user deletion). In these cases, only `user` at the root and -`user.target` must be populated. - -Example where "root" creates user "bob": - -[source,json] ------------ -{ - "user": { - "name": "root", - "id": "1", - "target": { - "name": "bob", - "id": "1002", - ... - } - } - "related": { "user": ["bob", "root"] } -} ------------ - -When there's a change of state to an existing user, `user.target` must be used -to capture the prior state of the user, and `user.changes` should list only -the changes that were performed. - -Example where "root" renames user "bob" to "bob.barker": - -[source,json] ------------ -{ - "user": { - "name": "root", - "id": "1", - "target": { - "name": "bob", - "id": "1002" - }, - "changes": { - "name": "bob.barker" - } - }, - "related": { "user": ["bob", "bob.barker", "root"] } -} ------------ - -You'll note in the example above that unmodified attributes like the user ID are -not repeated under `user.changes.*`, since they didn't change. - -[discrete] -[[ecs-user-usage-combining]] -====== Combining IAM and Privilege Change - -We've covered above how `user.target` and `user.changes` can be used at the same time. -If privilege escalation is also present in the same IAM event, `user.effective` -should of course be used as well. - -Here's the "rename" example from the IAM section above. In the following example, -we know "alice" is escalating privileges to "root", in order to modify user "bob": - -[source,json] ------------ -{ - "user": { - "name": "alice", - "id": "1001", - "effective": { - "name": "root", - "id": "1" - }, - "target": { - "name": "bob", - "id": "1002" - }, - "changes": { - "name": "bob.barker" - } - }, - "related": { "user": ["alice", "bob", "bob.barker", "root"] } -} ------------ - -[discrete] -[[ecs-user-usage-reuse-subtleties]] -====== Subtleties around field reuse - -Most cases of field reuse in ECS are reusing a field set inside a different field set. -Two examples of this are: - -* reusing `group` in `user`, resulting in the `user.group.*` fields, or -* reusing `user` in `destination`, resulting in the `destination.user.*` fields, - which also include `destination.user.group.*`. - -The `user` fields can also be reused within `user` as different names, -representing the role of each relevant user. Examples are the `user.target.*` or `user.effective.*` fields. - -However, it's important to note that `user` fields reused within -`user` are _not carried around anywhere else_. -Let's illustrate the various permutations of what's valid and what is not. - -[options="header"] -|===== -| Field | Validity | Notes - -| `user.group.*` | Valid | Normal reuse. -| `destination.user.group.*` | Valid | The `group` reuse gets carried around when `user` is reused elsewhere. -Populate only if relevant to the event. - -| `user.target.group.*`, `user.effective.group.*`, `user.changes.group.*` -| Valid -| The `group` reuse gets carried around even when `user` is reused within itself. -Populate only if relevant to the event. - -| `destination.user.target.*`, `destination.user.effective.*`, `destination.user.changes.*` -| *Invalid* -| The `user` fields reused within `user` are not carried around anywhere else. -The same rule applies when `user` is reused under `source`, `client` and `server`. - -|===== - - -[discrete] -[[ecs-user-usage-pivoting]] -===== Pivoting via related.user - -In all events in this page, we've populated the `related.user` fields. - -Any event that has users in it should always populate the array field `related.user` -with all usernames seen in the event; including event names that appear in custom fields. -Note that this field is not a nesting of all user fields, -it's a flat array meant to contain user identifiers. - -Taking the example from `user.changes` again, we can see that no matter the role -of the each user (before/after privilege escalation, affected user, username after rename), they are all present in `related.user`: - -[source,json] ------------ -{ - "user": { - "name": "alice", - "id": "1001", - "effective": { - "name": "root", - "id": "1" - }, - "target": { - "name": "bob", - "id": "1002" - }, - "changes": { - "name": "bob.barker" - } - }, - "related": { "user": ["alice", "root", "bob", "bob.barker"] } -} ------------ - -Like the other fields in the <> field set, `related.user` is meant to facilitate -pivoting. For example, if you have a suspicion about user "bob.barker", searching -for this name in `related.user` will give you all events related to this user, whether -it's the creation / rename of the user, or events where this user was active in a system. - -[discrete] -[[ecs-user-usage-mappings]] -===== Mapping Examples - -For examples of mapping events from various sources, you can look at -https://github.com/elastic/ecs/blob/master/rfcs/text/0007-multiple-users.md#source-data[RFC 0007 in section Source Data]. diff --git a/docs/using-ecs/images/ecs-getting-started-dashboard.png b/docs/images/ecs-getting-started-dashboard.png similarity index 100% rename from docs/using-ecs/images/ecs-getting-started-dashboard.png rename to docs/images/ecs-getting-started-dashboard.png diff --git a/docs/migrating/images/kib-create-pipeline-from-csv-preview.png b/docs/images/kib-create-pipeline-from-csv-preview.png similarity index 100% rename from docs/migrating/images/kib-create-pipeline-from-csv-preview.png rename to docs/images/kib-create-pipeline-from-csv-preview.png diff --git a/docs/migrating/images/kib-create-pipeline-from-csv.png b/docs/images/kib-create-pipeline-from-csv.png similarity index 100% rename from docs/migrating/images/kib-create-pipeline-from-csv.png rename to docs/images/kib-create-pipeline-from-csv.png diff --git a/docs/index.asciidoc b/docs/index.asciidoc deleted file mode 100644 index d1f7ab1dce..0000000000 --- a/docs/index.asciidoc +++ /dev/null @@ -1,74 +0,0 @@ -= Elastic Common Schema (ECS) Reference -:doctype: book -:ecs: ECS - -include::{asciidoc-dir}/../../shared/versions/stack/current.asciidoc[] -include::{asciidoc-dir}/../../shared/attributes.asciidoc[] - -:ecs_repo: https://github.com/elastic/ecs/ -:ecs_github_repo_link: {ecs_repo}blob/{source_branch} -:ecs_issue: {ecs_repo}issues/ -:ecs_pull: {ecs_repo}pull/ - -[[ecs-reference]] -== Overview - -This is the documentation of ECS version 9.1.0-dev. - -[float] -=== What is ECS? - -The Elastic Common Schema (ECS) is an open source specification, -developed with support from the Elastic user community. -ECS defines a common set of fields to be used when storing event data in Elasticsearch, -such as logs and metrics. - -ECS specifies field names and Elasticsearch datatypes for each field, -and provides descriptions and example usage. -ECS also groups fields into ECS levels, which are used to signal how much a field -is expected to be present. You can learn more about ECS levels in <>. -Finally, ECS also provides a set of naming guidelines for adding custom fields. - -The goal of ECS is to enable and encourage users of Elasticsearch to normalize their event data, -so that they can better analyze, visualize, and correlate the data represented in their events. -ECS has been scoped to accommodate a wide variety of events, spanning: - -- *Event sources*: whether the source of your event is an Elastic product, - a third- party product, or a custom application built by your organization. -- *Ingestion architectures*: whether the ingestion path for your events includes Beats processors, - Logstash, Elasticsearch ingest node, all of the above, or none of the above. -- *Consumers*: whether consumed by API, Kibana queries, dashboards, apps, or other means. - -[float] -=== New to ECS? - -If you're new to ECS and looking for an introduction to its benefits and examples of the -core concepts, <> is a great place to start. - -[float] -=== My events don't map with ECS - -ECS is a permissive schema. If your events have additional data that cannot be -mapped to ECS, you can simply add them to your events, using custom field names. - - -[float] -=== Maturity - -ECS improvements are released following https://semver.org/[Semantic Versioning]. -Major ECS releases are planned to be aligned with major Elastic Stack releases. - -Any feedback on the general structure, missing fields, or existing fields is appreciated. -For contributions please read the -https://github.com/elastic/ecs/blob/master/CONTRIBUTING.md[Contribution -Guidelines]. - - - -include::using-ecs/index.asciidoc[] -include::fields/fields.asciidoc[] -include::fields/field-values.asciidoc[] -include::migrating/index.asciidoc[] -include::opentelemetry/index.asciidoc[] -include::additional-info/index.asciidoc[] -include::release-notes/index.asciidoc[] diff --git a/docs/migrating/converting.asciidoc b/docs/migrating/converting.asciidoc deleted file mode 100644 index 099ee86711..0000000000 --- a/docs/migrating/converting.asciidoc +++ /dev/null @@ -1,205 +0,0 @@ -[[ecs-converting]] -=== Map custom data to ECS - -A common schema helps you correlate and use data from various sources. - -Fields for most Elastic modules and solutions (version 7.0 and later) are mapped -to the Elastic Common Schema. You may want to map data from other -implementations to ECS to help you correlate data across all of your products -and solutions. - -[float] -[[ecs-converting-before-you-start]] -==== Before you start - -Before you start a conversion, be sure that you understand the basics below. - -[float] -[[core-or-ext]] -===== Core and extended levels - -Make sure you understand the distinction between Core and Extended fields, -as explained in the <>. - -Core and Extended fields are documented in the <> or, for -a single page representation of all fields, please see the -{ecs_github_repo_link}/generated/csv/fields.csv[generated CSV of fields]. - -[float] -[[ecs-conv]] -===== An approach to mapping an existing implementation - -Here's the recommended approach for converting an existing implementation to {ecs}. - -. Review each field in your original event and map it to the relevant ECS field. - - - Start by mapping your field to the relevant ECS Core field. - - If a relevant ECS Core field does not exist, map your field to the relevant ECS Extended field. - - If no relevant ECS Extended field exists, consider keeping your field with its original details, - or possibly renaming it using ECS naming guidelines and attempt to map one - or more of your original event fields to it. - -. Review each ECS Core field, and attempt to populate it. - - - Review your original event data again - - Consider populating the field based on additional meta-data such as static - information (e.g. add `event.category:authentication` even if your auth events - don't mention the word "authentication") - - Consider capturing additional environment meta-data, such as information about the - host, container or cloud instance. - -. Review other extended fields from any field set you are already using, and - attempt to populate it as well. - -. Set `ecs.version` to the version of the schema you are conforming to. This will - allow you to upgrade your sources, pipelines and content (like dashboards) - smoothly in the future. - -[float] -[[ecs-conv-spreadsheet]] -===== Use a spreadsheet to plan your migration - -Using a spreadsheet to plan the migration from pre-existing source fields to ECS -is a common practice. It's a good way to address each of your fields -methodically among colleagues. - -To help you plan your migration, Elastic offers a -https://ela.st/sample-pipeline-mapping[spreadsheet template]. You can use a CSV -version of this spreadsheet to -<>. This is the easiest and most consistent way to map your custom data -to ECS, regardless of your ingest method. - -[float] -[[ecs-map-custom-data-to-ecs-es-pipeline]] -==== Map custom data to ECS using an {es} ingest pipeline - -Use {kib}'s **Create pipeline from CSV** feature to create an {es} ingest -pipeline from a CSV file that maps custom data to ECS fields. - -Before you start, ensure you meet the -{ref}/ingest.html#ingest-prerequisites[prerequisites] to create ingest pipelines -in {kib}. - -. Download or make a copy of the -https://ela.st/sample-pipeline-mapping[spreadsheet template]. - -. Use the spreadsheet to map your custom data to ECS fields. While you can -include additional columns, {kib} only processes the following supported -columns. Other columns are ignored. -+ -.**Supported columns** -[role="child_attributes"] -[%collapsible] -==== -`source_field`:: -(Required) JSON field key from your custom data. Supports dot notation. Rows -with an empty `source_field` are skipped. - -`destination_field`:: -(Required) ECS field name. Supports dot notation. To perform a -`format_action` without renaming the field, leave `destination_field` empty. -+ -If the `destination field` is `@timestamp`, a `format_action` of -`parse_timestamp` and a `timestamp_format` of `UNIX_MS` are used, regardless of -any provided values. This helps prevent downstream conversion problems. - -`format_action`:: -(Optional) Conversion to apply to the field value. -+ -[%collapsible%open] -.Valid values -===== -(empty):: -No conversion. - -`parse_timestamp`:: -Formats a date or time value. To specify a format, use `timestamp_format`. - -`to_array`:: -Converts to an array. - -`to_boolean`:: -Converts to a boolean. - -`to_float`:: -Converts to a floating point number. - -`to_integer`:: -Converts to an integer - -`to_string`:: -Converts to a string. - -`lowercase`:: -Converts to lowercase. - -`uppercase`:: -Converts to uppercase. -===== - -`timestamp_format`:: -(Optional) Time and date format to use with the `parse_timestamp` format action. -Valid values are `UNIX`, `UNIX_MS`, `ISO8601`, `TAI64N`, and -{ref}/mapping-date-format.html[Java time patterns]. Defaults to `UNIX_MS`. - -`copy_action`:: -(Optional) Action to take on the `source_field`. Valid values are: -+ -[%collapsible%open] -.Valid values -===== -(empty):: -(Default) Uses the default action. You'll specify the default action later -on {kib}'s **Create pipeline from CSV** page. - -`copy`:: -Makes a copy of the `source_field` to use as the `destination_field`. -The final document contains both fields. -+ -`rename`:: -Renames the `source_field` to the `destination_field`. The final document only -contains the `destination_field`. -===== -==== - -. Save and export your spreadsheet as a CSV file. -+ -NOTE: {kib}'s **Create pipeline from CSV** feature only supports CSV files up to -100 MB. - -. In {kib}, open the main menu and click **Stack Management > Ingest Pipelines > -Create pipeline > New pipeline from CSV**. -+ -[role="screenshot"] -image::images/kib-create-pipeline-from-csv.png[Create Pipeline from CSV in Kibana,align="center"] - -. On the **Create pipeline from CSV** page, upload your CSV file. - -. Under **Default action**, select the **Copy field name** or **Rename field** -option. -+ -For the **Copy field name** option, {kib} makes a copy of the `source_field` to -use as the `destination_field` by default. The final document contains both -fields. -+ -For the **Rename field** option, renames the `source_field` to the -`destination_field` by default. The final document only contains the -`destination_field`. -+ -You can override this default using the `copy_action` column of your CSV. - -. Click **Process CSV**. -+ -{kib} displays a JSON preview of the ingest pipeline generated from your CSV -file. -+ -[role="screenshot"] -image::images/kib-create-pipeline-from-csv-preview.png[Preview pipeline from CSV in Kibana,align="center"] - -. To create the pipeline, click **Continue to create pipeline**. - -. On the **Create pipeline** page, you can add additional ingest processors to -your pipeline. -+ -When you're done, click **Create pipeline**. diff --git a/docs/migrating/index.asciidoc b/docs/migrating/index.asciidoc deleted file mode 100644 index 655cc2fc3f..0000000000 --- a/docs/migrating/index.asciidoc +++ /dev/null @@ -1,11 +0,0 @@ -[[migrating-to-ecs]] -== Migrating to ECS - -There are multiple ways to reap the benefit of ECS. -The simplest is to use <>. - -If you have a custom pipeline or application you would like to convert to ECS, -please have a look at <>. - -include::products-solutions.asciidoc[] -include::converting.asciidoc[] diff --git a/docs/migrating/products-solutions.asciidoc b/docs/migrating/products-solutions.asciidoc deleted file mode 100644 index cea4c957dc..0000000000 --- a/docs/migrating/products-solutions.asciidoc +++ /dev/null @@ -1,18 +0,0 @@ -[[ecs-products-solutions]] -=== Products and Solutions that Support ECS - -The following Elastic products support ECS out of the box, as of version 7.0: - -* {beats-ref}/beats-reference.html[{beats}] -* {apm-guide-ref}/apm-overview.html[APM] -* {security-guide}/es-overview.html[Elastic Security] -** {security-guide}/siem-field-reference.html[Elastic Security Field Reference] - a list of ECS fields used in the Security app -* https://www.elastic.co/products/endpoint-security[Elastic Endpoint Security -Server] -* {observability-guide}/monitor-logs.html[Log Monitoring] -* Log formatters that support ECS out of the box for various languages can be found - https://github.com/elastic/ecs-logging/blob/master/README.md[here]. -* {observability-guide}/analyze-metrics.html[Metrics Monitoring] -* {ls}' {es} output has an {logstash-ref}/plugins-outputs-elasticsearch.html#_compatibility_with_the_elastic_common_schema_ecs[ECS compatibility mode] - -// TODO Insert community & partner solutions here diff --git a/docs/opentelemetry/index.asciidoc b/docs/opentelemetry/index.asciidoc deleted file mode 100644 index 89f5997701..0000000000 --- a/docs/opentelemetry/index.asciidoc +++ /dev/null @@ -1,69 +0,0 @@ -[[ecs-opentelemetry]] -== ECS & OpenTelemetry - -In April 2023, Elastic donated ECS to OpenTelemetry and together with the OTel community https://opentelemetry.io/blog/2023/ecs-otel-semconv-convergence/[jointly announced] -the intention to achieve convergence of ECS and https://opentelemetry.io/docs/specs/semconv/[OTel Semantic Conventions (SemConv)]. -This donation should be seen as a directional decision for the evolution of both standards -rather than a single event that merged both schemas into a single standard. - -While both schemes complement each other in large areas and thus offer mutual added value and great potential for convergence, -it is important to understand that in some areas convergence is not achievable due to conceptual differences or consistency reasons. -For example, while ECS contains a few fields (e.g. https://www.elastic.co/guide/en/ecs/current/ecs-container.html#field-container-disk-read-bytes[container.disk.read.bytes]) -that are meant to be used in a metric-like way, metrics in OpenTelemetry are following a completely different data model (with metric name, type, dimensions, etc.). -Also, in OTel semantic conventions there are some stable (or quasi-stable) attributes that have semantically equivalent, stable fields in ECS, however, with different field names. -This kind of differences require explicit handling to achieve compatibility. - -[float] -[[ecs-opentelemetry-relation]] -=== Relation between ECS and Semantic Conventions - -The https://github.com/elastic/ecs/tree/main/schemas[ECS schema files] contain an explicit mapping between ECS fields and corresponding OTel semnatic convention attributes. -This can be used to generate tooling for compatibility between ECS and semnatic conventions (e.g. alias fields in Elasticsearch). -The relation between individual ECS fields and corresponding OTel semantic conventions attributes follows one of the following categories: - -[%header] -|==== -| Category | Description - -| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match] -| The name of the ECS field is identical to the SemConv attribute name and has (practically) the same semantics. - -| image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent] -| The ECS field name is different but has the same semantics as the corresponding SemConv attribute. -For this type of relation aliasing approaches (e.g. https://www.elastic.co/guide/en/elasticsearch/reference/current/field-alias.html[Elasticsearch field aliases]) can be used to achieve compatibility between ECS and OTel SemConv. - -| image:https://img.shields.io/badge/related-efc20d?style=flat[relation,title=related] -| The ECS field name is different and has related - yet different - semantics as the corresponding SemConv attribute. -An aliasing approach is not sufficient to resolve compatibility for this type of relation. - -| image:https://img.shields.io/badge/conflict-910000?style=flat[relation,title=conflict] -| The ECS field name is the same as an OTel SemConv namespace or an attribute that has significantly different semantics. - -| image:https://img.shields.io/badge/OTLP-ffdcb2?style=flat[relation,title=OTLP] -| The ECS field has a corresponding representation in https://github.com/open-telemetry/opentelemetry-proto[OpenTelemetry's protocol definition]. - -| image:https://img.shields.io/badge/metric-cb00cb?style=flat[relation,title=metric] -| For this ECS field there is a corresponding metric defined in OTel SemConv. - -| image:https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat[relation,title=na] -| The ECS field is not applicable in the context of OTel or won't be aligned due to significant, conceptual conflict with OTel concepts in that area. -|==== - - - -The following documentation pages provide an overview and more details on the alignment between ECS and OTel semantic conventions: - -- <> -- <> - - - -[[ecs-otel-alignment-overview]] -=== OTel Alignment Overview - -include::otel-mapping-summary.asciidoc[] - -[[ecs-otel-alignment-details]] -=== Field & Attributes Alignment - -include::otel-fields-mapping.asciidoc[] diff --git a/docs/opentelemetry/otel-fields-mapping.asciidoc b/docs/opentelemetry/otel-fields-mapping.asciidoc deleted file mode 100644 index d7d2d112f2..0000000000 --- a/docs/opentelemetry/otel-fields-mapping.asciidoc +++ /dev/null @@ -1,2568 +0,0 @@ -//// -This file is automatically generated. Don't edit it manually! -//// -The following table gives an overview of mappings between individual ECS fields (in ECS version `9.1.0-dev`) -and corresponding OTel semantic convention attributes (in SemConv version `1.29.0`). - -[%header] -|==== -| ECS Field | Relation | OTel Semantic Conventions Attribute | Stability - -[[otel-mapping-namespace-base]] -4+e| *Base Fields* - -.3+| -[[otel-mapping-for-timestamp]] -<> - - - -^| image:https://img.shields.io/badge/OTLP-ffdcb2?style=flat[relation,title=otlp,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://github.com/search?q=repo%3Aopen-telemetry%2Fopentelemetry-proto+%22\+time_unix_nano+%22&type=code[time_unix_nano] + - - -^|image:https://img.shields.io/badge/✔-93c93e?style=flat[stable,title=stable] - -// =============================================================== - - -^| image:https://img.shields.io/badge/OTLP-ffdcb2?style=flat[relation,title=otlp,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://github.com/search?q=repo%3Aopen-telemetry%2Fopentelemetry-proto+%22\+observed_time_unix_nano+%22&type=code[observed_time_unix_nano] + - - -^|image:https://img.shields.io/badge/✔-93c93e?style=flat[stable,title=stable] - -// =============================================================== - - -^| image:https://img.shields.io/badge/OTLP-ffdcb2?style=flat[relation,title=otlp,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://github.com/search?q=repo%3Aopen-telemetry%2Fopentelemetry-proto+%22\+start_time_unix_nano+%22&type=code[start_time_unix_nano] + - -Note: On logs, events and metrics, `time_unix_nano` defines the time of the event. For logs, `observed_time_unix_nano` defines the time when the event was observed by the collection system. On spans, `start_time_unix_nano` represents the start time of the span. - - - -^|image:https://img.shields.io/badge/✔-93c93e?style=flat[stable,title=stable] - -// =============================================================== -.2+| -[[otel-mapping-for-labels]] -<> - - - -^| image:https://img.shields.io/badge/related-efc20d?style=flat[relation,title=related,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/k8s/#k8s-pod-label[k8s.pod.label] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== - - -^| image:https://img.shields.io/badge/related-efc20d?style=flat[relation,title=related,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/container/#container-label[container.label] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-message]] -<> - - - -^| image:https://img.shields.io/badge/OTLP-ffdcb2?style=flat[relation,title=otlp,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://github.com/search?q=repo%3Aopen-telemetry%2Fopentelemetry-proto+%22\+body+%22&type=code[body] + - -Note: The `body` in OTLP is of type `Any` and can be either an unstructured log message or a structured event. - - -^|image:https://img.shields.io/badge/✔-93c93e?style=flat[stable,title=stable] - -// =============================================================== -[[otel-mapping-namespace-client]] -4+e| *Client Fields* - -.1+| -[[otel-mapping-for-client-address]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/client/#client-address[client.address] + - - -^|image:https://img.shields.io/badge/✔-93c93e?style=flat[stable,title=stable] - -// =============================================================== -.1+| -[[otel-mapping-for-client-port]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/client/#client-port[client.port] + - - -^|image:https://img.shields.io/badge/✔-93c93e?style=flat[stable,title=stable] - -// =============================================================== -[[otel-mapping-namespace-cloud]] -4+e| *Cloud Fields* - -.1+| -[[otel-mapping-for-cloud-account-id]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/cloud/#cloud-account-id[cloud.account.id] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-cloud-availability-zone]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/cloud/#cloud-availability-zone[cloud.availability_zone] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-cloud-provider]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/cloud/#cloud-provider[cloud.provider] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-cloud-region]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/cloud/#cloud-region[cloud.region] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-cloud-service-name]] -<> - - - -^| image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/cloud/#cloud-platform[cloud.platform] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -[[otel-mapping-namespace-container]] -4+e| *Container Fields* - -.1+| -[[otel-mapping-for-container-cpu-usage]] -<> - - - -^| image:https://img.shields.io/badge/metric-cb00cb?style=flat[relation,title=metric,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--\+semconv+metric.container.cpu.usage+--%3E%22&type=code[container.cpu.usage] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-container-id]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/container/#container-id[container.id] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-container-image-hash-all]] -<> - - - -^| image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/container/#container-image-repo-digests[container.image.repo_digests] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-container-image-name]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/container/#container-image-name[container.image.name] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-container-image-tag]] -<> - - - -^| image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/container/#container-image-tags[container.image.tags] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-container-labels]] -<> - - - -^| image:https://img.shields.io/badge/related-efc20d?style=flat[relation,title=related,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/container/#container-label[container.label] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-container-memory-usage]] -<> - - - -^| image:https://img.shields.io/badge/metric-cb00cb?style=flat[relation,title=metric,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--\+semconv+metric.container.memory.usage+--%3E%22&type=code[container.memory.usage] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-container-name]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/container/#container-name[container.name] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-container-runtime]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/container/#container-runtime[container.runtime] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -[[otel-mapping-namespace-data_stream]] -4+e| *Data Stream Fields* - -.1+| -[[otel-mapping-for-data-stream-dataset]] -<> [beta] - - - -^| image:https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat[relation,title=not-applicable,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| Not applicable. + - - -^| - -// =============================================================== -.1+| -[[otel-mapping-for-data-stream-namespace]] -<> [beta] - - - -^| image:https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat[relation,title=not-applicable,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| Not applicable. + - - -^| - -// =============================================================== -.1+| -[[otel-mapping-for-data-stream-type]] -<> [beta] - - - -^| image:https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat[relation,title=not-applicable,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| Not applicable. + - - -^| - -// =============================================================== -[[otel-mapping-namespace-destination]] -4+e| *Destination Fields* - -.1+| -[[otel-mapping-for-destination-address]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/destination/#destination-address[destination.address] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-destination-port]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/destination/#destination-port[destination.port] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -[[otel-mapping-namespace-device]] -4+e| *Device Fields* - -.1+| -[[otel-mapping-for-device-id]] -<> [beta] - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/device/#device-id[device.id] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-device-manufacturer]] -<> [beta] - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/device/#device-manufacturer[device.manufacturer] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-device-model-identifier]] -<> [beta] - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/device/#device-model-identifier[device.model.identifier] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-device-model-name]] -<> [beta] - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/device/#device-model-name[device.model.name] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -[[otel-mapping-namespace-dns]] -4+e| *DNS Fields* - -.1+| -[[otel-mapping-for-dns-question-name]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/dns/#dns-question-name[dns.question.name] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -[[otel-mapping-namespace-ecs]] -4+e| *ECS Fields* - -.1+| -[[otel-mapping-for-ecs-version]] -<> - - - -^| image:https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat[relation,title=not-applicable,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| Not applicable. + - - -^| - -// =============================================================== -[[otel-mapping-namespace-error]] -4+e| *Error Fields* - -.1+| -[[otel-mapping-for-error-message]] -<> - - - -^| image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/exception/#exception-message[exception.message] + - - -^|image:https://img.shields.io/badge/✔-93c93e?style=flat[stable,title=stable] - -// =============================================================== -.1+| -[[otel-mapping-for-error-stack-trace]] -<> - - - -^| image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/exception/#exception-stacktrace[exception.stacktrace] + - - -^|image:https://img.shields.io/badge/✔-93c93e?style=flat[stable,title=stable] - -// =============================================================== -.1+| -[[otel-mapping-for-error-type]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/error/#error-type[error.type] + - - -^|image:https://img.shields.io/badge/✔-93c93e?style=flat[stable,title=stable] - -// =============================================================== -[[otel-mapping-namespace-faas]] -4+e| *FaaS Fields* - -.1+| -[[otel-mapping-for-faas-coldstart]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/faas/#faas-coldstart[faas.coldstart] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-faas-execution]] -<> - - - -^| image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/faas/#faas-invocation-id[faas.invocation_id] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-faas-name]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/faas/#faas-name[faas.name] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-faas-trigger-type]] -<> - - - -^| image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/faas/#faas-trigger[faas.trigger] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-faas-version]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/faas/#faas-version[faas.version] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -[[otel-mapping-namespace-file]] -4+e| *File Fields* - -.1+| -[[otel-mapping-for-file-accessed]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-accessed[file.accessed] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-file-attributes]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-attributes[file.attributes] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-file-created]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-created[file.created] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-file-ctime]] -<> - - - -^| image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-changed[file.changed] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-file-directory]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-directory[file.directory] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-file-extension]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-extension[file.extension] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-file-fork-name]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-fork-name[file.fork_name] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-file-gid]] -<> - - - -^| image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-group-id[file.group.id] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-file-group]] -<> - - - -^| image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-group-name[file.group.name] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-file-inode]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-inode[file.inode] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-file-mode]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-mode[file.mode] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-file-mtime]] -<> - - - -^| image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-modified[file.modified] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-file-name]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-name[file.name] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-file-owner]] -<> - - - -^| image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-owner-name[file.owner.name] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-file-path]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-path[file.path] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-file-size]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-size[file.size] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-file-target-path]] -<> - - - -^| image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-symbolic-link-target-path[file.symbolic_link.target_path] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-file-uid]] -<> - - - -^| image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-owner-id[file.owner.id] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -[[otel-mapping-namespace-geo]] -4+e| *Geo Fields* - -.1+| -[[otel-mapping-for-geo-city-name]] -<> - - - -^| image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/geo/#geo-locality-name[geo.locality.name] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-geo-continent-code]] -<> - - - -^| image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/geo/#geo-continent-code[geo.continent.code] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-geo-country-iso-code]] -<> - - - -^| image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/geo/#geo-country-iso-code[geo.country.iso_code] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.2+| -[[otel-mapping-for-geo-location]] -<> - - - -^| image:https://img.shields.io/badge/related-efc20d?style=flat[relation,title=related,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/geo/#geo-location-lat[geo.location.lat] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== - - -^| image:https://img.shields.io/badge/related-efc20d?style=flat[relation,title=related,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/geo/#geo-location-lon[geo.location.lon] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-geo-postal-code]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/geo/#geo-postal-code[geo.postal_code] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-geo-region-iso-code]] -<> - - - -^| image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/geo/#geo-region-iso-code[geo.region.iso_code] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -[[otel-mapping-namespace-host]] -4+e| *Host Fields* - -.1+| -[[otel-mapping-for-host-architecture]] -<> - - - -^| image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/host/#host-arch[host.arch] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-host-cpu-usage]] -<> - - - -^| image:https://img.shields.io/badge/metric-cb00cb?style=flat[relation,title=metric,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--\+semconv+metric.system.cpu.utilization+--%3E%22&type=code[system.cpu.utilization] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-host-disk-read-bytes]] -<> - - - -^| image:https://img.shields.io/badge/metric-cb00cb?style=flat[relation,title=metric,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--\+semconv+metric.system.disk.io+--%3E%22&type=code[system.disk.io] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-host-disk-write-bytes]] -<> - - - -^| image:https://img.shields.io/badge/metric-cb00cb?style=flat[relation,title=metric,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--\+semconv+metric.system.disk.io+--%3E%22&type=code[system.disk.io] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-host-id]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/host/#host-id[host.id] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-host-ip]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/host/#host-ip[host.ip] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-host-mac]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/host/#host-mac[host.mac] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-host-name]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/host/#host-name[host.name] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-host-network-egress-bytes]] -<> - - - -^| image:https://img.shields.io/badge/metric-cb00cb?style=flat[relation,title=metric,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--\+semconv+metric.system.network.io+--%3E%22&type=code[system.network.io] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-host-network-egress-packets]] -<> - - - -^| image:https://img.shields.io/badge/metric-cb00cb?style=flat[relation,title=metric,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--\+semconv+metric.system.network.packets+--%3E%22&type=code[system.network.packets] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-host-network-ingress-bytes]] -<> - - - -^| image:https://img.shields.io/badge/metric-cb00cb?style=flat[relation,title=metric,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--\+semconv+metric.system.network.io+--%3E%22&type=code[system.network.io] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-host-network-ingress-packets]] -<> - - - -^| image:https://img.shields.io/badge/metric-cb00cb?style=flat[relation,title=metric,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--\+semconv+metric.system.network.packets+--%3E%22&type=code[system.network.packets] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-host-type]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/host/#host-type[host.type] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-host-uptime]] -<> - - - -^| image:https://img.shields.io/badge/metric-cb00cb?style=flat[relation,title=metric,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--\+semconv+metric.system.uptime+--%3E%22&type=code[system.uptime] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -[[otel-mapping-namespace-http]] -4+e| *HTTP Fields* - -.1+| -[[otel-mapping-for-http-request-body-bytes]] -<> - - - -^| image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/http/#http-request-body-size[http.request.body.size] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-http-request-bytes]] -<> - - - -^| image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/http/#http-request-size[http.request.size] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.2+| -[[otel-mapping-for-http-request-method]] -<> - - - -^| image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/http/#http-request-method-original[http.request.method_original] + - - -^|image:https://img.shields.io/badge/✔-93c93e?style=flat[stable,title=stable] - -// =============================================================== - - -^| image:https://img.shields.io/badge/conflict-910000?style=flat[relation,title=conflict,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/http/#http-request-method[http.request.method] + - -Note: `http.request.method` in SemConv is the known, normalized, upper case value of the request method, other than the ECS' `http.request.method` that retains casing from the original event. - - - -^|image:https://img.shields.io/badge/✔-93c93e?style=flat[stable,title=stable] - -// =============================================================== -.1+| -[[otel-mapping-for-http-response-body-bytes]] -<> - - - -^| image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/http/#http-response-body-size[http.response.body.size] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-http-response-bytes]] -<> - - - -^| image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/http/#http-response-size[http.response.size] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-http-response-status-code]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/http/#http-response-status-code[http.response.status_code] + - - -^|image:https://img.shields.io/badge/✔-93c93e?style=flat[stable,title=stable] - -// =============================================================== -.2+| -[[otel-mapping-for-http-version]] -<> - - - -^| image:https://img.shields.io/badge/related-efc20d?style=flat[relation,title=related,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/network/#network-protocol-name[network.protocol.name] + - - -^|image:https://img.shields.io/badge/✔-93c93e?style=flat[stable,title=stable] - -// =============================================================== - - -^| image:https://img.shields.io/badge/related-efc20d?style=flat[relation,title=related,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/network/#network-protocol-version[network.protocol.version] + - -Note: In OTel SemConv, `network.protocol.version` specifies the HTTP version if the value of `network.protocol.name` is `http`. - - - -^|image:https://img.shields.io/badge/✔-93c93e?style=flat[stable,title=stable] - -// =============================================================== -[[otel-mapping-namespace-log]] -4+e| *Log Fields* - -.1+| -[[otel-mapping-for-log-file-path]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/log/#log-file-path[log.file.path] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-log-level]] -<> - - - -^| image:https://img.shields.io/badge/OTLP-ffdcb2?style=flat[relation,title=otlp,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://github.com/search?q=repo%3Aopen-telemetry%2Fopentelemetry-proto+%22\+severity_text+%22&type=code[severity_text] + - - -^|image:https://img.shields.io/badge/✔-93c93e?style=flat[stable,title=stable] - -// =============================================================== -[[otel-mapping-namespace-network]] -4+e| *Network Fields* - -.1+| -[[otel-mapping-for-network-protocol]] -<> - - - -^| image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/network/#network-protocol-name[network.protocol.name] + - - -^|image:https://img.shields.io/badge/✔-93c93e?style=flat[stable,title=stable] - -// =============================================================== -.1+| -[[otel-mapping-for-network-transport]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/network/#network-transport[network.transport] + - - -^|image:https://img.shields.io/badge/✔-93c93e?style=flat[stable,title=stable] - -// =============================================================== -.1+| -[[otel-mapping-for-network-type]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/network/#network-type[network.type] + - - -^|image:https://img.shields.io/badge/✔-93c93e?style=flat[stable,title=stable] - -// =============================================================== -[[otel-mapping-namespace-os]] -4+e| *Operating System Fields* - -.1+| -[[otel-mapping-for-os-full]] -<> - - - -^| image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/os/#os-description[os.description] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-os-name]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/os/#os-name[os.name] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-os-type]] -<> - - - -^| image:https://img.shields.io/badge/conflict-910000?style=flat[relation,title=conflict,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/os/#os-type[os.type] + - -Note: The expected values in ECS for `os.type` do not correspond with the values defined in semantic conventions! - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-os-version]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/os/#os-version[os.version] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -[[otel-mapping-namespace-process]] -4+e| *Process Fields* - -.1+| -[[otel-mapping-for-process-args]] -<> - - - -^| image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-command-args[process.command_args] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-process-args-count]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-args-count[process.args_count] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-process-command-line]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-command-line[process.command_line] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-process-executable]] -<> - - - -^| image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-executable-path[process.executable.path] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -process.real_user.id - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-real-user-id[process.real_user.id] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -process.saved_user.id - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-saved-user-id[process.saved_user.id] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -process.user.id - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-user-id[process.user.id] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-process-interactive]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-interactive[process.interactive] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -process.real_user.name - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-real-user-name[process.real_user.name] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -process.saved_user.name - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-saved-user-name[process.saved_user.name] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -process.user.name - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-user-name[process.user.name] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -process.group_leader.pid - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-group-leader-pid[process.group_leader.pid] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-process-pid]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-pid[process.pid] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -process.session_leader.pid - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-session-leader-pid[process.session_leader.pid] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-process-title]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-title[process.title] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-process-uptime]] -<> - - - -^| image:https://img.shields.io/badge/metric-cb00cb?style=flat[relation,title=metric,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--\+semconv+metric.process.uptime+--%3E%22&type=code[process.uptime] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-process-vpid]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-vpid[process.vpid] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-process-working-directory]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-working-directory[process.working_directory] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -[[otel-mapping-namespace-related]] -4+e| *Related Fields* - -.1+| -[[otel-mapping-for-related-hash]] -<> - - - -^| image:https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat[relation,title=not-applicable,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| Not applicable. + - - -^| - -// =============================================================== -.1+| -[[otel-mapping-for-related-hosts]] -<> - - - -^| image:https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat[relation,title=not-applicable,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| Not applicable. + - - -^| - -// =============================================================== -.1+| -[[otel-mapping-for-related-ip]] -<> - - - -^| image:https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat[relation,title=not-applicable,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| Not applicable. + - - -^| - -// =============================================================== -.1+| -[[otel-mapping-for-related-user]] -<> - - - -^| image:https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat[relation,title=not-applicable,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| Not applicable. + - - -^| - -// =============================================================== -[[otel-mapping-namespace-server]] -4+e| *Server Fields* - -.1+| -[[otel-mapping-for-server-address]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/server/#server-address[server.address] + - - -^|image:https://img.shields.io/badge/✔-93c93e?style=flat[stable,title=stable] - -// =============================================================== -.1+| -[[otel-mapping-for-server-port]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/server/#server-port[server.port] + - - -^|image:https://img.shields.io/badge/✔-93c93e?style=flat[stable,title=stable] - -// =============================================================== -[[otel-mapping-namespace-service]] -4+e| *Service Fields* - -.1+| -[[otel-mapping-for-service-environment]] -<> [beta] - - - -^| image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/deployment/#deployment-environment-name[deployment.environment.name] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-service-name]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/service/#service-name[service.name] + - - -^|image:https://img.shields.io/badge/✔-93c93e?style=flat[stable,title=stable] - -// =============================================================== -.1+| -[[otel-mapping-for-service-node-name]] -<> - - - -^| image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/service/#service-instance-id[service.instance.id] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-service-version]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/service/#service-version[service.version] + - - -^|image:https://img.shields.io/badge/✔-93c93e?style=flat[stable,title=stable] - -// =============================================================== -[[otel-mapping-namespace-source]] -4+e| *Source Fields* - -.1+| -[[otel-mapping-for-source-address]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/source/#source-address[source.address] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-source-port]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/source/#source-port[source.port] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -[[otel-mapping-namespace-tls]] -4+e| *TLS Fields* - -.1+| -[[otel-mapping-for-tls-cipher]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-cipher[tls.cipher] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-tls-client-certificate]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-certificate[tls.client.certificate] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-tls-client-certificate-chain]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-certificate-chain[tls.client.certificate_chain] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-tls-client-hash-md5]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-hash-md5[tls.client.hash.md5] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-tls-client-hash-sha1]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-hash-sha1[tls.client.hash.sha1] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-tls-client-hash-sha256]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-hash-sha256[tls.client.hash.sha256] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-tls-client-issuer]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-issuer[tls.client.issuer] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-tls-client-ja3]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-ja3[tls.client.ja3] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-tls-client-not-after]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-not-after[tls.client.not_after] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-tls-client-not-before]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-not-before[tls.client.not_before] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-tls-client-subject]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-subject[tls.client.subject] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-tls-client-supported-ciphers]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-supported-ciphers[tls.client.supported_ciphers] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-tls-curve]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-curve[tls.curve] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-tls-established]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-established[tls.established] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-tls-next-protocol]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-next-protocol[tls.next_protocol] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-tls-resumed]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-resumed[tls.resumed] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-tls-server-certificate]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-certificate[tls.server.certificate] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-tls-server-certificate-chain]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-certificate-chain[tls.server.certificate_chain] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-tls-server-hash-md5]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-hash-md5[tls.server.hash.md5] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-tls-server-hash-sha1]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-hash-sha1[tls.server.hash.sha1] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-tls-server-hash-sha256]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-hash-sha256[tls.server.hash.sha256] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-tls-server-issuer]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-issuer[tls.server.issuer] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-tls-server-ja3s]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-ja3s[tls.server.ja3s] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-tls-server-not-after]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-not-after[tls.server.not_after] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-tls-server-not-before]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-not-before[tls.server.not_before] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-tls-server-subject]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-subject[tls.server.subject] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -[[otel-mapping-namespace-tracing]] -4+e| *Tracing Fields* - -.1+| -[[otel-mapping-for-span-id]] -<> - - - -^| image:https://img.shields.io/badge/OTLP-ffdcb2?style=flat[relation,title=otlp,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://github.com/search?q=repo%3Aopen-telemetry%2Fopentelemetry-proto+%22\+span_id+%22&type=code[span_id] + - - -^|image:https://img.shields.io/badge/✔-93c93e?style=flat[stable,title=stable] - -// =============================================================== -.1+| -[[otel-mapping-for-trace-id]] -<> - - - -^| image:https://img.shields.io/badge/OTLP-ffdcb2?style=flat[relation,title=otlp,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://github.com/search?q=repo%3Aopen-telemetry%2Fopentelemetry-proto+%22\+trace_id+%22&type=code[trace_id] + - - -^|image:https://img.shields.io/badge/✔-93c93e?style=flat[stable,title=stable] - -// =============================================================== -.1+| -[[otel-mapping-for-transaction-id]] -<> - - - -^| image:https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat[relation,title=not-applicable,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| Not applicable. + - - -^| - -// =============================================================== -[[otel-mapping-namespace-url]] -4+e| *URL Fields* - -.1+| -[[otel-mapping-for-url-domain]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-domain[url.domain] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-url-extension]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-extension[url.extension] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-url-fragment]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-fragment[url.fragment] + - - -^|image:https://img.shields.io/badge/✔-93c93e?style=flat[stable,title=stable] - -// =============================================================== -.1+| -[[otel-mapping-for-url-full]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-full[url.full] + - - -^|image:https://img.shields.io/badge/✔-93c93e?style=flat[stable,title=stable] - -// =============================================================== -.1+| -[[otel-mapping-for-url-original]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-original[url.original] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-url-path]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-path[url.path] + - - -^|image:https://img.shields.io/badge/✔-93c93e?style=flat[stable,title=stable] - -// =============================================================== -.1+| -[[otel-mapping-for-url-port]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-port[url.port] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-url-query]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-query[url.query] + - - -^|image:https://img.shields.io/badge/✔-93c93e?style=flat[stable,title=stable] - -// =============================================================== -.1+| -[[otel-mapping-for-url-registered-domain]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-registered-domain[url.registered_domain] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-url-scheme]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-scheme[url.scheme] + - - -^|image:https://img.shields.io/badge/✔-93c93e?style=flat[stable,title=stable] - -// =============================================================== -.1+| -[[otel-mapping-for-url-subdomain]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-subdomain[url.subdomain] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-url-top-level-domain]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-top-level-domain[url.top_level_domain] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -[[otel-mapping-namespace-user]] -4+e| *User Fields* - -.1+| -[[otel-mapping-for-user-email]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/user/#user-email[user.email] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-user-full-name]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/user/#user-full-name[user.full_name] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-user-hash]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/user/#user-hash[user.hash] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-user-id]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/user/#user-id[user.id] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-user-name]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/user/#user-name[user.name] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-user-roles]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/user/#user-roles[user.roles] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -[[otel-mapping-namespace-user_agent]] -4+e| *User agent Fields* - -.1+| -[[otel-mapping-for-user-agent-name]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/user_agent/#user-agent-name[user_agent.name] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -.1+| -[[otel-mapping-for-user-agent-original]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/user_agent/#user-agent-original[user_agent.original] + - - -^|image:https://img.shields.io/badge/✔-93c93e?style=flat[stable,title=stable] - -// =============================================================== -.1+| -[[otel-mapping-for-user-agent-version]] -<> - - - -^| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match,link=ecs-opentelemetry.html#ecs-opentelemetry-relation] - -<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/user_agent/#user-agent-version[user_agent.version] + - - -^|image:https://img.shields.io/badge/✘-fed10c?style=flat[experimental,title=experimental] - -// =============================================================== -|==== \ No newline at end of file diff --git a/docs/opentelemetry/otel-mapping-summary.asciidoc b/docs/opentelemetry/otel-mapping-summary.asciidoc deleted file mode 100644 index 3e766382d8..0000000000 --- a/docs/opentelemetry/otel-mapping-summary.asciidoc +++ /dev/null @@ -1,1164 +0,0 @@ - -//// -This file is automatically generated. Don't edit it manually! -//// -The following table summarizes the alignment status by namespaces between ECS in version `9.1.0-dev` -and OpenTelemetry semantic conventions in version `1.29.0`. - -[cols="10*"] -|==== -h| -9+^h| Number of Fields / Attributes - -h| Namespace -^h| ECS -^h| OTel -^h| image:https://img.shields.io/badge/match-93c93e?style=flat[relation,title=match] -^h| image:https://img.shields.io/badge/equivalent-1ba9f5?style=flat[relation,title=equivalent] -^h| image:https://img.shields.io/badge/related-efc20d?style=flat[relation,title=related] -^h| image:https://img.shields.io/badge/conflict-910000?style=flat[relation,title=conflict] -^h| image:https://img.shields.io/badge/metric-cb00cb?style=flat[relation,title=metric] -^h| image:https://img.shields.io/badge/OTLP-ffdcb2?style=flat[relation,title=OTLP] -^h| image:https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat[relation,title=na] - - -| Agent -^| <> -^| · -^| · -^| · -^| · -^| · -^| · -^| · -^| · - - -| Android -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/android[1] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| artifact -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/artifact[7] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| Autonomous System -^| <> -^| · -^| · -^| · -^| · -^| · -^| · -^| · -^| · - - -| ASP.NET Core -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/aspnetcore[7] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| General AWS -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/aws[42] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| az -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/az[2] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| Base -^| <> -^| · -^| · -^| · -^| 2 -^| · -^| · -^| 4 -^| · - - -| Browser -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/browser[4] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| cicd -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/cicd[6] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| Client -^| <> -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/client[2] -^| 2 -^| · -^| · -^| · -^| · -^| · -^| · - - -| Cloud -^| <> -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/cloud[6] -^| 4 -^| 1 -^| · -^| · -^| · -^| · -^| · - - -| CloudEvents -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/cloudevents[5] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| cloudfoundry -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/cloudfoundry[11] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| Code -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/code[6] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| Code Signature -^| <> -^| · -^| · -^| · -^| · -^| · -^| · -^| · -^| · - - -| Container -^| <> -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/container[13] -^| 4 -^| 2 -^| 1 -^| · -^| 2 -^| · -^| · - - -| CPU -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/cpu[1] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| Data Stream -^| <> -^| · -^| · -^| · -^| · -^| · -^| · -^| · -^| 3 - - -| General Database -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/db[27] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| Deployment -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/deployment[4] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| Destination -^| <> -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/destination[2] -^| 2 -^| · -^| · -^| · -^| · -^| · -^| · - - -| Device -^| <> -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/device[4] -^| 4 -^| · -^| · -^| · -^| · -^| · -^| · - - -| Disk -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/disk[1] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| DLL -^| <> -^| · -^| · -^| · -^| · -^| · -^| · -^| · -^| · - - -| DNS -^| <> -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/dns[1] -^| 1 -^| · -^| · -^| · -^| · -^| · -^| · - - -| .NET -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/dotnet[1] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| ECS -^| <> -^| · -^| · -^| · -^| · -^| · -^| · -^| · -^| 1 - - -| ELF Header -^| <> -^| · -^| · -^| · -^| · -^| · -^| · -^| · -^| · - - -| Email -^| <> -^| · -^| · -^| · -^| · -^| · -^| · -^| · -^| · - - -| Error -^| <> -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/error[1] -^| 1 -^| 2 -^| · -^| · -^| · -^| · -^| · - - -| Event -^| <> -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/event[1] -^| · -^| · -^| · -^| · -^| · -^| · -^| · - - -| Exception -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/exception[4] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| FaaS -^| <> -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/faas[16] -^| 3 -^| 2 -^| · -^| · -^| · -^| · -^| · - - -| Feature Flag -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/feature-flag[8] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| File -^| <> -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/file[18] -^| 11 -^| 7 -^| · -^| · -^| · -^| · -^| · - - -| GCP Client -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/gcp[5] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| GenAI -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/gen-ai[22] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| Geo -^| <> -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/geo[7] -^| 1 -^| 4 -^| 2 -^| · -^| · -^| · -^| · - - -| Go -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/go[1] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| GraphQL -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/graphql[3] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| Group -^| <> -^| · -^| · -^| · -^| · -^| · -^| · -^| · -^| · - - -| Hash -^| <> -^| · -^| · -^| · -^| · -^| · -^| · -^| · -^| · - - -| Heroku -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/heroku[3] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| Host -^| <> -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/host[15] -^| 5 -^| 1 -^| · -^| · -^| 8 -^| · -^| · - - -| HTTP -^| <> -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/http[12] -^| 1 -^| 5 -^| 2 -^| 1 -^| · -^| · -^| · - - -| hw -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/hw[5] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| Interface -^| <> -^| · -^| · -^| · -^| · -^| · -^| · -^| · -^| · - - -| Java Virtual Machine (JVM) -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/jvm[7] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| Kubernetes -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/k8s[26] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| linux -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/linux[1] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| Log -^| <> -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/log[7] -^| 1 -^| · -^| · -^| · -^| · -^| 1 -^| · - - -| Mach-O Header -^| <> -^| · -^| · -^| · -^| · -^| · -^| · -^| · -^| · - - -| General Messaging -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/messaging[37] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| Network -^| <> -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/network[16] -^| 2 -^| 1 -^| · -^| · -^| · -^| · -^| · - - -| Node.js -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/nodejs[1] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| Observer -^| <> -^| · -^| · -^| · -^| · -^| · -^| · -^| · -^| · - - -| Open Container Initiative (OCI) -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/oci[1] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| OpenTracing -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/opentracing[1] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| Orchestrator -^| <> -^| · -^| · -^| · -^| · -^| · -^| · -^| · -^| · - - -| Organization -^| <> -^| · -^| · -^| · -^| · -^| · -^| · -^| · -^| · - - -| Operating System -^| <> -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/os[5] -^| 2 -^| 1 -^| · -^| 1 -^| · -^| · -^| · - - -| OTel -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/otel[4] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| Package -^| <> -^| · -^| · -^| · -^| · -^| · -^| · -^| · -^| · - - -| PE Header -^| <> -^| · -^| · -^| · -^| · -^| · -^| · -^| · -^| · - - -| Peer -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/peer[1] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| Process -^| <> -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/process[33] -^| 15 -^| 2 -^| · -^| · -^| 1 -^| · -^| · - - -| profile -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/profile[1] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| Registry -^| <> -^| · -^| · -^| · -^| · -^| · -^| · -^| · -^| · - - -| Related -^| <> -^| · -^| · -^| · -^| · -^| · -^| · -^| · -^| 4 - - -| Risk information -^| <> -^| · -^| · -^| · -^| · -^| · -^| · -^| · -^| · - - -| Remote Procedure Call (RPC) -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/rpc[17] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| Rule -^| <> -^| · -^| · -^| · -^| · -^| · -^| · -^| · -^| · - - -| Server -^| <> -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/server[2] -^| 2 -^| · -^| · -^| · -^| · -^| · -^| · - - -| Service -^| <> -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/service[4] -^| 2 -^| 2 -^| · -^| · -^| · -^| · -^| · - - -| Session -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/session[2] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| SignalR -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/signalr[2] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| Source -^| <> -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/source[2] -^| 2 -^| · -^| · -^| · -^| · -^| · -^| · - - -| General System -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/system[12] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| Telemetry -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/telemetry[5] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| test -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/test[4] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| Thread -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/thread[2] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| Threat -^| <> -^| · -^| · -^| · -^| · -^| · -^| · -^| · -^| · - - -| TLS -^| <> -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls[28] -^| 26 -^| · -^| · -^| · -^| · -^| · -^| · - - -| Tracing -^| <> -^| · -^| · -^| · -^| · -^| · -^| · -^| 2 -^| 1 - - -| URL -^| <> -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/url[13] -^| 12 -^| · -^| · -^| · -^| · -^| · -^| · - - -| User -^| <> -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/user[6] -^| 6 -^| · -^| · -^| · -^| · -^| · -^| · - - -| User agent -^| <> -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/user-agent[4] -^| 3 -^| · -^| · -^| · -^| · -^| · -^| · - - -| V8 JS -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/v8js[2] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| vcs -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/vcs[13] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| VLAN -^| <> -^| · -^| · -^| · -^| · -^| · -^| · -^| · -^| · - - -| Volume -^| <> -^| · -^| · -^| · -^| · -^| · -^| · -^| · -^| · - - -| Vulnerability -^| <> -^| · -^| · -^| · -^| · -^| · -^| · -^| · -^| · - - -| Web Engine -^| · -^| https://opentelemetry.io/docs/specs/semconv/attributes-registry/webengine[3] -^| · -^| · -^| · -^| · -^| · -^| · -^| - - -| x509 Certificate -^| <> -^| · -^| · -^| · -^| · -^| · -^| · -^| · -^| · - -|==== diff --git a/docs/reference/ecs-additional-information.md b/docs/reference/ecs-additional-information.md new file mode 100644 index 0000000000..929d853040 --- /dev/null +++ b/docs/reference/ecs-additional-information.md @@ -0,0 +1,19 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-additional-information.html +applies_to: + stack: all + serverless: all +navigation_title: Additional information +--- + +# Additional ECS information [ecs-additional-information] + +* [Questions and Answers](/reference/ecs-faq.md) +* [Contributing to ECS](/reference/ecs-contributing.md) +* [Generated Artifacts](/reference/ecs-artifacts.md) + + + + + diff --git a/docs/reference/ecs-agent.md b/docs/reference/ecs-agent.md new file mode 100644 index 0000000000..abd1934a29 --- /dev/null +++ b/docs/reference/ecs-agent.md @@ -0,0 +1,26 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-agent.html +applies_to: + stack: all + serverless: all +--- + +# Agent fields [ecs-agent] + +The agent fields contain the data about the software entity, if any, that collects, detects, or observes events on a host, or takes measurements on a host. + +Examples include Beats. Agents may also run on observers. ECS agent.* fields shall be populated with details of the agent running on the host or observer where the event happened or the measurement was taken. + + +## Agent field details [_agent_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-agent-build-original$$$[agent.build.original](#field-agent-build-original) | Extended build information for the agent.

This field is intended to contain any build information that a data source may provide, no specific formatting is required.

type: keyword

example: `metricbeat version 7.6.0 (amd64), libbeat 7.6.0 [6a23e8f8f30f5001ba344e4e54d8d9cb82cb107c built 2020-02-05 23:10:10 +0000 UTC]`
| core | +| $$$field-agent-ephemeral-id$$$[agent.ephemeral_id](#field-agent-ephemeral-id) | Ephemeral identifier of this agent (if one exists).

This id normally changes across restarts, but `agent.id` does not.

type: keyword

example: `8a4f500f`
| extended | +| $$$field-agent-id$$$[agent.id](#field-agent-id) | Unique identifier of this agent (if one exists).

Example: For Beats this would be beat.id.

type: keyword

example: `8a4f500d`
| core | +| $$$field-agent-name$$$[agent.name](#field-agent-name) | Custom name of the agent.

This is a name that can be given to an agent. This can be helpful if for example two Filebeat instances are running on the same host but a human readable separation is needed on which Filebeat instance data is coming from.

type: keyword

example: `foo`
| core | +| $$$field-agent-type$$$[agent.type](#field-agent-type) | Type of the agent.

The agent type always stays the same and should be given by the agent used. In case of Filebeat the agent would always be Filebeat also if two Filebeat instances are run on the same machine.

type: keyword

example: `filebeat`
| core | +| $$$field-agent-version$$$[agent.version](#field-agent-version) | Version of the agent.

type: keyword

example: `6.0.0-rc2`
| core | + diff --git a/docs/reference/ecs-allowed-values-event-category.md b/docs/reference/ecs-allowed-values-event-category.md new file mode 100644 index 0000000000..ba87c0b491 --- /dev/null +++ b/docs/reference/ecs-allowed-values-event-category.md @@ -0,0 +1,230 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-allowed-values-event-category.html +applies_to: + stack: all + serverless: all +navigation_title: event.category +--- + +# ECS categorization field: event.category [ecs-allowed-values-event-category] + +This is one of four ECS Categorization Fields, and indicates the second level in the ECS category hierarchy. + +`event.category` represents the "big buckets" of ECS categories. For example, filtering on `event.category:process` yields all events relating to process activity. This field is closely related to `event.type`, which is used as a subcategory. + +This field is an array. This will allow proper categorization of some events that fall in multiple categories. + +**Allowed values** + +* [api](#ecs-event-category-api) +* [authentication](#ecs-event-category-authentication) +* [configuration](#ecs-event-category-configuration) +* [database](#ecs-event-category-database) +* [driver](#ecs-event-category-driver) +* [email](#ecs-event-category-email) +* [file](#ecs-event-category-file) +* [host](#ecs-event-category-host) +* [iam](#ecs-event-category-iam) +* [intrusion_detection](#ecs-event-category-intrusion_detection) +* [library](#ecs-event-category-library) +* [malware](#ecs-event-category-malware) +* [network](#ecs-event-category-network) +* [package](#ecs-event-category-package) +* [process](#ecs-event-category-process) +* [registry](#ecs-event-category-registry) +* [session](#ecs-event-category-session) +* [threat](#ecs-event-category-threat) +* [vulnerability](#ecs-event-category-vulnerability) +* [web](#ecs-event-category-web) + + +## api [ecs-event-category-api] + +Events in this category annotate API calls that occured on a system. Typical sources for those events could be from the Operating System level through the native libraries (for example Windows Win32, Linux libc, etc.), or managed sources of events (such as ETW, syslog), but can also include network protocols (such as SOAP, RPC, Websocket, REST, etc.) + +**Expected event types for category api:** + +access, admin, allowed, change, creation, deletion, denied, end, info, start, user + + +## authentication [ecs-event-category-authentication] + +Events in this category are related to the challenge and response process in which credentials are supplied and verified to allow the creation of a session. Common sources for these logs are Windows event logs and ssh logs. Visualize and analyze events in this category to look for failed logins, and other authentication-related activity. + +**Expected event types for category authentication:** + +start, end, info + + +## configuration [ecs-event-category-configuration] + +Events in the configuration category have to deal with creating, modifying, or deleting the settings or parameters of an application, process, or system. + +Example sources include security policy change logs, configuration auditing logging, and system integrity monitoring. + +**Expected event types for category configuration:** + +access, change, creation, deletion, info + + +## database [ecs-event-category-database] + +The database category denotes events and metrics relating to a data storage and retrieval system. Note that use of this category is not limited to relational database systems. Examples include event logs from MS SQL, MySQL, Elasticsearch, MongoDB, etc. Use this category to visualize and analyze database activity such as accesses and changes. + +**Expected event types for category database:** + +access, change, info, error + + +## driver [ecs-event-category-driver] + +Events in the driver category have to do with operating system device drivers and similar software entities such as Windows drivers, kernel extensions, kernel modules, etc. + +Use events and metrics in this category to visualize and analyze driver-related activity and status on hosts. + +**Expected event types for category driver:** + +change, end, info, start + + +## email [ecs-event-category-email] + +This category is used for events relating to email messages, email attachments, and email network or protocol activity. + +Emails events can be produced by email security gateways, mail transfer agents, email cloud service providers, or mail server monitoring applications. + +**Expected event types for category email:** + +info + + +## file [ecs-event-category-file] + +Relating to a set of information that has been created on, or has existed on a filesystem. Use this category of events to visualize and analyze the creation, access, and deletions of files. Events in this category can come from both host-based and network-based sources. An example source of a network-based detection of a file transfer would be the Zeek file.log. + +**Expected event types for category file:** + +access, change, creation, deletion, info + + +## host [ecs-event-category-host] + +Use this category to visualize and analyze information such as host inventory or host lifecycle events. + +Most of the events in this category can usually be observed from the outside, such as from a hypervisor or a control plane’s point of view. Some can also be seen from within, such as "start" or "end". + +Note that this category is for information about hosts themselves; it is not meant to capture activity "happening on a host". + +**Expected event types for category host:** + +access, change, end, info, start + + +## iam [ecs-event-category-iam] + +Identity and access management (IAM) events relating to users, groups, and administration. Use this category to visualize and analyze IAM-related logs and data from active directory, LDAP, Okta, Duo, and other IAM systems. + +**Expected event types for category iam:** + +admin, change, creation, deletion, group, info, user + + +## intrusion_detection [ecs-event-category-intrusion_detection] + +Relating to intrusion detections from IDS/IPS systems and functions, both network and host-based. Use this category to visualize and analyze intrusion detection alerts from systems such as Snort, Suricata, and Palo Alto threat detections. + +**Expected event types for category intrusion_detection:** + +allowed, denied, info + + +## library [ecs-event-category-library] + +Events in this category refer to the loading of a library, such as (dll / so / dynlib), into a process. Use this category to visualize and analyze library loading related activity on hosts. Keep in mind that driver related activity will be captured under the "driver" category above. + +**Expected event types for category library:** + +start + + +## malware [ecs-event-category-malware] + +Malware detection events and alerts. Use this category to visualize and analyze malware detections from EDR/EPP systems such as Elastic Endpoint Security, Symantec Endpoint Protection, Crowdstrike, and network IDS/IPS systems such as Suricata, or other sources of malware-related events such as Palo Alto Networks threat logs and Wildfire logs. + +**Expected event types for category malware:** + +info + + +## network [ecs-event-category-network] + +Relating to all network activity, including network connection lifecycle, network traffic, and essentially any event that includes an IP address. Many events containing decoded network protocol transactions fit into this category. Use events in this category to visualize or analyze counts of network ports, protocols, addresses, geolocation information, etc. + +**Expected event types for category network:** + +access, allowed, connection, denied, end, info, protocol, start + + +## package [ecs-event-category-package] + +Relating to software packages installed on hosts. Use this category to visualize and analyze inventory of software installed on various hosts, or to determine host vulnerability in the absence of vulnerability scan data. + +**Expected event types for category package:** + +access, change, deletion, info, installation, start + + +## process [ecs-event-category-process] + +Use this category of events to visualize and analyze process-specific information such as lifecycle events or process ancestry. + +**Expected event types for category process:** + +access, change, end, info, start + + +## registry [ecs-event-category-registry] + +Having to do with settings and assets stored in the Windows registry. Use this category to visualize and analyze activity such as registry access and modifications. + +**Expected event types for category registry:** + +access, change, creation, deletion + + +## session [ecs-event-category-session] + +The session category is applied to events and metrics regarding logical persistent connections to hosts and services. Use this category to visualize and analyze interactive or automated persistent connections between assets. Data for this category may come from Windows Event logs, SSH logs, or stateless sessions such as HTTP cookie-based sessions, etc. + +**Expected event types for category session:** + +start, end, info + + +## threat [ecs-event-category-threat] + +Use this category to visualize and analyze events describing threat actors' targets, motives, or behaviors. + +**Expected event types for category threat:** + +indicator + + +## vulnerability [ecs-event-category-vulnerability] + +Relating to vulnerability scan results. Use this category to analyze vulnerabilities detected by Tenable, Qualys, internal scanners, and other vulnerability management sources. + +**Expected event types for category vulnerability:** + +info + + +## web [ecs-event-category-web] + +Relating to web server access. Use this category to create a dashboard of web server/proxy activity from apache, IIS, nginx web servers, etc. Note: events from network observers such as Zeek http log may also be included in this category. + +**Expected event types for category web:** + +access, error, info + diff --git a/docs/reference/ecs-allowed-values-event-kind.md b/docs/reference/ecs-allowed-values-event-kind.md new file mode 100644 index 0000000000..b5ba1c5e57 --- /dev/null +++ b/docs/reference/ecs-allowed-values-event-kind.md @@ -0,0 +1,94 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-allowed-values-event-kind.html +applies_to: + stack: all + serverless: all +navigation_title: event.kind +--- + +# ECS categorization field: event.kind [ecs-allowed-values-event-kind] + +This is one of four ECS Categorization Fields, and indicates the highest level in the ECS category hierarchy. + +`event.kind` gives high-level information about what type of information the event contains, without being specific to the contents of the event. For example, values of this field distinguish alert events from metric events. + +The value of this field can be used to inform how these kinds of events should be handled. They may warrant different retention, different access control, it may also help understand whether the data is coming in at a regular interval or not. + +**Allowed values** + +* [alert](#ecs-event-kind-alert) +* [asset](#ecs-event-kind-asset) +* [enrichment](#ecs-event-kind-enrichment) +* [event](#ecs-event-kind-event) +* [metric](#ecs-event-kind-metric) +* [state](#ecs-event-kind-state) +* [pipeline_error](#ecs-event-kind-pipeline_error) +* [signal](#ecs-event-kind-signal) + + +## alert [ecs-event-kind-alert] + +This value indicates an event such as an alert or notable event, triggered by a detection rule executing externally to the Elastic Stack. + +`event.kind:alert` is often populated for events coming from firewalls, intrusion detection systems, endpoint detection and response systems, and so on. + +This value is not used by Elastic solutions for alert documents that are created by rules executing within the Kibana alerting framework. + + +## asset [ecs-event-kind-asset] + +::::{warning} +These fields are in beta and are subject to change. +:::: + +This value indicates events whose primary purpose is to store an inventory of assets/entities and their attributes. Assets/entities are objects (such as users and hosts) that are expected to be subjects of detailed analysis within the system. + +Examples include lists of user identities or accounts ingested from directory services such as Active Directory (AD), inventory of hosts pulled from configuration management databases (CMDB), and lists of cloud storage buckets pulled from cloud provider APIs. + +This value is used by Elastic Security for asset management solutions. `event.kind: asset` is not used for normal system events or logs that are coming from an asset/entity, nor is it used for system events or logs coming from a directory or CMDB system. + + +## enrichment [ecs-event-kind-enrichment] + +The `enrichment` value indicates an event collected to provide additional context, often to other events. + +An example is collecting indicators of compromise (IOCs) from a threat intelligence provider with the intent to use those values to enrich other events. The IOC events from the intelligence provider should be categorized as `event.kind:enrichment`. + + +## event [ecs-event-kind-event] + +This value is the most general and most common value for this field. It is used to represent events that indicate that something happened. + + +## metric [ecs-event-kind-metric] + +This value is used to indicate that this event describes a numeric measurement taken at given point in time. + +Examples include CPU utilization, memory usage, or device temperature. + +Metric events are often collected on a predictable frequency, such as once every few seconds, or once a minute, but can also be used to describe ad-hoc numeric metric queries. + + +## state [ecs-event-kind-state] + +The state value is similar to metric, indicating that this event describes a measurement taken at given point in time, except that the measurement does not result in a numeric value, but rather one of a fixed set of categorical values that represent conditions or states. + +Examples include periodic events reporting Elasticsearch cluster state (green/yellow/red), the state of a TCP connection (open, closed, fin_wait, etc.), the state of a host with respect to a software vulnerability (vulnerable, not vulnerable), and the state of a system regarding compliance with a regulatory standard (compliant, not compliant). + +Note that an event that describes a change of state would not use `event.kind:state`, but instead would use *event.kind:event* since a state change fits the more general event definition of something that happened. + +State events are often collected on a predictable frequency, such as once every few seconds, once a minute, once an hour, or once a day, but can also be used to describe ad-hoc state queries. + + +## pipeline_error [ecs-event-kind-pipeline_error] + +This value indicates that an error occurred during the ingestion of this event, and that event data may be missing, inconsistent, or incorrect. `event.kind:pipeline_error` is often associated with parsing errors. + + +## signal [ecs-event-kind-signal] + +This value is used by Elastic solutions (e.g., Security, Observability) for alert documents that are created by rules executing within the Kibana alerting framework. + +Usage of this value is reserved, and data ingestion pipelines must not populate `event.kind` with the value "signal". + diff --git a/docs/reference/ecs-allowed-values-event-outcome.md b/docs/reference/ecs-allowed-values-event-outcome.md new file mode 100644 index 0000000000..2baec3e308 --- /dev/null +++ b/docs/reference/ecs-allowed-values-event-outcome.md @@ -0,0 +1,42 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-allowed-values-event-outcome.html +applies_to: + stack: all + serverless: all +navigation_title: event.outcome +--- + +# ECS categorization field: event.outcome [ecs-allowed-values-event-outcome] + +This is one of four ECS Categorization Fields, and indicates the lowest level in the ECS category hierarchy. + +`event.outcome` simply denotes whether the event represents a success or a failure from the perspective of the entity that produced the event. + +Note that when a single transaction is described in multiple events, each event may populate different values of `event.outcome`, according to their perspective. + +Also note that in the case of a compound event (a single event that contains multiple logical events), this field should be populated with the value that best captures the overall success or failure from the perspective of the event producer. + +Further note that not all events will have an associated outcome. For example, this field is generally not populated for metric events, events with `event.type:info`, or any events for which an outcome does not make logical sense. + +**Allowed values** + +* [failure](#ecs-event-outcome-failure) +* [success](#ecs-event-outcome-success) +* [unknown](#ecs-event-outcome-unknown) + + +## failure [ecs-event-outcome-failure] + +Indicates that this event describes a failed result. A common example is `event.category:file AND event.type:access AND event.outcome:failure` to indicate that a file access was attempted, but was not successful. + + +## success [ecs-event-outcome-success] + +Indicates that this event describes a successful result. A common example is `event.category:file AND event.type:create AND event.outcome:success` to indicate that a file was successfully created. + + +## unknown [ecs-event-outcome-unknown] + +Indicates that this event describes only an attempt for which the result is unknown from the perspective of the event producer. For example, if the event contains information only about the request side of a transaction that results in a response, populating `event.outcome:unknown` in the request event is appropriate. The unknown value should not be used when an outcome doesn’t make logical sense for the event. In such cases `event.outcome` should not be populated. + diff --git a/docs/reference/ecs-allowed-values-event-type.md b/docs/reference/ecs-allowed-values-event-type.md new file mode 100644 index 0000000000..8d5e09cd3d --- /dev/null +++ b/docs/reference/ecs-allowed-values-event-type.md @@ -0,0 +1,124 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-allowed-values-event-type.html +applies_to: + stack: all + serverless: all +navigation_title: event.type +--- + +# ECS categorization field: event.type [ecs-allowed-values-event-type] + +This is one of four ECS Categorization Fields, and indicates the third level in the ECS category hierarchy. + +`event.type` represents a categorization "sub-bucket" that, when used along with the `event.category` field values, enables filtering events down to a level appropriate for single visualization. + +This field is an array. This will allow proper categorization of some events that fall in multiple event types. + +**Allowed values** + +* [access](#ecs-event-type-access) +* [admin](#ecs-event-type-admin) +* [allowed](#ecs-event-type-allowed) +* [change](#ecs-event-type-change) +* [connection](#ecs-event-type-connection) +* [creation](#ecs-event-type-creation) +* [deletion](#ecs-event-type-deletion) +* [denied](#ecs-event-type-denied) +* [end](#ecs-event-type-end) +* [error](#ecs-event-type-error) +* [group](#ecs-event-type-group) +* [indicator](#ecs-event-type-indicator) +* [info](#ecs-event-type-info) +* [installation](#ecs-event-type-installation) +* [protocol](#ecs-event-type-protocol) +* [start](#ecs-event-type-start) +* [user](#ecs-event-type-user) + + +## access [ecs-event-type-access] + +The access event type is used for the subset of events within a category that indicate that something was accessed. Common examples include `event.category:database AND event.type:access`, or `event.category:file AND event.type:access`. Note for file access, both directory listings and file opens should be included in this subcategory. You can further distinguish access operations using the ECS `event.action` field. + + +## admin [ecs-event-type-admin] + +The admin event type is used for the subset of events within a category that are related to admin objects. For example, administrative changes within an IAM framework that do not specifically affect a user or group (e.g., adding new applications to a federation solution or connecting discrete forests in Active Directory) would fall into this subcategory. Common example: `event.category:iam AND event.type:change AND event.type:admin`. You can further distinguish admin operations using the ECS `event.action` field. + + +## allowed [ecs-event-type-allowed] + +The allowed event type is used for the subset of events within a category that indicate that something was allowed. Common examples include `event.category:network AND event.type:connection AND event.type:allowed` (to indicate a network firewall event for which the firewall disposition was to allow the connection to complete) and `event.category:intrusion_detection AND event.type:allowed` (to indicate a network intrusion prevention system event for which the IPS disposition was to allow the connection to complete). You can further distinguish allowed operations using the ECS `event.action` field, populating with values of your choosing, such as "allow", "detect", or "pass". + + +## change [ecs-event-type-change] + +The change event type is used for the subset of events within a category that indicate that something has changed. If semantics best describe an event as modified, then include them in this subcategory. Common examples include `event.category:process AND event.type:change`, and `event.category:file AND event.type:change`. You can further distinguish change operations using the ECS `event.action` field. + + +## connection [ecs-event-type-connection] + +Used primarily with `event.category:network` this value is used for the subset of network traffic that includes sufficient information for the event to be included in flow or connection analysis. Events in this subcategory will contain at least source and destination IP addresses, source and destination TCP/UDP ports, and will usually contain counts of bytes and/or packets transferred. Events in this subcategory may contain unidirectional or bidirectional information, including summary information. Use this subcategory to visualize and analyze network connections. Flow analysis, including Netflow, IPFIX, and other flow-related events fit in this subcategory. Note that firewall events from many Next-Generation Firewall (NGFW) devices will also fit into this subcategory. A common filter for flow/connection information would be `event.category:network AND event.type:connection AND event.type:end` (to view or analyze all completed network connections, ignoring mid-flow reports). You can further distinguish connection events using the ECS `event.action` field, populating with values of your choosing, such as "timeout", or "reset". + + +## creation [ecs-event-type-creation] + +The "creation" event type is used for the subset of events within a category that indicate that something was created. A common example is `event.category:file AND event.type:creation`. + + +## deletion [ecs-event-type-deletion] + +The deletion event type is used for the subset of events within a category that indicate that something was deleted. A common example is `event.category:file AND event.type:deletion` to indicate that a file has been deleted. + + +## denied [ecs-event-type-denied] + +The denied event type is used for the subset of events within a category that indicate that something was denied. Common examples include `event.category:network AND event.type:denied` (to indicate a network firewall event for which the firewall disposition was to deny the connection) and `event.category:intrusion_detection AND event.type:denied` (to indicate a network intrusion prevention system event for which the IPS disposition was to deny the connection to complete). You can further distinguish denied operations using the ECS `event.action` field, populating with values of your choosing, such as "blocked", "dropped", or "quarantined". + + +## end [ecs-event-type-end] + +The end event type is used for the subset of events within a category that indicate something has ended. A common example is `event.category:process AND event.type:end`. + + +## error [ecs-event-type-error] + +The error event type is used for the subset of events within a category that indicate or describe an error. A common example is `event.category:database AND event.type:error`. Note that pipeline errors that occur during the event ingestion process should not use this `event.type` value. Instead, they should use `event.kind:pipeline_error`. + + +## group [ecs-event-type-group] + +The group event type is used for the subset of events within a category that are related to group objects. Common example: `event.category:iam AND event.type:creation AND event.type:group`. You can further distinguish group operations using the ECS `event.action` field. + + +## indicator [ecs-event-type-indicator] + +The indicator event type is used for the subset of events within a category that contain details about indicators of compromise (IOCs). + +A common example is `event.category:threat AND event.type:indicator`. + + +## info [ecs-event-type-info] + +The info event type is used for the subset of events within a category that indicate that they are purely informational, and don’t report a state change, or any type of action. For example, an initial run of a file integrity monitoring system (FIM), where an agent reports all files under management, would fall into the "info" subcategory. Similarly, an event containing a dump of all currently running processes (as opposed to reporting that a process started/ended) would fall into the "info" subcategory. An additional common examples is `event.category:intrusion_detection AND event.type:info`. + + +## installation [ecs-event-type-installation] + +The installation event type is used for the subset of events within a category that indicate that something was installed. A common example is `event.category:package` AND `event.type:installation`. + + +## protocol [ecs-event-type-protocol] + +The protocol event type is used for the subset of events within a category that indicate that they contain protocol details or analysis, beyond simply identifying the protocol. Generally, network events that contain specific protocol details will fall into this subcategory. A common example is `event.category:network AND event.type:protocol AND event.type:connection AND event.type:end` (to indicate that the event is a network connection event sent at the end of a connection that also includes a protocol detail breakdown). Note that events that only indicate the name or id of the protocol should not use the protocol value. Further note that when the protocol subcategory is used, the identified protocol is populated in the ECS `network.protocol` field. + + +## start [ecs-event-type-start] + +The start event type is used for the subset of events within a category that indicate something has started. A common example is `event.category:process AND event.type:start`. + + +## user [ecs-event-type-user] + +The user event type is used for the subset of events within a category that are related to user objects. Common example: `event.category:iam AND event.type:deletion AND event.type:user`. You can further distinguish user operations using the ECS `event.action` field. + diff --git a/docs/reference/ecs-artifacts.md b/docs/reference/ecs-artifacts.md new file mode 100644 index 0000000000..e5b089c51e --- /dev/null +++ b/docs/reference/ecs-artifacts.md @@ -0,0 +1,14 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-artifacts.html +applies_to: + stack: all + serverless: all +--- + +# Generated artifacts [ecs-artifacts] + +ECS maintains a collection of artifacts which are generated based on the schema. Examples include Elasticsearch index templates, CSV, and Beats field mappings. The maintained artifacts can be found in the [ECS Github repo](https://github.com/elastic/ecs/blob/master/generated#artifacts-generated-from-ecs). + +Users can generate custom versions of these artifacts using the ECS project’s tooling. See the tooling [usage documentation](https://github.com/elastic/ecs/blob/master/USAGE.md) for more detail. + diff --git a/docs/reference/ecs-as.md b/docs/reference/ecs-as.md new file mode 100644 index 0000000000..2fbac6d5ba --- /dev/null +++ b/docs/reference/ecs-as.md @@ -0,0 +1,34 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-as.html +applies_to: + stack: all + serverless: all +--- + +# Autonomous system fields [ecs-as] + +An autonomous system (AS) is a collection of connected Internet Protocol (IP) routing prefixes under the control of one or more network operators on behalf of a single administrative entity or domain that presents a common, clearly defined routing policy to the internet. + + +## Autonomous system field details [_autonomous_system_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-as-number$$$[as.number](#field-as-number) | Unique number allocated to the autonomous system. The autonomous system number (ASN) uniquely identifies each network on the Internet.

type: long

example: `15169`
| extended | +| $$$field-as-organization-name$$$[as.organization.name](#field-as-organization-name) | Organization name.

type: keyword

Multi-fields:

* as.organization.name.text (type: match_only_text)

example: `Google LLC`
| extended | + + +## Field reuse [_field_reuse] + +The `as` fields are expected to be nested at: + +* `client.as` +* `destination.as` +* `server.as` +* `source.as` +* `threat.enrichments.indicator.as` +* `threat.indicator.as` + +Note also that the `as` fields are not expected to be used directly at the root of the events. + diff --git a/docs/reference/ecs-base.md b/docs/reference/ecs-base.md new file mode 100644 index 0000000000..15038dd98b --- /dev/null +++ b/docs/reference/ecs-base.md @@ -0,0 +1,22 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-base.html +applies_to: + stack: all + serverless: all +--- + +# Base fields [ecs-base] + +The `base` field set contains all fields which are at the root of the events. These fields are common across all types of events. + + +## Base field details [_base_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-timestamp$$$[@timestamp](#field-timestamp) | Date/time when the event originated.

This is the date/time extracted from the event, typically representing when the event was generated by the source.

If the event source has no original timestamp, this value is typically populated by the first time the event was received by the pipeline.

Required field for all events.

type: date

example: `2016-05-23T08:05:34.853Z`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/OTLP-ffdcb2?style=flat "otlp") [time_unix_nano](https://github.com/search?q=repo%3Aopen-telemetry%2Fopentelemetry-proto+%22+time_unix_nano+%22&type=code)

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/OTLP-ffdcb2?style=flat "otlp") [observed_time_unix_nano](https://github.com/search?q=repo%3Aopen-telemetry%2Fopentelemetry-proto+%22+observed_time_unix_nano+%22&type=code)

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/OTLP-ffdcb2?style=flat "otlp") [start_time_unix_nano](https://github.com/search?q=repo%3Aopen-telemetry%2Fopentelemetry-proto+%22+start_time_unix_nano+%22&type=code)

Note: On logs, events and metrics, `time_unix_nano` defines the time of the event. For logs, `observed_time_unix_nano` defines the time when the event was observed by the collection system. On spans, `start_time_unix_nano` represents the start time of the span.
| core | +| $$$field-labels$$$[labels](#field-labels) | Custom key/value pairs.

Can be used to add meta information to events. Should not contain nested objects. All values are stored as keyword.

Example: `docker` and `k8s` labels.

type: object

example: `{"application": "foo-bar", "env": "production"}`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/related-efc20d?style=flat "related") [k8s.pod.label](https://opentelemetry.io/docs/specs/semconv/attributes-registry/k8s/#k8s-pod-label)

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/related-efc20d?style=flat "related") [container.label](https://opentelemetry.io/docs/specs/semconv/attributes-registry/container/#container-label)
| core | +| $$$field-message$$$[message](#field-message) | For log events the message field contains the log message, optimized for viewing in a log viewer.

For structured logs without an original message field, other fields can be concatenated to form a human-readable summary of the event.

If multiple messages exist, they can be combined into one message.

type: match_only_text

example: `Hello World`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/OTLP-ffdcb2?style=flat "otlp") [body](https://github.com/search?q=repo%3Aopen-telemetry%2Fopentelemetry-proto+%22+body+%22&type=code)

Note: The `body` in OTLP is of type `Any` and can be either an unstructured log message or a structured event.
| core | +| $$$field-tags$$$[tags](#field-tags) | List of keywords used to tag each event.

type: keyword

Note: this field should contain an array of values.

example: `["production", "env2"]`
| core | + diff --git a/docs/reference/ecs-category-field-values-reference.md b/docs/reference/ecs-category-field-values-reference.md new file mode 100644 index 0000000000..0a7f1dd969 --- /dev/null +++ b/docs/reference/ecs-category-field-values-reference.md @@ -0,0 +1,37 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-category-field-values-reference.html +applies_to: + stack: all + serverless: all +--- + +# ECS categorization fields [ecs-category-field-values-reference] + +At a high level, ECS provides fields to classify events in two different ways: "Where it’s from" (e.g., `event.module`, `event.dataset`, `agent.type`, `observer.type`, etc.), and "What it is." The categorization fields hold the "What it is" information, independent of the source of the events. + +ECS defines four categorization fields for this purpose, each of which falls under the `event.*` field set. + + +## Categorization fields [ecs-category-fields] + +* [event.kind](/reference/ecs-allowed-values-event-kind.md) +* [event.category](/reference/ecs-allowed-values-event-category.md) +* [event.type](/reference/ecs-allowed-values-event-type.md) +* [event.outcome](/reference/ecs-allowed-values-event-outcome.md) + +::::{note} +If your events don’t match any of these categorization values, you should leave the fields empty. This will ensure you can start populating the fields once the appropriate categorization values are published, in a later release. +:::: + + + +## Categorization usage [ecs-category-usage] + +[Using the categorization fields](/reference/ecs-using-categorization-fields.md) contains examples combining the categorization fields to classify different types of events. + + + + + + diff --git a/docs/reference/ecs-client.md b/docs/reference/ecs-client.md new file mode 100644 index 0000000000..04edd06271 --- /dev/null +++ b/docs/reference/ecs-client.md @@ -0,0 +1,46 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-client.html +applies_to: + stack: all + serverless: all +--- + +# Client fields [ecs-client] + +A client is defined as the initiator of a network connection for events regarding sessions, connections, or bidirectional flow records. + +For TCP events, the client is the initiator of the TCP connection that sends the SYN packet(s). For other protocols, the client is generally the initiator or requestor in the network transaction. Some systems use the term "originator" to refer the client in TCP connections. The client fields describe details about the system acting as the client in the network event. Client fields are usually populated in conjunction with server fields. Client fields are generally not populated for packet-level events. + +Client / server representations can add semantic context to an exchange, which is helpful to visualize the data in certain situations. If your context falls in that category, you should still ensure that source and destination are filled appropriately. + + +## Client field details [_client_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-client-address$$$[client.address](#field-client-address) | Some event client addresses are defined ambiguously. The event will sometimes list an IP, a domain or a unix socket. You should always store the raw address in the `.address` field.

Then it should be duplicated to `.ip` or `.domain`, depending on which one it is.

type: keyword

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [client.address](https://opentelemetry.io/docs/specs/semconv/attributes-registry/client/#client-address)
| extended | +| $$$field-client-bytes$$$[client.bytes](#field-client-bytes) | Bytes sent from the client to the server.

type: long

example: `184`
| core | +| $$$field-client-domain$$$[client.domain](#field-client-domain) | The domain name of the client system.

This value may be a host name, a fully qualified domain name, or another host naming format. The value may derive from the original event or be added from enrichment.

type: keyword

example: `foo.example.com`
| core | +| $$$field-client-ip$$$[client.ip](#field-client-ip) | IP address of the client (IPv4 or IPv6).

type: ip
| core | +| $$$field-client-mac$$$[client.mac](#field-client-mac) | MAC address of the client.

The notation format from RFC 7042 is suggested: Each octet (that is, 8-bit byte) is represented by two [uppercase] hexadecimal digits giving the value of the octet as an unsigned integer. Successive octets are separated by a hyphen.

type: keyword

example: `00-00-5E-00-53-23`
| core | +| $$$field-client-nat-ip$$$[client.nat.ip](#field-client-nat-ip) | Translated IP of source based NAT sessions (e.g. internal client to internet).

Typically connections traversing load balancers, firewalls, or routers.

type: ip
| extended | +| $$$field-client-nat-port$$$[client.nat.port](#field-client-nat-port) | Translated port of source based NAT sessions (e.g. internal client to internet).

Typically connections traversing load balancers, firewalls, or routers.

type: long
| extended | +| $$$field-client-packets$$$[client.packets](#field-client-packets) | Packets sent from the client to the server.

type: long

example: `12`
| core | +| $$$field-client-port$$$[client.port](#field-client-port) | Port of the client.

type: long

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [client.port](https://opentelemetry.io/docs/specs/semconv/attributes-registry/client/#client-port)
| core | +| $$$field-client-registered-domain$$$[client.registered_domain](#field-client-registered-domain) | The highest registered client domain, stripped of the subdomain.

For example, the registered domain for "foo.example.com" is "example.com".

This value can be determined precisely with a list like the public suffix list ([https://publicsuffix.org](https://publicsuffix.org)). Trying to approximate this by simply taking the last two labels will not work well for TLDs such as "co.uk".

type: keyword

example: `example.com`
| extended | +| $$$field-client-subdomain$$$[client.subdomain](#field-client-subdomain) | The subdomain portion of a fully qualified domain name includes all of the names except the host name under the registered_domain. In a partially qualified domain, or if the the qualification level of the full name cannot be determined, subdomain contains all of the names below the registered domain.

For example the subdomain portion of "www.east.mydomain.co.uk" is "east". If the domain has multiple levels of subdomain, such as "sub2.sub1.example.com", the subdomain field should contain "sub2.sub1", with no trailing period.

type: keyword

example: `east`
| extended | +| $$$field-client-top-level-domain$$$[client.top_level_domain](#field-client-top-level-domain) | The effective top level domain (eTLD), also known as the domain suffix, is the last part of the domain name. For example, the top level domain for example.com is "com".

This value can be determined precisely with a list like the public suffix list ([https://publicsuffix.org](https://publicsuffix.org)). Trying to approximate this by simply taking the last label will not work well for effective TLDs such as "co.uk".

type: keyword

example: `co.uk`
| extended | + + +## Field reuse [_field_reuse_2] + + +### Field sets that can be nested under client [ecs-client-nestings] + +| Location | Field Set | Description | +| --- | --- | --- | +| `client.as.*` | [as](/reference/ecs-as.md) | Fields describing an Autonomous System (Internet routing prefix). | +| `client.geo.*` | [geo](/reference/ecs-geo.md) | Fields describing a location. | +| `client.user.*` | [user](/reference/ecs-user.md) | Fields to describe the user relevant to the event. | + diff --git a/docs/reference/ecs-cloud-usage.md b/docs/reference/ecs-cloud-usage.md new file mode 100644 index 0000000000..d19a5cbb63 --- /dev/null +++ b/docs/reference/ecs-cloud-usage.md @@ -0,0 +1,76 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-cloud-usage.html +applies_to: + stack: all + serverless: all +--- + +# Cloud fields usage and examples [ecs-cloud-usage] + +Here are the subjects covered in this page. + +* [Field reuse](#ecs-cloud-usage-field-reuse) + + * [Cloud fields at the root of an event](#ecs-cloud-usage-cloud-at-root) + * [Describing external cloud resources in an invocation relationship](#ecs-cloud-usage-origin-target) + + + +## Field reuse [ecs-cloud-usage-field-reuse] + +The cloud fields can be used to * [describe the cloud resource an event comes from](#ecs-cloud-usage-cloud-at-root) * or to [describe the context of an external cloud resource](#ecs-cloud-usage-origin-target) that has a direct invocation relationship to the observed service or resource + + +### Cloud fields at the root of an event [ecs-cloud-usage-cloud-at-root] + +Use the cloud fields at the root of an event to describe the cloud resource the event primarily relates to. An example for this use case is a log entry being recorded for a service that is deployed in a cloud environment: + +```json +{ + "cloud": { + "provider": "aws", + "region": "us-east-1", + "service": { "name": "ec2" } + } +} +``` + + +### Describing external cloud resources in an invocation relationship [ecs-cloud-usage-origin-target] + +Managed cloud services can be in an invocation relationship to the observed service (i.e. service for which the corresponding event is captured). For instance, an observed service running on AWS Lambda can be invoked through an AWS API Gateway. Another example is an observed service that invokes an external cloud service (e.g. AWS Simple Email Service). In the context of an invocation relationship, cloud fields can be nested under `cloud.origin.*` and `cloud.target.*`, respectively, to capture the cloud context on origin or target cloud services from the perspective of an observed service. This concept is similar to [nesting of service fields](/reference/ecs-service-usage.md#ecs-service-usage-origin-target) under `service.origin.*` and `service.target.*`. + +Let’s consider an exemplary event that represents an inbound AWS Lambda invocation coming from an AWS API Gateway. Use the following `cloud.origin.*` nesting to describe the API Gateway service from the perspective of the AWS Lambda service: + +```json +{ + "service": { <1> + "name": "MyLambdaFunction", + "version": "1.0.0", + "origin": { <2> + "name": "MyGateway", + "version" "2.0", + } + }, + "cloud": { <3> + "provider": "aws", + "region": "us-east-1", + "service": { "name": "lambda" }, + "origin": { <4> + "provider": "aws", + "region": "eu-west-1", + "service": { "name": "apigateway" } + } + } +} +``` + +1. Describes the observed AWS Lambda function +2. Describes the API Gateway service where the inbound request comes from +3. Describes the cloud context of the observed AWS Lambda function +4. Describes the cloud context of the API Gateway service where the inbound request comes from + + +Note that `cloud.origin.*` and `cloud.target.*` fields should only be used on events that represent an invocation relationship. + diff --git a/docs/reference/ecs-cloud.md b/docs/reference/ecs-cloud.md new file mode 100644 index 0000000000..7fc7b7f144 --- /dev/null +++ b/docs/reference/ecs-cloud.md @@ -0,0 +1,57 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-cloud.html +applies_to: + stack: all + serverless: all +--- + +# Cloud fields [ecs-cloud] + +Fields related to the cloud or infrastructure the events are coming from. + + +## Cloud field details [_cloud_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-cloud-account-id$$$[cloud.account.id](#field-cloud-account-id) | The cloud account or organization id used to identify different entities in a multi-tenant environment.

Examples: AWS account id, Google Cloud ORG Id, or other unique identifier.

type: keyword

example: `666777888999`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [cloud.account.id](https://opentelemetry.io/docs/specs/semconv/attributes-registry/cloud/#cloud-account-id)
| extended | +| $$$field-cloud-account-name$$$[cloud.account.name](#field-cloud-account-name) | The cloud account name or alias used to identify different entities in a multi-tenant environment.

Examples: AWS account name, Google Cloud ORG display name.

type: keyword

example: `elastic-dev`
| extended | +| $$$field-cloud-availability-zone$$$[cloud.availability_zone](#field-cloud-availability-zone) | Availability zone in which this host, resource, or service is located.

type: keyword

example: `us-east-1c`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [cloud.availability_zone](https://opentelemetry.io/docs/specs/semconv/attributes-registry/cloud/#cloud-availability-zone)
| extended | +| $$$field-cloud-instance-id$$$[cloud.instance.id](#field-cloud-instance-id) | Instance ID of the host machine.

type: keyword

example: `i-1234567890abcdef0`
| extended | +| $$$field-cloud-instance-name$$$[cloud.instance.name](#field-cloud-instance-name) | Instance name of the host machine.

type: keyword
| extended | +| $$$field-cloud-machine-type$$$[cloud.machine.type](#field-cloud-machine-type) | Machine type of the host machine.

type: keyword

example: `t2.medium`
| extended | +| $$$field-cloud-project-id$$$[cloud.project.id](#field-cloud-project-id) | The cloud project identifier.

Examples: Google Cloud Project id, Azure Project id.

type: keyword

example: `my-project`
| extended | +| $$$field-cloud-project-name$$$[cloud.project.name](#field-cloud-project-name) | The cloud project name.

Examples: Google Cloud Project name, Azure Project name.

type: keyword

example: `my project`
| extended | +| $$$field-cloud-provider$$$[cloud.provider](#field-cloud-provider) | Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean.

type: keyword

example: `aws`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [cloud.provider](https://opentelemetry.io/docs/specs/semconv/attributes-registry/cloud/#cloud-provider)
| extended | +| $$$field-cloud-region$$$[cloud.region](#field-cloud-region) | Region in which this host, resource, or service is located.

type: keyword

example: `us-east-1`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [cloud.region](https://opentelemetry.io/docs/specs/semconv/attributes-registry/cloud/#cloud-region)
| extended | +| $$$field-cloud-service-name$$$[cloud.service.name](#field-cloud-service-name) | The cloud service name is intended to distinguish services running on different platforms within a provider, eg AWS EC2 vs Lambda, GCP GCE vs App Engine, Azure VM vs App Server.

Examples: app engine, app service, cloud run, fargate, lambda.

type: keyword

example: `lambda`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") [cloud.platform](https://opentelemetry.io/docs/specs/semconv/attributes-registry/cloud/#cloud-platform)
| extended | + + +## Field reuse [_field_reuse_3] + +The `cloud` fields are expected to be nested at: + +* `cloud.origin` +* `cloud.target` + +Note also that the `cloud` fields may be used directly at the root of the events. + + +### Field sets that can be nested under Cloud [ecs-cloud-nestings] + +::::{warning} +These fields are in beta and are subject to change. +:::: + +| Location | Field Set | Description | +| --- | --- | --- | +| `cloud.origin.*` | `cloud` | Provides the cloud information of the origin entity in case of an incoming request or event. | +| `cloud.target.*` | `cloud` | Provides the cloud information of the target entity in case of an outgoing request or event. | + + +## Cloud field usage [_cloud_field_usage] + +For usage and examples of the cloud fields, please see the [Cloud fields usage and examples](/reference/ecs-cloud-usage.md) section. + + diff --git a/docs/reference/ecs-code_signature.md b/docs/reference/ecs-code_signature.md new file mode 100644 index 0000000000..e34676da0f --- /dev/null +++ b/docs/reference/ecs-code_signature.md @@ -0,0 +1,53 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-code_signature.html +applies_to: + stack: all + serverless: all +--- + +# Code signature fields [ecs-code_signature] + +These fields contain information about binary code signatures. + + +## Code signature field details [_code_signature_field_details] + +::::{tab-set} + +:::{tab-item} Fields + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-code-signature-digest-algorithm$$$[code_signature.digest_algorithm](#field-code-signature-digest-algorithm) | The hashing algorithm used to sign the process.

This value can distinguish signatures when a file is signed multiple times by the same signer but with a different digest algorithm.

type: keyword

example: `sha256`
| extended | +| $$$field-code-signature-exists$$$[code_signature.exists](#field-code-signature-exists) | Boolean to capture if a signature is present.

type: boolean

example: `true`
| core | +| $$$field-code-signature-signing-id$$$[code_signature.signing_id](#field-code-signature-signing-id) | The identifier used to sign the process.

This is used to identify the application manufactured by a software vendor. The field is relevant to Apple *OS only.

type: keyword

example: `com.apple.xpc.proxy`
| extended | +| $$$field-code-signature-status$$$[code_signature.status](#field-code-signature-status) | Additional information about the certificate status.

This is useful for logging cryptographic errors with the certificate validity or trust status. Leave unpopulated if the validity or trust of the certificate was unchecked.

type: keyword

example: `ERROR_UNTRUSTED_ROOT`
| extended | +| $$$field-code-signature-subject-name$$$[code_signature.subject_name](#field-code-signature-subject-name) | Subject name of the code signer

type: keyword

example: `Microsoft Corporation`
| core | +| $$$field-code-signature-team-id$$$[code_signature.team_id](#field-code-signature-team-id) | The team identifier used to sign the process.

This is used to identify the team or vendor of a software product. The field is relevant to Apple *OS only.

type: keyword

example: `EQHXZ8M8AV`
| extended | +| $$$field-code-signature-timestamp$$$[code_signature.timestamp](#field-code-signature-timestamp) | Date and time when the code signature was generated and signed.

type: date

example: `2021-01-01T12:10:30Z`
| extended | +| $$$field-code-signature-trusted$$$[code_signature.trusted](#field-code-signature-trusted) | Stores the trust status of the certificate chain.

Validating the trust of the certificate chain may be complicated, and this field should only be populated by tools that actively check the status.

type: boolean

example: `true`
| extended | +| $$$field-code-signature-valid$$$[code_signature.valid](#field-code-signature-valid) | Boolean to capture if the digital signature is verified against the binary content.

Leave unpopulated if a certificate was unchecked.

type: boolean

example: `true`
| extended | + +::: + +:::{tab-item} Beta Fields + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-code-signature-flags$$$[code_signature.flags](#field-code-signature-flags) | The flags used to sign the process.

type: keyword

example: `570522385`
| extended | + + +::: +:::: + +## Field reuse [_field_reuse_4] + +The `code_signature` fields are expected to be nested at: + +* `dll.code_signature` +* `file.code_signature` +* `process.code_signature` + +Note also that the `code_signature` fields are not expected to be used directly at the root of the events. + diff --git a/docs/reference/ecs-container.md b/docs/reference/ecs-container.md new file mode 100644 index 0000000000..367a93b132 --- /dev/null +++ b/docs/reference/ecs-container.md @@ -0,0 +1,34 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-container.html +applies_to: + stack: all + serverless: all +--- + +# Container fields [ecs-container] + +Container fields are used for meta information about the specific container that is the source of information. + +These fields help correlate data based containers from any runtime. + + +## Container field details [_container_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-container-cpu-usage$$$[container.cpu.usage](#field-container-cpu-usage) | Percent CPU used which is normalized by the number of CPU cores and it ranges from 0 to 1. Scaling factor: 1000.

type: scaled_float

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/metric-cb00cb?style=flat "metric") [container.cpu.usage](https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--+semconv+metric.container.cpu.usage+--%3E%22&type=code)
| extended | +| $$$field-container-disk-read-bytes$$$[container.disk.read.bytes](#field-container-disk-read-bytes) | The total number of bytes (gauge) read successfully (aggregated from all disks) since the last metric collection.

type: long
| extended | +| $$$field-container-disk-write-bytes$$$[container.disk.write.bytes](#field-container-disk-write-bytes) | The total number of bytes (gauge) written successfully (aggregated from all disks) since the last metric collection.

type: long
| extended | +| $$$field-container-id$$$[container.id](#field-container-id) | Unique container id.

type: keyword

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [container.id](https://opentelemetry.io/docs/specs/semconv/attributes-registry/container/#container-id)
| core | +| $$$field-container-image-hash-all$$$[container.image.hash.all](#field-container-image-hash-all) | An array of digests of the image the container was built on. Each digest consists of the hash algorithm and value in this format: `algorithm:value`. Algorithm names should align with the field names in the ECS hash field set.

type: keyword

Note: this field should contain an array of values.

example: `[sha256:f8fefc80e3273dc756f288a63945820d6476ad64883892c771b5e2ece6bf1b26]`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") [container.image.repo_digests](https://opentelemetry.io/docs/specs/semconv/attributes-registry/container/#container-image-repo-digests)
| extended | +| $$$field-container-image-name$$$[container.image.name](#field-container-image-name) | Name of the image the container was built on.

type: keyword

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [container.image.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/container/#container-image-name)
| extended | +| $$$field-container-image-tag$$$[container.image.tag](#field-container-image-tag) | Container image tags.

type: keyword

Note: this field should contain an array of values.

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") [container.image.tags](https://opentelemetry.io/docs/specs/semconv/attributes-registry/container/#container-image-tags)
| extended | +| $$$field-container-labels$$$[container.labels](#field-container-labels) | Image labels.

type: object

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/related-efc20d?style=flat "related") [container.label](https://opentelemetry.io/docs/specs/semconv/attributes-registry/container/#container-label)
| extended | +| $$$field-container-memory-usage$$$[container.memory.usage](#field-container-memory-usage) | Memory usage percentage and it ranges from 0 to 1. Scaling factor: 1000.

type: scaled_float

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/metric-cb00cb?style=flat "metric") [container.memory.usage](https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--+semconv+metric.container.memory.usage+--%3E%22&type=code)
| extended | +| $$$field-container-name$$$[container.name](#field-container-name) | Container name.

type: keyword

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [container.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/container/#container-name)
| extended | +| $$$field-container-network-egress-bytes$$$[container.network.egress.bytes](#field-container-network-egress-bytes) | The number of bytes (gauge) sent out on all network interfaces by the container since the last metric collection.

type: long
| extended | +| $$$field-container-network-ingress-bytes$$$[container.network.ingress.bytes](#field-container-network-ingress-bytes) | The number of bytes received (gauge) on all network interfaces by the container since the last metric collection.

type: long
| extended | +| $$$field-container-runtime$$$[container.runtime](#field-container-runtime) | Runtime managing this container.

type: keyword

example: `docker`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [container.runtime](https://opentelemetry.io/docs/specs/semconv/attributes-registry/container/#container-runtime)
| extended | +| $$$field-container-security-context-privileged$$$[container.security_context.privileged](#field-container-security-context-privileged) | Indicates whether the container is running in privileged mode.

type: boolean
| extended | + diff --git a/docs/reference/ecs-contributing.md b/docs/reference/ecs-contributing.md new file mode 100644 index 0000000000..d8095f0812 --- /dev/null +++ b/docs/reference/ecs-contributing.md @@ -0,0 +1,14 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-contributing.html +applies_to: + stack: all + serverless: all +--- + +# Contributing to ECS [ecs-contributing] + +All information related to ECS is versioned in the [elastic/ecs](https://github.com/elastic/ecs) repository. All changes to ECS happen through Pull Requests submitted through Git. + +See the [Contribution guidelines](https://github.com/elastic/ecs/blob/master/CONTRIBUTING.md). + diff --git a/docs/reference/ecs-conventions.md b/docs/reference/ecs-conventions.md new file mode 100644 index 0000000000..e15e88a3d6 --- /dev/null +++ b/docs/reference/ecs-conventions.md @@ -0,0 +1,61 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-conventions.html +applies_to: + stack: all + serverless: all +--- + +# Conventions [ecs-conventions] + +The implementation of ECS follows a few conventions. Understanding them will help you understand ECS better. + + +## Datatype for integers [_datatype_for_integers] + +Unless otherwise noted, the datatype used for integer fields should be `long`. + + +## IDs and most codes are keywords, not integers [_ids_and_most_codes_are_keywords_not_integers] + +Despite the fact that IDs and codes (such as error codes) are often integers, this is not always the case. Since we want to make it possible to map as many systems and data sources to ECS as possible, we default to using the `keyword` type for IDs and codes. + +Some specific kinds of codes are always integers, like HTTP status codes. If those have a specific corresponding specific field (as HTTP status does), its type can safely be an integer type. But generic fields like `error.code` cannot have this guarantee, and are therefore `keyword`. + + +## Text indexing and multi-fields [_text_indexing_and_multi_fields] + +Elasticsearch can index text using datatypes: + +* **`text`** Text indexing allows for full text search, or searching arbitrary words that are part of the field. See [Text datatype](elasticsearch://reference/elasticsearch/mapping-reference/text.md) in the {{es}} Reference Guide. +* **`keyword`** Keyword indexing offers faster exact match filtering, prefix search (like autocomplete), and makes aggregations (like {{kib}} visualizations) possible. See the {{es}} Reference Guide for more information on [exact match filtering](elasticsearch://reference/query-languages/query-dsl-term-query.md), [prefix search](elasticsearch://reference/query-languages/query-dsl-prefix-query.md), or [aggregations](elasticsearch://reference/data-analysis/aggregations/index.md). + + +## Default Elasticsearch convention for indexing text fields [_default_elasticsearch_convention_for_indexing_text_fields] + +Unless your index mapping or index template specifies otherwise (as the [ECS index template](/reference/ecs-artifacts.md) does), Elasticsearch indexes a text field as `text` at the canonical field name, and indexes a second time as `keyword`, nested in a multi-field. + +Default Elasticsearch convention: + +* Canonical field: `myfield` is `text` +* Multi-field: `myfield.keyword` is `keyword` + + +## ECS convention for indexing text fields [_ecs_convention_for_indexing_text_fields] + +ECS flips the above convention around. + +For monitoring use cases, `keyword` indexing is needed almost exclusively, with full text search needed on very few fields. Moreover, indexing for full text search on lots of fields, where it’s not expected to be used is wasteful of resources. + +Given these two premises, ECS defaults all text indexing to `keyword` datatype (with very few exceptions). Any use case that requires full text search indexing on additional fields can add a [multi-field](elasticsearch://reference/elasticsearch/mapping-reference/multi-fields.md) for full text search. Doing so does not conflict with ECS, as the canonical field name will remain `keyword` indexed. + +So the ECS multi-field convention for text is: + +* Canonical field: `myfield` is `keyword` +* Multi-field: `myfield.text` is `text` + +**Exceptions** + +The only exceptions to this convention are fields `message` and `error.message`, which are indexed for full text search only, with no multi-field. These two fields don’t follow the new convention because they are deemed too big of a breaking change with these two widely used fields in Beats. + +Any future field that will be indexed for full text search in ECS will however follow the multi-field convention where `text` indexing is nested in the multi-field. diff --git a/docs/reference/ecs-converting.md b/docs/reference/ecs-converting.md new file mode 100644 index 0000000000..9e67bb8b4a --- /dev/null +++ b/docs/reference/ecs-converting.md @@ -0,0 +1,158 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-converting.html +applies_to: + stack: all + serverless: all +--- + +# Map custom data to ECS [ecs-converting] + +A common schema helps you correlate and use data from various sources. + +Fields for most Elastic modules and solutions (version 7.0 and later) are mapped to the Elastic Common Schema. You may want to map data from other implementations to ECS to help you correlate data across all of your products and solutions. + + +## Before you start [ecs-converting-before-you-start] + +Before you start a conversion, be sure that you understand the basics below. + + +### Core and extended levels [core-or-ext] + +Make sure you understand the distinction between Core and Extended fields, as explained in the [Guidelines and Best Practices](/reference/ecs-guidelines.md). + +Core and Extended fields are documented in the [*ECS Field Reference*](/reference/ecs-field-reference.md) or, for a single page representation of all fields, please see the [generated CSV of fields](https://github.com/elastic/ecs/blob/master/generated/csv/fields.csv). + + +### An approach to mapping an existing implementation [ecs-conv] + +Here’s the recommended approach for converting an existing implementation to ECS. + +1. Review each field in your original event and map it to the relevant ECS field. + + * Start by mapping your field to the relevant ECS Core field. + * If a relevant ECS Core field does not exist, map your field to the relevant ECS Extended field. + * If no relevant ECS Extended field exists, consider keeping your field with its original details, or possibly renaming it using ECS naming guidelines and attempt to map one or more of your original event fields to it. + +2. Review each ECS Core field, and attempt to populate it. + + * Review your original event data again + * Consider populating the field based on additional meta-data such as static information (e.g. add `event.category:authentication` even if your auth events don’t mention the word "authentication") + * Consider capturing additional environment meta-data, such as information about the host, container or cloud instance. + +3. Review other extended fields from any field set you are already using, and attempt to populate it as well. +4. Set `ecs.version` to the version of the schema you are conforming to. This will allow you to upgrade your sources, pipelines and content (like dashboards) smoothly in the future. + + +### Use a spreadsheet to plan your migration [ecs-conv-spreadsheet] + +Using a spreadsheet to plan the migration from pre-existing source fields to ECS is a common practice. It’s a good way to address each of your fields methodically among colleagues. + +To help you plan your migration, Elastic offers a [spreadsheet template](https://ela.st/sample-pipeline-mapping). You can use a CSV version of this spreadsheet to [automatically create an {{es}} ingest pipeline](#ecs-map-custom-data-to-ecs-es-pipeline). This is the easiest and most consistent way to map your custom data to ECS, regardless of your ingest method. + + +## Map custom data to ECS using an {{es}} ingest pipeline [ecs-map-custom-data-to-ecs-es-pipeline] + +Use {{kib}}'s **Create pipeline from CSV** feature to create an {{es}} ingest pipeline from a CSV file that maps custom data to ECS fields. + +Before you start, ensure you meet the [prerequisites](docs-content://manage-data/ingest/transform-enrich/ingest-pipelines.md#ingest-prerequisites) to create ingest pipelines in {{kib}}. + +1. Download or make a copy of the [spreadsheet template](https://ela.st/sample-pipeline-mapping). +2. Use the spreadsheet to map your custom data to ECS fields. While you can include additional columns, {{kib}} only processes the following supported columns. Other columns are ignored. + +:::::{dropdown} Supported columns +`source_field` +: (Required) JSON field key from your custom data. Supports dot notation. Rows with an empty `source_field` are skipped. + + `destination_field` +: (Required) ECS field name. Supports dot notation. To perform a `format_action` without renaming the field, leave `destination_field` empty. + +If the `destination field` is `@timestamp`, a `format_action` of `parse_timestamp` and a `timestamp_format` of `UNIX_MS` are used, regardless of any provided values. This helps prevent downstream conversion problems. + +`format_action` +: (Optional) Conversion to apply to the field value. + +::::{dropdown} Valid values +(empty) +: No conversion. + +`parse_timestamp` +: Formats a date or time value. To specify a format, use `timestamp_format`. + +`to_array` +: Converts to an array. + +`to_boolean` +: Converts to a boolean. + +`to_float` +: Converts to a floating point number. + +`to_integer` +: Converts to an integer + +`to_string` +: Converts to a string. + +`lowercase` +: Converts to lowercase. + +`uppercase` +: Converts to uppercase. +:::: + +`timestamp_format` +: (Optional) Time and date format to use with the `parse_timestamp` format action. Valid values are `UNIX`, `UNIX_MS`, `ISO8601`, `TAI64N`, and [Java time patterns](elasticsearch://reference/elasticsearch/mapping-reference/mapping-date-format.md). Defaults to `UNIX_MS`. + +`copy_action` +: (Optional) Action to take on the `source_field`. Valid values are: + +::::{dropdown} Valid values +(empty) +: (Default) Uses the default action. You’ll specify the default action later on {{kib}}'s **Create pipeline from CSV** page. + +`copy` +: Makes a copy of the `source_field` to use as the `destination_field`. The final document contains both fields. + +`rename` +: Renames the `source_field` to the `destination_field`. The final document only contains the `destination_field`. + +:::: +::::: + +3. Save and export your spreadsheet as a CSV file. + + ::::{note} + {{kib}}'s **Create pipeline from CSV** feature only supports CSV files up to 100 MB. + :::: + +4. In {{kib}}, open the main menu and click **Stack Management > Ingest Pipelines > Create pipeline > New pipeline from CSV**. + + :::{image} ../images/kib-create-pipeline-from-csv.png + :alt: Create Pipeline from CSV in Kibana + :class: screenshot + ::: + +5. On the **Create pipeline from CSV** page, upload your CSV file. +6. Under **Default action**, select the **Copy field name** or **Rename field** option. + + For the **Copy field name** option, {{kib}} makes a copy of the `source_field` to use as the `destination_field` by default. The final document contains both fields. + + For the **Rename field** option, renames the `source_field` to the `destination_field` by default. The final document only contains the `destination_field`. + + You can override this default using the `copy_action` column of your CSV. + +7. Click **Process CSV**. + + {{kib}} displays a JSON preview of the ingest pipeline generated from your CSV file. + + :::{image} ../images/kib-create-pipeline-from-csv-preview.png + :alt: Preview pipeline from CSV in Kibana + :class: screenshot + ::: + +8. To create the pipeline, click **Continue to create pipeline**. +9. On the **Create pipeline** page, you can add additional ingest processors to your pipeline. + + When you’re done, click **Create pipeline**. diff --git a/docs/reference/ecs-custom-fields-in-ecs.md b/docs/reference/ecs-custom-fields-in-ecs.md new file mode 100644 index 0000000000..8be39fdf26 --- /dev/null +++ b/docs/reference/ecs-custom-fields-in-ecs.md @@ -0,0 +1,95 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-custom-fields-in-ecs.html +applies_to: + stack: all + serverless: all +--- + +# Custom fields [ecs-custom-fields-in-ecs] + +ECS defines fields, their datatypes and their usage, and classifies them in "core" and "extended" levels. + +However, ECS does not define anything about custom fields. By definition, they are additional fields, exactly as the user or the integration defines them, independently of ECS. + +Users and integrations are welcome to capture additional information in their events, as custom fields. This flexibility is by design, and ensures that no one is ever blocked by something not being supported by ECS yet. + +ECS is under active development, however. Adding custom fields carries a small risk of conflicting with a future ECS field. There are ways of modeling custom fields that will lead to lower chances of conflict with future versions of ECS. This section outlines a few of these strategies. + + +## Modeling to reduce chances of conflict [_modeling_to_reduce_chances_of_conflict] + + +### The `labels` field [_the_labels_field] + +Any time a data source has a few extra fields that can be modeled with the `keyword` data type, the simplest way to capture them is with the ECS field `labels`. + +Example: + +```json +{ "labels": { "foo_id": "beef42", "env": "production" }, + "message": "...", + "event": { ... } +} +``` + +If `labels` doesn’t work for your use case, here’s a few more tips to avoid conflicts. + + +### Proper names [_proper_names] + +ECS tries to model information by using the name of concepts, and avoids proper names such as tool names, project names or company names. By extension, nesting custom fields under a proper name is a relatively safe approach to adding custom fields. This is the approach taken by Filebeats modules, for example. + +As an example, an HTTP log from HAProxy will contain typical HTTP information, as well as proxy details and statistics. The standard HTTP information can be captured in the ECS field sets `http` and `url`, and the extra details in a custom `haproxy` section: + +```json +{ "http": { "request": { "method": "get", ... }, + "response": { "status_code": 200, ... } }, + "url": { "original": "/favicon.ico", ... }, + "haproxy": { "frontend_name": "myfrontend", "backend_name": "mybackend_prod", + "backend_queue": 0, ... } +} +``` + + +### Capitalization [_capitalization] + +ECS strives for a consistent feel by using nesting to group related concepts, and underscores to join words. Following these guidelines for custom fields ensures you preserve the same consistent experience throughout your schemas. + +Note however, that breaking away from these guidelines for your custom fields can be used to your advantage. It can be a good way to differentiate between ECS fields and custom fields. Since ECS doesn’t use capitalization for field names, this approach virtually guarantees that custom fields will not conflict with future ECS fields. + +Common proxy concepts could modelled via a capitalized, but generic concept name. + +HAProxy example: + +```json +{ "http": { "request": { "method": "get", ... } }, + "url": { "original": "/favicon.ico", ... }, + "Proxy": { "FrontendName": "myfrontend", "BackendName": "mybackend_prod" }, + "event": { "module": "haproxy" } +} +``` + +NGINX example: + +```json +{ "http": { "request": { "method": "get", ... } }, + "url": { "original": "/favicon.ico", ... }, + "Proxy": { "FrontendName": "another_frontend", "BackendName": "another_backend_prod" }, + "event": { "module": "nginx" } +} +``` + +The above demonstrates that using a common concept name in custom fields can still be beneficial to correlate among multiple sources that populate them. Using capitalization ensures a future version of ECS that defines a `proxy` field set will not conflict. + +Here’s a sample event, during a migration from the custom field, to using a new equivalent ECS field set: + +```json +{ "http": { "request": { "method": "get", ... } }, + "Proxy": { "FrontendName": "myfrontend", "BackendName": "mybackend_prod" }, + "proxy": { "frontend_name": "myfrontend", "backend_name": "mybackend_prod" } +} +``` + +The above will look strange during the migration. However the ability to start populating ECS fields while custom fields are still present in your events makes it possible to decouple the upgrade to a new version of ECS from the time you adjust your pipelines and analysis content. + diff --git a/docs/reference/ecs-data_stream.md b/docs/reference/ecs-data_stream.md new file mode 100644 index 0000000000..4df9650ff8 --- /dev/null +++ b/docs/reference/ecs-data_stream.md @@ -0,0 +1,30 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-data_stream.html +applies_to: + stack: all + serverless: all +--- + +# Data stream fields [ecs-data_stream] + +The data_stream fields take part in defining the new data stream naming scheme. + +In the new data stream naming scheme the value of the data stream fields combine to the name of the actual data stream in the following manner: `{data_stream.type}-{data_stream.dataset}-{data_stream.namespace}`. This means the fields can only contain characters that are valid as part of names of data streams. More details about this can be found in this [blog post](https://www.elastic.co/blog/an-introduction-to-the-elastic-data-stream-naming-scheme). + +An Elasticsearch data stream consists of one or more backing indices, and a data stream name forms part of the backing indices names. Due to this convention, data streams must also follow index naming restrictions. For example, data stream names cannot include `\`, `/`, `*`, `?`, `"`, `<`, `>`, `|`, ` ` (space character), `,`, or `#`. Please see the Elasticsearch reference for additional [restrictions](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-create). + +::::{warning} +These fields are in beta and are subject to change. +:::: + + + +## Data stream field details [_data_stream_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-data-stream-dataset$$$[data_stream.dataset](#field-data-stream-dataset) | The field can contain anything that makes sense to signify the source of the data.

Examples include `nginx.access`, `prometheus`, `endpoint` etc. For data streams that otherwise fit, but that do not have dataset set we use the value "generic" for the dataset value. `event.dataset` should have the same value as `data_stream.dataset`.

Beyond the Elasticsearch data stream naming criteria noted above, the `dataset` value has additional restrictions:

- Must not contain `-`
- No longer than 100 characters

type: constant_keyword

example: `nginx.access`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat "not-applicable") Not applicable.
| extended | +| $$$field-data-stream-namespace$$$[data_stream.namespace](#field-data-stream-namespace) | A user defined namespace. Namespaces are useful to allow grouping of data.

Many users already organize their indices this way, and the data stream naming scheme now provides this best practice as a default. Many users will populate this field with `default`. If no value is used, it falls back to `default`.

Beyond the Elasticsearch index naming criteria noted above, `namespace` value has the additional restrictions:

- Must not contain `-`
- No longer than 100 characters

type: constant_keyword

example: `production`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat "not-applicable") Not applicable.
| extended | +| $$$field-data-stream-type$$$[data_stream.type](#field-data-stream-type) | An overarching type for the data stream.

Currently allowed values are "logs" and "metrics". We expect to also add "traces" and "synthetics" in the near future.

type: constant_keyword

example: `logs`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat "not-applicable") Not applicable.
| extended | + diff --git a/docs/reference/ecs-destination.md b/docs/reference/ecs-destination.md new file mode 100644 index 0000000000..39ceeae2a8 --- /dev/null +++ b/docs/reference/ecs-destination.md @@ -0,0 +1,44 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-destination.html +applies_to: + stack: all + serverless: all +--- + +# Destination fields [ecs-destination] + +Destination fields capture details about the receiver of a network exchange/packet. These fields are populated from a network event, packet, or other event containing details of a network transaction. + +Destination fields are usually populated in conjunction with source fields. The source and destination fields are considered the baseline and should always be filled if an event contains source and destination details from a network transaction. If the event also contains identification of the client and server roles, then the client and server fields should also be populated. + + +## Destination field details [_destination_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-destination-address$$$[destination.address](#field-destination-address) | Some event destination addresses are defined ambiguously. The event will sometimes list an IP, a domain or a unix socket. You should always store the raw address in the `.address` field.

Then it should be duplicated to `.ip` or `.domain`, depending on which one it is.

type: keyword

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [destination.address](https://opentelemetry.io/docs/specs/semconv/attributes-registry/destination/#destination-address)
| extended | +| $$$field-destination-bytes$$$[destination.bytes](#field-destination-bytes) | Bytes sent from the destination to the source.

type: long

example: `184`
| core | +| $$$field-destination-domain$$$[destination.domain](#field-destination-domain) | The domain name of the destination system.

This value may be a host name, a fully qualified domain name, or another host naming format. The value may derive from the original event or be added from enrichment.

type: keyword

example: `foo.example.com`
| core | +| $$$field-destination-ip$$$[destination.ip](#field-destination-ip) | IP address of the destination (IPv4 or IPv6).

type: ip
| core | +| $$$field-destination-mac$$$[destination.mac](#field-destination-mac) | MAC address of the destination.

The notation format from RFC 7042 is suggested: Each octet (that is, 8-bit byte) is represented by two [uppercase] hexadecimal digits giving the value of the octet as an unsigned integer. Successive octets are separated by a hyphen.

type: keyword

example: `00-00-5E-00-53-23`
| core | +| $$$field-destination-nat-ip$$$[destination.nat.ip](#field-destination-nat-ip) | Translated ip of destination based NAT sessions (e.g. internet to private DMZ)

Typically used with load balancers, firewalls, or routers.

type: ip
| extended | +| $$$field-destination-nat-port$$$[destination.nat.port](#field-destination-nat-port) | Port the source session is translated to by NAT Device.

Typically used with load balancers, firewalls, or routers.

type: long
| extended | +| $$$field-destination-packets$$$[destination.packets](#field-destination-packets) | Packets sent from the destination to the source.

type: long

example: `12`
| core | +| $$$field-destination-port$$$[destination.port](#field-destination-port) | Port of the destination.

type: long

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [destination.port](https://opentelemetry.io/docs/specs/semconv/attributes-registry/destination/#destination-port)
| core | +| $$$field-destination-registered-domain$$$[destination.registered_domain](#field-destination-registered-domain) | The highest registered destination domain, stripped of the subdomain.

For example, the registered domain for "foo.example.com" is "example.com".

This value can be determined precisely with a list like the public suffix list ([https://publicsuffix.org](https://publicsuffix.org)). Trying to approximate this by simply taking the last two labels will not work well for TLDs such as "co.uk".

type: keyword

example: `example.com`
| extended | +| $$$field-destination-subdomain$$$[destination.subdomain](#field-destination-subdomain) | The subdomain portion of a fully qualified domain name includes all of the names except the host name under the registered_domain. In a partially qualified domain, or if the the qualification level of the full name cannot be determined, subdomain contains all of the names below the registered domain.

For example the subdomain portion of "www.east.mydomain.co.uk" is "east". If the domain has multiple levels of subdomain, such as "sub2.sub1.example.com", the subdomain field should contain "sub2.sub1", with no trailing period.

type: keyword

example: `east`
| extended | +| $$$field-destination-top-level-domain$$$[destination.top_level_domain](#field-destination-top-level-domain) | The effective top level domain (eTLD), also known as the domain suffix, is the last part of the domain name. For example, the top level domain for example.com is "com".

This value can be determined precisely with a list like the public suffix list ([https://publicsuffix.org](https://publicsuffix.org)). Trying to approximate this by simply taking the last label will not work well for effective TLDs such as "co.uk".

type: keyword

example: `co.uk`
| extended | + + +## Field reuse [_field_reuse_5] + + +### Field sets that can be nested under destination [ecs-destination-nestings] + +| Location | Field Set | Description | +| --- | --- | --- | +| `destination.as.*` | [as](/reference/ecs-as.md) | Fields describing an Autonomous System (Internet routing prefix). | +| `destination.geo.*` | [geo](/reference/ecs-geo.md) | Fields describing a location. | +| `destination.user.*` | [user](/reference/ecs-user.md) | Fields to describe the user relevant to the event. | + diff --git a/docs/reference/ecs-device.md b/docs/reference/ecs-device.md new file mode 100644 index 0000000000..b2fe62c291 --- /dev/null +++ b/docs/reference/ecs-device.md @@ -0,0 +1,44 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-device.html +applies_to: + stack: all + serverless: all +--- + +# Device fields [ecs-device] + +Fields that describe a device instance and its characteristics. Data collected for applications and processes running on a (mobile) device can be enriched with these fields to describe the identity, type and other characteristics of the device. + +This field group definition is based on the Device namespace of the OpenTelemetry Semantic Conventions ([https://opentelemetry.io/docs/reference/specification/resource/semantic_conventions/device/](https://opentelemetry.io/docs/reference/specification/resource/semantic_conventions/device/)). + +::::{warning} +These fields are in beta and are subject to change. +:::: + + + +## Device field details [_device_field_details] + +::::{tab-set} + +:::{tab-item} Fields + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-device-id$$$[device.id](#field-device-id) | The unique identifier of a device. The identifier must not change across application sessions but stay fixed for an instance of a (mobile) device.

On iOS, this value must be equal to the vendor identifier ([https://developer.apple.com/documentation/uikit/uidevice/1620059-identifierforvendor](https://developer.apple.com/documentation/uikit/uidevice/1620059-identifierforvendor)). On Android, this value must be equal to the Firebase Installation ID or a globally unique UUID which is persisted across sessions in your application.

For GDPR and data protection law reasons this identifier should not carry information that would allow to identify a user.

type: keyword

example: `00000000-54b3-e7c7-0000-000046bffd97`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [device.id](https://opentelemetry.io/docs/specs/semconv/attributes-registry/device/#device-id)
| extended | +| $$$field-device-manufacturer$$$[device.manufacturer](#field-device-manufacturer) | The vendor name of the device manufacturer.

type: keyword

example: `Samsung`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [device.manufacturer](https://opentelemetry.io/docs/specs/semconv/attributes-registry/device/#device-manufacturer)
| extended | +| $$$field-device-model-identifier$$$[device.model.identifier](#field-device-model-identifier) | The machine readable identifier of the device model.

type: keyword

example: `SM-G920F`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [device.model.identifier](https://opentelemetry.io/docs/specs/semconv/attributes-registry/device/#device-model-identifier)
| extended | +| $$$field-device-model-name$$$[device.model.name](#field-device-model-name) | The human readable marketing name of the device model.

type: keyword

example: `Samsung Galaxy S6`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [device.model.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/device/#device-model-name)
| extended | + +::: + +:::{tab-item} Beta Fields + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-device-serial-number$$$[device.serial_number](#field-device-serial-number) | The unique serial number serves as a distinct identifier for each device, aiding in inventory management and device authentication.

type: keyword

example: `DJGAQS4CW5`
| core | + +::: +:::: + diff --git a/docs/reference/ecs-dll.md b/docs/reference/ecs-dll.md new file mode 100644 index 0000000000..2f2d738405 --- /dev/null +++ b/docs/reference/ecs-dll.md @@ -0,0 +1,38 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-dll.html +applies_to: + stack: all + serverless: all +--- + +# DLL fields [ecs-dll] + +These fields contain information about code libraries dynamically loaded into processes. + +Many operating systems refer to "shared code libraries" with different names, but this field set refers to all of the following: + +* Dynamic-link library (`.dll`) commonly used on Windows +* Shared Object (`.so`) commonly used on Unix-like operating systems +* Dynamic library (`.dylib`) commonly used on macOS + + +## DLL field details [_dll_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-dll-name$$$[dll.name](#field-dll-name) | Name of the library.

This generally maps to the name of the file on disk.

type: keyword

example: `kernel32.dll`
| core | +| $$$field-dll-path$$$[dll.path](#field-dll-path) | Full file path of the library.

type: keyword

example: `C:\Windows\System32\kernel32.dll`
| extended | + + +## Field reuse [_field_reuse_6] + + +### Field sets that can be nested under DLL [ecs-dll-nestings] + +| Location | Field Set | Description | +| --- | --- | --- | +| `dll.code_signature.*` | [code_signature](/reference/ecs-code_signature.md) | These fields contain information about binary code signatures. | +| `dll.hash.*` | [hash](/reference/ecs-hash.md) | Hashes, usually file hashes. | +| `dll.pe.*` | [pe](/reference/ecs-pe.md) | These fields contain Windows Portable Executable (PE) metadata. | + diff --git a/docs/reference/ecs-dns.md b/docs/reference/ecs-dns.md new file mode 100644 index 0000000000..b5b951c0fd --- /dev/null +++ b/docs/reference/ecs-dns.md @@ -0,0 +1,38 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-dns.html +applies_to: + stack: all + serverless: all +--- + +# DNS fields [ecs-dns] + +Fields describing DNS queries and answers. + +DNS events should either represent a single DNS query prior to getting answers (`dns.type:query`) or they should represent a full exchange and contain the query details as well as all of the answers that were provided for this query (`dns.type:answer`). + + +## DNS field details [_dns_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-dns-answers$$$[dns.answers](#field-dns-answers) | An array containing an object for each answer section returned by the server.

The main keys that should be present in these objects are defined by ECS. Records that have more information may contain more keys than what ECS defines.

Not all DNS data sources give all details about DNS answers. At minimum, answer objects must contain the `data` key. If more information is available, map as much of it to ECS as possible, and add any additional fields to the answer objects as custom fields.

type: object

Note: this field should contain an array of values.
| extended | +| $$$field-dns-answers-class$$$[dns.answers.class](#field-dns-answers-class) | The class of DNS data contained in this resource record.

type: keyword

example: `IN`
| extended | +| $$$field-dns-answers-data$$$[dns.answers.data](#field-dns-answers-data) | The data describing the resource.

The meaning of this data depends on the type and class of the resource record.

type: keyword

example: `10.10.10.10`
| extended | +| $$$field-dns-answers-name$$$[dns.answers.name](#field-dns-answers-name) | The domain name to which this resource record pertains.

If a chain of CNAME is being resolved, each answer’s `name` should be the one that corresponds with the answer’s `data`. It should not simply be the original `question.name` repeated.

type: keyword

example: `www.example.com`
| extended | +| $$$field-dns-answers-ttl$$$[dns.answers.ttl](#field-dns-answers-ttl) | The time interval in seconds that this resource record may be cached before it should be discarded. Zero values mean that the data should not be cached.

type: long

example: `180`
| extended | +| $$$field-dns-answers-type$$$[dns.answers.type](#field-dns-answers-type) | The type of data contained in this resource record.

type: keyword

example: `CNAME`
| extended | +| $$$field-dns-header-flags$$$[dns.header_flags](#field-dns-header-flags) | Array of 2 letter DNS header flags.

Expected values for this field:

* `AA`
* `TC`
* `RD`
* `RA`
* `AD`
* `CD`
* `DO`

type: keyword

Note: this field should contain an array of values.

example: `["RD", "RA"]`
| extended | +| $$$field-dns-id$$$[dns.id](#field-dns-id) | The DNS packet identifier assigned by the program that generated the query. The identifier is copied to the response.

type: keyword

example: `62111`
| extended | +| $$$field-dns-op-code$$$[dns.op_code](#field-dns-op-code) | The DNS operation code that specifies the kind of query in the message. This value is set by the originator of a query and copied into the response.

type: keyword

example: `QUERY`
| extended | +| $$$field-dns-question-class$$$[dns.question.class](#field-dns-question-class) | The class of records being queried.

type: keyword

example: `IN`
| extended | +| $$$field-dns-question-name$$$[dns.question.name](#field-dns-question-name) | The name being queried.

If the name field contains non-printable characters (below 32 or above 126), those characters should be represented as escaped base 10 integers (\DDD). Back slashes and quotes should be escaped. Tabs, carriage returns, and line feeds should be converted to \t, \r, and \n respectively.

type: keyword

example: `www.example.com`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [dns.question.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/dns/#dns-question-name)
| extended | +| $$$field-dns-question-registered-domain$$$[dns.question.registered_domain](#field-dns-question-registered-domain) | The highest registered domain, stripped of the subdomain.

For example, the registered domain for "foo.example.com" is "example.com".

This value can be determined precisely with a list like the public suffix list ([https://publicsuffix.org](https://publicsuffix.org)). Trying to approximate this by simply taking the last two labels will not work well for TLDs such as "co.uk".

type: keyword

example: `example.com`
| extended | +| $$$field-dns-question-subdomain$$$[dns.question.subdomain](#field-dns-question-subdomain) | The subdomain is all of the labels under the registered_domain.

If the domain has multiple levels of subdomain, such as "sub2.sub1.example.com", the subdomain field should contain "sub2.sub1", with no trailing period.

type: keyword

example: `www`
| extended | +| $$$field-dns-question-top-level-domain$$$[dns.question.top_level_domain](#field-dns-question-top-level-domain) | The effective top level domain (eTLD), also known as the domain suffix, is the last part of the domain name. For example, the top level domain for example.com is "com".

This value can be determined precisely with a list like the public suffix list ([https://publicsuffix.org](https://publicsuffix.org)). Trying to approximate this by simply taking the last label will not work well for effective TLDs such as "co.uk".

type: keyword

example: `co.uk`
| extended | +| $$$field-dns-question-type$$$[dns.question.type](#field-dns-question-type) | The type of record being queried.

type: keyword

example: `AAAA`
| extended | +| $$$field-dns-resolved-ip$$$[dns.resolved_ip](#field-dns-resolved-ip) | Array containing all IPs seen in `answers.data`.

The `answers` array can be difficult to use, because of the variety of data formats it can contain. Extracting all IP addresses seen in there to `dns.resolved_ip` makes it possible to index them as IP addresses, and makes them easier to visualize and query for.

type: ip

Note: this field should contain an array of values.

example: `["10.10.10.10", "10.10.10.11"]`
| extended | +| $$$field-dns-response-code$$$[dns.response_code](#field-dns-response-code) | The DNS response code.

type: keyword

example: `NOERROR`
| extended | +| $$$field-dns-type$$$[dns.type](#field-dns-type) | The type of DNS event captured, query or answer.

If your source of DNS events only gives you DNS queries, you should only create dns events of type `dns.type:query`.

If your source of DNS events gives you answers as well, you should create one event per query (optionally as soon as the query is seen). And a second event containing all query details as well as an array of answers.

type: keyword

example: `answer`
| extended | + diff --git a/docs/reference/ecs-ecs.md b/docs/reference/ecs-ecs.md new file mode 100644 index 0000000000..9ee3a6fcd1 --- /dev/null +++ b/docs/reference/ecs-ecs.md @@ -0,0 +1,19 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-ecs.html +applies_to: + stack: all + serverless: all +--- + +# ECS fields [ecs-ecs] + +Meta-information specific to ECS. + + +## ECS field details [_ecs_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-ecs-version$$$[ecs.version](#field-ecs-version) | ECS version this event conforms to. `ecs.version` is a required field and must exist in all events.

When querying across multiple indices — which may conform to slightly different ECS versions — this field lets integrations adjust to the schema version of the events.

type: keyword

example: `1.0.0`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat "not-applicable") Not applicable.
| core | + diff --git a/docs/reference/ecs-elf.md b/docs/reference/ecs-elf.md new file mode 100644 index 0000000000..93d520ab52 --- /dev/null +++ b/docs/reference/ecs-elf.md @@ -0,0 +1,71 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-elf.html +applies_to: + stack: all + serverless: all +--- + +# ELF header fields [ecs-elf] + +These fields contain Linux Executable Linkable Format (ELF) metadata. + +::::{warning} +These fields are in beta and are subject to change. +:::: + + + +## ELF header field details [_elf_header_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-elf-architecture$$$[elf.architecture](#field-elf-architecture) | Machine architecture of the ELF file.

type: keyword

example: `x86-64`
| extended | +| $$$field-elf-byte-order$$$[elf.byte_order](#field-elf-byte-order) | Byte sequence of ELF file.

type: keyword

example: `Little Endian`
| extended | +| $$$field-elf-cpu-type$$$[elf.cpu_type](#field-elf-cpu-type) | CPU type of the ELF file.

type: keyword

example: `Intel`
| extended | +| $$$field-elf-creation-date$$$[elf.creation_date](#field-elf-creation-date) | Extracted when possible from the file’s metadata. Indicates when it was built or compiled. It can also be faked by malware creators.

type: date
| extended | +| $$$field-elf-exports$$$[elf.exports](#field-elf-exports) | List of exported element names and types.

type: flattened

Note: this field should contain an array of values.
| extended | +| $$$field-elf-go-import-hash$$$[elf.go_import_hash](#field-elf-go-import-hash) | A hash of the Go language imports in an ELF file excluding standard library imports. An import hash can be used to fingerprint binaries even after recompilation or other code-level transformations have occurred, which would change more traditional hash values.

The algorithm used to calculate the Go symbol hash and a reference implementation are available here: [https://github.com/elastic/toutoumomoma](https://github.com/elastic/toutoumomoma)

type: keyword

example: `10bddcb4cee42080f76c88d9ff964491`
| extended | +| $$$field-elf-go-imports$$$[elf.go_imports](#field-elf-go-imports) | List of imported Go language element names and types.

type: flattened
| extended | +| $$$field-elf-go-imports-names-entropy$$$[elf.go_imports_names_entropy](#field-elf-go-imports-names-entropy) | Shannon entropy calculation from the list of Go imports.

type: long
| extended | +| $$$field-elf-go-imports-names-var-entropy$$$[elf.go_imports_names_var_entropy](#field-elf-go-imports-names-var-entropy) | Variance for Shannon entropy calculation from the list of Go imports.

type: long
| extended | +| $$$field-elf-go-stripped$$$[elf.go_stripped](#field-elf-go-stripped) | Set to true if the file is a Go executable that has had its symbols stripped or obfuscated and false if an unobfuscated Go executable.

type: boolean
| extended | +| $$$field-elf-header-abi-version$$$[elf.header.abi_version](#field-elf-header-abi-version) | Version of the ELF Application Binary Interface (ABI).

type: keyword
| extended | +| $$$field-elf-header-class$$$[elf.header.class](#field-elf-header-class) | Header class of the ELF file.

type: keyword
| extended | +| $$$field-elf-header-data$$$[elf.header.data](#field-elf-header-data) | Data table of the ELF header.

type: keyword
| extended | +| $$$field-elf-header-entrypoint$$$[elf.header.entrypoint](#field-elf-header-entrypoint) | Header entrypoint of the ELF file.

type: long
| extended | +| $$$field-elf-header-object-version$$$[elf.header.object_version](#field-elf-header-object-version) | "0x1" for original ELF files.

type: keyword
| extended | +| $$$field-elf-header-os-abi$$$[elf.header.os_abi](#field-elf-header-os-abi) | Application Binary Interface (ABI) of the Linux OS.

type: keyword
| extended | +| $$$field-elf-header-type$$$[elf.header.type](#field-elf-header-type) | Header type of the ELF file.

type: keyword
| extended | +| $$$field-elf-header-version$$$[elf.header.version](#field-elf-header-version) | Version of the ELF header.

type: keyword
| extended | +| $$$field-elf-import-hash$$$[elf.import_hash](#field-elf-import-hash) | A hash of the imports in an ELF file. An import hash can be used to fingerprint binaries even after recompilation or other code-level transformations have occurred, which would change more traditional hash values.

This is an ELF implementation of the Windows PE imphash.

type: keyword

example: `d41d8cd98f00b204e9800998ecf8427e`
| extended | +| $$$field-elf-imports$$$[elf.imports](#field-elf-imports) | List of imported element names and types.

type: flattened

Note: this field should contain an array of values.
| extended | +| $$$field-elf-imports-names-entropy$$$[elf.imports_names_entropy](#field-elf-imports-names-entropy) | Shannon entropy calculation from the list of imported element names and types.

type: long
| extended | +| $$$field-elf-imports-names-var-entropy$$$[elf.imports_names_var_entropy](#field-elf-imports-names-var-entropy) | Variance for Shannon entropy calculation from the list of imported element names and types.

type: long
| extended | +| $$$field-elf-sections$$$[elf.sections](#field-elf-sections) | An array containing an object for each section of the ELF file.

The keys that should be present in these objects are defined by sub-fields underneath `elf.sections.*`.

type: nested

Note: this field should contain an array of values.
| extended | +| $$$field-elf-sections-chi2$$$[elf.sections.chi2](#field-elf-sections-chi2) | Chi-square probability distribution of the section.

type: long
| extended | +| $$$field-elf-sections-entropy$$$[elf.sections.entropy](#field-elf-sections-entropy) | Shannon entropy calculation from the section.

type: long
| extended | +| $$$field-elf-sections-flags$$$[elf.sections.flags](#field-elf-sections-flags) | ELF Section List flags.

type: keyword
| extended | +| $$$field-elf-sections-name$$$[elf.sections.name](#field-elf-sections-name) | ELF Section List name.

type: keyword
| extended | +| $$$field-elf-sections-physical-offset$$$[elf.sections.physical_offset](#field-elf-sections-physical-offset) | ELF Section List offset.

type: keyword
| extended | +| $$$field-elf-sections-physical-size$$$[elf.sections.physical_size](#field-elf-sections-physical-size) | ELF Section List physical size.

type: long
| extended | +| $$$field-elf-sections-type$$$[elf.sections.type](#field-elf-sections-type) | ELF Section List type.

type: keyword
| extended | +| $$$field-elf-sections-var-entropy$$$[elf.sections.var_entropy](#field-elf-sections-var-entropy) | Variance for Shannon entropy calculation from the section.

type: long
| extended | +| $$$field-elf-sections-virtual-address$$$[elf.sections.virtual_address](#field-elf-sections-virtual-address) | ELF Section List virtual address.

type: long
| extended | +| $$$field-elf-sections-virtual-size$$$[elf.sections.virtual_size](#field-elf-sections-virtual-size) | ELF Section List virtual size.

type: long
| extended | +| $$$field-elf-segments$$$[elf.segments](#field-elf-segments) | An array containing an object for each segment of the ELF file.

The keys that should be present in these objects are defined by sub-fields underneath `elf.segments.*`.

type: nested

Note: this field should contain an array of values.
| extended | +| $$$field-elf-segments-sections$$$[elf.segments.sections](#field-elf-segments-sections) | ELF object segment sections.

type: keyword
| extended | +| $$$field-elf-segments-type$$$[elf.segments.type](#field-elf-segments-type) | ELF object segment type.

type: keyword
| extended | +| $$$field-elf-shared-libraries$$$[elf.shared_libraries](#field-elf-shared-libraries) | List of shared libraries used by this ELF object.

type: keyword

Note: this field should contain an array of values.
| extended | +| $$$field-elf-telfhash$$$[elf.telfhash](#field-elf-telfhash) | telfhash symbol hash for ELF file.

type: keyword
| extended | + + +## Field reuse [_field_reuse_7] + +The `elf` fields are expected to be nested at: + +* `file.elf` +* `process.elf` + +Note also that the `elf` fields are not expected to be used directly at the root of the events. + diff --git a/docs/reference/ecs-email.md b/docs/reference/ecs-email.md new file mode 100644 index 0000000000..78faa5ed57 --- /dev/null +++ b/docs/reference/ecs-email.md @@ -0,0 +1,49 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-email.html +applies_to: + stack: all + serverless: all +--- + +# Email fields [ecs-email] + +Event details relating to an email transaction. + +This field set focuses on the email message header, body, and attachments. Network protocols that send and receive email messages such as SMTP are outside the scope of the `email.*` fields. + + +## Email field details [_email_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-email-attachments$$$[email.attachments](#field-email-attachments) | A list of objects describing the attachment files sent along with an email message.

type: nested

Note: this field should contain an array of values.
| extended | +| $$$field-email-attachments-file-extension$$$[email.attachments.file.extension](#field-email-attachments-file-extension) | Attachment file extension, excluding the leading dot.

type: keyword

example: `txt`
| extended | +| $$$field-email-attachments-file-mime-type$$$[email.attachments.file.mime_type](#field-email-attachments-file-mime-type) | The MIME media type of the attachment.

This value will typically be extracted from the `Content-Type` MIME header field.

type: keyword

example: `text/plain`
| extended | +| $$$field-email-attachments-file-name$$$[email.attachments.file.name](#field-email-attachments-file-name) | Name of the attachment file including the file extension.

type: keyword

example: `attachment.txt`
| extended | +| $$$field-email-attachments-file-size$$$[email.attachments.file.size](#field-email-attachments-file-size) | Attachment file size in bytes.

type: long

example: `64329`
| extended | +| $$$field-email-bcc-address$$$[email.bcc.address](#field-email-bcc-address) | The email address of BCC recipient

type: keyword

Note: this field should contain an array of values.

example: `bcc.user1@example.com`
| extended | +| $$$field-email-cc-address$$$[email.cc.address](#field-email-cc-address) | The email address of CC recipient

type: keyword

Note: this field should contain an array of values.

example: `cc.user1@example.com`
| extended | +| $$$field-email-content-type$$$[email.content_type](#field-email-content-type) | Information about how the message is to be displayed.

Typically a MIME type.

type: keyword

example: `text/plain`
| extended | +| $$$field-email-delivery-timestamp$$$[email.delivery_timestamp](#field-email-delivery-timestamp) | The date and time when the email message was received by the service or client.

type: date

example: `2020-11-10T22:12:34.8196921Z`
| extended | +| $$$field-email-direction$$$[email.direction](#field-email-direction) | The direction of the message based on the sending and receiving domains.

type: keyword

example: `inbound`
| extended | +| $$$field-email-from-address$$$[email.from.address](#field-email-from-address) | The email address of the sender, typically from the RFC 5322 `From:` header field.

type: keyword

Note: this field should contain an array of values.

example: `sender@example.com`
| extended | +| $$$field-email-local-id$$$[email.local_id](#field-email-local-id) | Unique identifier given to the email by the source that created the event.

Identifier is not persistent across hops.

type: keyword

example: `c26dbea0-80d5-463b-b93c-4e8b708219ce`
| extended | +| $$$field-email-message-id$$$[email.message_id](#field-email-message-id) | Identifier from the RFC 5322 `Message-ID:` email header that refers to a particular email message.

type: wildcard

example: `81ce15$8r2j59@mail01.example.com`
| extended | +| $$$field-email-origination-timestamp$$$[email.origination_timestamp](#field-email-origination-timestamp) | The date and time the email message was composed. Many email clients will fill in this value automatically when the message is sent by a user.

type: date

example: `2020-11-10T22:12:34.8196921Z`
| extended | +| $$$field-email-reply-to-address$$$[email.reply_to.address](#field-email-reply-to-address) | The address that replies should be delivered to based on the value in the RFC 5322 `Reply-To:` header.

type: keyword

Note: this field should contain an array of values.

example: `reply.here@example.com`
| extended | +| $$$field-email-sender-address$$$[email.sender.address](#field-email-sender-address) | Per RFC 5322, specifies the address responsible for the actual transmission of the message.

type: keyword
| extended | +| $$$field-email-subject$$$[email.subject](#field-email-subject) | A brief summary of the topic of the message.

type: keyword

Multi-fields:

* email.subject.text (type: match_only_text)

example: `Please see this important message.`
| extended | +| $$$field-email-to-address$$$[email.to.address](#field-email-to-address) | The email address of recipient

type: keyword

Note: this field should contain an array of values.

example: `user1@example.com`
| extended | +| $$$field-email-x-mailer$$$[email.x_mailer](#field-email-x-mailer) | The name of the application that was used to draft and send the original email message.

type: keyword

example: `Spambot v2.5`
| extended | + + +## Field reuse [_field_reuse_8] + + +### Field sets that can be nested under email [ecs-email-nestings] + +| Location | Field Set | Description | +| --- | --- | --- | +| `email.attachments.file.hash.*` | [hash](/reference/ecs-hash.md) | Hashes, usually file hashes. | + diff --git a/docs/reference/ecs-error.md b/docs/reference/ecs-error.md new file mode 100644 index 0000000000..07ad291d45 --- /dev/null +++ b/docs/reference/ecs-error.md @@ -0,0 +1,25 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-error.html +applies_to: + stack: all + serverless: all +--- + +# Error fields [ecs-error] + +These fields can represent errors of any kind. + +Use them for errors that happen while fetching events or in cases where the event itself contains an error. + + +## Error field details [_error_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-error-code$$$[error.code](#field-error-code) | Error code describing the error.

type: keyword
| core | +| $$$field-error-id$$$[error.id](#field-error-id) | Unique identifier for the error.

type: keyword
| core | +| $$$field-error-message$$$[error.message](#field-error-message) | Error message.

type: match_only_text

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") [exception.message](https://opentelemetry.io/docs/specs/semconv/attributes-registry/exception/#exception-message)
| core | +| $$$field-error-stack-trace$$$[error.stack_trace](#field-error-stack-trace) | The stack trace of this error in plain text.

type: wildcard

Multi-fields:

* error.stack_trace.text (type: match_only_text)

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") [exception.stacktrace](https://opentelemetry.io/docs/specs/semconv/attributes-registry/exception/#exception-stacktrace)
| extended | +| $$$field-error-type$$$[error.type](#field-error-type) | The type of the error, for example the class name of the exception.

type: keyword

example: `java.lang.NullPointerException`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [error.type](https://opentelemetry.io/docs/specs/semconv/attributes-registry/error/#error-type)
| extended | + diff --git a/docs/reference/ecs-event.md b/docs/reference/ecs-event.md new file mode 100644 index 0000000000..bcb2386f47 --- /dev/null +++ b/docs/reference/ecs-event.md @@ -0,0 +1,46 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-event.html +applies_to: + stack: all + serverless: all +--- + +# Event fields [ecs-event] + +The event fields are used for context information about the log or metric event itself. + +A log is defined as an event containing details of something that happened. Log events must include the time at which the thing happened. Examples of log events include a process starting on a host, a network packet being sent from a source to a destination, or a network connection between a client and a server being initiated or closed. A metric is defined as an event containing one or more numerical measurements and the time at which the measurement was taken. Examples of metric events include memory pressure measured on a host and device temperature. See the `event.kind` definition in this section for additional details about metric and state events. + + +## Event field details [_event_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-event-action$$$[event.action](#field-event-action) | The action captured by the event.

This describes the information in the event. It is more specific than `event.category`. Examples are `group-add`, `process-started`, `file-created`. The value is normally defined by the implementer.

type: keyword

example: `user-password-change`
| core | +| $$$field-event-agent-id-status$$$[event.agent_id_status](#field-event-agent-id-status) | Agents are normally responsible for populating the `agent.id` field value. If the system receiving events is capable of validating the value based on authentication information for the client then this field can be used to reflect the outcome of that validation.

For example if the agent’s connection is authenticated with mTLS and the client cert contains the ID of the agent to which the cert was issued then the `agent.id` value in events can be checked against the certificate. If the values match then `event.agent_id_status: verified` is added to the event, otherwise one of the other allowed values should be used.

If no validation is performed then the field should be omitted.

The allowed values are:

`verified` - The `agent.id` field value matches expected value obtained from auth metadata.

`mismatch` - The `agent.id` field value does not match the expected value obtained from auth metadata.

`missing` - There was no `agent.id` field in the event to validate.

`auth_metadata_missing` - There was no auth metadata or it was missing information about the agent ID.

type: keyword

example: `verified`
| extended | +| $$$field-event-category$$$[event.category](#field-event-category) | This is one of four ECS Categorization Fields, and indicates the second level in the ECS category hierarchy.

`event.category` represents the "big buckets" of ECS categories. For example, filtering on `event.category:process` yields all events relating to process activity. This field is closely related to `event.type`, which is used as a subcategory.

This field is an array. This will allow proper categorization of some events that fall in multiple categories.

type: keyword

Note: this field should contain an array of values.

**Important**: The field value must be one of the following:

api, authentication, configuration, database, driver, email, file, host, iam, intrusion_detection, library, malware, network, package, process, registry, session, threat, vulnerability, web

To learn more about when to use which value, visit the page [allowed values for event.category](/reference/ecs-allowed-values-event-category.md)
| core | +| $$$field-event-code$$$[event.code](#field-event-code) | Identification code for this event, if one exists.

Some event sources use event codes to identify messages unambiguously, regardless of message language or wording adjustments over time. An example of this is the Windows Event ID.

type: keyword

example: `4648`
| extended | +| $$$field-event-created$$$[event.created](#field-event-created) | `event.created` contains the date/time when the event was first read by an agent, or by your pipeline.

This field is distinct from `@timestamp` in that `@timestamp` typically contain the time extracted from the original event.

In most situations, these two timestamps will be slightly different. The difference can be used to calculate the delay between your source generating an event, and the time when your agent first processed it. This can be used to monitor your agent’s or pipeline’s ability to keep up with your event source.

In case the two timestamps are identical, `@timestamp` should be used.

type: date

example: `2016-05-23T08:05:34.857Z`
| core | +| $$$field-event-dataset$$$[event.dataset](#field-event-dataset) | Name of the dataset.

If an event source publishes more than one type of log or events (e.g. access log, error log), the dataset is used to specify which one the event comes from.

It’s recommended but not required to start the dataset name with the module name, followed by a dot, then the dataset name.

type: keyword

example: `apache.access`
| core | +| $$$field-event-duration$$$[event.duration](#field-event-duration) | Duration of the event in nanoseconds.

If `event.start` and `event.end` are known this value should be the difference between the end and start time.

type: long
| core | +| $$$field-event-end$$$[event.end](#field-event-end) | `event.end` contains the date when the event ended or when the activity was last observed.

type: date
| extended | +| $$$field-event-hash$$$[event.hash](#field-event-hash) | Hash (perhaps logstash fingerprint) of raw field to be able to demonstrate log integrity.

type: keyword

example: `123456789012345678901234567890ABCD`
| extended | +| $$$field-event-id$$$[event.id](#field-event-id) | Unique ID to describe the event.

type: keyword

example: `8a4f500d`
| core | +| $$$field-event-ingested$$$[event.ingested](#field-event-ingested) | Timestamp when an event arrived in the central data store.

This is different from `@timestamp`, which is when the event originally occurred. It’s also different from `event.created`, which is meant to capture the first time an agent saw the event.

In normal conditions, assuming no tampering, the timestamps should chronologically look like this: `@timestamp` < `event.created` < `event.ingested`.

type: date

example: `2016-05-23T08:05:35.101Z`
| core | +| $$$field-event-kind$$$[event.kind](#field-event-kind) | This is one of four ECS Categorization Fields, and indicates the highest level in the ECS category hierarchy.

`event.kind` gives high-level information about what type of information the event contains, without being specific to the contents of the event. For example, values of this field distinguish alert events from metric events.

The value of this field can be used to inform how these kinds of events should be handled. They may warrant different retention, different access control, it may also help understand whether the data is coming in at a regular interval or not.

type: keyword

**Important**: The field value must be one of the following:

alert, asset, enrichment, event, metric, state, pipeline_error, signal

To learn more about when to use which value, visit the page [allowed values for event.kind](/reference/ecs-allowed-values-event-kind.md)
| core | +| $$$field-event-module$$$[event.module](#field-event-module) | Name of the module this data is coming from.

If your monitoring agent supports the concept of modules or plugins to process events of a given source (e.g. Apache logs), `event.module` should contain the name of this module.

type: keyword

example: `apache`
| core | +| $$$field-event-original$$$[event.original](#field-event-original) | Raw text message of entire event. Used to demonstrate log integrity or where the full log message (before splitting it up in multiple parts) may be required, e.g. for reindex.

This field is not indexed and doc_values are disabled. It cannot be searched, but it can be retrieved from `_source`. If users wish to override this and index this field, please see `Field data types` in the `Elasticsearch Reference`.

type: keyword

example: `Sep 19 08:26:10 host CEF:0|Security| threatmanager|1.0|100| worm successfully stopped|10|src=10.0.0.1 dst=2.1.2.2spt=1232`
| core | +| $$$field-event-outcome$$$[event.outcome](#field-event-outcome) | This is one of four ECS Categorization Fields, and indicates the lowest level in the ECS category hierarchy.

`event.outcome` simply denotes whether the event represents a success or a failure from the perspective of the entity that produced the event.

Note that when a single transaction is described in multiple events, each event may populate different values of `event.outcome`, according to their perspective.

Also note that in the case of a compound event (a single event that contains multiple logical events), this field should be populated with the value that best captures the overall success or failure from the perspective of the event producer.

Further note that not all events will have an associated outcome. For example, this field is generally not populated for metric events, events with `event.type:info`, or any events for which an outcome does not make logical sense.

type: keyword

**Important**: The field value must be one of the following:

failure, success, unknown

To learn more about when to use which value, visit the page [allowed values for event.outcome](/reference/ecs-allowed-values-event-outcome.md)
| core | +| $$$field-event-provider$$$[event.provider](#field-event-provider) | Source of the event.

Event transports such as Syslog or the Windows Event Log typically mention the source of an event. It can be the name of the software that generated the event (e.g. Sysmon, httpd), or of a subsystem of the operating system (kernel, Microsoft-Windows-Security-Auditing).

type: keyword

example: `kernel`
| extended | +| $$$field-event-reason$$$[event.reason](#field-event-reason) | Reason why this event happened, according to the source.

This describes the why of a particular action or outcome captured in the event. Where `event.action` captures the action from the event, `event.reason` describes why that action was taken. For example, a web proxy with an `event.action` which denied the request may also populate `event.reason` with the reason why (e.g. `blocked site`).

type: keyword

example: `Terminated an unexpected process`
| extended | +| $$$field-event-reference$$$[event.reference](#field-event-reference) | Reference URL linking to additional information about this event.

This URL links to a static definition of this event. Alert events, indicated by `event.kind:alert`, are a common use case for this field.

type: keyword

example: `https://system.example.com/event/#0001234`
| extended | +| $$$field-event-risk-score$$$[event.risk_score](#field-event-risk-score) | Risk score or priority of the event (e.g. security solutions). Use your system’s original value here.

type: float
| core | +| $$$field-event-risk-score-norm$$$[event.risk_score_norm](#field-event-risk-score-norm) | Normalized risk score or priority of the event, on a scale of 0 to 100.

This is mainly useful if you use more than one system that assigns risk scores, and you want to see a normalized value across all systems.

type: float
| extended | +| $$$field-event-sequence$$$[event.sequence](#field-event-sequence) | Sequence number of the event.

The sequence number is a value published by some event sources, to make the exact ordering of events unambiguous, regardless of the timestamp precision.

type: long
| extended | +| $$$field-event-severity$$$[event.severity](#field-event-severity) | The numeric severity of the event according to your event source.

What the different severity values mean can be different between sources and use cases. It’s up to the implementer to make sure severities are consistent across events from the same source.

The Syslog severity belongs in `log.syslog.severity.code`. `event.severity` is meant to represent the severity according to the event source (e.g. firewall, IDS). If the event source does not publish its own severity, you may optionally copy the `log.syslog.severity.code` to `event.severity`.

type: long

example: `7`
| core | +| $$$field-event-start$$$[event.start](#field-event-start) | `event.start` contains the date when the event started or when the activity was first observed.

type: date
| extended | +| $$$field-event-timezone$$$[event.timezone](#field-event-timezone) | This field should be populated when the event’s timestamp does not include timezone information already (e.g. default Syslog timestamps). It’s optional otherwise.

Acceptable timezone formats are: a canonical ID (e.g. "Europe/Amsterdam"), abbreviated (e.g. "EST") or an HH:mm differential (e.g. "-05:00").

type: keyword
| extended | +| $$$field-event-type$$$[event.type](#field-event-type) | This is one of four ECS Categorization Fields, and indicates the third level in the ECS category hierarchy.

`event.type` represents a categorization "sub-bucket" that, when used along with the `event.category` field values, enables filtering events down to a level appropriate for single visualization.

This field is an array. This will allow proper categorization of some events that fall in multiple event types.

type: keyword

Note: this field should contain an array of values.

**Important**: The field value must be one of the following:

access, admin, allowed, change, connection, creation, deletion, denied, end, error, group, indicator, info, installation, protocol, start, user

To learn more about when to use which value, visit the page [allowed values for event.type](/reference/ecs-allowed-values-event-type.md)
| core | +| $$$field-event-url$$$[event.url](#field-event-url) | URL linking to an external system to continue investigation of this event.

This URL links to another system where in-depth investigation of the specific occurrence of this event can take place. Alert events, indicated by `event.kind:alert`, are a common use case for this field.

type: keyword

example: `https://mysystem.example.com/alert/5271dedb-f5b0-4218-87f0-4ac4870a38fe`
| extended | + diff --git a/docs/reference/ecs-faas.md b/docs/reference/ecs-faas.md new file mode 100644 index 0000000000..41b78c0640 --- /dev/null +++ b/docs/reference/ecs-faas.md @@ -0,0 +1,30 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-faas.html +applies_to: + stack: all + serverless: all +--- + +# FaaS fields [ecs-faas] + +The user fields describe information about the function as a service (FaaS) that is relevant to the event. + +::::{warning} +These fields are in beta and are subject to change. +:::: + + + +## FaaS field details [_faas_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-faas-coldstart$$$[faas.coldstart](#field-faas-coldstart) | Boolean value indicating a cold start of a function.

type: boolean

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [faas.coldstart](https://opentelemetry.io/docs/specs/semconv/attributes-registry/faas/#faas-coldstart)
| extended | +| $$$field-faas-execution$$$[faas.execution](#field-faas-execution) | The execution ID of the current function execution.

type: keyword

example: `af9d5aa4-a685-4c5f-a22b-444f80b3cc28`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") [faas.invocation_id](https://opentelemetry.io/docs/specs/semconv/attributes-registry/faas/#faas-invocation-id)
| extended | +| $$$field-faas-id$$$[faas.id](#field-faas-id) | The unique identifier of a serverless function.

For AWS Lambda it’s the function ARN (Amazon Resource Name) without a version or alias suffix.

type: keyword

example: `arn:aws:lambda:us-west-2:123456789012:function:my-function`
| extended | +| $$$field-faas-name$$$[faas.name](#field-faas-name) | The name of a serverless function.

type: keyword

example: `my-function`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [faas.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/faas/#faas-name)
| extended | +| $$$field-faas-trigger-request-id$$$[faas.trigger.request_id](#field-faas-trigger-request-id) | The ID of the trigger request , message, event, etc.

type: keyword

example: `123456789`
| extended | +| $$$field-faas-trigger-type$$$[faas.trigger.type](#field-faas-trigger-type) | The trigger for the function execution.

Expected values for this field:

- `http`
- `pubsub`
- `datasource`
- `timer`
- `other`

type: keyword

example: `http`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") [faas.trigger](https://opentelemetry.io/docs/specs/semconv/attributes-registry/faas/#faas-trigger)
| extended | +| $$$field-faas-version$$$[faas.version](#field-faas-version) | The version of a serverless function.

type: keyword

example: `123`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [faas.version](https://opentelemetry.io/docs/specs/semconv/attributes-registry/faas/#faas-version)
| extended | + diff --git a/docs/reference/ecs-faq.md b/docs/reference/ecs-faq.md new file mode 100644 index 0000000000..9f05cff6b5 --- /dev/null +++ b/docs/reference/ecs-faq.md @@ -0,0 +1,87 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-faq.html +applies_to: + stack: all + serverless: all +--- + +# Questions and answers [ecs-faq] + + +## What are the benefits of using ECS? [ecs-benefits] + +The benefits to a user adopting these fields and names in their clusters are: + +* **Data correlation.** Ability to easily correlate data from the same or different sources, including: + + * data from metrics, logs, and application performance management (apm) tools + * data from the same machines/hosts + * data from the same service + +* **Ease of recall.** Improved ability to remember commonly used field names (because there is a single set, not a set per data source) +* **Ease of deduction.** Improved ability to deduce field names (because the field naming follows a small number of rules with few exceptions) +* **Reuse.** Ability to re-use analysis content (searches, visualizations, dashboards, alerts, reports, and machine learning jobs) across multiple data sources +* **Future proofing.** Ability to use any future Elastic-provided analysis content in your environment without modifications + + +## What if I have fields that conflict with ECS? [conflict] + +The [rename processor](elasticsearch://reference/ingestion-tools/enrich-processor/rename-processor.md) can help you resolve field conflicts. For example, imagine that you already have a field called "user," but ECS employs `user` as an object. You can use the rename processor on ingest time to rename your field to the matching ECS field. If your field does not match ECS, you can rename your field to `user.value` instead. + + +## What if my events have additional fields? [addl-fields] + +Events may contain fields in addition to ECS fields. These fields can follow the ECS naming and writing rules, but this is not a requirement. + + +## Why does ECS use a dot notation instead of an underline notation? [dot-notation] + +There are two common key formats for ingesting data into Elasticsearch: + +* Dot notation: `user.firstname: Nicolas`, `user.lastname: Ruflin` +* Underline notation: `user_firstname: Nicolas`, `user_lastname: Ruflin` + +ECS uses the dot notation to represent nested objects. + + +### What is the difference between the two notations? [notation-diff] + +Ingesting `user.firstname: Nicolas` and `user.lastname: Ruflin` is identical to ingesting the following JSON: + +```json + "user": { + "firstname": "Nicolas", + "lastname": "Ruflin" + } +``` + +In Elasticsearch, `user` is represented as an [object datatype](elasticsearch://reference/elasticsearch/mapping-reference/object.md). In the case of the underline notation, both are just [string datatypes](elasticsearch://reference/elasticsearch/mapping-reference/field-data-types.md). + + +### Advantages of dot notation [dot-adv] + +With dot notation, each prefix in Elasticsearch is an object. Each object can have [parameters](elasticsearch://reference/elasticsearch/mapping-reference/object.md#object-params) that control how fields inside the object are treated. In the context of ECS, for example, these parameters would allow you to disable dynamic property creation for certain prefixes. + +Individual objects give you more flexibility on both the ingest and the event sides. In Elasticsearch, for example, you can use the remove processor to drop complete objects instead of selecting each key inside. You don’t have to know ahead of time which keys will be in an object. + +In Beats, you can simplify the creation of events. For example, you can treat each object as an object (or struct in Golang), which makes constructing and modifying each part of the final event easier. + + +### Disadvantage of dot notation [dot-disadv] + +In Elasticsearch, each key can have only one type. For example, if `user` is an `object`, you can’t use it as a `keyword` type in the same index, like `{"user": "nicolas ruflin"}`. This restriction can be an issue in certain datasets. For the ECS data itself, this is not an issue because all fields are predefined. + + +### What if I already use the underline notation? [underline] + +As long as there are no conflicts, underline notation and ECS dot notation can coexist in the same document. + + +## What if I want to use a different data type from the same field type family? [type-interop] + +In Elasticsearch, field types are grouped by family. Types in the same family support the same search functionality but may have different space usage or performance characteristics. For example, both `keyword` and `wildcard` types are members of the `keyword` family, and `text` and `match_only_text` are members of the `text` family. + +The field types defined in ECS provide the best default experience for most users. However, a different type from the same family can replace the default defined in ECS if required for a specific use cases. Users should understand any potential performance or storage differences before changing from a default field type. + +The Elasticsearch [mapping types](elasticsearch://reference/elasticsearch/mapping-reference/field-data-types.md) section has more information about type families. diff --git a/docs/reference/ecs-field-reference.md b/docs/reference/ecs-field-reference.md new file mode 100644 index 0000000000..28cf698029 --- /dev/null +++ b/docs/reference/ecs-field-reference.md @@ -0,0 +1,128 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html +applies_to: + stack: all + serverless: all +--- + +# ECS field reference [ecs-field-reference] + +This is the documentation of ECS version 9.1.0-dev. + +ECS defines multiple groups of related fields. They are called "field sets". The [Base](/reference/ecs-base.md) field set is the only one whose fields are defined at the root of the event. + +All other field sets are defined as objects in {{es}}, under which all fields are defined. + +For a single page representation of all fields, please see the [generated CSV of fields](https://github.com/elastic/ecs/blob/master/generated/csv/fields.csv). + + +## Field sets [ecs-fieldsets] + +| Field Set | Description | +| --- | --- | +| [Base](/reference/ecs-base.md) | All fields defined directly at the root of the events. | +| [Agent](/reference/ecs-agent.md) | Fields about the monitoring agent. | +| [Autonomous system](/reference/ecs-as.md) | Fields describing an Autonomous System (Internet routing prefix). | +| [Client](/reference/ecs-client.md) | Fields about the client side of a network connection, used with server. | +| [Cloud](/reference/ecs-cloud.md) | Fields about the cloud resource. | +| [Code signature](/reference/ecs-code_signature.md) | These fields contain information about binary code signatures. | +| [Container](/reference/ecs-container.md) | Fields describing the container that generated this event. | +| [Data stream](/reference/ecs-data_stream.md) | The data_stream fields take part in defining the new data stream naming scheme. | +| [Destination](/reference/ecs-destination.md) | Fields about the destination side of a network connection, used with source. | +| [Device](/reference/ecs-device.md) | Fields characterizing a (mobile) device a process or application is running on. | +| [DLL](/reference/ecs-dll.md) | These fields contain information about code libraries dynamically loaded into processes. | +| [DNS](/reference/ecs-dns.md) | Fields describing DNS queries and answers. | +| [ECS](/reference/ecs-ecs.md) | Meta-information specific to ECS. | +| [ELF header](/reference/ecs-elf.md) | These fields contain Linux Executable Linkable Format (ELF) metadata. | +| [Email](/reference/ecs-email.md) | Describes an email transaction. | +| [Error](/reference/ecs-error.md) | Fields about errors of any kind. | +| [Event](/reference/ecs-event.md) | Fields breaking down the event details. | +| [FaaS](/reference/ecs-faas.md) | Fields describing functions as a service. | +| [File](/reference/ecs-file.md) | Fields describing files. | +| [Geo](/reference/ecs-geo.md) | Fields describing a location. | +| [Group](/reference/ecs-group.md) | User’s group relevant to the event. | +| [Hash](/reference/ecs-hash.md) | Hashes, usually file hashes. | +| [Host](/reference/ecs-host.md) | Fields describing the relevant computing instance. | +| [HTTP](/reference/ecs-http.md) | Fields describing an HTTP request. | +| [Interface](/reference/ecs-interface.md) | Fields to describe observer interface information. | +| [Log](/reference/ecs-log.md) | Details about the event’s logging mechanism. | +| [Mach-O header](/reference/ecs-macho.md) | These fields contain Mac OS Mach Object file format (Mach-O) metadata. | +| [Network](/reference/ecs-network.md) | Fields describing the communication path over which the event happened. | +| [Observer](/reference/ecs-observer.md) | Fields describing an entity observing the event from outside the host. | +| [Orchestrator](/reference/ecs-orchestrator.md) | Fields relevant to container orchestrators. | +| [Organization](/reference/ecs-organization.md) | Fields describing the organization or company the event is associated with. | +| [Operating system](/reference/ecs-os.md) | OS fields contain information about the operating system. | +| [Package](/reference/ecs-package.md) | These fields contain information about an installed software package. | +| [PE header](/reference/ecs-pe.md) | These fields contain Windows Portable Executable (PE) metadata. | +| [Process](/reference/ecs-process.md) | These fields contain information about a process. | +| [Registry](/reference/ecs-registry.md) | Fields related to Windows Registry operations. | +| [Related](/reference/ecs-related.md) | Fields meant to facilitate pivoting around a piece of data. | +| [Risk information](/reference/ecs-risk.md) | Fields for describing risk score and level. | +| [Rule](/reference/ecs-rule.md) | Fields to capture details about rules used to generate alerts or other notable events. | +| [Server](/reference/ecs-server.md) | Fields about the server side of a network connection, used with client. | +| [Service](/reference/ecs-service.md) | Fields describing the service for or from which the data was collected. | +| [Source](/reference/ecs-source.md) | Fields about the source side of a network connection, used with destination. | +| [Threat](/reference/ecs-threat.md) | Fields to classify events and alerts according to a threat taxonomy. | +| [TLS](/reference/ecs-tls.md) | Fields describing a TLS connection. | +| [Tracing](/reference/ecs-tracing.md) | Fields related to distributed tracing. | +| [URL](/reference/ecs-url.md) | Fields that let you store URLs in various forms. | +| [User](/reference/ecs-user.md) | Fields to describe the user relevant to the event. | +| [User agent](/reference/ecs-user_agent.md) | Fields to describe a browser user_agent string. | +| [VLAN](/reference/ecs-vlan.md) | Fields to describe observed VLAN information. | +| [Volume](/reference/ecs-volume.md) | Fields related to storage volume details. | +| [Vulnerability](/reference/ecs-vulnerability.md) | Fields to describe the vulnerability relevant to an event. | +| [x509 certificate](/reference/ecs-x509.md) | These fields contain x509 certificate metadata. | + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/reference/ecs-file.md b/docs/reference/ecs-file.md new file mode 100644 index 0000000000..741854dcaf --- /dev/null +++ b/docs/reference/ecs-file.md @@ -0,0 +1,79 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-file.html +applies_to: + stack: all + serverless: all +--- + +# File fields [ecs-file] + +A file is defined as a set of information that has been created on, or has existed on a filesystem. + +File objects can be associated with host events, network events, and/or file events (e.g., those produced by File Integrity Monitoring [FIM] products or services). File fields provide details about the affected file associated with the event or metric. + + +## File field details [_file_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-file-accessed$$$[file.accessed](#field-file-accessed) | Last time the file was accessed.

Note that not all filesystems keep track of access time.

type: date

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [file.accessed](https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-accessed)
| extended | +| $$$field-file-attributes$$$[file.attributes](#field-file-attributes) | Array of file attributes.

Attributes names will vary by platform. Here’s a non-exhaustive list of values that are expected in this field: archive, compressed, directory, encrypted, execute, hidden, read, readonly, system, write.

type: keyword

Note: this field should contain an array of values.

example: `["readonly", "system"]`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [file.attributes](https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-attributes)
| extended | +| $$$field-file-created$$$[file.created](#field-file-created) | File creation time.

Note that not all filesystems store the creation time.

type: date

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [file.created](https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-created)
| extended | +| $$$field-file-ctime$$$[file.ctime](#field-file-ctime) | Last time the file attributes or metadata changed.

Note that changes to the file content will update `mtime`. This implies `ctime` will be adjusted at the same time, since `mtime` is an attribute of the file.

type: date

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") [file.changed](https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-changed)
| extended | +| $$$field-file-device$$$[file.device](#field-file-device) | Device that is the source of the file.

type: keyword

example: `sda`
| extended | +| $$$field-file-directory$$$[file.directory](#field-file-directory) | Directory where the file is located. It should include the drive letter, when appropriate.

type: keyword

example: `/home/alice`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [file.directory](https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-directory)
| extended | +| $$$field-file-drive-letter$$$[file.drive_letter](#field-file-drive-letter) | Drive letter where the file is located. This field is only relevant on Windows.

The value should be uppercase, and not include the colon.

type: keyword

example: `C`
| extended | +| $$$field-file-extension$$$[file.extension](#field-file-extension) | File extension, excluding the leading dot.

Note that when the file name has multiple extensions (example.tar.gz), only the last one should be captured ("gz", not "tar.gz").

type: keyword

example: `png`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [file.extension](https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-extension)
| extended | +| $$$field-file-fork-name$$$[file.fork_name](#field-file-fork-name) | A fork is additional data associated with a filesystem object.

On Linux, a resource fork is used to store additional data with a filesystem object. A file always has at least one fork for the data portion, and additional forks may exist.

On NTFS, this is analogous to an Alternate Data Stream (ADS), and the default data stream for a file is just called $DATA. Zone.Identifier is commonly used by Windows to track contents downloaded from the Internet. An ADS is typically of the form: `C:\path\to\filename.extension:some_fork_name`, and `some_fork_name` is the value that should populate `fork_name`. `filename.extension` should populate `file.name`, and `extension` should populate `file.extension`. The full path, `file.path`, will include the fork name.

type: keyword

example: `Zone.Identifer`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [file.fork_name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-fork-name)
| extended | +| $$$field-file-gid$$$[file.gid](#field-file-gid) | Primary group ID (GID) of the file.

type: keyword

example: `1001`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") [file.group.id](https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-group-id)
| extended | +| $$$field-file-group$$$[file.group](#field-file-group) | Primary group name of the file.

type: keyword

example: `alice`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") [file.group.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-group-name)
| extended | +| $$$field-file-inode$$$[file.inode](#field-file-inode) | Inode representing the file in the filesystem.

type: keyword

example: `256383`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [file.inode](https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-inode)
| extended | +| $$$field-file-mime-type$$$[file.mime_type](#field-file-mime-type) | MIME type should identify the format of the file or stream of bytes using [IANA official types](https://www.iana.org/assignments/media-types/media-types.xhtml), where possible. When more than one type is applicable, the most specific type should be used.

type: keyword
| extended | +| $$$field-file-mode$$$[file.mode](#field-file-mode) | Mode of the file in octal representation.

type: keyword

example: `0640`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [file.mode](https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-mode)
| extended | +| $$$field-file-mtime$$$[file.mtime](#field-file-mtime) | Last time the file content was modified.

type: date

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") [file.modified](https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-modified)
| extended | +| $$$field-file-name$$$[file.name](#field-file-name) | Name of the file including the extension, without the directory.

type: keyword

example: `example.png`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [file.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-name)
| extended | +| $$$field-file-owner$$$[file.owner](#field-file-owner) | File owner’s username.

type: keyword

example: `alice`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") [file.owner.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-owner-name)
| extended | +| $$$field-file-path$$$[file.path](#field-file-path) | Full path to the file, including the file name. It should include the drive letter, when appropriate.

type: keyword

Multi-fields:

- file.path.text (type: match_only_text)

example: `/home/alice/example.png`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [file.path](https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-path)
| extended | +| $$$field-file-size$$$[file.size](#field-file-size) | File size in bytes.

Only relevant when `file.type` is "file".

type: long

example: `16384`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [file.size](https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-size)
| extended | +| $$$field-file-target-path$$$[file.target_path](#field-file-target-path) | Target path for symlinks.

type: keyword

Multi-fields:

- file.target_path.text (type: match_only_text)

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") [file.symbolic_link.target_path](https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-symbolic-link-target-path)
| extended | +| $$$field-file-type$$$[file.type](#field-file-type) | File type (file, dir, or symlink).

type: keyword

example: `file`
| extended | +| $$$field-file-uid$$$[file.uid](#field-file-uid) | The user ID (UID) or security identifier (SID) of the file owner.

type: keyword

example: `1001`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") [file.owner.id](https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-owner-id)
| extended | + + +## Field reuse [_field_reuse_9] + +The `file` fields are expected to be nested at: + +* `threat.enrichments.indicator.file` +* `threat.indicator.file` + +Note also that the `file` fields may be used directly at the root of the events. + + +### Field sets that can be nested under file [ecs-file-nestings] + +::::{tab-set} + +:::{tab-item} Field set + +| Location | Field Set | Description | +| --- | --- | --- | +| `file.code_signature.*` | [code_signature](/reference/ecs-code_signature.md) | These fields contain information about binary code signatures. | +| `file.hash.*` | [hash](/reference/ecs-hash.md) | Hashes, usually file hashes. | +| `file.pe.*` | [pe](/reference/ecs-pe.md) | These fields contain Windows Portable Executable (PE) metadata. | +| `file.x509.*` | [x509](/reference/ecs-x509.md) | These fields contain x509 certificate metadata. | + + +::: + +:::{tab-item} Beta Field set + +| Location | Field Set | Description | +| --- | --- | --- | +| `file.elf.*` | [elf](/reference/ecs-elf.md) | These fields contain Linux Executable Linkable Format (ELF) metadata. | +| `file.macho.*` | [macho](/reference/ecs-macho.md) | These fields contain Mac OS Mach Object file format (Mach-O) metadata. | + +::: +:::: + diff --git a/docs/reference/ecs-geo.md b/docs/reference/ecs-geo.md new file mode 100644 index 0000000000..1536d5a057 --- /dev/null +++ b/docs/reference/ecs-geo.md @@ -0,0 +1,47 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-geo.html +applies_to: + stack: all + serverless: all +--- + +# Geo fields [ecs-geo] + +Geo fields can carry data about a specific location related to an event. + +This geolocation information can be derived from techniques such as Geo IP, or be user-supplied. + + +## Geo field details [_geo_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-geo-city-name$$$[geo.city_name](#field-geo-city-name) | City name.

type: keyword

example: `Montreal`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") [geo.locality.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/geo/#geo-locality-name)
| core | +| $$$field-geo-continent-code$$$[geo.continent_code](#field-geo-continent-code) | Two-letter code representing continent’s name.

type: keyword

example: `NA`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") [geo.continent.code](https://opentelemetry.io/docs/specs/semconv/attributes-registry/geo/#geo-continent-code)
| core | +| $$$field-geo-continent-name$$$[geo.continent_name](#field-geo-continent-name) | Name of the continent.

type: keyword

example: `North America`
| core | +| $$$field-geo-country-iso-code$$$[geo.country_iso_code](#field-geo-country-iso-code) | Country ISO code.

type: keyword

example: `CA`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") [geo.country.iso_code](https://opentelemetry.io/docs/specs/semconv/attributes-registry/geo/#geo-country-iso-code)
| core | +| $$$field-geo-country-name$$$[geo.country_name](#field-geo-country-name) | Country name.

type: keyword

example: `Canada`
| core | +| $$$field-geo-location$$$[geo.location](#field-geo-location) | Longitude and latitude.

type: geo_point

example: `{ "lon": -73.614830, "lat": 45.505918 }`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/related-efc20d?style=flat "related") [geo.location.lat](https://opentelemetry.io/docs/specs/semconv/attributes-registry/geo/#geo-location-lat)

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/related-efc20d?style=flat "related") [geo.location.lon](https://opentelemetry.io/docs/specs/semconv/attributes-registry/geo/#geo-location-lon)
| core | +| $$$field-geo-name$$$[geo.name](#field-geo-name) | User-defined description of a location, at the level of granularity they care about.

Could be the name of their data centers, the floor number, if this describes a local physical entity, city names.

Not typically used in automated geolocation.

type: keyword

example: `boston-dc`
| extended | +| $$$field-geo-postal-code$$$[geo.postal_code](#field-geo-postal-code) | Postal code associated with the location.

Values appropriate for this field may also be known as a postcode or ZIP code and will vary widely from country to country.

type: keyword

example: `94040`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [geo.postal_code](https://opentelemetry.io/docs/specs/semconv/attributes-registry/geo/#geo-postal-code)
| core | +| $$$field-geo-region-iso-code$$$[geo.region_iso_code](#field-geo-region-iso-code) | Region ISO code.

type: keyword

example: `CA-QC`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") [geo.region.iso_code](https://opentelemetry.io/docs/specs/semconv/attributes-registry/geo/#geo-region-iso-code)
| core | +| $$$field-geo-region-name$$$[geo.region_name](#field-geo-region-name) | Region name.

type: keyword

example: `Quebec`
| core | +| $$$field-geo-timezone$$$[geo.timezone](#field-geo-timezone) | The time zone of the location, such as IANA time zone name.

type: keyword

example: `America/Argentina/Buenos_Aires`
| core | + + +## Field reuse [_field_reuse_10] + +The `geo` fields are expected to be nested at: + +* `client.geo` +* `destination.geo` +* `host.geo` +* `observer.geo` +* `server.geo` +* `source.geo` +* `threat.enrichments.indicator.geo` +* `threat.indicator.geo` + +Note also that the `geo` fields are not expected to be used directly at the root of the events. + diff --git a/docs/reference/ecs-getting-started.md b/docs/reference/ecs-getting-started.md new file mode 100644 index 0000000000..db55e1ae83 --- /dev/null +++ b/docs/reference/ecs-getting-started.md @@ -0,0 +1,119 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-getting-started.html +applies_to: + stack: all + serverless: all +--- + +# Getting started [ecs-getting-started] + +ECS enables and encourages users to normalize event data in order to better analyze, visualize, and correlate their events. Collected events can be normalized at ingest time, consistently searched across indices, and visualized predictably. + +Note that when adopting an Elastic solution, such as [Observability](https://www.elastic.co/observability) or [Security](https://www.elastic.co/security), all events will map to ECS out of the box. Elastic provides an extensive set of [integrations](https://www.elastic.co/integrations) to simplify ingesting your data sources. + +If you rely on custom data pipelines and/or building content around specific needs, ECS can still help to alleviate the challenges of searching, analyzing, and visualizing across your data. Let’s see how using a common schema can simplify the search experience, and then take a look at how an event’s contents can be mapped to ECS field sets. + + +## Simplified search [_simplified_search] + +With ECS defining a normalized schema across all of your data sources, querying against those sources is simplified. Consider searching for a particular source IP address prior to adopting ECS. All the various data sources and their field mappings would need to be considered in your query: + +```sh +src:10.42.42.42 OR client_ip:10.42.42.42 OR apache.access.remote_ip:10.42.42.42 OR +context.user.ip:10.42.42.42 OR src_ip:10.42.42.42 +``` + +With all sources mapped to ECS, the query becomes much simpler: + +```sh +source.ip:10.42.42.42 +``` + +Not only does this simplify writing queries, but saved queries shared with other users become much more obvious. To gain familiarity with ECS fields, you can also take a look at the [*ECS Field Reference*](/reference/ecs-field-reference.md) section. + + +## Unified visualizations [_unified_visualizations] + +With normalized data from different data sources, building insightful visualizations across sources is simple. From a single, centralized dashboard, events from web servers, IDS/IPS devices, and firewalls can be aggregated and visualized, and enhanced with drill-downs, and pivoting for delving into deeper investigations. Centralized monitoring of diverse data sources is straightforward with normalized ECS data. + +![Simplify visualization using ECS](../images/ecs-getting-started-dashboard.png "") + + +## Translating data sources [_translating_data_sources] + +To align events to ECS, some sort of parsing will usually be necessary to transform the contents of the original event into the relevant ECS fields. Depending on how you’ve designed your Elastic Stack data ingestion pipelines, the amount of work to parse your events will vary. + +For example, an Apache web server log event: + +```sh +10.42.42.42 - - [15/Jul/2020:20:48:32 +0000] "GET /content HTTP/1.1" 200 2571 "-" +"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko) +Chrome/83.0.4103.106 Safari/537.36" +``` + +In order to map this event to ECS, the contents of the event is associated with the appropriate ECS fields. + +| Field Name | Value | +| --- | --- | +| @timestamp | `2020-07-15T20:48:32.000Z` | +| event.original | 10.42.42.42 - - [15/Jul/2020:20:48:32 +0000] "GET /content HTTP/1.1" 200 2571 "-""Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko)Chrome/83.0.4103.106 Safari/537.36 | +| http.request.method | GET | +| http.response.body.bytes | 2571 | +| http.response.status_code | 200 | +| http.version | 1.1 | +| message | GET /content HTTP/1.1" 200 2571 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36 | +| source.address | 10.42.42.42 | +| source.ip | 10.42.42.42 | +| url.original | `/content` | +| user_agent.original | `Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36` | + +Beyond extracting values that are present in the original event, we also populate other fields to provide additional context about the event itself. + +* `ecs.version`: States which version of ECS the ingest pipeline was developed against. +* `event.dataset` and `event.module`: Answers "where is this event from" and are expected to have a hardcoded value per pipeline, per source. +* `event.kind`, `event.category`, `event.type`, and `event.outcome`: The [*ECS Categorization Fields*](/reference/ecs-category-field-values-reference.md) should also be hardcoded using knowledge of each type of event the source emits. The contents of these fields are limited to the specifically allowed values detailed in the ECS documentation. + +| Field Name | Value | +| --- | --- | +| ecs.version | 1.5.0 | +| event.module | apache | +| event.dataset | apache.access | +| event.kind | event | +| event.category | [ "network", "web" ] | +| event.type | [ "access" ] | +| event.outcome | success | + +Lastly, existing field values can be interpreted or enriched using a processor, with the results populating additional fields in the final event. + +* The `user_agent` processor extracts details from the original user agent string, `user_agent.original`. +* IP fields like `source.ip` can provide enrichment using the `geoip` processor to add information about the location and autonomous system number (ASN) associated with an IP address. +* The `registered domain` processor reads a field containing a hostname and writes the registered domain to another field +* Event collectors, such as [Beats](beats://reference/index.md), can enrich each event with metadata from the machine’s hosting provider (cloud) and/or from the host machine (host). + +Here are some examples of additional fields processed by metadata or parser processors. + +| Field Name | Value | Processor | +| --- | --- | --- | +| host.architecture | x86_64 | `add_host_metadata` | +| host.hostname | mbp.example.com | `add_host_metadata` | +| host.ip | [ "192.168.1.100" ] | `add_host_metadata` | +| host.os.family | darwin | `add_host_metadata` | +| host.os.kernel | 19.4.0 | `add_host_metadata` | +| host.os.name | Mac OS X | `add_host_metadata` | +| host.os.version | 10.15.4 | `add_host_metadata` | +| user_agent.name | Chrome | `user_agent` | +| user_agent.os.full | Mac OS X 10.15.4 | `user_agent` | +| user_agent.os.name | Mac OS X | `user_agent` | +| user_agent.os.version | 10.15.4 | `user_agent` | +| user_agent.version | 83.0.4103.106 | `user_agent` | + + +## Field mapping reference guides [_field_mapping_reference_guides] + +We’ve covered at a high level how to map your events to ECS. Now if you’d like your events to render well in the Elastic solutions, check out the reference guides below to learn more about each: + +* [Log monitoring field reference](docs-content://reference/observability/fields-and-object-schemas/logs-app-fields.md) +* [Metrics monitoring field reference](docs-content://reference/observability/fields-and-object-schemas/metrics-app-fields.md) +* [Elastic Security field reference](docs-content://reference/security/fields-and-object-schemas/siem-field-reference.md) + diff --git a/docs/reference/ecs-group.md b/docs/reference/ecs-group.md new file mode 100644 index 0000000000..d3d3219e01 --- /dev/null +++ b/docs/reference/ecs-group.md @@ -0,0 +1,35 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-group.html +applies_to: + stack: all + serverless: all +--- + +# Group fields [ecs-group] + +The group fields are meant to represent groups that are relevant to the event. + + +## Group field details [_group_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-group-domain$$$[group.domain](#field-group-domain) | Name of the directory the group is a member of.

For example, an LDAP or Active Directory domain name.

type: keyword
| extended | +| $$$field-group-id$$$[group.id](#field-group-id) | Unique identifier for the group on the system/platform.

type: keyword
| extended | +| $$$field-group-name$$$[group.name](#field-group-name) | Name of the group.

type: keyword
| extended | + + +## Field reuse [_field_reuse_11] + +The `group` fields are expected to be nested at: + +* `process.attested_groups` +* `process.group` +* `process.real_group` +* `process.saved_group` +* `process.supplemental_groups` +* `user.group` + +Note also that the `group` fields may be used directly at the root of the events. + diff --git a/docs/reference/ecs-guidelines.md b/docs/reference/ecs-guidelines.md new file mode 100644 index 0000000000..d712cd622f --- /dev/null +++ b/docs/reference/ecs-guidelines.md @@ -0,0 +1,61 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-guidelines.html +--- + +# Guidelines and best practices [ecs-guidelines] + +The ECS schema serves best when you follow schema guidelines and best practices. + + +## ECS field levels [_ecs_field_levels] + +ECS defines "Core" and "Extended" fields. + +* **Core fields.** Fields that are most common across all use cases are defined as **core fields**. + + These generalized fields are used by analysis content (searches, visualizations, dashboards, alerts, machine learning jobs, reports) across use cases. Analysis content designed to operate on these fields should work properly on data from any relevant source. + + Focus on populating these fields first. + +* **Extended fields.** Any field that is not a core field is defined as an **extended field**. Extended fields may apply to more narrow use cases, or may be more open to interpretation depending on the use case. Extended fields are more likely to change over time. + +Each ECS [field](/reference/ecs-field-reference.md) in a table is identified as core or extended. + + +## General guidelines [_general_guidelines] + +* The document MUST have the `@timestamp` field. +* Use the [data types](elasticsearch://reference/elasticsearch/mapping-reference/field-data-types.md) defined for an ECS field. +* Use the `ecs.version` field to define which version of ECS is used. +* Map as many fields as possible to ECS. + + +### Guidelines for field names [_guidelines_for_field_names] + +* **Field names must be lower case** +* **Combine words using underscore** +* **No special characters except underscore** +* **Use present tense** unless field describes historical information. +* **Use singular and plural names properly** to reflect the field content. + + * For example, use `requests_per_sec` rather than `request_per_sec`. + +* **Use prefixes for all fields**, except for the base fields. + + * For example, all `host` fields are prefixed with `host.`. Such a grouping is called a field set. + +* **Nest fields inside a field set** with dots + + * The document structure should be nested JSON objects. If you use Beats or Logstash, the nesting of JSON objects is done for you automatically. If you’re ingesting to Elasticsearch using the API, your fields must be nested objects, not strings containing dots. + * See [Why does ECS use a dot notation instead of an underline notation?](/reference/ecs-faq.md#dot-notation) for more details. + +* **General to specific**. Organise the nesting of field sets from general to specific, to allow grouping fields into objects with a prefix like `host.*`. +* **Avoid repetition** or stuttering of words + + * If part of the field name is already in the name of the field set, avoid repeating it. Example: `host.host_ip` should be `host.ip`. + * Exceptions can be made, when changing the name of the field would break a strong convention in the community. Example: `host.hostname` is an exception to this rule. + +* **Avoid abbreviations when possible** + + * Exceptions can be made, when the name used for the concept is too strongly in favor of the abbreviation. Example: `ip` fields, or field sets such as `os`, `geo`. diff --git a/docs/reference/ecs-hash.md b/docs/reference/ecs-hash.md new file mode 100644 index 0000000000..833bc5441a --- /dev/null +++ b/docs/reference/ecs-hash.md @@ -0,0 +1,55 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-hash.html +applies_to: + stack: all + serverless: all +--- + +# Hash fields [ecs-hash] + +The hash fields represent different bitwise hash algorithms and their values. + +Field names for common hashes (e.g. MD5, SHA1) are predefined. Add fields for other hashes by lowercasing the hash algorithm name and using underscore separators as appropriate (snake case, e.g. sha3_512). + +Note that this fieldset is used for common hashes that may be computed over a range of generic bytes. Entity-specific hashes such as ja3 or imphash are placed in the fieldsets to which they relate (tls and pe, respectively). + + +## Hash field details [_hash_field_details] + +::::{tab-set} + +:::{tab-item} Fields + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-hash-md5$$$[hash.md5](#field-hash-md5) | MD5 hash.

type: keyword
| extended | +| $$$field-hash-sha1$$$[hash.sha1](#field-hash-sha1) | SHA1 hash.

type: keyword
| extended | +| $$$field-hash-sha256$$$[hash.sha256](#field-hash-sha256) | SHA256 hash.

type: keyword
| extended | +| $$$field-hash-sha384$$$[hash.sha384](#field-hash-sha384) | SHA384 hash.

type: keyword
| extended | +| $$$field-hash-sha512$$$[hash.sha512](#field-hash-sha512) | SHA512 hash.

type: keyword
| extended | +| $$$field-hash-ssdeep$$$[hash.ssdeep](#field-hash-ssdeep) | SSDEEP hash.

type: keyword
| extended | +| $$$field-hash-tlsh$$$[hash.tlsh](#field-hash-tlsh) | TLSH hash.

type: keyword
| extended | + +::: + +:::{tab-item} Beta Fields + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-hash-cdhash$$$[hash.cdhash](#field-hash-cdhash) | Code directory hash, utilized to uniquely identify and authenticate the integrity of the executable code.

type: keyword

example: `3783b4052fd474dbe30676b45c329e7a6d44acd9`
| extended | + +::: +:::: + +## Field reuse [_field_reuse_12] + +The `hash` fields are expected to be nested at: + +* `dll.hash` +* `email.attachments.file.hash` +* `file.hash` +* `process.hash` + +Note also that the `hash` fields are not expected to be used directly at the root of the events. + diff --git a/docs/reference/ecs-host.md b/docs/reference/ecs-host.md new file mode 100644 index 0000000000..10438acba5 --- /dev/null +++ b/docs/reference/ecs-host.md @@ -0,0 +1,64 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-host.html +applies_to: + stack: all + serverless: all +--- + +# Host fields [ecs-host] + +A host is defined as a general computing instance. + +ECS host.* fields should be populated with details about the host on which the event happened, or from which the measurement was taken. Host types include hardware, virtual machines, Docker containers, and Kubernetes nodes. + + +## Host field details [_host_field_details] + +::::{tab-set} + +:::{tab-item} Fields + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-host-architecture$$$[host.architecture](#field-host-architecture) | Operating system architecture.

type: keyword

example: `x86_64`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") [host.arch](https://opentelemetry.io/docs/specs/semconv/attributes-registry/host/#host-arch)
| core | +| $$$field-host-cpu-usage$$$[host.cpu.usage](#field-host-cpu-usage) | Percent CPU used which is normalized by the number of CPU cores and it ranges from 0 to 1.

Scaling factor: 1000.

For example: For a two core host, this value should be the average of the two cores, between 0 and 1.

type: scaled_float

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/metric-cb00cb?style=flat "metric") [system.cpu.utilization](https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--+semconv+metric.system.cpu.utilization+--%3E%22&type=code)
| extended | +| $$$field-host-disk-read-bytes$$$[host.disk.read.bytes](#field-host-disk-read-bytes) | The total number of bytes (gauge) read successfully (aggregated from all disks) since the last metric collection.

type: long

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/metric-cb00cb?style=flat "metric") [system.disk.io](https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--+semconv+metric.system.disk.io+--%3E%22&type=code)
| extended | +| $$$field-host-disk-write-bytes$$$[host.disk.write.bytes](#field-host-disk-write-bytes) | The total number of bytes (gauge) written successfully (aggregated from all disks) since the last metric collection.

type: long

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/metric-cb00cb?style=flat "metric") [system.disk.io](https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--+semconv+metric.system.disk.io+--%3E%22&type=code)
| extended | +| $$$field-host-domain$$$[host.domain](#field-host-domain) | Name of the domain of which the host is a member.

For example, on Windows this could be the host’s Active Directory domain or NetBIOS domain name. For Linux this could be the domain of the host’s LDAP provider.

type: keyword

example: `CONTOSO`
| extended | +| $$$field-host-hostname$$$[host.hostname](#field-host-hostname) | Hostname of the host.

It normally contains what the `hostname` command returns on the host machine.

type: keyword
| core | +| $$$field-host-id$$$[host.id](#field-host-id) | Unique host id.

As hostname is not always unique, use values that are meaningful in your environment.

Example: The current usage of `beat.name`.

type: keyword

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [host.id](https://opentelemetry.io/docs/specs/semconv/attributes-registry/host/#host-id)
| core | +| $$$field-host-ip$$$[host.ip](#field-host-ip) | Host ip addresses.

type: ip

Note: this field should contain an array of values.

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [host.ip](https://opentelemetry.io/docs/specs/semconv/attributes-registry/host/#host-ip)
| core | +| $$$field-host-mac$$$[host.mac](#field-host-mac) | Host MAC addresses.

The notation format from RFC 7042 is suggested: Each octet (that is, 8-bit byte) is represented by two [uppercase] hexadecimal digits giving the value of the octet as an unsigned integer. Successive octets are separated by a hyphen.

type: keyword

Note: this field should contain an array of values.

example: `["00-00-5E-00-53-23", "00-00-5E-00-53-24"]`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [host.mac](https://opentelemetry.io/docs/specs/semconv/attributes-registry/host/#host-mac)
| core | +| $$$field-host-name$$$[host.name](#field-host-name) | Name of the host.

It can contain what hostname returns on Unix systems, the fully qualified domain name (FQDN), or a name specified by the user. The recommended value is the lowercase FQDN of the host.

type: keyword

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [host.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/host/#host-name)
| core | +| $$$field-host-network-egress-bytes$$$[host.network.egress.bytes](#field-host-network-egress-bytes) | The number of bytes (gauge) sent out on all network interfaces by the host since the last metric collection.

type: long

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/metric-cb00cb?style=flat "metric") [system.network.io](https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--+semconv+metric.system.network.io+--%3E%22&type=code)
| extended | +| $$$field-host-network-egress-packets$$$[host.network.egress.packets](#field-host-network-egress-packets) | The number of packets (gauge) sent out on all network interfaces by the host since the last metric collection.

type: long

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/metric-cb00cb?style=flat "metric") [system.network.packets](https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--+semconv+metric.system.network.packets+--%3E%22&type=code)
| extended | +| $$$field-host-network-ingress-bytes$$$[host.network.ingress.bytes](#field-host-network-ingress-bytes) | The number of bytes received (gauge) on all network interfaces by the host since the last metric collection.

type: long

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/metric-cb00cb?style=flat "metric") [system.network.io](https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--+semconv+metric.system.network.io+--%3E%22&type=code)
| extended | +| $$$field-host-network-ingress-packets$$$[host.network.ingress.packets](#field-host-network-ingress-packets) | The number of packets (gauge) received on all network interfaces by the host since the last metric collection.

type: long

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/metric-cb00cb?style=flat "metric") [system.network.packets](https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--+semconv+metric.system.network.packets+--%3E%22&type=code)
| extended | +| $$$field-host-type$$$[host.type](#field-host-type) | Type of host.

For Cloud providers this can be the machine type like `t2.medium`. If vm, this could be the container, for example, or other information meaningful in your environment.

type: keyword

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [host.type](https://opentelemetry.io/docs/specs/semconv/attributes-registry/host/#host-type)
| core | +| $$$field-host-uptime$$$[host.uptime](#field-host-uptime) | Seconds the host has been up.

type: long

example: `1325`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/metric-cb00cb?style=flat "metric") [system.uptime](https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--+semconv+metric.system.uptime+--%3E%22&type=code)
| extended | + +::: + +:::{tab-item} Beta Fields + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-host-boot-id$$$[host.boot.id](#field-host-boot-id) | Linux boot uuid taken from /proc/sys/kernel/random/boot_id. Note the boot_id value from /proc may or may not be the same in containers as on the host. Some container runtimes will bind mount a new boot_id value onto the proc file in each container.

type: keyword

example: `88a1f0ed-5ae5-41ee-af6b-41921c311872`
| extended | +| $$$field-host-pid-ns-ino$$$[host.pid_ns_ino](#field-host-pid-ns-ino) | This is the inode number of the namespace in the namespace file system (nsfs). Unsigned int inum in include/linux/ns_common.h.

type: keyword

example: `256383`
| extended | + +::: +:::: + + +## Field reuse [_field_reuse_13] + + +### Field sets that can be nested under host [ecs-host-nestings] + +| Location | Field Set | Description | +| --- | --- | --- | +| `host.geo.*` | [geo](/reference/ecs-geo.md) | Fields describing a location. | +| `host.os.*` | [os](/reference/ecs-os.md) | OS fields contain information about the operating system. | +| `host.risk.*` | [risk](/reference/ecs-risk.md) | Fields for describing risk score and level. | + diff --git a/docs/reference/ecs-http.md b/docs/reference/ecs-http.md new file mode 100644 index 0000000000..00635507bb --- /dev/null +++ b/docs/reference/ecs-http.md @@ -0,0 +1,31 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-http.html +applies_to: + stack: all + serverless: all +--- + +# HTTP fields [ecs-http] + +Fields related to HTTP activity. Use the `url` field set to store the url of the request. + + +## HTTP field details [_http_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-http-request-body-bytes$$$[http.request.body.bytes](#field-http-request-body-bytes) | Size in bytes of the request body.

type: long

example: `887`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") [http.request.body.size](https://opentelemetry.io/docs/specs/semconv/attributes-registry/http/#http-request-body-size)
| extended | +| $$$field-http-request-body-content$$$[http.request.body.content](#field-http-request-body-content) | The full HTTP request body.

type: wildcard

Multi-fields:

- http.request.body.content.text (type: match_only_text)

example: `Hello world`
| extended | +| $$$field-http-request-bytes$$$[http.request.bytes](#field-http-request-bytes) | Total size in bytes of the request (body and headers).

type: long

example: `1437`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") [http.request.size](https://opentelemetry.io/docs/specs/semconv/attributes-registry/http/#http-request-size)
| extended | +| $$$field-http-request-id$$$[http.request.id](#field-http-request-id) | A unique identifier for each HTTP request to correlate logs between clients and servers in transactions.

The id may be contained in a non-standard HTTP header, such as `X-Request-ID` or `X-Correlation-ID`.

type: keyword

example: `123e4567-e89b-12d3-a456-426614174000`
| extended | +| $$$field-http-request-method$$$[http.request.method](#field-http-request-method) | HTTP request method.

The value should retain its casing from the original event. For example, `GET`, `get`, and `GeT` are all considered valid values for this field.

type: keyword

example: `POST`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") [http.request.method_original](https://opentelemetry.io/docs/specs/semconv/attributes-registry/http/#http-request-method-original)

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/conflict-910000?style=flat "conflict") [http.request.method](https://opentelemetry.io/docs/specs/semconv/attributes-registry/http/#http-request-method)

Note: `http.request.method` in SemConv is the known, normalized, upper case value of the request method, other than the ECS' `http.request.method` that retains casing from the original event.
| extended | +| $$$field-http-request-mime-type$$$[http.request.mime_type](#field-http-request-mime-type) | Mime type of the body of the request.

This value must only be populated based on the content of the request body, not on the `Content-Type` header. Comparing the mime type of a request with the request’s Content-Type header can be helpful in detecting threats or misconfigured clients.

type: keyword

example: `image/gif`
| extended | +| $$$field-http-request-referrer$$$[http.request.referrer](#field-http-request-referrer) | Referrer for this HTTP request.

type: keyword

example: `https://blog.example.com/`
| extended | +| $$$field-http-response-body-bytes$$$[http.response.body.bytes](#field-http-response-body-bytes) | Size in bytes of the response body.

type: long

example: `887`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") [http.response.body.size](https://opentelemetry.io/docs/specs/semconv/attributes-registry/http/#http-response-body-size)
| extended | +| $$$field-http-response-body-content$$$[http.response.body.content](#field-http-response-body-content) | The full HTTP response body.

type: wildcard

Multi-fields:

- http.response.body.content.text (type: match_only_text)

example: `Hello world`
| extended | +| $$$field-http-response-bytes$$$[http.response.bytes](#field-http-response-bytes) | Total size in bytes of the response (body and headers).

type: long

example: `1437`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") [http.response.size](https://opentelemetry.io/docs/specs/semconv/attributes-registry/http/#http-response-size)
| extended | +| $$$field-http-response-mime-type$$$[http.response.mime_type](#field-http-response-mime-type) | Mime type of the body of the response.

This value must only be populated based on the content of the response body, not on the `Content-Type` header. Comparing the mime type of a response with the response’s Content-Type header can be helpful in detecting misconfigured servers.

type: keyword

example: `image/gif`
| extended | +| $$$field-http-response-status-code$$$[http.response.status_code](#field-http-response-status-code) | HTTP response status code.

type: long

example: `404`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [http.response.status_code](https://opentelemetry.io/docs/specs/semconv/attributes-registry/http/#http-response-status-code)
| extended | +| $$$field-http-version$$$[http.version](#field-http-version) | HTTP version.

type: keyword

example: `1.1`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/related-efc20d?style=flat "related") [network.protocol.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/network/#network-protocol-name)

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/related-efc20d?style=flat "related") [network.protocol.version](https://opentelemetry.io/docs/specs/semconv/attributes-registry/network/#network-protocol-version)

Note: In OTel SemConv, `network.protocol.version` specifies the HTTP version if the value of `network.protocol.name` is `http`.
| extended | + diff --git a/docs/reference/ecs-interface.md b/docs/reference/ecs-interface.md new file mode 100644 index 0000000000..992cc3d5df --- /dev/null +++ b/docs/reference/ecs-interface.md @@ -0,0 +1,31 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-interface.html +applies_to: + stack: all + serverless: all +--- + +# Interface fields [ecs-interface] + +The interface fields are used to record ingress and egress interface information when reported by an observer (e.g. firewall, router, load balancer) in the context of the observer handling a network connection. In the case of a single observer interface (e.g. network sensor on a span port) only the observer.ingress information should be populated. + + +## Interface field details [_interface_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-interface-alias$$$[interface.alias](#field-interface-alias) | Interface alias as reported by the system, typically used in firewall implementations for e.g. inside, outside, or dmz logical interface naming.

type: keyword

example: `outside`
| extended | +| $$$field-interface-id$$$[interface.id](#field-interface-id) | Interface ID as reported by an observer (typically SNMP interface ID).

type: keyword

example: `10`
| extended | +| $$$field-interface-name$$$[interface.name](#field-interface-name) | Interface name as reported by the system.

type: keyword

example: `eth0`
| extended | + + +## Field reuse [_field_reuse_14] + +The `interface` fields are expected to be nested at: + +* `observer.egress.interface` +* `observer.ingress.interface` + +Note also that the `interface` fields are not expected to be used directly at the root of the events. + diff --git a/docs/reference/ecs-log.md b/docs/reference/ecs-log.md new file mode 100644 index 0000000000..c2e19a6fad --- /dev/null +++ b/docs/reference/ecs-log.md @@ -0,0 +1,40 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-log.html +applies_to: + stack: all + serverless: all +--- + +# Log fields [ecs-log] + +Details about the event’s logging mechanism or logging transport. + +The log.* fields are typically populated with details about the logging mechanism used to create and/or transport the event. For example, syslog details belong under `log.syslog.*`. + +The details specific to your event source are typically not logged under `log.*`, but rather in `event.*` or in other ECS fields. + + +## Log field details [_log_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-log-file-path$$$[log.file.path](#field-log-file-path) | Full path to the log file this event came from, including the file name. It should include the drive letter, when appropriate.

If the event wasn’t read from a log file, do not populate this field.

type: keyword

example: `/var/log/fun-times.log`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [log.file.path](https://opentelemetry.io/docs/specs/semconv/attributes-registry/log/#log-file-path)
| extended | +| $$$field-log-level$$$[log.level](#field-log-level) | Original log level of the log event.

If the source of the event provides a log level or textual severity, this is the one that goes in `log.level`. If your source doesn’t specify one, you may put your event transport’s severity here (e.g. Syslog severity).

Some examples are `warn`, `err`, `i`, `informational`.

type: keyword

example: `error`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/OTLP-ffdcb2?style=flat "otlp") [severity_text](https://github.com/search?q=repo%3Aopen-telemetry%2Fopentelemetry-proto+%22+severity_text+%22&type=code)
| core | +| $$$field-log-logger$$$[log.logger](#field-log-logger) | The name of the logger inside an application. This is usually the name of the class which initialized the logger, or can be a custom name.

type: keyword

example: `org.elasticsearch.bootstrap.Bootstrap`
| core | +| $$$field-log-origin-file-line$$$[log.origin.file.line](#field-log-origin-file-line) | The line number of the file containing the source code which originated the log event.

type: long

example: `42`
| extended | +| $$$field-log-origin-file-name$$$[log.origin.file.name](#field-log-origin-file-name) | The name of the file containing the source code which originated the log event.

Note that this field is not meant to capture the log file. The correct field to capture the log file is `log.file.path`.

type: keyword

example: `Bootstrap.java`
| extended | +| $$$field-log-origin-function$$$[log.origin.function](#field-log-origin-function) | The name of the function or method which originated the log event.

type: keyword

example: `init`
| extended | +| $$$field-log-syslog$$$[log.syslog](#field-log-syslog) | The Syslog metadata of the event, if the event was transmitted via Syslog. Please see RFCs 5424 or 3164.

type: object
| extended | +| $$$field-log-syslog-appname$$$[log.syslog.appname](#field-log-syslog-appname) | The device or application that originated the Syslog message, if available.

type: keyword

example: `sshd`
| extended | +| $$$field-log-syslog-facility-code$$$[log.syslog.facility.code](#field-log-syslog-facility-code) | The Syslog numeric facility of the log event, if available.

According to RFCs 5424 and 3164, this value should be an integer between 0 and 23.

type: long

example: `23`
| extended | +| $$$field-log-syslog-facility-name$$$[log.syslog.facility.name](#field-log-syslog-facility-name) | The Syslog text-based facility of the log event, if available.

type: keyword

example: `local7`
| extended | +| $$$field-log-syslog-hostname$$$[log.syslog.hostname](#field-log-syslog-hostname) | The hostname, FQDN, or IP of the machine that originally sent the Syslog message. This is sourced from the hostname field of the syslog header. Depending on the environment, this value may be different from the host that handled the event, especially if the host handling the events is acting as a collector.

type: keyword

example: `example-host`
| extended | +| $$$field-log-syslog-msgid$$$[log.syslog.msgid](#field-log-syslog-msgid) | An identifier for the type of Syslog message, if available. Only applicable for RFC 5424 messages.

type: keyword

example: `ID47`
| extended | +| $$$field-log-syslog-priority$$$[log.syslog.priority](#field-log-syslog-priority) | Syslog numeric priority of the event, if available.

According to RFCs 5424 and 3164, the priority is 8 * facility + severity. This number is therefore expected to contain a value between 0 and 191.

type: long

example: `135`
| extended | +| $$$field-log-syslog-procid$$$[log.syslog.procid](#field-log-syslog-procid) | The process name or ID that originated the Syslog message, if available.

type: keyword

example: `12345`
| extended | +| $$$field-log-syslog-severity-code$$$[log.syslog.severity.code](#field-log-syslog-severity-code) | The Syslog numeric severity of the log event, if available.

If the event source publishing via Syslog provides a different numeric severity value (e.g. firewall, IDS), your source’s numeric severity should go to `event.severity`. If the event source does not specify a distinct severity, you can optionally copy the Syslog severity to `event.severity`.

type: long

example: `3`
| extended | +| $$$field-log-syslog-severity-name$$$[log.syslog.severity.name](#field-log-syslog-severity-name) | The Syslog numeric severity of the log event, if available.

If the event source publishing via Syslog provides a different severity value (e.g. firewall, IDS), your source’s text severity should go to `log.level`. If the event source does not specify a distinct severity, you can optionally copy the Syslog severity to `log.level`.

type: keyword

example: `Error`
| extended | +| $$$field-log-syslog-structured-data$$$[log.syslog.structured_data](#field-log-syslog-structured-data) | Structured data expressed in RFC 5424 messages, if available. These are key-value pairs formed from the structured data portion of the syslog message, as defined in RFC 5424 Section 6.3.

type: flattened
| extended | +| $$$field-log-syslog-version$$$[log.syslog.version](#field-log-syslog-version) | The version of the Syslog protocol specification. Only applicable for RFC 5424 messages.

type: keyword

example: `1`
| extended | + diff --git a/docs/reference/ecs-macho.md b/docs/reference/ecs-macho.md new file mode 100644 index 0000000000..4c72e636ee --- /dev/null +++ b/docs/reference/ecs-macho.md @@ -0,0 +1,49 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-macho.html +applies_to: + stack: all + serverless: all +--- + +# Mach-O header fields [ecs-macho] + +These fields contain Mac OS Mach Object file format (Mach-O) metadata. + +::::{warning} +These fields are in beta and are subject to change. +:::: + + + +## Mach-O header field details [_mach_o_header_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-macho-go-import-hash$$$[macho.go_import_hash](#field-macho-go-import-hash) | A hash of the Go language imports in a Mach-O file excluding standard library imports. An import hash can be used to fingerprint binaries even after recompilation or other code-level transformations have occurred, which would change more traditional hash values.

The algorithm used to calculate the Go symbol hash and a reference implementation are available here: [https://github.com/elastic/toutoumomoma](https://github.com/elastic/toutoumomoma)

type: keyword

example: `10bddcb4cee42080f76c88d9ff964491`
| extended | +| $$$field-macho-go-imports$$$[macho.go_imports](#field-macho-go-imports) | List of imported Go language element names and types.

type: flattened
| extended | +| $$$field-macho-go-imports-names-entropy$$$[macho.go_imports_names_entropy](#field-macho-go-imports-names-entropy) | Shannon entropy calculation from the list of Go imports.

type: long
| extended | +| $$$field-macho-go-imports-names-var-entropy$$$[macho.go_imports_names_var_entropy](#field-macho-go-imports-names-var-entropy) | Variance for Shannon entropy calculation from the list of Go imports.

type: long
| extended | +| $$$field-macho-go-stripped$$$[macho.go_stripped](#field-macho-go-stripped) | Set to true if the file is a Go executable that has had its symbols stripped or obfuscated and false if an unobfuscated Go executable.

type: boolean
| extended | +| $$$field-macho-import-hash$$$[macho.import_hash](#field-macho-import-hash) | A hash of the imports in a Mach-O file. An import hash can be used to fingerprint binaries even after recompilation or other code-level transformations have occurred, which would change more traditional hash values.

This is a synonym for symhash.

type: keyword

example: `d41d8cd98f00b204e9800998ecf8427e`
| extended | +| $$$field-macho-imports$$$[macho.imports](#field-macho-imports) | List of imported element names and types.

type: flattened

Note: this field should contain an array of values.
| extended | +| $$$field-macho-imports-names-entropy$$$[macho.imports_names_entropy](#field-macho-imports-names-entropy) | Shannon entropy calculation from the list of imported element names and types.

type: long
| extended | +| $$$field-macho-imports-names-var-entropy$$$[macho.imports_names_var_entropy](#field-macho-imports-names-var-entropy) | Variance for Shannon entropy calculation from the list of imported element names and types.

type: long
| extended | +| $$$field-macho-sections$$$[macho.sections](#field-macho-sections) | An array containing an object for each section of the Mach-O file.

The keys that should be present in these objects are defined by sub-fields underneath `macho.sections.*`.

type: nested

Note: this field should contain an array of values.
| extended | +| $$$field-macho-sections-entropy$$$[macho.sections.entropy](#field-macho-sections-entropy) | Shannon entropy calculation from the section.

type: long
| extended | +| $$$field-macho-sections-name$$$[macho.sections.name](#field-macho-sections-name) | Mach-O Section List name.

type: keyword
| extended | +| $$$field-macho-sections-physical-size$$$[macho.sections.physical_size](#field-macho-sections-physical-size) | Mach-O Section List physical size.

type: long
| extended | +| $$$field-macho-sections-var-entropy$$$[macho.sections.var_entropy](#field-macho-sections-var-entropy) | Variance for Shannon entropy calculation from the section.

type: long
| extended | +| $$$field-macho-sections-virtual-size$$$[macho.sections.virtual_size](#field-macho-sections-virtual-size) | Mach-O Section List virtual size. This is always the same as `physical_size`.

type: long
| extended | +| $$$field-macho-symhash$$$[macho.symhash](#field-macho-symhash) | A hash of the imports in a Mach-O file. An import hash can be used to fingerprint binaries even after recompilation or other code-level transformations have occurred, which would change more traditional hash values.

This is a Mach-O implementation of the Windows PE imphash

type: keyword

example: `d3ccf195b62a9279c3c19af1080497ec`
| extended | + + +## Field reuse [_field_reuse_15] + +The `macho` fields are expected to be nested at: + +* `file.macho` +* `process.macho` + +Note also that the `macho` fields are not expected to be used directly at the root of the events. + diff --git a/docs/reference/ecs-mapping-network-events.md b/docs/reference/ecs-mapping-network-events.md new file mode 100644 index 0000000000..7a3ca77695 --- /dev/null +++ b/docs/reference/ecs-mapping-network-events.md @@ -0,0 +1,222 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-mapping-network-events.html +applies_to: + stack: all + serverless: all +--- + +# Mapping network events [ecs-mapping-network-events] + +Network events capture the details of one device communicating with another. The initiator is referred to as the source, and the recipient as the destination. Depending on the data source, a network event can contain details of addresses, protocols, headers, and device roles. + +This guide describes the different field sets available for network-related events in ECS and provides direction on the ECS best practices for mapping to them. + + +### Source and destination baseline [_source_and_destination_baseline] + +When an event contains details about the sending and receiving hosts, the baseline for capturing these values will be the [source](/reference/ecs-source.md) and [destination](/reference/ecs-destination.md) fields. + +Some events may also indicate each host’s role in the exchange: client or server. When this information is available, the [client](/reference/ecs-client.md) and [server](/reference/ecs-server.md) fields should be used *in addition to* the `source` and `destination` fields. The fields and values mapped under `source`/`destination` should be copied under `client`/`server`. + + +### Network event mapping example [_network_event_mapping_example] + +Below is a DNS network event. The source device (`192.168.86.222`) makes a DNS query, acting as the client and the DNS server is the destination (`192.168.86.1`). + +Note this event contains additional details that would populate additional fields (such as the [DNS Fields](/reference/ecs-dns.md)) if this was a complete mapping example. These additional fields are omitted here to focus on the network details. + +```json +{ + "ts":1599775747.53056, + "uid":"CYqFPH3nOAa0kPxA0d", + "id.orig_h":"192.168.86.222", + "id.orig_p":54162, + "id.resp_h":"192.168.86.1", + "id.resp_p":53, + "proto":"udp", + "trans_id":28899, + "rtt":0.02272200584411621, + "query":"example.com", + "qclass":1, + "qclass_name":"C_INTERNET", + "qtype":1, + "qtype_name":"A", + "rcode":0, + "rcode_name":"NOERROR", + "AA":false, + "TC":false, + "RD":true, + "RA":true, + "Z":0, + "answers":["93.184.216.34"], + "TTLs":[21209.0], + "rejected":false +} +``` + + +### Source and destination fields [_source_and_destination_fields] + +First, the `source.*` and `destination.*` field sets are populated: + +```json + "source": { + "ip": "192.168.86.222", + "port": 54162 + } +``` + +```json + "destination": { + "ip": "192.168.86.1", + "port": 53 + } +``` + + +### Client and server fields [_client_and_server_fields] + +Looking back at the original event, it shows the source device is the DNS client and the destination device is the DNS server. The values mapped under `source` and `destination` are copied and mapped under `client` and `server`, respectively: + +```json + "client": { + "ip": "192.168.86.222", + "port": 54162 + } +``` + +```json + "server": { + "ip": "192.168.86.1", + "port": 53 + } +``` + +Mapping both pairs of field sets gives query visibility of the same network transaction in two ways. + +* `source.ip:192.168.86.222` returns all events sourced from `192.168.86.222`, regardless its role in a transaction +* `client.ip:192.168.86.222` returns all events with host `192.168.86.222` acting as a client + +The same applies for the `destination` and `server` fields: + +* `destination.ip:192.168.86.1` returns all events destined to `192.168.86.1` +* `server.ip:192.168.86.1` returns all events with `192.168.86.1` acting as the server + +It’s important to note that while the values for the `source` and `destination` fields may reverse between events in a single network transaction, the values for `client` and `server` typically will not. The following two tables demonstrate how two DNS transactions involving two clients and one server would map to `source.ip`/`destination.ip` vs. `client.ip`/`server.ip`: + +| source.ip | destination.ip | event | +| --- | --- | --- | +| 192.168.86.222 | 192.168.86.1 | DNS query request 1 | +| 192.168.86.1 | 192.168.86.222 | DNS answer response 1 | +| 192.168.86.42 | 192.168.86.1 | DNS answer request 2 | +| 192.168.86.1 | 192.168.86.42 | DNS answer request 2 | + +| client.ip | server.ip | event | +| --- | --- | --- | +| 192.168.86.222 | 192.168.86.1 | DNS query request 1 | +| 192.168.86.222 | 192.168.86.1 | DNS answer response 1 | +| 192.168.86.42 | 192.168.86.1 | DNS query request 2 | +| 192.168.86.42 | 192.168.86.1 | DNS answer response 2 | + + +## Related fields [_related_fields_2] + +The `related.ip` field captures all the IPs present in the event in a single array: + +```json + "related": { + "ip": [ + "192.168.86.222", + "192.168.86.1", + "93.184.216.34" + ] + } +``` + +The [related fields](/reference/ecs-related.md) are meant to facilitate pivoting. Since these IP addresses can appear in many different fields (`source.ip`, `destination.ip`, `client.ip`, `server.ip`, etc.), you can search for the IP trivially no matter what field it appears using a single query, e.g. `related.ip:192.168.86.222`. + +Network events are not only limited to using `related.ip`. If hostnames or other host identifiers were present in the event, `related.hosts` should be populated too. + + +### Categorization using event fields [_categorization_using_event_fields] + +When considering the [event categorization fields](/reference/ecs-category-field-values-reference.md), the `category` and `type` fields are populated using their respective allowed values which best classify the source network event. + +```json + "event": { + "category": [ + "network" + ], + "type": [ + "connection", + "protocol" + ], + "kind": "event" + } +``` + +Most [event.category](/reference/ecs-allowed-values-event-category.md)/[event.type](/reference/ecs-allowed-values-event-type.md) ECS pairings are complete on their own. However, the pairing of `event.category:network` and `event.type:protocol` is an exception. When these two fields/value pairs both used to categorize an event, the `network.protocol` field should also be populated: + +```json + "network": { + "protocol": "dns", + "type": "ipv4", + "transport": "udp" + } +``` + + +### Result [_result] + +Putting everything together covered so far, we have a final ECS-mapped event: + +```json +{ + "event": { + "category": [ + "network" + ], + "type": [ + "connection", + "protocol" + ], + "kind": "event" + }, + "network": { + "protocol": "dns", + "type": "ipv4", + "transport": "udp" + }, + "source": { + "ip": "192.168.86.222", + "port": 54162 + }, + "destination": { + "ip": "192.168.86.1", + "port": 53 + }, + "client": { + "ip": "192.168.86.222", + "port": 64734 + }, + "server": { + "ip": "192.168.86.1", + "port": 53 + }, + "related": { + "ip": [ + "192.168.86.222", + "192.168.86.1", + "93.184.216.34" + ] + }, + "dns": { ... }, <1> + "zeek": { "ts":1599775747.53056, ... } <2> +} +``` + +1. Again, not diving into the DNS fields here but included for completeness. +2. Original fields can optionally be kept around as custom fields. + + diff --git a/docs/reference/ecs-network.md b/docs/reference/ecs-network.md new file mode 100644 index 0000000000..61a8525ccc --- /dev/null +++ b/docs/reference/ecs-network.md @@ -0,0 +1,43 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-network.html +applies_to: + stack: all + serverless: all +--- + +# Network fields [ecs-network] + +The network is defined as the communication path over which a host or network event happens. + +The network.* fields should be populated with details about the network activity associated with an event. + + +## Network field details [_network_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-network-application$$$[network.application](#field-network-application) | When a specific application or service is identified from network connection details (source/dest IPs, ports, certificates, or wire format), this field captures the application’s or service’s name.

For example, the original event identifies the network connection being from a specific web service in a `https` network connection, like `facebook` or `twitter`.

The field value must be normalized to lowercase for querying.

type: keyword

example: `aim`
| extended | +| $$$field-network-bytes$$$[network.bytes](#field-network-bytes) | Total bytes transferred in both directions.

If `source.bytes` and `destination.bytes` are known, `network.bytes` is their sum.

type: long

example: `368`
| core | +| $$$field-network-community-id$$$[network.community_id](#field-network-community-id) | A hash of source and destination IPs and ports, as well as the protocol used in a communication. This is a tool-agnostic standard to identify flows.

Learn more at [https://github.com/corelight/community-id-spec](https://github.com/corelight/community-id-spec).

type: keyword

example: `1:hO+sN4H+MG5MY/8hIrXPqc4ZQz0=`
| extended | +| $$$field-network-direction$$$[network.direction](#field-network-direction) | Direction of the network traffic.

When mapping events from a host-based monitoring context, populate this field from the host’s point of view, using the values "ingress" or "egress".

When mapping events from a network or perimeter-based monitoring context, populate this field from the point of view of the network perimeter, using the values "inbound", "outbound", "internal" or "external".

Note that "internal" is not crossing perimeter boundaries, and is meant to describe communication between two hosts within the perimeter. Note also that "external" is meant to describe traffic between two hosts that are external to the perimeter. This could for example be useful for ISPs or VPN service providers.

Expected values for this field:

- `ingress`
- `egress`
- `inbound`
- `outbound`
- `internal`
- `external`
- `unknown`

type: keyword

example: `inbound`
| core | +| $$$field-network-forwarded-ip$$$[network.forwarded_ip](#field-network-forwarded-ip) | Host IP address when the source IP address is the proxy.

type: ip

example: `192.1.1.2`
| core | +| $$$field-network-iana-number$$$[network.iana_number](#field-network-iana-number) | IANA Protocol Number ([https://www.iana.org/assignments/protocol-numbers/protocol-numbers.xhtml](https://www.iana.org/assignments/protocol-numbers/protocol-numbers.xhtml)). Standardized list of protocols. This aligns well with NetFlow and sFlow related logs which use the IANA Protocol Number.

type: keyword

example: `6`
| extended | +| $$$field-network-inner$$$[network.inner](#field-network-inner) | Network.inner fields are added in addition to network.vlan fields to describe the innermost VLAN when q-in-q VLAN tagging is present. Allowed fields include vlan.id and vlan.name. Inner vlan fields are typically used when sending traffic with multiple 802.1q encapsulations to a network sensor (e.g. Zeek, Wireshark.)

type: object
| extended | +| $$$field-network-name$$$[network.name](#field-network-name) | Name given by operators to sections of their network.

type: keyword

example: `Guest Wifi`
| extended | +| $$$field-network-packets$$$[network.packets](#field-network-packets) | Total packets transferred in both directions.

If `source.packets` and `destination.packets` are known, `network.packets` is their sum.

type: long

example: `24`
| core | +| $$$field-network-protocol$$$[network.protocol](#field-network-protocol) | In the OSI Model this would be the Application Layer protocol. For example, `http`, `dns`, or `ssh`.

The field value must be normalized to lowercase for querying.

type: keyword

example: `http`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") [network.protocol.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/network/#network-protocol-name)
| core | +| $$$field-network-transport$$$[network.transport](#field-network-transport) | Same as network.iana_number, but instead using the Keyword name of the transport layer (udp, tcp, ipv6-icmp, etc.)

The field value must be normalized to lowercase for querying.

type: keyword

example: `tcp`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [network.transport](https://opentelemetry.io/docs/specs/semconv/attributes-registry/network/#network-transport)
| core | +| $$$field-network-type$$$[network.type](#field-network-type) | In the OSI Model this would be the Network Layer. ipv4, ipv6, ipsec, pim, etc

The field value must be normalized to lowercase for querying.

type: keyword

example: `ipv4`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [network.type](https://opentelemetry.io/docs/specs/semconv/attributes-registry/network/#network-type)
| core | + + +## Field reuse [_field_reuse_16] + + +### Field sets that can be nested under network [ecs-network-nestings] + +| Location | Field Set | Description | +| --- | --- | --- | +| `network.inner.vlan.*` | [vlan](/reference/ecs-vlan.md) | Fields to describe observed VLAN information. | +| `network.vlan.*` | [vlan](/reference/ecs-vlan.md) | Fields to describe observed VLAN information. | + diff --git a/docs/reference/ecs-observer.md b/docs/reference/ecs-observer.md new file mode 100644 index 0000000000..1e9bad0f6b --- /dev/null +++ b/docs/reference/ecs-observer.md @@ -0,0 +1,48 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-observer.html +applies_to: + stack: all + serverless: all +--- + +# Observer fields [ecs-observer] + +An observer is defined as a special network, security, or application device used to detect, observe, or create network, security, or application-related events and metrics. + +This could be a custom hardware appliance or a server that has been configured to run special network, security, or application software. Examples include firewalls, web proxies, intrusion detection/prevention systems, network monitoring sensors, web application firewalls, data loss prevention systems, and APM servers. The observer.* fields shall be populated with details of the system, if any, that detects, observes and/or creates a network, security, or application event or metric. Message queues and ETL components used in processing events or metrics are not considered observers in ECS. + + +## Observer field details [_observer_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-observer-egress$$$[observer.egress](#field-observer-egress) | Observer.egress holds information like interface number and name, vlan, and zone information to classify egress traffic. Single armed monitoring such as a network sensor on a span port should only use observer.ingress to categorize traffic.

type: object
| extended | +| $$$field-observer-egress-zone$$$[observer.egress.zone](#field-observer-egress-zone) | Network zone of outbound traffic as reported by the observer to categorize the destination area of egress traffic, e.g. Internal, External, DMZ, HR, Legal, etc.

type: keyword

example: `Public_Internet`
| extended | +| $$$field-observer-hostname$$$[observer.hostname](#field-observer-hostname) | Hostname of the observer.

type: keyword
| core | +| $$$field-observer-ingress$$$[observer.ingress](#field-observer-ingress) | Observer.ingress holds information like interface number and name, vlan, and zone information to classify ingress traffic. Single armed monitoring such as a network sensor on a span port should only use observer.ingress to categorize traffic.

type: object
| extended | +| $$$field-observer-ingress-zone$$$[observer.ingress.zone](#field-observer-ingress-zone) | Network zone of incoming traffic as reported by the observer to categorize the source area of ingress traffic. e.g. internal, External, DMZ, HR, Legal, etc.

type: keyword

example: `DMZ`
| extended | +| $$$field-observer-ip$$$[observer.ip](#field-observer-ip) | IP addresses of the observer.

type: ip

Note: this field should contain an array of values.
| core | +| $$$field-observer-mac$$$[observer.mac](#field-observer-mac) | MAC addresses of the observer.

The notation format from RFC 7042 is suggested: Each octet (that is, 8-bit byte) is represented by two [uppercase] hexadecimal digits giving the value of the octet as an unsigned integer. Successive octets are separated by a hyphen.

type: keyword

Note: this field should contain an array of values.

example: `["00-00-5E-00-53-23", "00-00-5E-00-53-24"]`
| core | +| $$$field-observer-name$$$[observer.name](#field-observer-name) | Custom name of the observer.

This is a name that can be given to an observer. This can be helpful for example if multiple firewalls of the same model are used in an organization.

If no custom name is needed, the field can be left empty.

type: keyword

example: `1_proxySG`
| extended | +| $$$field-observer-product$$$[observer.product](#field-observer-product) | The product name of the observer.

type: keyword

example: `s200`
| extended | +| $$$field-observer-serial-number$$$[observer.serial_number](#field-observer-serial-number) | Observer serial number.

type: keyword
| extended | +| $$$field-observer-type$$$[observer.type](#field-observer-type) | The type of the observer the data is coming from.

There is no predefined list of observer types. Some examples are `forwarder`, `firewall`, `ids`, `ips`, `proxy`, `poller`, `sensor`, `APM server`.

type: keyword

example: `firewall`
| core | +| $$$field-observer-vendor$$$[observer.vendor](#field-observer-vendor) | Vendor name of the observer.

type: keyword

example: `Symantec`
| core | +| $$$field-observer-version$$$[observer.version](#field-observer-version) | Observer version.

type: keyword
| core | + + +## Field reuse [_field_reuse_17] + + +### Field sets that can be nested under observer [ecs-observer-nestings] + +| Location | Field Set | Description | +| --- | --- | --- | +| `observer.egress.interface.*` | [interface](/reference/ecs-interface.md) | Fields to describe observer interface information. | +| `observer.egress.vlan.*` | [vlan](/reference/ecs-vlan.md) | Fields to describe observed VLAN information. | +| `observer.geo.*` | [geo](/reference/ecs-geo.md) | Fields describing a location. | +| `observer.ingress.interface.*` | [interface](/reference/ecs-interface.md) | Fields to describe observer interface information. | +| `observer.ingress.vlan.*` | [vlan](/reference/ecs-vlan.md) | Fields to describe observed VLAN information. | +| `observer.os.*` | [os](/reference/ecs-os.md) | OS fields contain information about the operating system. | + diff --git a/docs/reference/ecs-opentelemetry.md b/docs/reference/ecs-opentelemetry.md new file mode 100644 index 0000000000..ebb6f97a21 --- /dev/null +++ b/docs/reference/ecs-opentelemetry.md @@ -0,0 +1,33 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/master/ecs-opentelemetry.html +--- + +# ECS & OpenTelemetry [ecs-opentelemetry] + +In April 2023, Elastic donated ECS to OpenTelemetry and together with the OTel community [jointly announced](https://opentelemetry.io/blog/2023/ecs-otel-semconv-convergence/) the intention to achieve convergence of ECS and [OTel Semantic Conventions (SemConv)](https://opentelemetry.io/docs/specs/semconv/). This donation should be seen as a directional decision for the evolution of both standards rather than a single event that merged both schemas into a single standard. + +While both schemes complement each other in large areas and thus offer mutual added value and great potential for convergence, it is important to understand that in some areas convergence is not achievable due to conceptual differences or consistency reasons. For example, while ECS contains a few fields (e.g. [container.disk.read.bytes](/reference/ecs-container.md#field-container-disk-read-bytes)) that are meant to be used in a metric-like way, metrics in OpenTelemetry are following a completely different data model (with metric name, type, dimensions, etc.). Also, in OTel semantic conventions there are some stable (or quasi-stable) attributes that have semantically equivalent, stable fields in ECS, however, with different field names. This kind of differences require explicit handling to achieve compatibility. + + +## Relation between ECS and Semantic Conventions [ecs-opentelemetry-relation] + +The [ECS schema files](https://github.com/elastic/ecs/tree/main/schemas) contain an explicit mapping between ECS fields and corresponding OTel semnatic convention attributes. This can be used to generate tooling for compatibility between ECS and semnatic conventions (e.g. alias fields in Elasticsearch). The relation between individual ECS fields and corresponding OTel semantic conventions attributes follows one of the following categories: + +| Category | Description | +| --- | --- | +| ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | The name of the ECS field is identical to the SemConv attribute name and has (practically) the same semantics. | +| ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") | The ECS field name is different but has the same semantics as the corresponding SemConv attribute.For this type of relation aliasing approaches (e.g. [Elasticsearch field aliases](elasticsearch://reference/elasticsearch/mapping-reference/field-alias.md)) can be used to achieve compatibility between ECS and OTel SemConv. | +| ![relation](https://img.shields.io/badge/related-efc20d?style=flat "related") | The ECS field name is different and has related - yet different - semantics as the corresponding SemConv attribute.An aliasing approach is not sufficient to resolve compatibility for this type of relation. | +| ![relation](https://img.shields.io/badge/conflict-910000?style=flat "conflict") | The ECS field name is the same as an OTel SemConv namespace or an attribute that has significantly different semantics. | +| ![relation](https://img.shields.io/badge/OTLP-ffdcb2?style=flat "OTLP") | The ECS field has a corresponding representation in [OpenTelemetry’s protocol definition](https://github.com/open-telemetry/opentelemetry-proto). | +| ![relation](https://img.shields.io/badge/metric-cb00cb?style=flat "metric") | For this ECS field there is a corresponding metric defined in OTel SemConv. | +| ![relation](https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat "na") | The ECS field is not applicable in the context of OTel or won’t be aligned due to significant, conceptual conflict with OTel concepts in that area. | + +The following documentation pages provide an overview and more details on the alignment between ECS and OTel semantic conventions: + +* [OTel Alignment Overview](/reference/ecs-otel-alignment-overview.md) +* [Field & Attributes Alignment](/reference/ecs-otel-alignment-details.md) + + + diff --git a/docs/reference/ecs-orchestrator.md b/docs/reference/ecs-orchestrator.md new file mode 100644 index 0000000000..5641376931 --- /dev/null +++ b/docs/reference/ecs-orchestrator.md @@ -0,0 +1,33 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-orchestrator.html +applies_to: + stack: all + serverless: all +--- + +# Orchestrator fields [ecs-orchestrator] + +Fields that describe the resources which container orchestrators manage or act upon. + + +## Orchestrator field details [_orchestrator_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-orchestrator-api-version$$$[orchestrator.api_version](#field-orchestrator-api-version) | API version being used to carry out the action

type: keyword

example: `v1beta1`
| extended | +| $$$field-orchestrator-cluster-id$$$[orchestrator.cluster.id](#field-orchestrator-cluster-id) | Unique ID of the cluster.

type: keyword
| extended | +| $$$field-orchestrator-cluster-name$$$[orchestrator.cluster.name](#field-orchestrator-cluster-name) | Name of the cluster.

type: keyword
| extended | +| $$$field-orchestrator-cluster-url$$$[orchestrator.cluster.url](#field-orchestrator-cluster-url) | URL of the API used to manage the cluster.

type: keyword
| extended | +| $$$field-orchestrator-cluster-version$$$[orchestrator.cluster.version](#field-orchestrator-cluster-version) | The version of the cluster.

type: keyword
| extended | +| $$$field-orchestrator-namespace$$$[orchestrator.namespace](#field-orchestrator-namespace) | Namespace in which the action is taking place.

type: keyword

example: `kube-system`
| extended | +| $$$field-orchestrator-organization$$$[orchestrator.organization](#field-orchestrator-organization) | Organization affected by the event (for multi-tenant orchestrator setups).

type: keyword

example: `elastic`
| extended | +| $$$field-orchestrator-resource-annotation$$$[orchestrator.resource.annotation](#field-orchestrator-resource-annotation) | The list of annotations added to the resource.

type: keyword

Note: this field should contain an array of values.

example: `['key1:value1', 'key2:value2', 'key3:value3']`
| extended | +| $$$field-orchestrator-resource-id$$$[orchestrator.resource.id](#field-orchestrator-resource-id) | Unique ID of the resource being acted upon.

type: keyword
| extended | +| $$$field-orchestrator-resource-ip$$$[orchestrator.resource.ip](#field-orchestrator-resource-ip) | IP address assigned to the resource associated with the event being observed. In the case of a Kubernetes Pod, this array would contain only one element: the IP of the Pod (as opposed to the Node on which the Pod is running).

type: ip

Note: this field should contain an array of values.
| extended | +| $$$field-orchestrator-resource-label$$$[orchestrator.resource.label](#field-orchestrator-resource-label) | The list of labels added to the resource.

type: keyword

Note: this field should contain an array of values.

example: `['key1:value1', 'key2:value2', 'key3:value3']`
| extended | +| $$$field-orchestrator-resource-name$$$[orchestrator.resource.name](#field-orchestrator-resource-name) | Name of the resource being acted upon.

type: keyword

example: `test-pod-cdcws`
| extended | +| $$$field-orchestrator-resource-parent-type$$$[orchestrator.resource.parent.type](#field-orchestrator-resource-parent-type) | Type or kind of the parent resource associated with the event being observed. In Kubernetes, this will be the name of a built-in workload resource (e.g., Deployment, StatefulSet, DaemonSet).

type: keyword

example: `DaemonSet`
| extended | +| $$$field-orchestrator-resource-type$$$[orchestrator.resource.type](#field-orchestrator-resource-type) | Type of resource being acted upon.

type: keyword

example: `service`
| extended | +| $$$field-orchestrator-type$$$[orchestrator.type](#field-orchestrator-type) | Orchestrator cluster type (e.g. kubernetes, nomad or cloudfoundry).

type: keyword

example: `kubernetes`
| extended | + diff --git a/docs/reference/ecs-organization.md b/docs/reference/ecs-organization.md new file mode 100644 index 0000000000..ade1e0e280 --- /dev/null +++ b/docs/reference/ecs-organization.md @@ -0,0 +1,22 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-organization.html +applies_to: + stack: all + serverless: all +--- + +# Organization fields [ecs-organization] + +The organization fields enrich data with information about the company or entity the data is associated with. + +These fields help you arrange or filter data stored in an index by one or multiple organizations. + + +## Organization field details [_organization_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-organization-id$$$[organization.id](#field-organization-id) | Unique identifier for the organization.

type: keyword
| extended | +| $$$field-organization-name$$$[organization.name](#field-organization-name) | Organization name.

type: keyword

Multi-fields:

* organization.name.text (type: match_only_text)
| extended | + diff --git a/docs/reference/ecs-os.md b/docs/reference/ecs-os.md new file mode 100644 index 0000000000..1fbcc6c01c --- /dev/null +++ b/docs/reference/ecs-os.md @@ -0,0 +1,36 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-os.html +applies_to: + stack: all + serverless: all +--- + +# Operating system fields [ecs-os] + +The OS fields contain information about the operating system. + + +## Operating system field details [_operating_system_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-os-family$$$[os.family](#field-os-family) | OS family (such as redhat, debian, freebsd, windows).

type: keyword

example: `debian`
| extended | +| $$$field-os-full$$$[os.full](#field-os-full) | Operating system name, including the version or code name.

type: keyword

Multi-fields:

- os.full.text (type: match_only_text)

example: `Mac OS Mojave`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") [os.description](https://opentelemetry.io/docs/specs/semconv/attributes-registry/os/#os-description)
| extended | +| $$$field-os-kernel$$$[os.kernel](#field-os-kernel) | Operating system kernel version as a raw string.

type: keyword

example: `4.4.0-112-generic`
| extended | +| $$$field-os-name$$$[os.name](#field-os-name) | Operating system name, without the version.

type: keyword

Multi-fields:

- os.name.text (type: match_only_text)

example: `Mac OS X`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [os.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/os/#os-name)
| extended | +| $$$field-os-platform$$$[os.platform](#field-os-platform) | Operating system platform (such centos, ubuntu, windows).

type: keyword

example: `darwin`
| extended | +| $$$field-os-type$$$[os.type](#field-os-type) | Use the `os.type` field to categorize the operating system into one of the broad commercial families.

If the OS you’re dealing with is not listed as an expected value, the field should not be populated. Please let us know by opening an issue with ECS, to propose its addition.

Expected values for this field:

- `linux`
- `macos`
- `unix`
- `windows`
- `ios`
- `android`

type: keyword

example: `macos`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/conflict-910000?style=flat "conflict") [os.type](https://opentelemetry.io/docs/specs/semconv/attributes-registry/os/#os-type)

Note: The expected values in ECS for `os.type` do not correspond with the values defined in semantic conventions!
| extended | +| $$$field-os-version$$$[os.version](#field-os-version) | Operating system version as a raw string.

type: keyword

example: `10.14.1`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [os.version](https://opentelemetry.io/docs/specs/semconv/attributes-registry/os/#os-version)
| extended | + + +## Field reuse [_field_reuse_18] + +The `os` fields are expected to be nested at: + +* `host.os` +* `observer.os` +* `user_agent.os` + +Note also that the `os` fields are not expected to be used directly at the root of the events. + diff --git a/docs/reference/ecs-otel-alignment-details.md b/docs/reference/ecs-otel-alignment-details.md new file mode 100644 index 0000000000..9d001d539e --- /dev/null +++ b/docs/reference/ecs-otel-alignment-details.md @@ -0,0 +1,217 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/master/ecs-otel-alignment-details.html +--- + +# Field & Attributes Alignment [ecs-otel-alignment-details] + +The following table gives an overview of mappings between individual ECS fields (in ECS version `9.1.0-dev`) and corresponding OTel semantic convention attributes (in SemConv version `1.29.0`). + +| ECS Field | Relation | OTel Semantic Conventions Attribute | Stability $$$otel-mapping-namespace-base$$$ | +| --- | --- | --- | --- | +| **Base Fields** | +| $$$otel-mapping-for-timestamp$$$[@timestamp](/reference/ecs-base.md#field-timestamp) | ![relation](https://img.shields.io/badge/OTLP-ffdcb2?style=flat "otlp") | [time_unix_nano](https://github.com/search?q=repo%3Aopen-telemetry%2Fopentelemetry-proto+%22+time_unix_nano+%22&type=code)
| ![stable](https://img.shields.io/badge/✔-93c93e?style=flat "stable") | +| ![relation](https://img.shields.io/badge/OTLP-ffdcb2?style=flat "otlp") | [observed_time_unix_nano](https://github.com/search?q=repo%3Aopen-telemetry%2Fopentelemetry-proto+%22+observed_time_unix_nano+%22&type=code)
| ![stable](https://img.shields.io/badge/✔-93c93e?style=flat "stable") | +| ![relation](https://img.shields.io/badge/OTLP-ffdcb2?style=flat "otlp") | [start_time_unix_nano](https://github.com/search?q=repo%3Aopen-telemetry%2Fopentelemetry-proto+%22+start_time_unix_nano+%22&type=code)

Note: On logs, events and metrics, `time_unix_nano` defines the time of the event. For logs, `observed_time_unix_nano` defines the time when the event was observed by the collection system. On spans, `start_time_unix_nano` represents the start time of the span. | ![stable](https://img.shields.io/badge/✔-93c93e?style=flat "stable") | +| $$$otel-mapping-for-labels$$$[labels](/reference/ecs-base.md#field-labels) | ![relation](https://img.shields.io/badge/related-efc20d?style=flat "related") | [k8s.pod.label](https://opentelemetry.io/docs/specs/semconv/attributes-registry/k8s/#k8s-pod-label)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| ![relation](https://img.shields.io/badge/related-efc20d?style=flat "related") | [container.label](https://opentelemetry.io/docs/specs/semconv/attributes-registry/container/#container-label)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-message$$$[message](/reference/ecs-base.md#field-message) | ![relation](https://img.shields.io/badge/OTLP-ffdcb2?style=flat "otlp") | [body](https://github.com/search?q=repo%3Aopen-telemetry%2Fopentelemetry-proto+%22+body+%22&type=code)

Note: The `body` in OTLP is of type `Any` and can be either an unstructured log message or a structured event. | ![stable](https://img.shields.io/badge/✔-93c93e?style=flat "stable")
$$$otel-mapping-namespace-client$$$ | +| **Client Fields** | +| $$$otel-mapping-for-client-address$$$[client.address](/reference/ecs-client.md#field-client-address) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [client.address](https://opentelemetry.io/docs/specs/semconv/attributes-registry/client/#client-address)
| ![stable](https://img.shields.io/badge/✔-93c93e?style=flat "stable") | +| $$$otel-mapping-for-client-port$$$[client.port](/reference/ecs-client.md#field-client-port) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [client.port](https://opentelemetry.io/docs/specs/semconv/attributes-registry/client/#client-port)
| ![stable](https://img.shields.io/badge/✔-93c93e?style=flat "stable")
$$$otel-mapping-namespace-cloud$$$ | +| **Cloud Fields** | +| $$$otel-mapping-for-cloud-account-id$$$[cloud.account.id](/reference/ecs-cloud.md#field-cloud-account-id) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [cloud.account.id](https://opentelemetry.io/docs/specs/semconv/attributes-registry/cloud/#cloud-account-id)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-cloud-availability-zone$$$[cloud.availability_zone](/reference/ecs-cloud.md#field-cloud-availability-zone) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [cloud.availability_zone](https://opentelemetry.io/docs/specs/semconv/attributes-registry/cloud/#cloud-availability-zone)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-cloud-provider$$$[cloud.provider](/reference/ecs-cloud.md#field-cloud-provider) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [cloud.provider](https://opentelemetry.io/docs/specs/semconv/attributes-registry/cloud/#cloud-provider)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-cloud-region$$$[cloud.region](/reference/ecs-cloud.md#field-cloud-region) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [cloud.region](https://opentelemetry.io/docs/specs/semconv/attributes-registry/cloud/#cloud-region)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-cloud-service-name$$$[cloud.service.name](/reference/ecs-cloud.md#field-cloud-service-name) | ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") | [cloud.platform](https://opentelemetry.io/docs/specs/semconv/attributes-registry/cloud/#cloud-platform)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental")
$$$otel-mapping-namespace-container$$$ | +| **Container Fields** | +| $$$otel-mapping-for-container-cpu-usage$$$[container.cpu.usage](/reference/ecs-container.md#field-container-cpu-usage) | ![relation](https://img.shields.io/badge/metric-cb00cb?style=flat "metric") | [container.cpu.usage](https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--+semconv+metric.container.cpu.usage+--%3E%22&type=code)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-container-id$$$[container.id](/reference/ecs-container.md#field-container-id) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [container.id](https://opentelemetry.io/docs/specs/semconv/attributes-registry/container/#container-id)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-container-image-hash-all$$$[container.image.hash.all](/reference/ecs-container.md#field-container-image-hash-all) | ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") | [container.image.repo_digests](https://opentelemetry.io/docs/specs/semconv/attributes-registry/container/#container-image-repo-digests)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-container-image-name$$$[container.image.name](/reference/ecs-container.md#field-container-image-name) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [container.image.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/container/#container-image-name)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-container-image-tag$$$[container.image.tag](/reference/ecs-container.md#field-container-image-tag) | ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") | [container.image.tags](https://opentelemetry.io/docs/specs/semconv/attributes-registry/container/#container-image-tags)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-container-labels$$$[container.labels](/reference/ecs-container.md#field-container-labels) | ![relation](https://img.shields.io/badge/related-efc20d?style=flat "related") | [container.label](https://opentelemetry.io/docs/specs/semconv/attributes-registry/container/#container-label)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-container-memory-usage$$$[container.memory.usage](/reference/ecs-container.md#field-container-memory-usage) | ![relation](https://img.shields.io/badge/metric-cb00cb?style=flat "metric") | [container.memory.usage](https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--+semconv+metric.container.memory.usage+--%3E%22&type=code)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-container-name$$$[container.name](/reference/ecs-container.md#field-container-name) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [container.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/container/#container-name)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-container-runtime$$$[container.runtime](/reference/ecs-container.md#field-container-runtime) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [container.runtime](https://opentelemetry.io/docs/specs/semconv/attributes-registry/container/#container-runtime)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental")
$$$otel-mapping-namespace-data_stream$$$ | +| **Data Stream Fields** | +| $$$otel-mapping-for-data-stream-dataset$$$[data_stream.dataset](/reference/ecs-data_stream.md#field-data-stream-dataset) [beta] | ![relation](https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat "not-applicable") | Not applicable.
| | +| $$$otel-mapping-for-data-stream-namespace$$$[data_stream.namespace](/reference/ecs-data_stream.md#field-data-stream-namespace) [beta] | ![relation](https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat "not-applicable") | Not applicable.
| | +| $$$otel-mapping-for-data-stream-type$$$[data_stream.type](/reference/ecs-data_stream.md#field-data-stream-type) [beta] | ![relation](https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat "not-applicable") | Not applicable.
| $$$otel-mapping-namespace-destination$$$ | +| **Destination Fields** | +| $$$otel-mapping-for-destination-address$$$[destination.address](/reference/ecs-destination.md#field-destination-address) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [destination.address](https://opentelemetry.io/docs/specs/semconv/attributes-registry/destination/#destination-address)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-destination-port$$$[destination.port](/reference/ecs-destination.md#field-destination-port) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [destination.port](https://opentelemetry.io/docs/specs/semconv/attributes-registry/destination/#destination-port)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental")
$$$otel-mapping-namespace-device$$$ | +| **Device Fields** | +| $$$otel-mapping-for-device-id$$$[device.id](/reference/ecs-device.md#field-device-id) [beta] | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [device.id](https://opentelemetry.io/docs/specs/semconv/attributes-registry/device/#device-id)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-device-manufacturer$$$[device.manufacturer](/reference/ecs-device.md#field-device-manufacturer) [beta] | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [device.manufacturer](https://opentelemetry.io/docs/specs/semconv/attributes-registry/device/#device-manufacturer)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-device-model-identifier$$$[device.model.identifier](/reference/ecs-device.md#field-device-model-identifier) [beta] | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [device.model.identifier](https://opentelemetry.io/docs/specs/semconv/attributes-registry/device/#device-model-identifier)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-device-model-name$$$[device.model.name](/reference/ecs-device.md#field-device-model-name) [beta] | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [device.model.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/device/#device-model-name)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental")
$$$otel-mapping-namespace-dns$$$ | +| **DNS Fields** | +| $$$otel-mapping-for-dns-question-name$$$[dns.question.name](/reference/ecs-dns.md#field-dns-question-name) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [dns.question.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/dns/#dns-question-name)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental")
$$$otel-mapping-namespace-ecs$$$ | +| **ECS Fields** | +| $$$otel-mapping-for-ecs-version$$$[ecs.version](/reference/ecs-ecs.md#field-ecs-version) | ![relation](https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat "not-applicable") | Not applicable.
| $$$otel-mapping-namespace-error$$$ | +| **Error Fields** | +| $$$otel-mapping-for-error-message$$$[error.message](/reference/ecs-error.md#field-error-message) | ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") | [exception.message](https://opentelemetry.io/docs/specs/semconv/attributes-registry/exception/#exception-message)
| ![stable](https://img.shields.io/badge/✔-93c93e?style=flat "stable") | +| $$$otel-mapping-for-error-stack-trace$$$[error.stack_trace](/reference/ecs-error.md#field-error-stack-trace) | ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") | [exception.stacktrace](https://opentelemetry.io/docs/specs/semconv/attributes-registry/exception/#exception-stacktrace)
| ![stable](https://img.shields.io/badge/✔-93c93e?style=flat "stable") | +| $$$otel-mapping-for-error-type$$$[error.type](/reference/ecs-error.md#field-error-type) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [error.type](https://opentelemetry.io/docs/specs/semconv/attributes-registry/error/#error-type)
| ![stable](https://img.shields.io/badge/✔-93c93e?style=flat "stable")
$$$otel-mapping-namespace-faas$$$ | +| **FaaS Fields** | +| $$$otel-mapping-for-faas-coldstart$$$[faas.coldstart](/reference/ecs-faas.md#field-faas-coldstart) [beta] | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [faas.coldstart](https://opentelemetry.io/docs/specs/semconv/attributes-registry/faas/#faas-coldstart)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-faas-execution$$$[faas.execution](/reference/ecs-faas.md#field-faas-execution) [beta] | ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") | [faas.invocation_id](https://opentelemetry.io/docs/specs/semconv/attributes-registry/faas/#faas-invocation-id)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-faas-name$$$[faas.name](/reference/ecs-faas.md#field-faas-name) [beta] | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [faas.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/faas/#faas-name)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-faas-trigger-type$$$[faas.trigger.type](/reference/ecs-faas.md#field-faas-trigger-type) [beta] | ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") | [faas.trigger](https://opentelemetry.io/docs/specs/semconv/attributes-registry/faas/#faas-trigger)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-faas-version$$$[faas.version](/reference/ecs-faas.md#field-faas-version) [beta] | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [faas.version](https://opentelemetry.io/docs/specs/semconv/attributes-registry/faas/#faas-version)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental")
$$$otel-mapping-namespace-file$$$ | +| **File Fields** | +| $$$otel-mapping-for-file-accessed$$$[file.accessed](/reference/ecs-file.md#field-file-accessed) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [file.accessed](https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-accessed)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-file-attributes$$$[file.attributes](/reference/ecs-file.md#field-file-attributes) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [file.attributes](https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-attributes)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-file-created$$$[file.created](/reference/ecs-file.md#field-file-created) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [file.created](https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-created)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-file-ctime$$$[file.ctime](/reference/ecs-file.md#field-file-ctime) | ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") | [file.changed](https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-changed)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-file-directory$$$[file.directory](/reference/ecs-file.md#field-file-directory) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [file.directory](https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-directory)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-file-extension$$$[file.extension](/reference/ecs-file.md#field-file-extension) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [file.extension](https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-extension)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-file-fork-name$$$[file.fork_name](/reference/ecs-file.md#field-file-fork-name) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [file.fork_name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-fork-name)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-file-gid$$$[file.gid](/reference/ecs-file.md#field-file-gid) | ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") | [file.group.id](https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-group-id)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-file-group$$$[file.group](/reference/ecs-file.md#field-file-group) | ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") | [file.group.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-group-name)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-file-inode$$$[file.inode](/reference/ecs-file.md#field-file-inode) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [file.inode](https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-inode)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-file-mode$$$[file.mode](/reference/ecs-file.md#field-file-mode) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [file.mode](https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-mode)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-file-mtime$$$[file.mtime](/reference/ecs-file.md#field-file-mtime) | ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") | [file.modified](https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-modified)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-file-name$$$[file.name](/reference/ecs-file.md#field-file-name) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [file.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-name)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-file-owner$$$[file.owner](/reference/ecs-file.md#field-file-owner) | ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") | [file.owner.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-owner-name)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-file-path$$$[file.path](/reference/ecs-file.md#field-file-path) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [file.path](https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-path)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-file-size$$$[file.size](/reference/ecs-file.md#field-file-size) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [file.size](https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-size)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-file-target-path$$$[file.target_path](/reference/ecs-file.md#field-file-target-path) | ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") | [file.symbolic_link.target_path](https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-symbolic-link-target-path)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-file-uid$$$[file.uid](/reference/ecs-file.md#field-file-uid) | ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") | [file.owner.id](https://opentelemetry.io/docs/specs/semconv/attributes-registry/file/#file-owner-id)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental")
$$$otel-mapping-namespace-geo$$$ | +| **Geo Fields** | +| $$$otel-mapping-for-geo-city-name$$$[geo.city_name](/reference/ecs-geo.md#field-geo-city-name) | ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") | [geo.locality.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/geo/#geo-locality-name)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-geo-continent-code$$$[geo.continent_code](/reference/ecs-geo.md#field-geo-continent-code) | ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") | [geo.continent.code](https://opentelemetry.io/docs/specs/semconv/attributes-registry/geo/#geo-continent-code)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-geo-country-iso-code$$$[geo.country_iso_code](/reference/ecs-geo.md#field-geo-country-iso-code) | ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") | [geo.country.iso_code](https://opentelemetry.io/docs/specs/semconv/attributes-registry/geo/#geo-country-iso-code)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-geo-location$$$[geo.location](/reference/ecs-geo.md#field-geo-location) | ![relation](https://img.shields.io/badge/related-efc20d?style=flat "related") | [geo.location.lat](https://opentelemetry.io/docs/specs/semconv/attributes-registry/geo/#geo-location-lat)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| ![relation](https://img.shields.io/badge/related-efc20d?style=flat "related") | [geo.location.lon](https://opentelemetry.io/docs/specs/semconv/attributes-registry/geo/#geo-location-lon)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-geo-postal-code$$$[geo.postal_code](/reference/ecs-geo.md#field-geo-postal-code) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [geo.postal_code](https://opentelemetry.io/docs/specs/semconv/attributes-registry/geo/#geo-postal-code)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-geo-region-iso-code$$$[geo.region_iso_code](/reference/ecs-geo.md#field-geo-region-iso-code) | ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") | [geo.region.iso_code](https://opentelemetry.io/docs/specs/semconv/attributes-registry/geo/#geo-region-iso-code)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental")
$$$otel-mapping-namespace-host$$$ | +| **Host Fields** | +| $$$otel-mapping-for-host-architecture$$$[host.architecture](/reference/ecs-host.md#field-host-architecture) | ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") | [host.arch](https://opentelemetry.io/docs/specs/semconv/attributes-registry/host/#host-arch)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-host-cpu-usage$$$[host.cpu.usage](/reference/ecs-host.md#field-host-cpu-usage) | ![relation](https://img.shields.io/badge/metric-cb00cb?style=flat "metric") | [system.cpu.utilization](https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--+semconv+metric.system.cpu.utilization+--%3E%22&type=code)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-host-disk-read-bytes$$$[host.disk.read.bytes](/reference/ecs-host.md#field-host-disk-read-bytes) | ![relation](https://img.shields.io/badge/metric-cb00cb?style=flat "metric") | [system.disk.io](https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--+semconv+metric.system.disk.io+--%3E%22&type=code)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-host-disk-write-bytes$$$[host.disk.write.bytes](/reference/ecs-host.md#field-host-disk-write-bytes) | ![relation](https://img.shields.io/badge/metric-cb00cb?style=flat "metric") | [system.disk.io](https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--+semconv+metric.system.disk.io+--%3E%22&type=code)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-host-id$$$[host.id](/reference/ecs-host.md#field-host-id) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [host.id](https://opentelemetry.io/docs/specs/semconv/attributes-registry/host/#host-id)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-host-ip$$$[host.ip](/reference/ecs-host.md#field-host-ip) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [host.ip](https://opentelemetry.io/docs/specs/semconv/attributes-registry/host/#host-ip)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-host-mac$$$[host.mac](/reference/ecs-host.md#field-host-mac) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [host.mac](https://opentelemetry.io/docs/specs/semconv/attributes-registry/host/#host-mac)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-host-name$$$[host.name](/reference/ecs-host.md#field-host-name) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [host.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/host/#host-name)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-host-network-egress-bytes$$$[host.network.egress.bytes](/reference/ecs-host.md#field-host-network-egress-bytes) | ![relation](https://img.shields.io/badge/metric-cb00cb?style=flat "metric") | [system.network.io](https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--+semconv+metric.system.network.io+--%3E%22&type=code)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-host-network-egress-packets$$$[host.network.egress.packets](/reference/ecs-host.md#field-host-network-egress-packets) | ![relation](https://img.shields.io/badge/metric-cb00cb?style=flat "metric") | [system.network.packets](https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--+semconv+metric.system.network.packets+--%3E%22&type=code)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-host-network-ingress-bytes$$$[host.network.ingress.bytes](/reference/ecs-host.md#field-host-network-ingress-bytes) | ![relation](https://img.shields.io/badge/metric-cb00cb?style=flat "metric") | [system.network.io](https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--+semconv+metric.system.network.io+--%3E%22&type=code)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-host-network-ingress-packets$$$[host.network.ingress.packets](/reference/ecs-host.md#field-host-network-ingress-packets) | ![relation](https://img.shields.io/badge/metric-cb00cb?style=flat "metric") | [system.network.packets](https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--+semconv+metric.system.network.packets+--%3E%22&type=code)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-host-type$$$[host.type](/reference/ecs-host.md#field-host-type) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [host.type](https://opentelemetry.io/docs/specs/semconv/attributes-registry/host/#host-type)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-host-uptime$$$[host.uptime](/reference/ecs-host.md#field-host-uptime) | ![relation](https://img.shields.io/badge/metric-cb00cb?style=flat "metric") | [system.uptime](https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--+semconv+metric.system.uptime+--%3E%22&type=code)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental")
$$$otel-mapping-namespace-http$$$ | +| **HTTP Fields** | +| $$$otel-mapping-for-http-request-body-bytes$$$[http.request.body.bytes](/reference/ecs-http.md#field-http-request-body-bytes) | ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") | [http.request.body.size](https://opentelemetry.io/docs/specs/semconv/attributes-registry/http/#http-request-body-size)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-http-request-bytes$$$[http.request.bytes](/reference/ecs-http.md#field-http-request-bytes) | ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") | [http.request.size](https://opentelemetry.io/docs/specs/semconv/attributes-registry/http/#http-request-size)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-http-request-method$$$[http.request.method](/reference/ecs-http.md#field-http-request-method) | ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") | [http.request.method_original](https://opentelemetry.io/docs/specs/semconv/attributes-registry/http/#http-request-method-original)
| ![stable](https://img.shields.io/badge/✔-93c93e?style=flat "stable") | +| ![relation](https://img.shields.io/badge/conflict-910000?style=flat "conflict") | [http.request.method](https://opentelemetry.io/docs/specs/semconv/attributes-registry/http/#http-request-method)

Note: `http.request.method` in SemConv is the known, normalized, upper case value of the request method, other than the ECS' `http.request.method` that retains casing from the original event. | ![stable](https://img.shields.io/badge/✔-93c93e?style=flat "stable") | +| $$$otel-mapping-for-http-response-body-bytes$$$[http.response.body.bytes](/reference/ecs-http.md#field-http-response-body-bytes) | ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") | [http.response.body.size](https://opentelemetry.io/docs/specs/semconv/attributes-registry/http/#http-response-body-size)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-http-response-bytes$$$[http.response.bytes](/reference/ecs-http.md#field-http-response-bytes) | ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") | [http.response.size](https://opentelemetry.io/docs/specs/semconv/attributes-registry/http/#http-response-size)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-http-response-status-code$$$[http.response.status_code](/reference/ecs-http.md#field-http-response-status-code) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [http.response.status_code](https://opentelemetry.io/docs/specs/semconv/attributes-registry/http/#http-response-status-code)
| ![stable](https://img.shields.io/badge/✔-93c93e?style=flat "stable") | +| $$$otel-mapping-for-http-version$$$[http.version](/reference/ecs-http.md#field-http-version) | ![relation](https://img.shields.io/badge/related-efc20d?style=flat "related") | [network.protocol.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/network/#network-protocol-name)
| ![stable](https://img.shields.io/badge/✔-93c93e?style=flat "stable") | +| ![relation](https://img.shields.io/badge/related-efc20d?style=flat "related") | [network.protocol.version](https://opentelemetry.io/docs/specs/semconv/attributes-registry/network/#network-protocol-version)

Note: In OTel SemConv, `network.protocol.version` specifies the HTTP version if the value of `network.protocol.name` is `http`. | ![stable](https://img.shields.io/badge/✔-93c93e?style=flat "stable")
$$$otel-mapping-namespace-log$$$ | +| **Log Fields** | +| $$$otel-mapping-for-log-file-path$$$[log.file.path](/reference/ecs-log.md#field-log-file-path) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [log.file.path](https://opentelemetry.io/docs/specs/semconv/attributes-registry/log/#log-file-path)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-log-level$$$[log.level](/reference/ecs-log.md#field-log-level) | ![relation](https://img.shields.io/badge/OTLP-ffdcb2?style=flat "otlp") | [severity_text](https://github.com/search?q=repo%3Aopen-telemetry%2Fopentelemetry-proto+%22+severity_text+%22&type=code)
| ![stable](https://img.shields.io/badge/✔-93c93e?style=flat "stable")
$$$otel-mapping-namespace-network$$$ | +| **Network Fields** | +| $$$otel-mapping-for-network-protocol$$$[network.protocol](/reference/ecs-network.md#field-network-protocol) | ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") | [network.protocol.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/network/#network-protocol-name)
| ![stable](https://img.shields.io/badge/✔-93c93e?style=flat "stable") | +| $$$otel-mapping-for-network-transport$$$[network.transport](/reference/ecs-network.md#field-network-transport) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [network.transport](https://opentelemetry.io/docs/specs/semconv/attributes-registry/network/#network-transport)
| ![stable](https://img.shields.io/badge/✔-93c93e?style=flat "stable") | +| $$$otel-mapping-for-network-type$$$[network.type](/reference/ecs-network.md#field-network-type) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [network.type](https://opentelemetry.io/docs/specs/semconv/attributes-registry/network/#network-type)
| ![stable](https://img.shields.io/badge/✔-93c93e?style=flat "stable")
$$$otel-mapping-namespace-os$$$ | +| **Operating System Fields** | +| $$$otel-mapping-for-os-full$$$[os.full](/reference/ecs-os.md#field-os-full) | ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") | [os.description](https://opentelemetry.io/docs/specs/semconv/attributes-registry/os/#os-description)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-os-name$$$[os.name](/reference/ecs-os.md#field-os-name) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [os.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/os/#os-name)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-os-type$$$[os.type](/reference/ecs-os.md#field-os-type) | ![relation](https://img.shields.io/badge/conflict-910000?style=flat "conflict") | [os.type](https://opentelemetry.io/docs/specs/semconv/attributes-registry/os/#os-type)

Note: The expected values in ECS for `os.type` do not correspond with the values defined in semantic conventions! | ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-os-version$$$[os.version](/reference/ecs-os.md#field-os-version) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [os.version](https://opentelemetry.io/docs/specs/semconv/attributes-registry/os/#os-version)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental")
$$$otel-mapping-namespace-process$$$ | +| **Process Fields** | +| $$$otel-mapping-for-process-args$$$[process.args](/reference/ecs-process.md#field-process-args) | ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") | [process.command_args](https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-command-args)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-process-args-count$$$[process.args_count](/reference/ecs-process.md#field-process-args-count) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [process.args_count](https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-args-count)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-process-command-line$$$[process.command_line](/reference/ecs-process.md#field-process-command-line) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [process.command_line](https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-command-line)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-process-executable$$$[process.executable](/reference/ecs-process.md#field-process-executable) | ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") | [process.executable.path](https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-executable-path)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| process.real_user.id | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [process.real_user.id](https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-real-user-id)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| process.saved_user.id | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [process.saved_user.id](https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-saved-user-id)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| process.user.id | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [process.user.id](https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-user-id)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-process-interactive$$$[process.interactive](/reference/ecs-process.md#field-process-interactive) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [process.interactive](https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-interactive)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| process.real_user.name | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [process.real_user.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-real-user-name)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| process.saved_user.name | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [process.saved_user.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-saved-user-name)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| process.user.name | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [process.user.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-user-name)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| process.group_leader.pid | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [process.group_leader.pid](https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-group-leader-pid)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-process-pid$$$[process.pid](/reference/ecs-process.md#field-process-pid) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [process.pid](https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-pid)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| process.session_leader.pid | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [process.session_leader.pid](https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-session-leader-pid)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-process-title$$$[process.title](/reference/ecs-process.md#field-process-title) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [process.title](https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-title)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-process-uptime$$$[process.uptime](/reference/ecs-process.md#field-process-uptime) | ![relation](https://img.shields.io/badge/metric-cb00cb?style=flat "metric") | [process.uptime](https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--+semconv+metric.process.uptime+--%3E%22&type=code)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-process-vpid$$$[process.vpid](/reference/ecs-process.md#field-process-vpid) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [process.vpid](https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-vpid)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-process-working-directory$$$[process.working_directory](/reference/ecs-process.md#field-process-working-directory) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [process.working_directory](https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-working-directory)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental")
$$$otel-mapping-namespace-related$$$ | +| **Related Fields** | +| $$$otel-mapping-for-related-hash$$$[related.hash](/reference/ecs-related.md#field-related-hash) | ![relation](https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat "not-applicable") | Not applicable.
| | +| $$$otel-mapping-for-related-hosts$$$[related.hosts](/reference/ecs-related.md#field-related-hosts) | ![relation](https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat "not-applicable") | Not applicable.
| | +| $$$otel-mapping-for-related-ip$$$[related.ip](/reference/ecs-related.md#field-related-ip) | ![relation](https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat "not-applicable") | Not applicable.
| | +| $$$otel-mapping-for-related-user$$$[related.user](/reference/ecs-related.md#field-related-user) | ![relation](https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat "not-applicable") | Not applicable.
| $$$otel-mapping-namespace-server$$$ | +| **Server Fields** | +| $$$otel-mapping-for-server-address$$$[server.address](/reference/ecs-server.md#field-server-address) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [server.address](https://opentelemetry.io/docs/specs/semconv/attributes-registry/server/#server-address)
| ![stable](https://img.shields.io/badge/✔-93c93e?style=flat "stable") | +| $$$otel-mapping-for-server-port$$$[server.port](/reference/ecs-server.md#field-server-port) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [server.port](https://opentelemetry.io/docs/specs/semconv/attributes-registry/server/#server-port)
| ![stable](https://img.shields.io/badge/✔-93c93e?style=flat "stable")
$$$otel-mapping-namespace-service$$$ | +| **Service Fields** | +| $$$otel-mapping-for-service-environment$$$[service.environment](/reference/ecs-service.md#field-service-environment) [beta] | ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") | [deployment.environment.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/deployment/#deployment-environment-name)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-service-name$$$[service.name](/reference/ecs-service.md#field-service-name) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [service.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/service/#service-name)
| ![stable](https://img.shields.io/badge/✔-93c93e?style=flat "stable") | +| $$$otel-mapping-for-service-node-name$$$[service.node.name](/reference/ecs-service.md#field-service-node-name) | ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") | [service.instance.id](https://opentelemetry.io/docs/specs/semconv/attributes-registry/service/#service-instance-id)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-service-version$$$[service.version](/reference/ecs-service.md#field-service-version) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [service.version](https://opentelemetry.io/docs/specs/semconv/attributes-registry/service/#service-version)
| ![stable](https://img.shields.io/badge/✔-93c93e?style=flat "stable")
$$$otel-mapping-namespace-source$$$ | +| **Source Fields** | +| $$$otel-mapping-for-source-address$$$[source.address](/reference/ecs-source.md#field-source-address) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [source.address](https://opentelemetry.io/docs/specs/semconv/attributes-registry/source/#source-address)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-source-port$$$[source.port](/reference/ecs-source.md#field-source-port) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [source.port](https://opentelemetry.io/docs/specs/semconv/attributes-registry/source/#source-port)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental")
$$$otel-mapping-namespace-tls$$$ | +| **TLS Fields** | +| $$$otel-mapping-for-tls-cipher$$$[tls.cipher](/reference/ecs-tls.md#field-tls-cipher) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [tls.cipher](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-cipher)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-tls-client-certificate$$$[tls.client.certificate](/reference/ecs-tls.md#field-tls-client-certificate) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [tls.client.certificate](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-certificate)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-tls-client-certificate-chain$$$[tls.client.certificate_chain](/reference/ecs-tls.md#field-tls-client-certificate-chain) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [tls.client.certificate_chain](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-certificate-chain)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-tls-client-hash-md5$$$[tls.client.hash.md5](/reference/ecs-tls.md#field-tls-client-hash-md5) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [tls.client.hash.md5](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-hash-md5)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-tls-client-hash-sha1$$$[tls.client.hash.sha1](/reference/ecs-tls.md#field-tls-client-hash-sha1) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [tls.client.hash.sha1](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-hash-sha1)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-tls-client-hash-sha256$$$[tls.client.hash.sha256](/reference/ecs-tls.md#field-tls-client-hash-sha256) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [tls.client.hash.sha256](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-hash-sha256)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-tls-client-issuer$$$[tls.client.issuer](/reference/ecs-tls.md#field-tls-client-issuer) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [tls.client.issuer](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-issuer)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-tls-client-ja3$$$[tls.client.ja3](/reference/ecs-tls.md#field-tls-client-ja3) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [tls.client.ja3](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-ja3)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-tls-client-not-after$$$[tls.client.not_after](/reference/ecs-tls.md#field-tls-client-not-after) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [tls.client.not_after](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-not-after)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-tls-client-not-before$$$[tls.client.not_before](/reference/ecs-tls.md#field-tls-client-not-before) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [tls.client.not_before](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-not-before)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-tls-client-subject$$$[tls.client.subject](/reference/ecs-tls.md#field-tls-client-subject) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [tls.client.subject](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-subject)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-tls-client-supported-ciphers$$$[tls.client.supported_ciphers](/reference/ecs-tls.md#field-tls-client-supported-ciphers) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [tls.client.supported_ciphers](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-supported-ciphers)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-tls-curve$$$[tls.curve](/reference/ecs-tls.md#field-tls-curve) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [tls.curve](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-curve)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-tls-established$$$[tls.established](/reference/ecs-tls.md#field-tls-established) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [tls.established](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-established)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-tls-next-protocol$$$[tls.next_protocol](/reference/ecs-tls.md#field-tls-next-protocol) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [tls.next_protocol](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-next-protocol)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-tls-resumed$$$[tls.resumed](/reference/ecs-tls.md#field-tls-resumed) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [tls.resumed](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-resumed)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-tls-server-certificate$$$[tls.server.certificate](/reference/ecs-tls.md#field-tls-server-certificate) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [tls.server.certificate](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-certificate)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-tls-server-certificate-chain$$$[tls.server.certificate_chain](/reference/ecs-tls.md#field-tls-server-certificate-chain) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [tls.server.certificate_chain](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-certificate-chain)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-tls-server-hash-md5$$$[tls.server.hash.md5](/reference/ecs-tls.md#field-tls-server-hash-md5) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [tls.server.hash.md5](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-hash-md5)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-tls-server-hash-sha1$$$[tls.server.hash.sha1](/reference/ecs-tls.md#field-tls-server-hash-sha1) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [tls.server.hash.sha1](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-hash-sha1)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-tls-server-hash-sha256$$$[tls.server.hash.sha256](/reference/ecs-tls.md#field-tls-server-hash-sha256) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [tls.server.hash.sha256](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-hash-sha256)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-tls-server-issuer$$$[tls.server.issuer](/reference/ecs-tls.md#field-tls-server-issuer) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [tls.server.issuer](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-issuer)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-tls-server-ja3s$$$[tls.server.ja3s](/reference/ecs-tls.md#field-tls-server-ja3s) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [tls.server.ja3s](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-ja3s)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-tls-server-not-after$$$[tls.server.not_after](/reference/ecs-tls.md#field-tls-server-not-after) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [tls.server.not_after](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-not-after)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-tls-server-not-before$$$[tls.server.not_before](/reference/ecs-tls.md#field-tls-server-not-before) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [tls.server.not_before](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-not-before)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-tls-server-subject$$$[tls.server.subject](/reference/ecs-tls.md#field-tls-server-subject) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [tls.server.subject](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-subject)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental")
$$$otel-mapping-namespace-tracing$$$ | +| **Tracing Fields** | +| $$$otel-mapping-for-span-id$$$[span.id](/reference/ecs-tracing.md#field-span-id) | ![relation](https://img.shields.io/badge/OTLP-ffdcb2?style=flat "otlp") | [span_id](https://github.com/search?q=repo%3Aopen-telemetry%2Fopentelemetry-proto+%22+span_id+%22&type=code)
| ![stable](https://img.shields.io/badge/✔-93c93e?style=flat "stable") | +| $$$otel-mapping-for-trace-id$$$[trace.id](/reference/ecs-tracing.md#field-trace-id) | ![relation](https://img.shields.io/badge/OTLP-ffdcb2?style=flat "otlp") | [trace_id](https://github.com/search?q=repo%3Aopen-telemetry%2Fopentelemetry-proto+%22+trace_id+%22&type=code)
| ![stable](https://img.shields.io/badge/✔-93c93e?style=flat "stable") | +| $$$otel-mapping-for-transaction-id$$$[transaction.id](/reference/ecs-tracing.md#field-transaction-id) | ![relation](https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat "not-applicable") | Not applicable.
| $$$otel-mapping-namespace-url$$$ | +| **URL Fields** | +| $$$otel-mapping-for-url-domain$$$[url.domain](/reference/ecs-url.md#field-url-domain) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [url.domain](https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-domain)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-url-extension$$$[url.extension](/reference/ecs-url.md#field-url-extension) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [url.extension](https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-extension)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-url-fragment$$$[url.fragment](/reference/ecs-url.md#field-url-fragment) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [url.fragment](https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-fragment)
| ![stable](https://img.shields.io/badge/✔-93c93e?style=flat "stable") | +| $$$otel-mapping-for-url-full$$$[url.full](/reference/ecs-url.md#field-url-full) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [url.full](https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-full)
| ![stable](https://img.shields.io/badge/✔-93c93e?style=flat "stable") | +| $$$otel-mapping-for-url-original$$$[url.original](/reference/ecs-url.md#field-url-original) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [url.original](https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-original)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-url-path$$$[url.path](/reference/ecs-url.md#field-url-path) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [url.path](https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-path)
| ![stable](https://img.shields.io/badge/✔-93c93e?style=flat "stable") | +| $$$otel-mapping-for-url-port$$$[url.port](/reference/ecs-url.md#field-url-port) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [url.port](https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-port)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-url-query$$$[url.query](/reference/ecs-url.md#field-url-query) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [url.query](https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-query)
| ![stable](https://img.shields.io/badge/✔-93c93e?style=flat "stable") | +| $$$otel-mapping-for-url-registered-domain$$$[url.registered_domain](/reference/ecs-url.md#field-url-registered-domain) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [url.registered_domain](https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-registered-domain)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-url-scheme$$$[url.scheme](/reference/ecs-url.md#field-url-scheme) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [url.scheme](https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-scheme)
| ![stable](https://img.shields.io/badge/✔-93c93e?style=flat "stable") | +| $$$otel-mapping-for-url-subdomain$$$[url.subdomain](/reference/ecs-url.md#field-url-subdomain) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [url.subdomain](https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-subdomain)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-url-top-level-domain$$$[url.top_level_domain](/reference/ecs-url.md#field-url-top-level-domain) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [url.top_level_domain](https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-top-level-domain)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental")
$$$otel-mapping-namespace-user$$$ | +| **User Fields** | +| $$$otel-mapping-for-user-email$$$[user.email](/reference/ecs-user.md#field-user-email) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [user.email](https://opentelemetry.io/docs/specs/semconv/attributes-registry/user/#user-email)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-user-full-name$$$[user.full_name](/reference/ecs-user.md#field-user-full-name) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [user.full_name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/user/#user-full-name)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-user-hash$$$[user.hash](/reference/ecs-user.md#field-user-hash) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [user.hash](https://opentelemetry.io/docs/specs/semconv/attributes-registry/user/#user-hash)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-user-id$$$[user.id](/reference/ecs-user.md#field-user-id) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [user.id](https://opentelemetry.io/docs/specs/semconv/attributes-registry/user/#user-id)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-user-name$$$[user.name](/reference/ecs-user.md#field-user-name) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [user.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/user/#user-name)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-user-roles$$$[user.roles](/reference/ecs-user.md#field-user-roles) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [user.roles](https://opentelemetry.io/docs/specs/semconv/attributes-registry/user/#user-roles)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental")
$$$otel-mapping-namespace-user_agent$$$ | +| **User agent Fields** | +| $$$otel-mapping-for-user-agent-name$$$[user_agent.name](/reference/ecs-user_agent.md#field-user-agent-name) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [user_agent.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/user_agent/#user-agent-name)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | +| $$$otel-mapping-for-user-agent-original$$$[user_agent.original](/reference/ecs-user_agent.md#field-user-agent-original) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [user_agent.original](https://opentelemetry.io/docs/specs/semconv/attributes-registry/user_agent/#user-agent-original)
| ![stable](https://img.shields.io/badge/✔-93c93e?style=flat "stable") | +| $$$otel-mapping-for-user-agent-version$$$[user_agent.version](/reference/ecs-user_agent.md#field-user-agent-version) | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | [user_agent.version](https://opentelemetry.io/docs/specs/semconv/attributes-registry/user_agent/#user-agent-version)
| ![experimental](https://img.shields.io/badge/✘-fed10c?style=flat "experimental") | diff --git a/docs/reference/ecs-otel-alignment-overview.md b/docs/reference/ecs-otel-alignment-overview.md new file mode 100644 index 0000000000..71995d9afd --- /dev/null +++ b/docs/reference/ecs-otel-alignment-overview.md @@ -0,0 +1,109 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/master/ecs-otel-alignment-overview.html +--- + +# OTel Alignment Overview [ecs-otel-alignment-overview] + +The following table summarizes the alignment status by namespaces between ECS in version `9.1.0-dev` and OpenTelemetry semantic conventions in version `1.29.0`. + +| | | +| --- | --- | +| | Number of Fields / Attributes | +| Namespace | ECS | OTel | ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") | ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") | ![relation](https://img.shields.io/badge/related-efc20d?style=flat "related") | ![relation](https://img.shields.io/badge/conflict-910000?style=flat "conflict") | ![relation](https://img.shields.io/badge/metric-cb00cb?style=flat "metric") | ![relation](https://img.shields.io/badge/OTLP-ffdcb2?style=flat "OTLP") | ![relation](https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat "na") | +| Agent | [6](/reference/ecs-agent.md) | · | · | · | · | · | · | · | · | +| Android | · | [1](https://opentelemetry.io/docs/specs/semconv/attributes-registry/android) | · | · | · | · | · | · | | +| artifact | · | [7](https://opentelemetry.io/docs/specs/semconv/attributes-registry/artifact) | · | · | · | · | · | · | | +| Autonomous System | [2](/reference/ecs-as.md) | · | · | · | · | · | · | · | · | +| ASP.NET Core | · | [7](https://opentelemetry.io/docs/specs/semconv/attributes-registry/aspnetcore) | · | · | · | · | · | · | | +| General AWS | · | [42](https://opentelemetry.io/docs/specs/semconv/attributes-registry/aws) | · | · | · | · | · | · | | +| az | · | [2](https://opentelemetry.io/docs/specs/semconv/attributes-registry/az) | · | · | · | · | · | · | | +| Base | [4](/reference/ecs-base.md) | · | · | · | 2 | · | · | 4 | · | +| Browser | · | [4](https://opentelemetry.io/docs/specs/semconv/attributes-registry/browser) | · | · | · | · | · | · | | +| cicd | · | [6](https://opentelemetry.io/docs/specs/semconv/attributes-registry/cicd) | · | · | · | · | · | · | | +| Client | [12](/reference/ecs-client.md) | [2](https://opentelemetry.io/docs/specs/semconv/attributes-registry/client) | 2 | · | · | · | · | · | · | +| Cloud | [11](/reference/ecs-cloud.md) | [6](https://opentelemetry.io/docs/specs/semconv/attributes-registry/cloud) | 4 | 1 | · | · | · | · | · | +| CloudEvents | · | [5](https://opentelemetry.io/docs/specs/semconv/attributes-registry/cloudevents) | · | · | · | · | · | · | | +| cloudfoundry | · | [11](https://opentelemetry.io/docs/specs/semconv/attributes-registry/cloudfoundry) | · | · | · | · | · | · | | +| Code | · | [6](https://opentelemetry.io/docs/specs/semconv/attributes-registry/code) | · | · | · | · | · | · | | +| Code Signature | [10](/reference/ecs-code_signature.md) | · | · | · | · | · | · | · | · | +| Container | [14](/reference/ecs-container.md) | [13](https://opentelemetry.io/docs/specs/semconv/attributes-registry/container) | 4 | 2 | 1 | · | 2 | · | · | +| CPU | · | [1](https://opentelemetry.io/docs/specs/semconv/attributes-registry/cpu) | · | · | · | · | · | · | | +| Data Stream | [3](/reference/ecs-data_stream.md) | · | · | · | · | · | · | · | 3 | +| General Database | · | [27](https://opentelemetry.io/docs/specs/semconv/attributes-registry/db) | · | · | · | · | · | · | | +| Deployment | · | [4](https://opentelemetry.io/docs/specs/semconv/attributes-registry/deployment) | · | · | · | · | · | · | | +| Destination | [12](/reference/ecs-destination.md) | [2](https://opentelemetry.io/docs/specs/semconv/attributes-registry/destination) | 2 | · | · | · | · | · | · | +| Device | [5](/reference/ecs-device.md) | [4](https://opentelemetry.io/docs/specs/semconv/attributes-registry/device) | 4 | · | · | · | · | · | · | +| Disk | · | [1](https://opentelemetry.io/docs/specs/semconv/attributes-registry/disk) | · | · | · | · | · | · | | +| DLL | [2](/reference/ecs-dll.md) | · | · | · | · | · | · | · | · | +| DNS | [18](/reference/ecs-dns.md) | [1](https://opentelemetry.io/docs/specs/semconv/attributes-registry/dns) | 1 | · | · | · | · | · | · | +| .NET | · | [1](https://opentelemetry.io/docs/specs/semconv/attributes-registry/dotnet) | · | · | · | · | · | · | | +| ECS | [1](/reference/ecs-ecs.md) | · | · | · | · | · | · | · | 1 | +| ELF Header | [38](/reference/ecs-elf.md) | · | · | · | · | · | · | · | · | +| Email | [19](/reference/ecs-email.md) | · | · | · | · | · | · | · | · | +| Error | [5](/reference/ecs-error.md) | [1](https://opentelemetry.io/docs/specs/semconv/attributes-registry/error) | 1 | 2 | · | · | · | · | · | +| Event | [26](/reference/ecs-event.md) | [1](https://opentelemetry.io/docs/specs/semconv/attributes-registry/event) | · | · | · | · | · | · | · | +| Exception | · | [4](https://opentelemetry.io/docs/specs/semconv/attributes-registry/exception) | · | · | · | · | · | · | | +| FaaS | [7](/reference/ecs-faas.md) | [16](https://opentelemetry.io/docs/specs/semconv/attributes-registry/faas) | 3 | 2 | · | · | · | · | · | +| Feature Flag | · | [8](https://opentelemetry.io/docs/specs/semconv/attributes-registry/feature-flag) | · | · | · | · | · | · | | +| File | [22](/reference/ecs-file.md) | [18](https://opentelemetry.io/docs/specs/semconv/attributes-registry/file) | 11 | 7 | · | · | · | · | · | +| GCP Client | · | [5](https://opentelemetry.io/docs/specs/semconv/attributes-registry/gcp) | · | · | · | · | · | · | | +| GenAI | · | [22](https://opentelemetry.io/docs/specs/semconv/attributes-registry/gen-ai) | · | · | · | · | · | · | | +| Geo | [11](/reference/ecs-geo.md) | [7](https://opentelemetry.io/docs/specs/semconv/attributes-registry/geo) | 1 | 4 | 2 | · | · | · | · | +| Go | · | [1](https://opentelemetry.io/docs/specs/semconv/attributes-registry/go) | · | · | · | · | · | · | | +| GraphQL | · | [3](https://opentelemetry.io/docs/specs/semconv/attributes-registry/graphql) | · | · | · | · | · | · | | +| Group | [3](/reference/ecs-group.md) | · | · | · | · | · | · | · | · | +| Hash | [8](/reference/ecs-hash.md) | · | · | · | · | · | · | · | · | +| Heroku | · | [3](https://opentelemetry.io/docs/specs/semconv/attributes-registry/heroku) | · | · | · | · | · | · | | +| Host | [18](/reference/ecs-host.md) | [15](https://opentelemetry.io/docs/specs/semconv/attributes-registry/host) | 5 | 1 | · | · | 8 | · | · | +| HTTP | [13](/reference/ecs-http.md) | [12](https://opentelemetry.io/docs/specs/semconv/attributes-registry/http) | 1 | 5 | 2 | 1 | · | · | · | +| hw | · | [5](https://opentelemetry.io/docs/specs/semconv/attributes-registry/hw) | · | · | · | · | · | · | | +| Interface | [3](/reference/ecs-interface.md) | · | · | · | · | · | · | · | · | +| Java Virtual Machine (JVM) | · | [7](https://opentelemetry.io/docs/specs/semconv/attributes-registry/jvm) | · | · | · | · | · | · | | +| Kubernetes | · | [26](https://opentelemetry.io/docs/specs/semconv/attributes-registry/k8s) | · | · | · | · | · | · | | +| linux | · | [1](https://opentelemetry.io/docs/specs/semconv/attributes-registry/linux) | · | · | · | · | · | · | | +| Log | [18](/reference/ecs-log.md) | [7](https://opentelemetry.io/docs/specs/semconv/attributes-registry/log) | 1 | · | · | · | · | 1 | · | +| Mach-O Header | [16](/reference/ecs-macho.md) | · | · | · | · | · | · | · | · | +| General Messaging | · | [37](https://opentelemetry.io/docs/specs/semconv/attributes-registry/messaging) | · | · | · | · | · | · | | +| Network | [12](/reference/ecs-network.md) | [16](https://opentelemetry.io/docs/specs/semconv/attributes-registry/network) | 2 | 1 | · | · | · | · | · | +| Node.js | · | [1](https://opentelemetry.io/docs/specs/semconv/attributes-registry/nodejs) | · | · | · | · | · | · | | +| Observer | [13](/reference/ecs-observer.md) | · | · | · | · | · | · | · | · | +| Open Container Initiative (OCI) | · | [1](https://opentelemetry.io/docs/specs/semconv/attributes-registry/oci) | · | · | · | · | · | · | | +| OpenTracing | · | [1](https://opentelemetry.io/docs/specs/semconv/attributes-registry/opentracing) | · | · | · | · | · | · | | +| Orchestrator | [15](/reference/ecs-orchestrator.md) | · | · | · | · | · | · | · | · | +| Organization | [2](/reference/ecs-organization.md) | · | · | · | · | · | · | · | · | +| Operating System | [7](/reference/ecs-os.md) | [5](https://opentelemetry.io/docs/specs/semconv/attributes-registry/os) | 2 | 1 | · | 1 | · | · | · | +| OTel | · | [4](https://opentelemetry.io/docs/specs/semconv/attributes-registry/otel) | · | · | · | · | · | · | | +| Package | [13](/reference/ecs-package.md) | · | · | · | · | · | · | · | · | +| PE Header | [23](/reference/ecs-pe.md) | · | · | · | · | · | · | · | · | +| Peer | · | [1](https://opentelemetry.io/docs/specs/semconv/attributes-registry/peer) | · | · | · | · | · | · | | +| Process | [37](/reference/ecs-process.md) | [33](https://opentelemetry.io/docs/specs/semconv/attributes-registry/process) | 15 | 2 | · | · | 1 | · | · | +| profile | · | [1](https://opentelemetry.io/docs/specs/semconv/attributes-registry/profile) | · | · | · | · | · | · | | +| Registry | [7](/reference/ecs-registry.md) | · | · | · | · | · | · | · | · | +| Related | [4](/reference/ecs-related.md) | · | · | · | · | · | · | · | 4 | +| Risk information | [6](/reference/ecs-risk.md) | · | · | · | · | · | · | · | · | +| Remote Procedure Call (RPC) | · | [17](https://opentelemetry.io/docs/specs/semconv/attributes-registry/rpc) | · | · | · | · | · | · | | +| Rule | [10](/reference/ecs-rule.md) | · | · | · | · | · | · | · | · | +| Server | [12](/reference/ecs-server.md) | [2](https://opentelemetry.io/docs/specs/semconv/attributes-registry/server) | 2 | · | · | · | · | · | · | +| Service | [11](/reference/ecs-service.md) | [4](https://opentelemetry.io/docs/specs/semconv/attributes-registry/service) | 2 | 2 | · | · | · | · | · | +| Session | · | [2](https://opentelemetry.io/docs/specs/semconv/attributes-registry/session) | · | · | · | · | · | · | | +| SignalR | · | [2](https://opentelemetry.io/docs/specs/semconv/attributes-registry/signalr) | · | · | · | · | · | · | | +| Source | [12](/reference/ecs-source.md) | [2](https://opentelemetry.io/docs/specs/semconv/attributes-registry/source) | 2 | · | · | · | · | · | · | +| General System | · | [12](https://opentelemetry.io/docs/specs/semconv/attributes-registry/system) | · | · | · | · | · | · | | +| Telemetry | · | [5](https://opentelemetry.io/docs/specs/semconv/attributes-registry/telemetry) | · | · | · | · | · | · | | +| test | · | [4](https://opentelemetry.io/docs/specs/semconv/attributes-registry/test) | · | · | · | · | · | · | | +| Thread | · | [2](https://opentelemetry.io/docs/specs/semconv/attributes-registry/thread) | · | · | · | · | · | · | | +| Threat | [65](/reference/ecs-threat.md) | · | · | · | · | · | · | · | · | +| TLS | [29](/reference/ecs-tls.md) | [28](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls) | 26 | · | · | · | · | · | · | +| Tracing | [3](/reference/ecs-tracing.md) | · | · | · | · | · | · | 2 | 1 | +| URL | [14](/reference/ecs-url.md) | [13](https://opentelemetry.io/docs/specs/semconv/attributes-registry/url) | 12 | · | · | · | · | · | · | +| User | [7](/reference/ecs-user.md) | [6](https://opentelemetry.io/docs/specs/semconv/attributes-registry/user) | 6 | · | · | · | · | · | · | +| User agent | [4](/reference/ecs-user_agent.md) | [4](https://opentelemetry.io/docs/specs/semconv/attributes-registry/user-agent) | 3 | · | · | · | · | · | · | +| V8 JS | · | [2](https://opentelemetry.io/docs/specs/semconv/attributes-registry/v8js) | · | · | · | · | · | · | | +| vcs | · | [13](https://opentelemetry.io/docs/specs/semconv/attributes-registry/vcs) | · | · | · | · | · | · | | +| VLAN | [2](/reference/ecs-vlan.md) | · | · | · | · | · | · | · | · | +| Volume | [16](/reference/ecs-volume.md) | · | · | · | · | · | · | · | · | +| Vulnerability | [13](/reference/ecs-vulnerability.md) | · | · | · | · | · | · | · | · | +| Web Engine | · | [3](https://opentelemetry.io/docs/specs/semconv/attributes-registry/webengine) | · | · | · | · | · | · | | +| x509 Certificate | [24](/reference/ecs-x509.md) | · | · | · | · | · | · | · | · | + diff --git a/docs/reference/ecs-package.md b/docs/reference/ecs-package.md new file mode 100644 index 0000000000..331493b21d --- /dev/null +++ b/docs/reference/ecs-package.md @@ -0,0 +1,31 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-package.html +applies_to: + stack: all + serverless: all +--- + +# Package fields [ecs-package] + +These fields contain information about an installed software package. It contains general information about a package, such as name, version or size. It also contains installation details, such as time or location. + + +## Package field details [_package_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-package-architecture$$$[package.architecture](#field-package-architecture) | Package architecture.

type: keyword

example: `x86_64`
| extended | +| $$$field-package-build-version$$$[package.build_version](#field-package-build-version) | Additional information about the build version of the installed package.

For example use the commit SHA of a non-released package.

type: keyword

example: `36f4f7e89dd61b0988b12ee000b98966867710cd`
| extended | +| $$$field-package-checksum$$$[package.checksum](#field-package-checksum) | Checksum of the installed package for verification.

type: keyword

example: `68b329da9893e34099c7d8ad5cb9c940`
| extended | +| $$$field-package-description$$$[package.description](#field-package-description) | Description of the package.

type: keyword

example: `Open source programming language to build simple/reliable/efficient software.`
| extended | +| $$$field-package-install-scope$$$[package.install_scope](#field-package-install-scope) | Indicating how the package was installed, e.g. user-local, global.

type: keyword

example: `global`
| extended | +| $$$field-package-installed$$$[package.installed](#field-package-installed) | Time when package was installed.

type: date
| extended | +| $$$field-package-license$$$[package.license](#field-package-license) | License under which the package was released.

Use a short name, e.g. the license identifier from SPDX License List where possible ([https://spdx.org/licenses/](https://spdx.org/licenses/)).

type: keyword

example: `Apache License 2.0`
| extended | +| $$$field-package-name$$$[package.name](#field-package-name) | Package name

type: keyword

example: `go`
| extended | +| $$$field-package-path$$$[package.path](#field-package-path) | Path where the package is installed.

type: keyword

example: `/usr/local/Cellar/go/1.12.9/`
| extended | +| $$$field-package-reference$$$[package.reference](#field-package-reference) | Home page or reference URL of the software in this package, if available.

type: keyword

example: `https://golang.org`
| extended | +| $$$field-package-size$$$[package.size](#field-package-size) | Package size in bytes.

type: long

example: `62231`
| extended | +| $$$field-package-type$$$[package.type](#field-package-type) | Type of package.

This should contain the package file type, rather than the package manager name. Examples: rpm, dpkg, brew, npm, gem, nupkg, jar.

type: keyword

example: `rpm`
| extended | +| $$$field-package-version$$$[package.version](#field-package-version) | Package version

type: keyword

example: `1.12.9`
| extended | + diff --git a/docs/reference/ecs-pe.md b/docs/reference/ecs-pe.md new file mode 100644 index 0000000000..303257fa5e --- /dev/null +++ b/docs/reference/ecs-pe.md @@ -0,0 +1,52 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-pe.html +applies_to: + stack: all + serverless: all +--- + +# PE header fields [ecs-pe] + +These fields contain Windows Portable Executable (PE) metadata. + + +## PE header field details [_pe_header_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-pe-architecture$$$[pe.architecture](#field-pe-architecture) | CPU architecture target for the file.

type: keyword

example: `x64`
| extended | +| $$$field-pe-company$$$[pe.company](#field-pe-company) | Internal company name of the file, provided at compile-time.

type: keyword

example: `Microsoft Corporation`
| extended | +| $$$field-pe-description$$$[pe.description](#field-pe-description) | Internal description of the file, provided at compile-time.

type: keyword

example: `Paint`
| extended | +| $$$field-pe-file-version$$$[pe.file_version](#field-pe-file-version) | Internal version of the file, provided at compile-time.

type: keyword

example: `6.3.9600.17415`
| extended | +| $$$field-pe-go-import-hash$$$[pe.go_import_hash](#field-pe-go-import-hash) | A hash of the Go language imports in a PE file excluding standard library imports. An import hash can be used to fingerprint binaries even after recompilation or other code-level transformations have occurred, which would change more traditional hash values.

The algorithm used to calculate the Go symbol hash and a reference implementation are available here: [https://github.com/elastic/toutoumomoma](https://github.com/elastic/toutoumomoma)

type: keyword

example: `10bddcb4cee42080f76c88d9ff964491`
| extended | +| $$$field-pe-go-imports$$$[pe.go_imports](#field-pe-go-imports) | List of imported Go language element names and types.

type: flattened
| extended | +| $$$field-pe-go-imports-names-entropy$$$[pe.go_imports_names_entropy](#field-pe-go-imports-names-entropy) | Shannon entropy calculation from the list of Go imports.

type: long
| extended | +| $$$field-pe-go-imports-names-var-entropy$$$[pe.go_imports_names_var_entropy](#field-pe-go-imports-names-var-entropy) | Variance for Shannon entropy calculation from the list of Go imports.

type: long
| extended | +| $$$field-pe-go-stripped$$$[pe.go_stripped](#field-pe-go-stripped) | Set to true if the file is a Go executable that has had its symbols stripped or obfuscated and false if an unobfuscated Go executable.

type: boolean
| extended | +| $$$field-pe-imphash$$$[pe.imphash](#field-pe-imphash) | A hash of the imports in a PE file. An imphash — or import hash — can be used to fingerprint binaries even after recompilation or other code-level transformations have occurred, which would change more traditional hash values.

Learn more at [https://www.fireeye.com/blog/threat-research/2014/01/tracking-malware-import-hashing.html](https://www.fireeye.com/blog/threat-research/2014/01/tracking-malware-import-hashing.html).

type: keyword

example: `0c6803c4e922103c4dca5963aad36ddf`
| extended | +| $$$field-pe-import-hash$$$[pe.import_hash](#field-pe-import-hash) | A hash of the imports in a PE file. An import hash can be used to fingerprint binaries even after recompilation or other code-level transformations have occurred, which would change more traditional hash values.

This is a synonym for imphash.

type: keyword

example: `d41d8cd98f00b204e9800998ecf8427e`
| extended | +| $$$field-pe-imports$$$[pe.imports](#field-pe-imports) | List of imported element names and types.

type: flattened

Note: this field should contain an array of values.
| extended | +| $$$field-pe-imports-names-entropy$$$[pe.imports_names_entropy](#field-pe-imports-names-entropy) | Shannon entropy calculation from the list of imported element names and types.

type: long
| extended | +| $$$field-pe-imports-names-var-entropy$$$[pe.imports_names_var_entropy](#field-pe-imports-names-var-entropy) | Variance for Shannon entropy calculation from the list of imported element names and types.

type: long
| extended | +| $$$field-pe-original-file-name$$$[pe.original_file_name](#field-pe-original-file-name) | Internal name of the file, provided at compile-time.

type: keyword

example: `MSPAINT.EXE`
| extended | +| $$$field-pe-pehash$$$[pe.pehash](#field-pe-pehash) | A hash of the PE header and data from one or more PE sections. An pehash can be used to cluster files by transforming structural information about a file into a hash value.

Learn more at [https://www.usenix.org/legacy/events/leet09/tech/full_papers/wicherski/wicherski_html/index.html](https://www.usenix.org/legacy/events/leet09/tech/full_papers/wicherski/wicherski_html/index.html).

type: keyword

example: `73ff189b63cd6be375a7ff25179a38d347651975`
| extended | +| $$$field-pe-product$$$[pe.product](#field-pe-product) | Internal product name of the file, provided at compile-time.

type: keyword

example: `Microsoft® Windows® Operating System`
| extended | +| $$$field-pe-sections$$$[pe.sections](#field-pe-sections) | An array containing an object for each section of the PE file.

The keys that should be present in these objects are defined by sub-fields underneath `pe.sections.*`.

type: nested

Note: this field should contain an array of values.
| extended | +| $$$field-pe-sections-entropy$$$[pe.sections.entropy](#field-pe-sections-entropy) | Shannon entropy calculation from the section.

type: long
| extended | +| $$$field-pe-sections-name$$$[pe.sections.name](#field-pe-sections-name) | PE Section List name.

type: keyword
| extended | +| $$$field-pe-sections-physical-size$$$[pe.sections.physical_size](#field-pe-sections-physical-size) | PE Section List physical size.

type: long
| extended | +| $$$field-pe-sections-var-entropy$$$[pe.sections.var_entropy](#field-pe-sections-var-entropy) | Variance for Shannon entropy calculation from the section.

type: long
| extended | +| $$$field-pe-sections-virtual-size$$$[pe.sections.virtual_size](#field-pe-sections-virtual-size) | PE Section List virtual size. This is always the same as `physical_size`.

type: long
| extended | + + +## Field reuse [_field_reuse_19] + +The `pe` fields are expected to be nested at: + +* `dll.pe` +* `file.pe` +* `process.pe` + +Note also that the `pe` fields are not expected to be used directly at the root of the events. + diff --git a/docs/reference/ecs-principles-design.md b/docs/reference/ecs-principles-design.md new file mode 100644 index 0000000000..90206361a3 --- /dev/null +++ b/docs/reference/ecs-principles-design.md @@ -0,0 +1,54 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-principles-design.html +applies_to: + stack: all + serverless: all +--- + +# Design principles [ecs-principles-design] + +The considerations here form the basis of ECS design. These principles help guide making the appropriate decisions for the project. + + +## A common schema [_a_common_schema] + +A goal of ECS is to maximize interoperability and reuse. When expanding the concepts represented in ECS, consider how broad or narrow the intended use cases are. + +Defining fields with narrow, lacking, or incorrect definitions limit future use. The best practice is to add the fewest fields to adequately capture an event. Adding more fields in the future is less complicated than changing or removing established ones. + +Also, avoid adding fields because a concept exists. For example, a network protocol specification may contain many features, but some are obscure and used infrequently. Finally, avoid standardization for standardization’s sake. + + +## Field sets are namespaces [_field_sets_are_namespaces] + +Field sets create independent schema sections for understanding a concept in isolation. + +Complex concepts may be better captured using nesting. A field set may contain several sub-components that make up a larger concept: `dns.question.class`, `dns.question.answer`, `dns.question.type`. + + +## Naming consistency [_naming_consistency] + +Consistent naming across the schema makes learning and memorizing field names easier. Do not limit terms with broad meaning to a single case. + +Examples: + +* Many concepts can include a `.name` or `.id` value (for example, `event.id`, `error.id`, `group.id`, `rule.id`, `user.id`). +* Several potential IP addresses in a single event (`source.ip`, `destination.ip`, `host.ip`). + + +## Reuse [_reuse] + +Introducing extra fields may seem necessary when adding or expanding a concept. But, try to use an existing field or reuse an existing field set to avoid duplicating fields. Leveraging consistent fields across event sources helps build more straightforward queries and visualizations. + +For example, imagine an app or framework produces a unique ID for each log it emits. Instead of adding a custom `.id` field specific to that app, consider the `event.id` field. + +Reusing fields simplifies capturing several entities of a type within a single event. One example, the `user.*` field set and the reuse `user.target.*` allow collecting the same detail about the acting and target users. Redefining the entire `user.*` field set is unnecessary. In limited use, consider an array of field set reuses if multiples of the same reuse need capturing. + + +## Custom fields are a feature [_custom_fields_are_a_feature] + +Many situations will need custom fields to fully capture the event contents. Users and integrations are encouraged to add custom fields to capture concepts not defined in ECS. Custom fields give users the flexibility to add fields for their internal use cases, less common concepts, and experimentation. + +Following the [best practices](/reference/ecs-custom-fields-in-ecs.md), users and integrations can create a path for future migration if a similar concept appears in ECS. + diff --git a/docs/reference/ecs-principles-implementation.md b/docs/reference/ecs-principles-implementation.md new file mode 100644 index 0000000000..600b743e86 --- /dev/null +++ b/docs/reference/ecs-principles-implementation.md @@ -0,0 +1,208 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-principles-implementation.html +applies_to: + stack: all + serverless: all +--- + +# Implementation patterns [ecs-principles-implementation] + +Normalizing data provides a more consistent view of events from various data sources. Following these conventions will help to better describe, discover, identify, and categorize events. + + +## Base fields [_base_fields] + +The group of individual fields residing outside any field set at the top-level of ECS are known as base fields. + +ECS events follow these conventions with the base fields: + +@timestamp +: All events must populate [`@timestamp`](/reference/ecs-base.md#field-timestamp) with the event’s original timestamp. + +message +: Most events should populate [`message`](/reference/ecs-base.md#field-message). + +ecs.version +: The referenced [`ecs.version`](/reference/ecs-ecs.md#field-ecs-version) used to develop the data mapping or ingest pipeline. This value helps detect when mappings update or fall behind. It can also help explain why a particular data source isn’t populating the same fields as another. + +tags and labels +: The [`tags`](/reference/ecs-base.md#field-tags) and [`labels`](/reference/ecs-base.md#field-labels) fields add simple metadata as `keyword` values. + + +## Host [_host] + +In ECS, the `host` is the computing instance where the event happened. A `host` can be a physical device, virtual machine, container, or cloud instance. + +The [`host.*`](/reference/ecs-host.md) field set contains common attributes for different computing instances. Certain host types have more fields to capture specific details, like `cloud.*` or `container.*`. + + +## Agent and observer [_agent_and_observer] + +An agent is software that collects, observes, measures, or detects the event. The [`agent.*`](/reference/ecs-agent.md) fields capture details about which agent entity captured the event, including the agent’s version. Examples of agents are Beats and Elastic Agent. + +An `observer` is an external monitoring or intermediary device, like a firewall, APM server, or web proxy. These devices monitor and detect network, security, application events. Capture the details for these device types in the [`observer.*`](/reference/ecs-observer.md) field set. + + +## Timestamps [_timestamps] + +ECS requires the `@timestamp` field on every event. Some events also contain extra timestamps to capture. + +@timestamp +: All events must populate [`@timestamp`](/reference/ecs-base.md#field-timestamp) with when the event originated. + +event.created +: The timestamp of when an agent or pipeline saw the event. + +event.ingested +: The timestamp of when an event arrived in the central data store, like Elasticsearch. + +These three timestamps should typically follow a chronological order: + +```sh +@timestamp < event.created < event.ingested +``` + +event.start +: This timestamp marks the beginning of the event activity. For example, in a network session, [`event.start`](/reference/ecs-event.md#field-event-start) is the timestamp of the first observed packet in the flow. + +event.end +: This timestamp marks the end of the activity. In a network flow, [`event.end`](/reference/ecs-event.md#field-event-end) is the timestamp of the last observed packet in the flow. + +event.duration +: The difference of `event.end` and `event.start`: + +```sh +event.duration = event.end - event.start +``` + + +## Origin [_origin] + +Specific `event.*` fields exist to capture where an event originated. + +event.provider +: Contains the name of the software or operating subsystem that generated the event. + +event.module +: If the ingest agent or pipeline has a concept of modules or plugins, populate [`event.module`](/reference/ecs-event.md#field-event-module) with the module or plugin name. + +event.dataset +: Used to define different types of logs or metrics from an event source. The recommended convention is `.`. For Apache web server access logs, the [`event.dataset`](/reference/ecs-event.md#field-event-dataset) value will be `apache.access`. + + +## Categorization [_categorization] + +The event categorization fields group similar events using allowed values for four fields: + +* `event.kind` +* `event.category` +* `event.type` +* `event.outcome` + +[Using the Categorization Fields](/reference/ecs-using-categorization-fields.md) covers more details on using these four fields together to categorize events. + + +## Enriching events [_enriching_events] + +A monitoring agent or ingest pipeline can add more details to the original event. ECS has many fields to hold these enrichment details. + + +## Lookups [_lookups] + +GeoIP +: Add information about the geographical location of an IPv4 or IPv6 address. Often used to populate the `geo.*` fields nested under network transaction fields like `source.*`, `destination.*`, `client.*`, and `server.*`. + +```json +{ + "source": { + "address": "8.8.8.8", + "ip": "8.8.8.8", + "geo": { + "continent_name": "North America", + "country_name": "United States", + "country_iso_code": "US", + "location": { "lat": 37.751, "lon": -97.822 } + } + } +} +``` + +Autonomous system number +: Autonomous System Number (ASN) database lookups determine the ASN associated with an IP address. + + +## Parsing [_parsing] + +User-agent +: Break the user-agent into individual fields. + +```json +{ + "user_agent": { + "user_agent": { + "name": "Chrome", + "original": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36", + "version": "51.0.2704.103", + "os": { + "name": "Mac OS X", + "version": "10.10.5", + "full": "Mac OS X 10.10.5", + "platform": "darwin", + "type": "macos" + }, + "device" : { + "name" : "Mac" + } + } + } +} +``` + +URL +: A URL can also break down into its discrete parts. + +```json +{ + "original" : "http://myusername:mypassword@www.example.com:80/foo.gif?key1=val1&key2=val2#fragment", + "url" : { + "path" : "/foo.gif", + "fragment" : "fragment", + "extension" : "gif", + "password" : "mypassword", + "original" : "http://myusername:mypassword@www.example.com:80/foo.gif?key1=val1&key2=val2#fragment", + "scheme" : "http", + "port" : 80, + "user_info" : "myusername:mypassword", + "domain" : "www.example.com", + "query" : "key1=val1&key2=val2", + "username" : "myusername" + } +} +``` + +Domain names +: Extract the registered domain (also known as the effective top-level domain plus one), sub-domain, and effective top-level domain from a fully-qualified domain name (FQDN). + +```json +{ + "fqdn": "www.example.ac.uk", + "url": { + "subdomain": "www", + "registered_domain": "example.ac.uk", + "top_level_domain": "ac.uk", + "domain": "www.example.ac.uk" +} +``` + + +## Related fields [_related_fields] + +Many events have similar content populating different fields: IP addresses, file hashes, hostnames. Pivot between these events using the [`related.*`](/reference/ecs-related.md) fields. + +For example, IP addresses found under the `host.*`, `source.*`, `destination.*`, `client.*`, and `server.*` fields sets and the `network.forwarded_ip` field. By adding all IP addresses in an event to the `related.ip` field, there is now a single field to search for a given IP regardless of what field it appeared: + +```sh +related.ip: ["10.42.42.42"] +``` + diff --git a/docs/reference/ecs-process.md b/docs/reference/ecs-process.md new file mode 100644 index 0000000000..0a12f5e8cd --- /dev/null +++ b/docs/reference/ecs-process.md @@ -0,0 +1,123 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-process.html +applies_to: + stack: all + serverless: all +--- + +# Process fields [ecs-process] + +These fields contain information about a process. + +These fields can help you correlate metrics information with a process id/name from a log message. The `process.pid` often stays in the metric itself and is copied to the global field for correlation. + + +## Process field details [_process_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-process-args$$$[process.args](#field-process-args) | Array of process arguments, starting with the absolute path to the executable.

May be filtered to protect sensitive information.

type: keyword

Note: this field should contain an array of values.

example: `["/usr/bin/ssh", "-l", "user", "10.0.0.16"]`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") [process.command_args](https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-command-args)
| extended | +| $$$field-process-args-count$$$[process.args_count](#field-process-args-count) | Length of the process.args array.

This field can be useful for querying or performing bucket analysis on how many arguments were provided to start a process. More arguments may be an indication of suspicious activity.

type: long

example: `4`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [process.args_count](https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-args-count)
| extended | +| $$$field-process-command-line$$$[process.command_line](#field-process-command-line) | Full command line that started the process, including the absolute path to the executable, and all arguments.

Some arguments may be filtered to protect sensitive information.

type: wildcard

Multi-fields:

- process.command_line.text (type: match_only_text)

example: `/usr/bin/ssh -l user 10.0.0.16`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [process.command_line](https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-command-line)
| extended | +| $$$field-process-end$$$[process.end](#field-process-end) | The time the process ended.

type: date

example: `2016-05-23T08:05:34.853Z`
| extended | +| $$$field-process-entity-id$$$[process.entity_id](#field-process-entity-id) | Unique identifier for the process.

The implementation of this is specified by the data source, but some examples of what could be used here are a process-generated UUID, Sysmon Process GUIDs, or a hash of some uniquely identifying components of a process.

Constructing a globally unique identifier is a common practice to mitigate PID reuse as well as to identify a specific process over time, across multiple monitored hosts.

type: keyword

example: `c2c455d9f99375d`
| extended | +| $$$field-process-entry-meta-type$$$[process.entry_meta.type](#field-process-entry-meta-type) | The entry type for the entry session leader. Values include: init(e.g systemd), sshd, ssm, kubelet, teleport, terminal, console

Note: This field is only set on process.session_leader.

type: keyword
| extended | +| $$$field-process-env-vars$$$[process.env_vars](#field-process-env-vars) | Array of environment variable bindings. Captured from a snapshot of the environment at the time of execution.

May be filtered to protect sensitive information.

type: keyword

Note: this field should contain an array of values.

example: `["PATH=/usr/local/bin:/usr/bin", "USER=ubuntu"]`
| extended | +| $$$field-process-executable$$$[process.executable](#field-process-executable) | Absolute path to the process executable.

type: keyword

Multi-fields:

- process.executable.text (type: match_only_text)

example: `/usr/bin/ssh`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") [process.executable.path](https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-executable-path)
| extended | +| $$$field-process-exit-code$$$[process.exit_code](#field-process-exit-code) | The exit code of the process, if this is a termination event.

The field should be absent if there is no exit code for the event (e.g. process start).

type: long

example: `137`
| extended | +| $$$field-process-interactive$$$[process.interactive](#field-process-interactive) | Whether the process is connected to an interactive shell.

Process interactivity is inferred from the processes file descriptors. If the character device for the controlling tty is the same as stdin and stderr for the process, the process is considered interactive.

Note: A non-interactive process can belong to an interactive session and is simply one that does not have open file descriptors reading the controlling TTY on FD 0 (stdin) or writing to the controlling TTY on FD 2 (stderr). A backgrounded process is still considered interactive if stdin and stderr are connected to the controlling TTY.

type: boolean

example: `True`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [process.interactive](https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-interactive)
| extended | +| $$$field-process-io$$$[process.io](#field-process-io) | A chunk of input or output (IO) from a single process.

This field only appears on the top level process object, which is the process that wrote the output or read the input.

type: object
| extended | +| $$$field-process-io-bytes-skipped$$$[process.io.bytes_skipped](#field-process-io-bytes-skipped) | An array of byte offsets and lengths denoting where IO data has been skipped.

type: object

Note: this field should contain an array of values.
| extended | +| $$$field-process-io-bytes-skipped-length$$$[process.io.bytes_skipped.length](#field-process-io-bytes-skipped-length) | The length of bytes skipped.

type: long
| extended | +| $$$field-process-io-bytes-skipped-offset$$$[process.io.bytes_skipped.offset](#field-process-io-bytes-skipped-offset) | The byte offset into this event’s io.text (or io.bytes in the future) where length bytes were skipped.

type: long
| extended | +| $$$field-process-io-max-bytes-per-process-exceeded$$$[process.io.max_bytes_per_process_exceeded](#field-process-io-max-bytes-per-process-exceeded) | If true, the process producing the output has exceeded the max_kilobytes_per_process configuration setting.

type: boolean
| extended | +| $$$field-process-io-text$$$[process.io.text](#field-process-io-text) | A chunk of output or input sanitized to UTF-8.

Best efforts are made to ensure complete lines are captured in these events. Assumptions should NOT be made that multiple lines will appear in the same event. TTY output may contain terminal control codes such as for cursor movement, so some string queries may not match due to terminal codes inserted between characters of a word.

type: wildcard
| extended | +| $$$field-process-io-total-bytes-captured$$$[process.io.total_bytes_captured](#field-process-io-total-bytes-captured) | The total number of bytes captured in this event.

type: long
| extended | +| $$$field-process-io-total-bytes-skipped$$$[process.io.total_bytes_skipped](#field-process-io-total-bytes-skipped) | The total number of bytes that were not captured due to implementation restrictions such as buffer size limits. Implementors should strive to ensure this value is always zero

type: long
| extended | +| $$$field-process-io-type$$$[process.io.type](#field-process-io-type) | The type of object on which the IO action (read or write) was taken.

Currently only *tty* is supported. Other types may be added in the future for *file* and *socket* support.

type: keyword
| extended | +| $$$field-process-name$$$[process.name](#field-process-name) | Process name.

Sometimes called program name or similar.

type: keyword

Multi-fields:

- process.name.text (type: match_only_text)

example: `ssh`
| extended | +| $$$field-process-pgid$$$[process.pgid](#field-process-pgid) | Deprecated for removal in next major version release. This field is superseded by `process.group_leader.pid`.

Identifier of the group of processes the process belongs to.

type: long
| extended | +| $$$field-process-pid$$$[process.pid](#field-process-pid) | Process id.

type: long

example: `4242`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [process.pid](https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-pid)
| core | +| $$$field-process-same-as-process$$$[process.same_as_process](#field-process-same-as-process) | This boolean is used to identify if a leader process is the same as the top level process.

For example, if `process.group_leader.same_as_process = true`, it means the process event in question is the leader of its process group. Details under `process.*` like `pid` would be the same under `process.group_leader.*` The same applies for both `process.session_leader` and `process.entry_leader`.

This field exists to the benefit of EQL and other rule engines since it’s not possible to compare equality between two fields in a single document. e.g `process.entity_id` = `process.group_leader.entity_id` (top level process is the process group leader) OR `process.entity_id` = `process.entry_leader.entity_id` (top level process is the entry session leader)

Instead these rules could be written like: `process.group_leader.same_as_process: true` OR `process.entry_leader.same_as_process: true`

Note: This field is only set on `process.entry_leader`, `process.session_leader` and `process.group_leader`.

type: boolean

example: `True`
| extended | +| $$$field-process-start$$$[process.start](#field-process-start) | The time the process started.

type: date

example: `2016-05-23T08:05:34.853Z`
| extended | +| $$$field-process-thread-capabilities-effective$$$[process.thread.capabilities.effective](#field-process-thread-capabilities-effective) | This is the set of capabilities used by the kernel to perform permission checks for the thread.

type: keyword

Note: this field should contain an array of values.

example: `["CAP_BPF", "CAP_SYS_ADMIN"]`
| extended | +| $$$field-process-thread-capabilities-permitted$$$[process.thread.capabilities.permitted](#field-process-thread-capabilities-permitted) | This is a limiting superset for the effective capabilities that the thread may assume.

type: keyword

Note: this field should contain an array of values.

example: `["CAP_BPF", "CAP_SYS_ADMIN"]`
| extended | +| $$$field-process-thread-id$$$[process.thread.id](#field-process-thread-id) | Thread ID.

type: long

example: `4242`
| extended | +| $$$field-process-thread-name$$$[process.thread.name](#field-process-thread-name) | Thread name.

type: keyword

example: `thread-0`
| extended | +| $$$field-process-title$$$[process.title](#field-process-title) | Process title.

The proctitle, some times the same as process name. Can also be different: for example a browser setting its title to the web page currently opened.

type: keyword

Multi-fields:

- process.title.text (type: match_only_text)

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [process.title](https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-title)
| extended | +| $$$field-process-tty$$$[process.tty](#field-process-tty) | Information about the controlling TTY device. If set, the process belongs to an interactive session.

type: object
| extended | +| $$$field-process-tty-char-device-major$$$[process.tty.char_device.major](#field-process-tty-char-device-major) | The major number identifies the driver associated with the device. The character device’s major and minor numbers can be algorithmically combined to produce the more familiar terminal identifiers such as "ttyS0" and "pts/0". For more details, please refer to the Linux kernel documentation.

type: long

example: `4`
| extended | +| $$$field-process-tty-char-device-minor$$$[process.tty.char_device.minor](#field-process-tty-char-device-minor) | The minor number is used only by the driver specified by the major number; other parts of the kernel don’t use it, and merely pass it along to the driver. It is common for a driver to control several devices; the minor number provides a way for the driver to differentiate among them.

type: long

example: `1`
| extended | +| $$$field-process-tty-columns$$$[process.tty.columns](#field-process-tty-columns) | The number of character columns per line. e.g terminal width

Terminal sizes can change, so this value reflects the maximum value for a given IO event. i.e. where event.action = *text_output*

type: long

example: `80`
| extended | +| $$$field-process-tty-rows$$$[process.tty.rows](#field-process-tty-rows) | The number of character rows in the terminal. e.g terminal height

Terminal sizes can change, so this value reflects the maximum value for a given IO event. i.e. where event.action = *text_output*

type: long

example: `24`
| extended | +| $$$field-process-uptime$$$[process.uptime](#field-process-uptime) | Seconds the process has been up.

type: long

example: `1325`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/metric-cb00cb?style=flat "metric") [process.uptime](https://github.com/search?q=repo%3Aopen-telemetry%2Fsemantic-conventions+%22%3C%21--+semconv+metric.process.uptime+--%3E%22&type=code)
| extended | +| $$$field-process-vpid$$$[process.vpid](#field-process-vpid) | Virtual process id.

The process id within a pid namespace. This is not necessarily unique across all processes on the host but it is unique within the process namespace that the process exists within.

type: long

example: `4242`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [process.vpid](https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-vpid)
| core | +| $$$field-process-working-directory$$$[process.working_directory](#field-process-working-directory) | The working directory of the process.

type: keyword

Multi-fields:

- process.working_directory.text (type: match_only_text)

example: `/home/alice`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [process.working_directory](https://opentelemetry.io/docs/specs/semconv/attributes-registry/process/#process-working-directory)
| extended | + + +## Field reuse [_field_reuse_20] + +The `process` fields are expected to be nested at: + +* `process.entry_leader` +* `process.entry_leader.parent` +* `process.entry_leader.parent.session_leader` +* `process.group_leader` +* `process.parent` +* `process.parent.group_leader` +* `process.previous` +* `process.responsible` +* `process.session_leader` +* `process.session_leader.parent` +* `process.session_leader.parent.session_leader` + +Note also that the `process` fields may be used directly at the root of the events. + + +### Field sets that can be nested under process [ecs-process-nestings] + +::::{tab-set} + +:::{tab-item} Field sets + +| Location | Field Set | Description | +| --- | --- | --- | +| `process.code_signature.*` | [code_signature](/reference/ecs-code_signature.md) | These fields contain information about binary code signatures. | +| `process.entry_leader.*` | `process` | First process from terminal or remote access via SSH, SSM, etc OR a service directly started by the init process. | +| `process.entry_leader.parent.*` | `process` | Information about the entry leader’s parent process. Only pid, start and entity_id fields are set. | +| `process.entry_leader.parent.session_leader.*` | `process` | Information about the parent session of the entry leader. Only pid, start and entity_id fields are set. | +| `process.entry_meta.source.*` | [source](/reference/ecs-source.md) | Remote client information such as ip, port and geo location. | +| `process.group.*` | [group](/reference/ecs-group.md) | The effective group (egid). | +| `process.group_leader.*` | `process` | Information about the process group leader. In some cases this may be the same as the top level process. | +| `process.hash.*` | [hash](/reference/ecs-hash.md) | Hashes, usually file hashes. | +| `process.parent.*` | `process` | Information about the parent process. | +| `process.parent.group_leader.*` | `process` | Information about the parent’s process group leader. Only pid, start and entity_id fields are set. | +| `process.pe.*` | [pe](/reference/ecs-pe.md) | These fields contain Windows Portable Executable (PE) metadata. | +| `process.previous.*` | `process` | An array of previous executions for the process, including the initial fork. Only executable and args are set.
Note: this reuse should contain an array of process field set objects. | +| `process.real_group.*` | [group](/reference/ecs-group.md) | The real group (rgid). | +| `process.real_user.*` | [user](/reference/ecs-user.md) | The real user (ruid). Identifies the real owner of the process. | +| `process.saved_group.*` | [group](/reference/ecs-group.md) | The saved group (sgid). | +| `process.saved_user.*` | [user](/reference/ecs-user.md) | The saved user (suid). | +| `process.session_leader.*` | `process` | Often the same as entry_leader. When it differs, it represents a session started within another session. e.g. using tmux | +| `process.session_leader.parent.*` | `process` | Information about the session leader’s parent process. Only pid, start and entity_id fields are set. | +| `process.session_leader.parent.session_leader.*` | `process` | Information about the parent session of the session leader. Only pid, start and entity_id fields are set. | +| `process.supplemental_groups.*` | [group](/reference/ecs-group.md) | An array of supplemental groups.
Note: this reuse should contain an array of group field set objects. | +| `process.user.*` | [user](/reference/ecs-user.md) | The effective user (euid). | + +::: + +:::{tab-item} Beta field sets + +| Location | Field Set | Description | +| --- | --- | --- | +| `process.attested_groups.*` | [group](/reference/ecs-group.md) | The externally attested groups based on an external source such as the Kube API.
Note: this reuse should contain an array of group field set objects. | +| `process.attested_user.*` | [user](/reference/ecs-user.md) | The externally attested user based on an external source such as the Kube API. | +| `process.elf.*` | [elf](/reference/ecs-elf.md) | These fields contain Linux Executable Linkable Format (ELF) metadata. | +| `process.macho.*` | [macho](/reference/ecs-macho.md) | These fields contain Mac OS Mach Object file format (Mach-O) metadata. | +| `process.responsible.*` | `process` | Responsible process in macOS tracks the originating process of an app, key for understanding permissions and hierarchy. | + + +::: +:::: + diff --git a/docs/reference/ecs-products-solutions.md b/docs/reference/ecs-products-solutions.md new file mode 100644 index 0000000000..28f82035a7 --- /dev/null +++ b/docs/reference/ecs-products-solutions.md @@ -0,0 +1,24 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-products-solutions.html +applies_to: + stack: all + serverless: all +--- + +# Products and solutions that support ECS [ecs-products-solutions] + +The following Elastic products support ECS out of the box, as of version 7.0: + +* [{{beats}}](beats://reference/index.md) +* [APM](docs-content://solutions/observability/apps/application-performance-monitoring-apm.md) +* [Elastic Security](docs-content://solutions/security.md) + + * [Elastic Security Field Reference](docs-content://reference/security/fields-and-object-schemas/siem-field-reference.md) - a list of ECS fields used in the Security app + +* [Elastic Endpoint Security Server](https://www.elastic.co/products/endpoint-security) +* [Log Monitoring](docs-content://solutions/observability/logs/explore-logs.md) +* Log formatters that support ECS out of the box for various languages can be found [here](https://github.com/elastic/ecs-logging/blob/master/README.md). +* [Metrics Monitoring](docs-content://solutions/observability/infra-and-hosts/analyze-infrastructure-host-metrics.md) +* {{ls}}' {{es}} output has an [ECS compatibility mode](logstash://reference/plugins-outputs-elasticsearch.md#_compatibility_with_the_elastic_common_schema_ecs) + diff --git a/docs/reference/ecs-registry.md b/docs/reference/ecs-registry.md new file mode 100644 index 0000000000..e7d6fc5ac2 --- /dev/null +++ b/docs/reference/ecs-registry.md @@ -0,0 +1,35 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-registry.html +applies_to: + stack: all + serverless: all +--- + +# Registry fields [ecs-registry] + +Fields related to Windows Registry operations. + + +## Registry field details [_registry_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-registry-data-bytes$$$[registry.data.bytes](#field-registry-data-bytes) | Original bytes written with base64 encoding.

For Windows registry operations, such as SetValueEx and RegQueryValueEx, this corresponds to the data pointed by `lp_data`. This is optional but provides better recoverability and should be populated for REG_BINARY encoded values.

type: keyword

example: `ZQBuAC0AVQBTAAAAZQBuAAAAAAA=`
| extended | +| $$$field-registry-data-strings$$$[registry.data.strings](#field-registry-data-strings) | Content when writing string types.

Populated as an array when writing string data to the registry. For single string registry types (REG_SZ, REG_EXPAND_SZ), this should be an array with one string. For sequences of string with REG_MULTI_SZ, this array will be variable length. For numeric data, such as REG_DWORD and REG_QWORD, this should be populated with the decimal representation (e.g `"1"`).

type: wildcard

Note: this field should contain an array of values.

example: `["C:\rta\red_ttp\bin\myapp.exe"]`
| core | +| $$$field-registry-data-type$$$[registry.data.type](#field-registry-data-type) | Standard registry type for encoding contents

type: keyword

example: `REG_SZ`
| core | +| $$$field-registry-hive$$$[registry.hive](#field-registry-hive) | Abbreviated name for the hive.

type: keyword

example: `HKLM`
| core | +| $$$field-registry-key$$$[registry.key](#field-registry-key) | Hive-relative path of keys.

type: keyword

example: `SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\winword.exe`
| core | +| $$$field-registry-path$$$[registry.path](#field-registry-path) | Full path, including hive, key and value

type: keyword

example: `HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\winword.exe\Debugger`
| core | +| $$$field-registry-value$$$[registry.value](#field-registry-value) | Name of the value written.

type: keyword

example: `Debugger`
| core | + + +## Field reuse [_field_reuse_21] + +The `registry` fields are expected to be nested at: + +* `threat.enrichments.indicator.registry` +* `threat.indicator.registry` + +Note also that the `registry` fields may be used directly at the root of the events. + diff --git a/docs/reference/ecs-related.md b/docs/reference/ecs-related.md new file mode 100644 index 0000000000..0f3d80a563 --- /dev/null +++ b/docs/reference/ecs-related.md @@ -0,0 +1,26 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-related.html +applies_to: + stack: all + serverless: all +--- + +# Related fields [ecs-related] + +This field set is meant to facilitate pivoting around a piece of data. + +Some pieces of information can be seen in many places in an ECS event. To facilitate searching for them, store an array of all seen values to their corresponding field in `related.`. + +A concrete example is IP addresses, which can be under host, observer, source, destination, client, server, and network.forwarded_ip. If you append all IPs to `related.ip`, you can then search for a given IP trivially, no matter where it appeared, by querying `related.ip:192.0.2.15`. + + +## Related field details [_related_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-related-hash$$$[related.hash](#field-related-hash) | All the hashes seen on your event. Populating this field, then using it to search for hashes can help in situations where you’re unsure what the hash algorithm is (and therefore which key name to search).

type: keyword

Note: this field should contain an array of values.

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat "not-applicable") Not applicable.
| extended | +| $$$field-related-hosts$$$[related.hosts](#field-related-hosts) | All hostnames or other host identifiers seen on your event. Example identifiers include FQDNs, domain names, workstation names, or aliases.

type: keyword

Note: this field should contain an array of values.

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat "not-applicable") Not applicable.
| extended | +| $$$field-related-ip$$$[related.ip](#field-related-ip) | All of the IPs seen on your event.

type: ip

Note: this field should contain an array of values.

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat "not-applicable") Not applicable.
| extended | +| $$$field-related-user$$$[related.user](#field-related-user) | All the user names or other user identifiers seen on the event.

type: keyword

Note: this field should contain an array of values.

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat "not-applicable") Not applicable.
| extended | + diff --git a/docs/reference/ecs-release-notes.md b/docs/reference/ecs-release-notes.md new file mode 100644 index 0000000000..437386e186 --- /dev/null +++ b/docs/reference/ecs-release-notes.md @@ -0,0 +1,30 @@ +--- +navigation_title: 'Release notes' +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-release-notes.html +--- + +# ECS release notes [ecs-release-notes] + +This section summarizes the changes in each release. + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/reference/ecs-risk.md b/docs/reference/ecs-risk.md new file mode 100644 index 0000000000..3fd43bc228 --- /dev/null +++ b/docs/reference/ecs-risk.md @@ -0,0 +1,39 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-risk.html +applies_to: + stack: all + serverless: all +--- + +# Risk information fields [ecs-risk] + +Fields for describing risk score and risk level of entities such as hosts and users. These fields are not allowed to be nested under `event.*`. Please continue to use `event.risk_score` and `event.risk_score_norm` for event risk. + +::::{warning} +These fields are in beta and are subject to change. +:::: + + + +## Risk information field details [_risk_information_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-risk-calculated-level$$$[risk.calculated_level](#field-risk-calculated-level) | A risk classification level calculated by an internal system as part of entity analytics and entity risk scoring.

type: keyword

example: `High`
| extended | +| $$$field-risk-calculated-score$$$[risk.calculated_score](#field-risk-calculated-score) | A risk classification score calculated by an internal system as part of entity analytics and entity risk scoring.

type: float

example: `880.73`
| extended | +| $$$field-risk-calculated-score-norm$$$[risk.calculated_score_norm](#field-risk-calculated-score-norm) | A risk classification score calculated by an internal system as part of entity analytics and entity risk scoring, and normalized to a range of 0 to 100.

type: float

example: `88.73`
| extended | +| $$$field-risk-static-level$$$[risk.static_level](#field-risk-static-level) | A risk classification level obtained from outside the system, such as from some external Threat Intelligence Platform.

type: keyword

example: `High`
| extended | +| $$$field-risk-static-score$$$[risk.static_score](#field-risk-static-score) | A risk classification score obtained from outside the system, such as from some external Threat Intelligence Platform.

type: float

example: `830.0`
| extended | +| $$$field-risk-static-score-norm$$$[risk.static_score_norm](#field-risk-static-score-norm) | A risk classification score obtained from outside the system, such as from some external Threat Intelligence Platform, and normalized to a range of 0 to 100.

type: float

example: `83.0`
| extended | + + +## Field reuse [_field_reuse_22] + +The `risk` fields are expected to be nested at: + +* `host.risk` +* `user.risk` + +Note also that the `risk` fields are not expected to be used directly at the root of the events. + diff --git a/docs/reference/ecs-rule.md b/docs/reference/ecs-rule.md new file mode 100644 index 0000000000..6605b846cc --- /dev/null +++ b/docs/reference/ecs-rule.md @@ -0,0 +1,30 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-rule.html +applies_to: + stack: all + serverless: all +--- + +# Rule fields [ecs-rule] + +Rule fields are used to capture the specifics of any observer or agent rules that generate alerts or other notable events. + +Examples of data sources that would populate the rule fields include: network admission control platforms, network or host IDS/IPS, network firewalls, web application firewalls, url filters, endpoint detection and response (EDR) systems, etc. + + +## Rule field details [_rule_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-rule-author$$$[rule.author](#field-rule-author) | Name, organization, or pseudonym of the author or authors who created the rule used to generate this event.

type: keyword

Note: this field should contain an array of values.

example: `["Star-Lord"]`
| extended | +| $$$field-rule-category$$$[rule.category](#field-rule-category) | A categorization value keyword used by the entity using the rule for detection of this event.

type: keyword

example: `Attempted Information Leak`
| extended | +| $$$field-rule-description$$$[rule.description](#field-rule-description) | The description of the rule generating the event.

type: keyword

example: `Block requests to public DNS over HTTPS / TLS protocols`
| extended | +| $$$field-rule-id$$$[rule.id](#field-rule-id) | A rule ID that is unique within the scope of an agent, observer, or other entity using the rule for detection of this event.

type: keyword

example: `101`
| extended | +| $$$field-rule-license$$$[rule.license](#field-rule-license) | Name of the license under which the rule used to generate this event is made available.

type: keyword

example: `Apache 2.0`
| extended | +| $$$field-rule-name$$$[rule.name](#field-rule-name) | The name of the rule or signature generating the event.

type: keyword

example: `BLOCK_DNS_over_TLS`
| extended | +| $$$field-rule-reference$$$[rule.reference](#field-rule-reference) | Reference URL to additional information about the rule used to generate this event.

The URL can point to the vendor’s documentation about the rule. If that’s not available, it can also be a link to a more general page describing this type of alert.

type: keyword

example: `https://en.wikipedia.org/wiki/DNS_over_TLS`
| extended | +| $$$field-rule-ruleset$$$[rule.ruleset](#field-rule-ruleset) | Name of the ruleset, policy, group, or parent category in which the rule used to generate this event is a member.

type: keyword

example: `Standard_Protocol_Filters`
| extended | +| $$$field-rule-uuid$$$[rule.uuid](#field-rule-uuid) | A rule ID that is unique within the scope of a set or group of agents, observers, or other entities using the rule for detection of this event.

type: keyword

example: `1100110011`
| extended | +| $$$field-rule-version$$$[rule.version](#field-rule-version) | The version / revision of the rule being used for analysis.

type: keyword

example: `1.1`
| extended | + diff --git a/docs/reference/ecs-server.md b/docs/reference/ecs-server.md new file mode 100644 index 0000000000..fab515cde8 --- /dev/null +++ b/docs/reference/ecs-server.md @@ -0,0 +1,46 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-server.html +applies_to: + stack: all + serverless: all +--- + +# Server fields [ecs-server] + +A server is defined as the responder in a network connection for events regarding sessions, connections, or bidirectional flow records. + +For TCP events, the server is the receiver of the initial SYN packet(s) of the TCP connection. For other protocols, the server is generally the responder in the network transaction. Some systems actually use the term "responder" to refer the server in TCP connections. The server fields describe details about the system acting as the server in the network event. Server fields are usually populated in conjunction with client fields. Server fields are generally not populated for packet-level events. + +Client / server representations can add semantic context to an exchange, which is helpful to visualize the data in certain situations. If your context falls in that category, you should still ensure that source and destination are filled appropriately. + + +## Server field details [_server_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-server-address$$$[server.address](#field-server-address) | Some event server addresses are defined ambiguously. The event will sometimes list an IP, a domain or a unix socket. You should always store the raw address in the `.address` field.

Then it should be duplicated to `.ip` or `.domain`, depending on which one it is.

type: keyword

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [server.address](https://opentelemetry.io/docs/specs/semconv/attributes-registry/server/#server-address)
| extended | +| $$$field-server-bytes$$$[server.bytes](#field-server-bytes) | Bytes sent from the server to the client.

type: long

example: `184`
| core | +| $$$field-server-domain$$$[server.domain](#field-server-domain) | The domain name of the server system.

This value may be a host name, a fully qualified domain name, or another host naming format. The value may derive from the original event or be added from enrichment.

type: keyword

example: `foo.example.com`
| core | +| $$$field-server-ip$$$[server.ip](#field-server-ip) | IP address of the server (IPv4 or IPv6).

type: ip
| core | +| $$$field-server-mac$$$[server.mac](#field-server-mac) | MAC address of the server.

The notation format from RFC 7042 is suggested: Each octet (that is, 8-bit byte) is represented by two [uppercase] hexadecimal digits giving the value of the octet as an unsigned integer. Successive octets are separated by a hyphen.

type: keyword

example: `00-00-5E-00-53-23`
| core | +| $$$field-server-nat-ip$$$[server.nat.ip](#field-server-nat-ip) | Translated ip of destination based NAT sessions (e.g. internet to private DMZ)

Typically used with load balancers, firewalls, or routers.

type: ip
| extended | +| $$$field-server-nat-port$$$[server.nat.port](#field-server-nat-port) | Translated port of destination based NAT sessions (e.g. internet to private DMZ)

Typically used with load balancers, firewalls, or routers.

type: long
| extended | +| $$$field-server-packets$$$[server.packets](#field-server-packets) | Packets sent from the server to the client.

type: long

example: `12`
| core | +| $$$field-server-port$$$[server.port](#field-server-port) | Port of the server.

type: long

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [server.port](https://opentelemetry.io/docs/specs/semconv/attributes-registry/server/#server-port)
| core | +| $$$field-server-registered-domain$$$[server.registered_domain](#field-server-registered-domain) | The highest registered server domain, stripped of the subdomain.

For example, the registered domain for "foo.example.com" is "example.com".

This value can be determined precisely with a list like the public suffix list ([https://publicsuffix.org](https://publicsuffix.org)). Trying to approximate this by simply taking the last two labels will not work well for TLDs such as "co.uk".

type: keyword

example: `example.com`
| extended | +| $$$field-server-subdomain$$$[server.subdomain](#field-server-subdomain) | The subdomain portion of a fully qualified domain name includes all of the names except the host name under the registered_domain. In a partially qualified domain, or if the the qualification level of the full name cannot be determined, subdomain contains all of the names below the registered domain.

For example the subdomain portion of "www.east.mydomain.co.uk" is "east". If the domain has multiple levels of subdomain, such as "sub2.sub1.example.com", the subdomain field should contain "sub2.sub1", with no trailing period.

type: keyword

example: `east`
| extended | +| $$$field-server-top-level-domain$$$[server.top_level_domain](#field-server-top-level-domain) | The effective top level domain (eTLD), also known as the domain suffix, is the last part of the domain name. For example, the top level domain for example.com is "com".

This value can be determined precisely with a list like the public suffix list ([https://publicsuffix.org](https://publicsuffix.org)). Trying to approximate this by simply taking the last label will not work well for effective TLDs such as "co.uk".

type: keyword

example: `co.uk`
| extended | + + +## Field reuse [_field_reuse_23] + + +### Field sets that can be nested under server [ecs-server-nestings] + +| Location | Field Set | Description | +| --- | --- | --- | +| `server.as.*` | [as](/reference/ecs-as.md) | Fields describing an Autonomous System (Internet routing prefix). | +| `server.geo.*` | [geo](/reference/ecs-geo.md) | Fields describing a location. | +| `server.user.*` | [user](/reference/ecs-user.md) | Fields to describe the user relevant to the event. | + diff --git a/docs/reference/ecs-service-usage.md b/docs/reference/ecs-service-usage.md new file mode 100644 index 0000000000..e9c17ab565 --- /dev/null +++ b/docs/reference/ecs-service-usage.md @@ -0,0 +1,93 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-service-usage.html +applies_to: + stack: all + serverless: all +--- + +# Service fields usage and examples [ecs-service-usage] + +Here are the subjects covered in this page. + +* [Field reuse](#ecs-service-usage-field-reuse) + + * [Service fields at the Root of an Event](#ecs-service-usage-service-at-root) + * [Describing external services in an invocation relationship](#ecs-service-usage-origin-target) + + + +## Field reuse [ecs-service-usage-field-reuse] + +The service fields can be used to * [describe the service for or from which the data was collected](#ecs-service-usage-service-at-root) (i.e. observed service) * or to [describe external services that have a direct invocation relationship](#ecs-service-usage-origin-target) to the observed service + + +### Service fields at the root of an event [ecs-service-usage-service-at-root] + +Use the service fields at the root of an event to describe the service the event primarily relates to. An example for this use case is a log entry being recorded for a particular service or appplication (e.g. `MyService`): + +```json +{ + "service": { <1> + "id": "d37e5ebfe0ae6c4972dbe9f0174a1637bb8247f6", + "name": "MyService", + "version": "1.0.0" + }, + ... +} +``` + +1. Describes the service for which the log entry is being captured + + + +### Describing external services in an invocation relationship [ecs-service-usage-origin-target] + +Multiple services can be in an invocation relationship. Where it is possible to apply [distributed tracing](docs-content://solutions/observability/apps/traces-2.md) on all the involved services describe the individual services [using root-level service fields](#ecs-service-usage-service-at-root) and use the [tracing fields](/reference/ecs-related.md) to represent the invocation relationship. + +There are situations when distributed tracing cannot be applied on some external services that are in an invocation relationship to an observed service. Let’s consider the example of a service `MyService` being deployed on a cloud provider with an upstream API gateway that passes through requests to `MyService` (with additional context information about the API gateway itself). To describe the API gateway as a service from the perspective of `MyService` one can self-nest the service fields under `service.origin`: + +```json +{ + "service": { <1> + "id": "d37e5ebfe0ae6c4972dbe9f0174a1637bb8247f6", + "name": "MyService", + "version": "1.0.0", + "origin": { <2> + "id": "api-gateway-46372994637e2b4567", + "name": "SomeGateway", + "version" "2.5.1", + } + }, + ... +} +``` + +1. Describes the observed service that receives the inbound request from an external service +2. Describes the origin external service of the inbound request + + +Similar to the usage of `service.origin` fields the service fields can be self-nested under `service.target.*` to describe an external target service for an outbound request: + +```json +{ + "service": { <1> + "id": "d37e5ebfe0ae6c4972dbe9f0174a1637bb8247f6", + "name": "MyService", + "version": "1.0.0", + "target": { <2> + "id": "sms-service-0xe6c4272dbeAf0134", + "name": "ManagedSMSService", + "version": "1.9.0", + } + }, + ... +} +``` + +1. Describes the observed service that emits the outbound request to an external service +2. Describes the target external service of the outbound request + + +Note that `service.origin.*` and `service.target.*` fields should only be used on events that represent an invocation relationship. + diff --git a/docs/reference/ecs-service.md b/docs/reference/ecs-service.md new file mode 100644 index 0000000000..cbaffb613b --- /dev/null +++ b/docs/reference/ecs-service.md @@ -0,0 +1,73 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-service.html +applies_to: + stack: all + serverless: all +--- + +# Service fields [ecs-service] + +The service fields describe the service for or from which the data was collected. + +These fields help you find and correlate logs for a specific service and version. + + +## Service field details [_service_field_details] + +::::{tab-set} + +:::{tab-item} Fields + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-service-address$$$[service.address](#field-service-address) | Address where data about this service was collected from.

This should be a URI, network address (ipv4:port or [ipv6]:port) or a resource path (sockets).

type: keyword

example: `172.26.0.2:5432`
| extended | +| $$$field-service-ephemeral-id$$$[service.ephemeral_id](#field-service-ephemeral-id) | Ephemeral identifier of this service (if one exists).

This id normally changes across restarts, but `service.id` does not.

type: keyword

example: `8a4f500f`
| extended | +| $$$field-service-id$$$[service.id](#field-service-id) | Unique identifier of the running service. If the service is comprised of many nodes, the `service.id` should be the same for all nodes.

This id should uniquely identify the service. This makes it possible to correlate logs and metrics for one specific service, no matter which particular node emitted the event.

Note that if you need to see the events from one specific host of the service, you should filter on that `host.name` or `host.id` instead.

type: keyword

example: `d37e5ebfe0ae6c4972dbe9f0174a1637bb8247f6`
| core | +| $$$field-service-name$$$[service.name](#field-service-name) | Name of the service data is collected from.

The name of the service is normally user given. This allows for distributed services that run on multiple hosts to correlate the related instances based on the name.

In the case of Elasticsearch the `service.name` could contain the cluster name. For Beats the `service.name` is by default a copy of the `service.type` field if no name is specified.

type: keyword

example: `elasticsearch-metrics`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [service.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/service/#service-name)
| core | +| $$$field-service-node-name$$$[service.node.name](#field-service-node-name) | Name of a service node.

This allows for two nodes of the same service running on the same host to be differentiated. Therefore, `service.node.name` should typically be unique across nodes of a given service.

In the case of Elasticsearch, the `service.node.name` could contain the unique node name within the Elasticsearch cluster. In cases where the service doesn’t have the concept of a node name, the host name or container name can be used to distinguish running instances that make up this service. If those do not provide uniqueness (e.g. multiple instances of the service running on the same host) - the node name can be manually set.

type: keyword

example: `instance-0000000016`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") [service.instance.id](https://opentelemetry.io/docs/specs/semconv/attributes-registry/service/#service-instance-id)
| extended | +| $$$field-service-node-role$$$[service.node.role](#field-service-node-role) | Deprecated for removal in next major version release. This field will be superseded by `node.roles`.

Role of a service node.

This allows for distinction between different running roles of the same service.

In the case of Kibana, the `service.node.role` could be `ui` or `background_tasks`.

In the case of Elasticsearch, the `service.node.role` could be `master` or `data`.

Other services could use this to distinguish between a `web` and `worker` role running as part of the service.

type: keyword

example: `background_tasks`
| extended | +| $$$field-service-node-roles$$$[service.node.roles](#field-service-node-roles) | Roles of a service node.

This allows for distinction between different running roles of the same service.

In the case of Kibana, the `service.node.role` could be `ui` or `background_tasks` or both.

In the case of Elasticsearch, the `service.node.role` could be `master` or `data` or both.

Other services could use this to distinguish between a `web` and `worker` role running as part of the service.

type: keyword

Note: this field should contain an array of values.

example: `["ui", "background_tasks"]`
| extended | +| $$$field-service-state$$$[service.state](#field-service-state) | Current state of the service.

type: keyword
| core | +| $$$field-service-type$$$[service.type](#field-service-type) | The type of the service data is collected from.

The type can be used to group and correlate logs and metrics from one service type.

Example: If logs or metrics are collected from Elasticsearch, `service.type` would be `elasticsearch`.

type: keyword

example: `elasticsearch`
| core | +| $$$field-service-version$$$[service.version](#field-service-version) | Version of the service the data was collected from.

This allows to look at a data set only for a specific version of a service.

type: keyword

example: `3.2.4`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [service.version](https://opentelemetry.io/docs/specs/semconv/attributes-registry/service/#service-version)
| core | + +::: + +:::{tab-item} Beta Fields + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-service-environment$$$[service.environment](#field-service-environment) | Identifies the environment where the service is running.

If the same service runs in different environments (production, staging, QA, development, etc.), the environment can identify other instances of the same service. Can also group services and applications from the same environment.

type: keyword

example: `production`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/equivalent-1ba9f5?style=flat "equivalent") [deployment.environment.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/deployment/#deployment-environment-name)
| extended | + +::: +:::: + + +## Field reuse [_field_reuse_24] + +The `service` fields are expected to be nested at: + +* `service.origin` +* `service.target` + +Note also that the `service` fields may be used directly at the root of the events. + + +### Field sets that can be nested under Service [ecs-service-nestings] + +::::{warning} +These fields are in beta and are subject to change. +:::: + +| Location | Field Set | Description | +| --- | --- | --- | +| `service.origin.*` | `service` | Describes the origin service in case of an incoming request or event. | +| `service.target.*` | `service` | Describes the target service in case of an outgoing request or event. | + + +## Service Field Usage [_service_field_usage] + +For usage and examples of the service fields, please see the [Service Fields Usage and Examples](/reference/ecs-service-usage.md) section. + + diff --git a/docs/reference/ecs-source.md b/docs/reference/ecs-source.md new file mode 100644 index 0000000000..26791ed601 --- /dev/null +++ b/docs/reference/ecs-source.md @@ -0,0 +1,50 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-source.html +applies_to: + stack: all + serverless: all +--- + +# Source fields [ecs-source] + +Source fields capture details about the sender of a network exchange/packet. These fields are populated from a network event, packet, or other event containing details of a network transaction. + +Source fields are usually populated in conjunction with destination fields. The source and destination fields are considered the baseline and should always be filled if an event contains source and destination details from a network transaction. If the event also contains identification of the client and server roles, then the client and server fields should also be populated. + + +## Source field details [_source_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-source-address$$$[source.address](#field-source-address) | Some event source addresses are defined ambiguously. The event will sometimes list an IP, a domain or a unix socket. You should always store the raw address in the `.address` field.

Then it should be duplicated to `.ip` or `.domain`, depending on which one it is.

type: keyword

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [source.address](https://opentelemetry.io/docs/specs/semconv/attributes-registry/source/#source-address)
| extended | +| $$$field-source-bytes$$$[source.bytes](#field-source-bytes) | Bytes sent from the source to the destination.

type: long

example: `184`
| core | +| $$$field-source-domain$$$[source.domain](#field-source-domain) | The domain name of the source system.

This value may be a host name, a fully qualified domain name, or another host naming format. The value may derive from the original event or be added from enrichment.

type: keyword

example: `foo.example.com`
| core | +| $$$field-source-ip$$$[source.ip](#field-source-ip) | IP address of the source (IPv4 or IPv6).

type: ip
| core | +| $$$field-source-mac$$$[source.mac](#field-source-mac) | MAC address of the source.

The notation format from RFC 7042 is suggested: Each octet (that is, 8-bit byte) is represented by two [uppercase] hexadecimal digits giving the value of the octet as an unsigned integer. Successive octets are separated by a hyphen.

type: keyword

example: `00-00-5E-00-53-23`
| core | +| $$$field-source-nat-ip$$$[source.nat.ip](#field-source-nat-ip) | Translated ip of source based NAT sessions (e.g. internal client to internet)

Typically connections traversing load balancers, firewalls, or routers.

type: ip
| extended | +| $$$field-source-nat-port$$$[source.nat.port](#field-source-nat-port) | Translated port of source based NAT sessions. (e.g. internal client to internet)

Typically used with load balancers, firewalls, or routers.

type: long
| extended | +| $$$field-source-packets$$$[source.packets](#field-source-packets) | Packets sent from the source to the destination.

type: long

example: `12`
| core | +| $$$field-source-port$$$[source.port](#field-source-port) | Port of the source.

type: long

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [source.port](https://opentelemetry.io/docs/specs/semconv/attributes-registry/source/#source-port)
| core | +| $$$field-source-registered-domain$$$[source.registered_domain](#field-source-registered-domain) | The highest registered source domain, stripped of the subdomain.

For example, the registered domain for "foo.example.com" is "example.com".

This value can be determined precisely with a list like the public suffix list ([https://publicsuffix.org](https://publicsuffix.org)). Trying to approximate this by simply taking the last two labels will not work well for TLDs such as "co.uk".

type: keyword

example: `example.com`
| extended | +| $$$field-source-subdomain$$$[source.subdomain](#field-source-subdomain) | The subdomain portion of a fully qualified domain name includes all of the names except the host name under the registered_domain. In a partially qualified domain, or if the the qualification level of the full name cannot be determined, subdomain contains all of the names below the registered domain.

For example the subdomain portion of "www.east.mydomain.co.uk" is "east". If the domain has multiple levels of subdomain, such as "sub2.sub1.example.com", the subdomain field should contain "sub2.sub1", with no trailing period.

type: keyword

example: `east`
| extended | +| $$$field-source-top-level-domain$$$[source.top_level_domain](#field-source-top-level-domain) | The effective top level domain (eTLD), also known as the domain suffix, is the last part of the domain name. For example, the top level domain for example.com is "com".

This value can be determined precisely with a list like the public suffix list ([https://publicsuffix.org](https://publicsuffix.org)). Trying to approximate this by simply taking the last label will not work well for effective TLDs such as "co.uk".

type: keyword

example: `co.uk`
| extended | + + +## Field reuse [_field_reuse_25] + +The `source` fields are expected to be nested at: + +* `process.entry_meta.source` + +Note also that the `source` fields may be used directly at the root of the events. + + +### Field sets that can be nested under source [ecs-source-nestings] + +| Location | Field Set | Description | +| --- | --- | --- | +| `source.as.*` | [as](/reference/ecs-as.md) | Fields describing an Autonomous System (Internet routing prefix). | +| `source.geo.*` | [geo](/reference/ecs-geo.md) | Fields describing a location. | +| `source.user.*` | [user](/reference/ecs-user.md) | Fields to describe the user relevant to the event. | + diff --git a/docs/fields/usage/threat.asciidoc b/docs/reference/ecs-threat-usage.md similarity index 78% rename from docs/fields/usage/threat.asciidoc rename to docs/reference/ecs-threat-usage.md index 7693250d30..388340392a 100644 --- a/docs/fields/usage/threat.asciidoc +++ b/docs/reference/ecs-threat-usage.md @@ -1,20 +1,21 @@ -[[ecs-threat-usage]] -==== Threat Fields Usage and Examples +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-threat-usage.html +applies_to: + stack: all + serverless: all +--- -The `threat.*` fields map threat indicators to ECS. The data helps detect -malicious events with indicator match rules and enrichment. +# Threat fields usage and examples [ecs-threat-usage] -[discrete] -[[ecs-threat-usage-indicators]] -===== Indicators +The `threat.*` fields map threat indicators to ECS. The data helps detect malicious events with indicator match rules and enrichment. -Threat intelligence indicators come from many sources in different structures. -Normalize these indicators using the ECS threat.indicator.* fields. Once -normalized, consistently query indicators from various sources and build -indicator match rules. -The below example is from an online database. It contains several network -indicators from a known malware site. +## Indicators [ecs-threat-usage-indicators] + +Threat intelligence indicators come from many sources in different structures. Normalize these indicators using the ECS threat.indicator.* fields. Once normalized, consistently query indicators from various sources and build indicator match rules. + +The below example is from an online database. It contains several network indicators from a known malware site. ```JSON { @@ -64,11 +65,13 @@ indicators from a known malware site. } } ``` -<1> Use the `enrichment` value for `event.kind`. -<2> Use the `threat` value for `event.category`. -<3> The event type is set to `indicator`. -<4> Capture indicator details at `threat.indicator.*`. -<5> Copy indicators to the relevant `related.*` fields. + +1. Use the `enrichment` value for `event.kind`. +2. Use the `threat` value for `event.category`. +3. The event type is set to `indicator`. +4. Capture indicator details at `threat.indicator.*`. +5. Copy indicators to the relevant `related.*` fields. + The following example maps a file-based indicator. @@ -117,15 +120,16 @@ The following example maps a file-based indicator. } } ``` -<1> Use the `file` value for `threat.indicator.type`. -<2> Capture file attributes at `threat.indicator.file.*`. -<3> Again, populate the `related.hash` field with the file hashes. -[discrete] -[[ecs-threat-usage-enrichments]] -===== Enrichments +1. Use the `file` value for `threat.indicator.type`. +2. Capture file attributes at `threat.indicator.file.*`. +3. Again, populate the `related.hash` field with the file hashes. + -Event enrichment searches for known threats using an event's values and, if found, adds those associated details. + +## Enrichments [ecs-threat-usage-enrichments] + +Event enrichment searches for known threats using an event’s values and, if found, adds those associated details. ```JSON { @@ -184,5 +188,8 @@ Event enrichment searches for known threats using an event's values and, if foun } } ``` -<1> Add each enrichment to a nested object under `threat.enrichments.*`. -<2> The `matched` object provides context about the indicators this event matched on. + +1. Add each enrichment to a nested object under `threat.enrichments.*`. +2. The `matched` object provides context about the indicators this event matched on. + + diff --git a/docs/reference/ecs-threat.md b/docs/reference/ecs-threat.md new file mode 100644 index 0000000000..9be13cdf2e --- /dev/null +++ b/docs/reference/ecs-threat.md @@ -0,0 +1,112 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-threat.html +applies_to: + stack: all + serverless: all +--- + +# Threat fields [ecs-threat] + +Fields to classify events and alerts according to a threat taxonomy such as the MITRE ATT&CK® framework. + +These fields are for users to classify alerts from all of their sources (e.g. IDS, NGFW, etc.) within a common taxonomy. The threat.tactic.* fields are meant to capture the high level category of the threat (e.g. "impact"). The threat.technique.* fields are meant to capture which kind of approach is used by this detected threat, to accomplish the goal (e.g. "endpoint denial of service"). + + +## Threat field details [_threat_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-threat-enrichments$$$[threat.enrichments](#field-threat-enrichments) | A list of associated indicators objects enriching the event, and the context of that association/enrichment.

type: nested

Note: this field should contain an array of values.
| extended | +| $$$field-threat-enrichments-indicator$$$[threat.enrichments.indicator](#field-threat-enrichments-indicator) | Object containing associated indicators enriching the event.

type: object
| extended | +| $$$field-threat-enrichments-indicator-confidence$$$[threat.enrichments.indicator.confidence](#field-threat-enrichments-indicator-confidence) | Identifies the vendor-neutral confidence rating using the None/Low/Medium/High scale defined in Appendix A of the STIX 2.1 framework. Vendor-specific confidence scales may be added as custom fields.

Expected values for this field:

- `Not Specified`
- `None`
- `Low`
- `Medium`
- `High`

type: keyword

example: `Medium`
| extended | +| $$$field-threat-enrichments-indicator-description$$$[threat.enrichments.indicator.description](#field-threat-enrichments-indicator-description) | Describes the type of action conducted by the threat.

type: keyword

example: `IP x.x.x.x was observed delivering the Angler EK.`
| extended | +| $$$field-threat-enrichments-indicator-email-address$$$[threat.enrichments.indicator.email.address](#field-threat-enrichments-indicator-email-address) | Identifies a threat indicator as an email address (irrespective of direction).

type: keyword

example: `phish@example.com`
| extended | +| $$$field-threat-enrichments-indicator-first-seen$$$[threat.enrichments.indicator.first_seen](#field-threat-enrichments-indicator-first-seen) | The date and time when intelligence source first reported sighting this indicator.

type: date

example: `2020-11-05T17:25:47.000Z`
| extended | +| $$$field-threat-enrichments-indicator-ip$$$[threat.enrichments.indicator.ip](#field-threat-enrichments-indicator-ip) | Identifies a threat indicator as an IP address (irrespective of direction).

type: ip

example: `1.2.3.4`
| extended | +| $$$field-threat-enrichments-indicator-last-seen$$$[threat.enrichments.indicator.last_seen](#field-threat-enrichments-indicator-last-seen) | The date and time when intelligence source last reported sighting this indicator.

type: date

example: `2020-11-05T17:25:47.000Z`
| extended | +| $$$field-threat-enrichments-indicator-marking-tlp$$$[threat.enrichments.indicator.marking.tlp](#field-threat-enrichments-indicator-marking-tlp) | Traffic Light Protocol sharing markings.

Expected values for this field:

- `WHITE`
- `CLEAR`
- `GREEN`
- `AMBER`
- `AMBER+STRICT`
- `RED`

type: keyword

example: `CLEAR`
| extended | +| $$$field-threat-enrichments-indicator-marking-tlp-version$$$[threat.enrichments.indicator.marking.tlp_version](#field-threat-enrichments-indicator-marking-tlp-version) | Traffic Light Protocol version.

type: keyword

example: `2.0`
| extended | +| $$$field-threat-enrichments-indicator-modified-at$$$[threat.enrichments.indicator.modified_at](#field-threat-enrichments-indicator-modified-at) | The date and time when intelligence source last modified information for this indicator.

type: date

example: `2020-11-05T17:25:47.000Z`
| extended | +| $$$field-threat-enrichments-indicator-name$$$[threat.enrichments.indicator.name](#field-threat-enrichments-indicator-name) | The display name indicator in an UI friendly format

URL, IP address, email address, registry key, port number, hash value, or other relevant name can serve as the display name.

type: keyword

example: `5.2.75.227`
| extended | +| $$$field-threat-enrichments-indicator-port$$$[threat.enrichments.indicator.port](#field-threat-enrichments-indicator-port) | Identifies a threat indicator as a port number (irrespective of direction).

type: long

example: `443`
| extended | +| $$$field-threat-enrichments-indicator-provider$$$[threat.enrichments.indicator.provider](#field-threat-enrichments-indicator-provider) | The name of the indicator’s provider.

type: keyword

example: `lrz_urlhaus`
| extended | +| $$$field-threat-enrichments-indicator-reference$$$[threat.enrichments.indicator.reference](#field-threat-enrichments-indicator-reference) | Reference URL linking to additional information about this indicator.

type: keyword

example: `https://system.example.com/indicator/0001234`
| extended | +| $$$field-threat-enrichments-indicator-scanner-stats$$$[threat.enrichments.indicator.scanner_stats](#field-threat-enrichments-indicator-scanner-stats) | Count of AV/EDR vendors that successfully detected malicious file or URL.

type: long

example: `4`
| extended | +| $$$field-threat-enrichments-indicator-sightings$$$[threat.enrichments.indicator.sightings](#field-threat-enrichments-indicator-sightings) | Number of times this indicator was observed conducting threat activity.

type: long

example: `20`
| extended | +| $$$field-threat-enrichments-indicator-type$$$[threat.enrichments.indicator.type](#field-threat-enrichments-indicator-type) | Type of indicator as represented by Cyber Observable in STIX 2.0.

Expected values for this field:

- `autonomous-system`
- `artifact`
- `directory`
- `domain-name`
- `email-addr`
- `file`
- `ipv4-addr`
- `ipv6-addr`
- `mac-addr`
- `mutex`
- `port`
- `process`
- `software`
- `url`
- `user-account`
- `windows-registry-key`
- `x509-certificate`

type: keyword

example: `ipv4-addr`
| extended | +| $$$field-threat-enrichments-matched-atomic$$$[threat.enrichments.matched.atomic](#field-threat-enrichments-matched-atomic) | Identifies the atomic indicator value that matched a local environment endpoint or network event.

type: keyword

example: `bad-domain.com`
| extended | +| $$$field-threat-enrichments-matched-field$$$[threat.enrichments.matched.field](#field-threat-enrichments-matched-field) | Identifies the field of the atomic indicator that matched a local environment endpoint or network event.

type: keyword

example: `file.hash.sha256`
| extended | +| $$$field-threat-enrichments-matched-id$$$[threat.enrichments.matched.id](#field-threat-enrichments-matched-id) | Identifies the _id of the indicator document enriching the event.

type: keyword

example: `ff93aee5-86a1-4a61-b0e6-0cdc313d01b5`
| extended | +| $$$field-threat-enrichments-matched-index$$$[threat.enrichments.matched.index](#field-threat-enrichments-matched-index) | Identifies the _index of the indicator document enriching the event.

type: keyword

example: `filebeat-8.0.0-2021.05.23-000011`
| extended | +| $$$field-threat-enrichments-matched-occurred$$$[threat.enrichments.matched.occurred](#field-threat-enrichments-matched-occurred) | Indicates when the indicator match was generated

type: date

example: `2021-10-05T17:00:58.326Z`
| extended | +| $$$field-threat-enrichments-matched-type$$$[threat.enrichments.matched.type](#field-threat-enrichments-matched-type) | Identifies the type of match that caused the event to be enriched with the given indicator

type: keyword

example: `indicator_match_rule`
| extended | +| $$$field-threat-feed-dashboard-id$$$[threat.feed.dashboard_id](#field-threat-feed-dashboard-id) | The saved object ID of the dashboard belonging to the threat feed for displaying dashboard links to threat feeds in Kibana.

type: keyword

example: `5ba16340-72e6-11eb-a3e3-b3cc7c78a70f`
| extended | +| $$$field-threat-feed-description$$$[threat.feed.description](#field-threat-feed-description) | Description of the threat feed in a UI friendly format.

type: keyword

example: `Threat feed from the AlienVault Open Threat eXchange network.`
| extended | +| $$$field-threat-feed-name$$$[threat.feed.name](#field-threat-feed-name) | The name of the threat feed in UI friendly format.

type: keyword

example: `AlienVault OTX`
| extended | +| $$$field-threat-feed-reference$$$[threat.feed.reference](#field-threat-feed-reference) | Reference information for the threat feed in a UI friendly format.

type: keyword

example: `https://otx.alienvault.com`
| extended | +| $$$field-threat-framework$$$[threat.framework](#field-threat-framework) | Name of the threat framework used to further categorize and classify the tactic and technique of the reported threat. Framework classification can be provided by detecting systems, evaluated at ingest time, or retrospectively tagged to events.

type: keyword

example: `MITRE ATT&CK`
| extended | +| $$$field-threat-group-alias$$$[threat.group.alias](#field-threat-group-alias) | The alias(es) of the group for a set of related intrusion activity that are tracked by a common name in the security community.

While not required, you can use a MITRE ATT&CK® group alias(es).

type: keyword

Note: this field should contain an array of values.

example: `[ "Magecart Group 6" ]`
| extended | +| $$$field-threat-group-id$$$[threat.group.id](#field-threat-group-id) | The id of the group for a set of related intrusion activity that are tracked by a common name in the security community.

While not required, you can use a MITRE ATT&CK® group id.

type: keyword

example: `G0037`
| extended | +| $$$field-threat-group-name$$$[threat.group.name](#field-threat-group-name) | The name of the group for a set of related intrusion activity that are tracked by a common name in the security community.

While not required, you can use a MITRE ATT&CK® group name.

type: keyword

example: `FIN6`
| extended | +| $$$field-threat-group-reference$$$[threat.group.reference](#field-threat-group-reference) | The reference URL of the group for a set of related intrusion activity that are tracked by a common name in the security community.

While not required, you can use a MITRE ATT&CK® group reference URL.

type: keyword

example: `https://attack.mitre.org/groups/G0037/`
| extended | +| $$$field-threat-indicator-confidence$$$[threat.indicator.confidence](#field-threat-indicator-confidence) | Identifies the vendor-neutral confidence rating using the None/Low/Medium/High scale defined in Appendix A of the STIX 2.1 framework. Vendor-specific confidence scales may be added as custom fields.

Expected values for this field:

- `Not Specified`
- `None`
- `Low`
- `Medium`
- `High`

type: keyword

example: `Medium`
| extended | +| $$$field-threat-indicator-description$$$[threat.indicator.description](#field-threat-indicator-description) | Describes the type of action conducted by the threat.

type: keyword

example: `IP x.x.x.x was observed delivering the Angler EK.`
| extended | +| $$$field-threat-indicator-email-address$$$[threat.indicator.email.address](#field-threat-indicator-email-address) | Identifies a threat indicator as an email address (irrespective of direction).

type: keyword

example: `phish@example.com`
| extended | +| $$$field-threat-indicator-first-seen$$$[threat.indicator.first_seen](#field-threat-indicator-first-seen) | The date and time when intelligence source first reported sighting this indicator.

type: date

example: `2020-11-05T17:25:47.000Z`
| extended | +| $$$field-threat-indicator-id$$$[threat.indicator.id](#field-threat-indicator-id) | The ID of the indicator used by this threat to conduct behavior commonly modeled using MITRE ATT&CK®. This field can have multiple values to allow for the identification of the same indicator across systems that use different ID formats.

While not required, a common approach is to use a STIX 2.x indicator ID.

type: keyword

Note: this field should contain an array of values.

example: `[indicator--d7008e06-ab86-415a-9803-3c81ce2d3c37]`
| extended | +| $$$field-threat-indicator-ip$$$[threat.indicator.ip](#field-threat-indicator-ip) | Identifies a threat indicator as an IP address (irrespective of direction).

type: ip

example: `1.2.3.4`
| extended | +| $$$field-threat-indicator-last-seen$$$[threat.indicator.last_seen](#field-threat-indicator-last-seen) | The date and time when intelligence source last reported sighting this indicator.

type: date

example: `2020-11-05T17:25:47.000Z`
| extended | +| $$$field-threat-indicator-marking-tlp$$$[threat.indicator.marking.tlp](#field-threat-indicator-marking-tlp) | Traffic Light Protocol sharing markings.

Expected values for this field:

- `WHITE`
- `CLEAR`
- `GREEN`
- `AMBER`
- `AMBER+STRICT`
- `RED`

type: keyword

example: `CLEAR`
| extended | +| $$$field-threat-indicator-marking-tlp-version$$$[threat.indicator.marking.tlp_version](#field-threat-indicator-marking-tlp-version) | Traffic Light Protocol version.

type: keyword

example: `2.0`
| extended | +| $$$field-threat-indicator-modified-at$$$[threat.indicator.modified_at](#field-threat-indicator-modified-at) | The date and time when intelligence source last modified information for this indicator.

type: date

example: `2020-11-05T17:25:47.000Z`
| extended | +| $$$field-threat-indicator-name$$$[threat.indicator.name](#field-threat-indicator-name) | The display name indicator in an UI friendly format

URL, IP address, email address, registry key, port number, hash value, or other relevant name can serve as the display name.

type: keyword

example: `5.2.75.227`
| extended | +| $$$field-threat-indicator-port$$$[threat.indicator.port](#field-threat-indicator-port) | Identifies a threat indicator as a port number (irrespective of direction).

type: long

example: `443`
| extended | +| $$$field-threat-indicator-provider$$$[threat.indicator.provider](#field-threat-indicator-provider) | The name of the indicator’s provider.

type: keyword

example: `lrz_urlhaus`
| extended | +| $$$field-threat-indicator-reference$$$[threat.indicator.reference](#field-threat-indicator-reference) | Reference URL linking to additional information about this indicator.

type: keyword

example: `https://system.example.com/indicator/0001234`
| extended | +| $$$field-threat-indicator-scanner-stats$$$[threat.indicator.scanner_stats](#field-threat-indicator-scanner-stats) | Count of AV/EDR vendors that successfully detected malicious file or URL.

type: long

example: `4`
| extended | +| $$$field-threat-indicator-sightings$$$[threat.indicator.sightings](#field-threat-indicator-sightings) | Number of times this indicator was observed conducting threat activity.

type: long

example: `20`
| extended | +| $$$field-threat-indicator-type$$$[threat.indicator.type](#field-threat-indicator-type) | Type of indicator as represented by Cyber Observable in STIX 2.0.

Expected values for this field:

- `autonomous-system`
- `artifact`
- `directory`
- `domain-name`
- `email-addr`
- `file`
- `ipv4-addr`
- `ipv6-addr`
- `mac-addr`
- `mutex`
- `port`
- `process`
- `software`
- `url`
- `user-account`
- `windows-registry-key`
- `x509-certificate`

type: keyword

example: `ipv4-addr`
| extended | +| $$$field-threat-software-alias$$$[threat.software.alias](#field-threat-software-alias) | The alias(es) of the software for a set of related intrusion activity that are tracked by a common name in the security community.

While not required, you can use a MITRE ATT&CK® associated software description.

type: keyword

Note: this field should contain an array of values.

example: `[ "X-Agent" ]`
| extended | +| $$$field-threat-software-id$$$[threat.software.id](#field-threat-software-id) | The id of the software used by this threat to conduct behavior commonly modeled using MITRE ATT&CK®.

While not required, you can use a MITRE ATT&CK® software id.

type: keyword

example: `S0552`
| extended | +| $$$field-threat-software-name$$$[threat.software.name](#field-threat-software-name) | The name of the software used by this threat to conduct behavior commonly modeled using MITRE ATT&CK®.

While not required, you can use a MITRE ATT&CK® software name.

type: keyword

example: `AdFind`
| extended | +| $$$field-threat-software-platforms$$$[threat.software.platforms](#field-threat-software-platforms) | The platforms of the software used by this threat to conduct behavior commonly modeled using MITRE ATT&CK®.

While not required, you can use MITRE ATT&CK® software platform values.

Expected values for this field:

- `AWS`
- `Azure`
- `Azure AD`
- `GCP`
- `Linux`
- `macOS`
- `Network`
- `Office 365`
- `SaaS`
- `Windows`

type: keyword

Note: this field should contain an array of values.

example: `[ "Windows" ]`
| extended | +| $$$field-threat-software-reference$$$[threat.software.reference](#field-threat-software-reference) | The reference URL of the software used by this threat to conduct behavior commonly modeled using MITRE ATT&CK®.

While not required, you can use a MITRE ATT&CK® software reference URL.

type: keyword

example: `https://attack.mitre.org/software/S0552/`
| extended | +| $$$field-threat-software-type$$$[threat.software.type](#field-threat-software-type) | The type of software used by this threat to conduct behavior commonly modeled using MITRE ATT&CK®.

While not required, you can use a MITRE ATT&CK® software type.

Expected values for this field:

- `Malware`
- `Tool`

type: keyword

example: `Tool`
| extended | +| $$$field-threat-tactic-id$$$[threat.tactic.id](#field-threat-tactic-id) | The id of tactic used by this threat. You can use a MITRE ATT&CK® tactic, for example. (ex. [https://attack.mitre.org/tactics/TA0002/](https://attack.mitre.org/tactics/TA0002/) )

type: keyword

Note: this field should contain an array of values.

example: `TA0002`
| extended | +| $$$field-threat-tactic-name$$$[threat.tactic.name](#field-threat-tactic-name) | Name of the type of tactic used by this threat. You can use a MITRE ATT&CK® tactic, for example. (ex. [https://attack.mitre.org/tactics/TA0002/](https://attack.mitre.org/tactics/TA0002/))

type: keyword

Note: this field should contain an array of values.

example: `Execution`
| extended | +| $$$field-threat-tactic-reference$$$[threat.tactic.reference](#field-threat-tactic-reference) | The reference url of tactic used by this threat. You can use a MITRE ATT&CK® tactic, for example. (ex. [https://attack.mitre.org/tactics/TA0002/](https://attack.mitre.org/tactics/TA0002/) )

type: keyword

Note: this field should contain an array of values.

example: `https://attack.mitre.org/tactics/TA0002/`
| extended | +| $$$field-threat-technique-id$$$[threat.technique.id](#field-threat-technique-id) | The id of technique used by this threat. You can use a MITRE ATT&CK® technique, for example. (ex. [https://attack.mitre.org/techniques/T1059/](https://attack.mitre.org/techniques/T1059/))

type: keyword

Note: this field should contain an array of values.

example: `T1059`
| extended | +| $$$field-threat-technique-name$$$[threat.technique.name](#field-threat-technique-name) | The name of technique used by this threat. You can use a MITRE ATT&CK® technique, for example. (ex. [https://attack.mitre.org/techniques/T1059/](https://attack.mitre.org/techniques/T1059/))

type: keyword

Multi-fields:

- threat.technique.name.text (type: match_only_text)

Note: this field should contain an array of values.

example: `Command and Scripting Interpreter`
| extended | +| $$$field-threat-technique-reference$$$[threat.technique.reference](#field-threat-technique-reference) | The reference url of technique used by this threat. You can use a MITRE ATT&CK® technique, for example. (ex. [https://attack.mitre.org/techniques/T1059/](https://attack.mitre.org/techniques/T1059/))

type: keyword

Note: this field should contain an array of values.

example: `https://attack.mitre.org/techniques/T1059/`
| extended | +| $$$field-threat-technique-subtechnique-id$$$[threat.technique.subtechnique.id](#field-threat-technique-subtechnique-id) | The full id of subtechnique used by this threat. You can use a MITRE ATT&CK® subtechnique, for example. (ex. [https://attack.mitre.org/techniques/T1059/001/](https://attack.mitre.org/techniques/T1059/001/))

type: keyword

Note: this field should contain an array of values.

example: `T1059.001`
| extended | +| $$$field-threat-technique-subtechnique-name$$$[threat.technique.subtechnique.name](#field-threat-technique-subtechnique-name) | The name of subtechnique used by this threat. You can use a MITRE ATT&CK® subtechnique, for example. (ex. [https://attack.mitre.org/techniques/T1059/001/](https://attack.mitre.org/techniques/T1059/001/))

type: keyword

Multi-fields:

- threat.technique.subtechnique.name.text (type: match_only_text)

Note: this field should contain an array of values.

example: `PowerShell`
| extended | +| $$$field-threat-technique-subtechnique-reference$$$[threat.technique.subtechnique.reference](#field-threat-technique-subtechnique-reference) | The reference url of subtechnique used by this threat. You can use a MITRE ATT&CK® subtechnique, for example. (ex. [https://attack.mitre.org/techniques/T1059/001/](https://attack.mitre.org/techniques/T1059/001/))

type: keyword

Note: this field should contain an array of values.

example: `https://attack.mitre.org/techniques/T1059/001/`
| extended | + + +## Field reuse [_field_reuse_26] + + +### Field sets that can be nested under threat [ecs-threat-nestings] + +| Location | Field Set | Description | +| --- | --- | --- | +| `threat.enrichments.indicator.as.*` | [as](/reference/ecs-as.md) | Fields describing an Autonomous System (Internet routing prefix). | +| `threat.enrichments.indicator.file.*` | [file](/reference/ecs-file.md) | Fields describing files. | +| `threat.enrichments.indicator.geo.*` | [geo](/reference/ecs-geo.md) | Fields describing a location. | +| `threat.enrichments.indicator.registry.*` | [registry](/reference/ecs-registry.md) | Fields related to Windows Registry operations. | +| `threat.enrichments.indicator.url.*` | [url](/reference/ecs-url.md) | Fields that let you store URLs in various forms. | +| `threat.enrichments.indicator.x509.*` | [x509](/reference/ecs-x509.md) | These fields contain x509 certificate metadata. | +| `threat.indicator.as.*` | [as](/reference/ecs-as.md) | Fields describing an Autonomous System (Internet routing prefix). | +| `threat.indicator.file.*` | [file](/reference/ecs-file.md) | Fields describing files. | +| `threat.indicator.geo.*` | [geo](/reference/ecs-geo.md) | Fields describing a location. | +| `threat.indicator.registry.*` | [registry](/reference/ecs-registry.md) | Fields related to Windows Registry operations. | +| `threat.indicator.url.*` | [url](/reference/ecs-url.md) | Fields that let you store URLs in various forms. | +| `threat.indicator.x509.*` | [x509](/reference/ecs-x509.md) | These fields contain x509 certificate metadata. | + + +## Threat field usage [_threat_field_usage] + +For usage and examples of the threat fields, please see the [Threat Fields Usage and Examples](/reference/ecs-threat-usage.md) section. + + diff --git a/docs/reference/ecs-tls.md b/docs/reference/ecs-tls.md new file mode 100644 index 0000000000..d0dc559c70 --- /dev/null +++ b/docs/reference/ecs-tls.md @@ -0,0 +1,58 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-tls.html +applies_to: + stack: all + serverless: all +--- + +# TLS fields [ecs-tls] + +Fields related to a TLS connection. These fields focus on the TLS protocol itself and intentionally avoids in-depth analysis of the related x.509 certificate files. + + +## TLS field details [_tls_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-tls-cipher$$$[tls.cipher](#field-tls-cipher) | String indicating the cipher used during the current connection.

type: keyword

example: `TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [tls.cipher](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-cipher)
| extended | +| $$$field-tls-client-certificate$$$[tls.client.certificate](#field-tls-client-certificate) | PEM-encoded stand-alone certificate offered by the client. This is usually mutually-exclusive of `client.certificate_chain` since this value also exists in that list.

type: keyword

example: `MII...`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [tls.client.certificate](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-certificate)
| extended | +| $$$field-tls-client-certificate-chain$$$[tls.client.certificate_chain](#field-tls-client-certificate-chain) | Array of PEM-encoded certificates that make up the certificate chain offered by the client. This is usually mutually-exclusive of `client.certificate` since that value should be the first certificate in the chain.

type: keyword

Note: this field should contain an array of values.

example: `["MII...", "MII..."]`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [tls.client.certificate_chain](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-certificate-chain)
| extended | +| $$$field-tls-client-hash-md5$$$[tls.client.hash.md5](#field-tls-client-hash-md5) | Certificate fingerprint using the MD5 digest of DER-encoded version of certificate offered by the client. For consistency with other hash values, this value should be formatted as an uppercase hash.

type: keyword

example: `0F76C7F2C55BFD7D8E8B8F4BFBF0C9EC`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [tls.client.hash.md5](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-hash-md5)
| extended | +| $$$field-tls-client-hash-sha1$$$[tls.client.hash.sha1](#field-tls-client-hash-sha1) | Certificate fingerprint using the SHA1 digest of DER-encoded version of certificate offered by the client. For consistency with other hash values, this value should be formatted as an uppercase hash.

type: keyword

example: `9E393D93138888D288266C2D915214D1D1CCEB2A`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [tls.client.hash.sha1](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-hash-sha1)
| extended | +| $$$field-tls-client-hash-sha256$$$[tls.client.hash.sha256](#field-tls-client-hash-sha256) | Certificate fingerprint using the SHA256 digest of DER-encoded version of certificate offered by the client. For consistency with other hash values, this value should be formatted as an uppercase hash.

type: keyword

example: `0687F666A054EF17A08E2F2162EAB4CBC0D265E1D7875BE74BF3C712CA92DAF0`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [tls.client.hash.sha256](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-hash-sha256)
| extended | +| $$$field-tls-client-issuer$$$[tls.client.issuer](#field-tls-client-issuer) | Distinguished name of subject of the issuer of the x.509 certificate presented by the client.

type: keyword

example: `CN=Example Root CA, OU=Infrastructure Team, DC=example, DC=com`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [tls.client.issuer](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-issuer)
| extended | +| $$$field-tls-client-ja3$$$[tls.client.ja3](#field-tls-client-ja3) | A hash that identifies clients based on how they perform an SSL/TLS handshake.

type: keyword

example: `d4e5b18d6b55c71272893221c96ba240`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [tls.client.ja3](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-ja3)
| extended | +| $$$field-tls-client-not-after$$$[tls.client.not_after](#field-tls-client-not-after) | Date/Time indicating when client certificate is no longer considered valid.

type: date

example: `2021-01-01T00:00:00.000Z`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [tls.client.not_after](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-not-after)
| extended | +| $$$field-tls-client-not-before$$$[tls.client.not_before](#field-tls-client-not-before) | Date/Time indicating when client certificate is first considered valid.

type: date

example: `1970-01-01T00:00:00.000Z`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [tls.client.not_before](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-not-before)
| extended | +| $$$field-tls-client-server-name$$$[tls.client.server_name](#field-tls-client-server-name) | Also called an SNI, this tells the server which hostname to which the client is attempting to connect to. When this value is available, it should get copied to `destination.domain`.

type: keyword

example: `www.elastic.co`
| extended | +| $$$field-tls-client-subject$$$[tls.client.subject](#field-tls-client-subject) | Distinguished name of subject of the x.509 certificate presented by the client.

type: keyword

example: `CN=myclient, OU=Documentation Team, DC=example, DC=com`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [tls.client.subject](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-subject)
| extended | +| $$$field-tls-client-supported-ciphers$$$[tls.client.supported_ciphers](#field-tls-client-supported-ciphers) | Array of ciphers offered by the client during the client hello.

type: keyword

Note: this field should contain an array of values.

example: `["TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384", "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384", "..."]`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [tls.client.supported_ciphers](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-client-supported-ciphers)
| extended | +| $$$field-tls-curve$$$[tls.curve](#field-tls-curve) | String indicating the curve used for the given cipher, when applicable.

type: keyword

example: `secp256r1`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [tls.curve](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-curve)
| extended | +| $$$field-tls-established$$$[tls.established](#field-tls-established) | Boolean flag indicating if the TLS negotiation was successful and transitioned to an encrypted tunnel.

type: boolean

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [tls.established](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-established)
| extended | +| $$$field-tls-next-protocol$$$[tls.next_protocol](#field-tls-next-protocol) | String indicating the protocol being tunneled. Per the values in the IANA registry ([https://www.iana.org/assignments/tls-extensiontype-values/tls-extensiontype-values.xhtml#alpn-protocol-ids](https://www.iana.org/assignments/tls-extensiontype-values/tls-extensiontype-values.xhtml#alpn-protocol-ids)), this string should be lower case.

type: keyword

example: `http/1.1`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [tls.next_protocol](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-next-protocol)
| extended | +| $$$field-tls-resumed$$$[tls.resumed](#field-tls-resumed) | Boolean flag indicating if this TLS connection was resumed from an existing TLS negotiation.

type: boolean

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [tls.resumed](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-resumed)
| extended | +| $$$field-tls-server-certificate$$$[tls.server.certificate](#field-tls-server-certificate) | PEM-encoded stand-alone certificate offered by the server. This is usually mutually-exclusive of `server.certificate_chain` since this value also exists in that list.

type: keyword

example: `MII...`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [tls.server.certificate](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-certificate)
| extended | +| $$$field-tls-server-certificate-chain$$$[tls.server.certificate_chain](#field-tls-server-certificate-chain) | Array of PEM-encoded certificates that make up the certificate chain offered by the server. This is usually mutually-exclusive of `server.certificate` since that value should be the first certificate in the chain.

type: keyword

Note: this field should contain an array of values.

example: `["MII...", "MII..."]`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [tls.server.certificate_chain](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-certificate-chain)
| extended | +| $$$field-tls-server-hash-md5$$$[tls.server.hash.md5](#field-tls-server-hash-md5) | Certificate fingerprint using the MD5 digest of DER-encoded version of certificate offered by the server. For consistency with other hash values, this value should be formatted as an uppercase hash.

type: keyword

example: `0F76C7F2C55BFD7D8E8B8F4BFBF0C9EC`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [tls.server.hash.md5](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-hash-md5)
| extended | +| $$$field-tls-server-hash-sha1$$$[tls.server.hash.sha1](#field-tls-server-hash-sha1) | Certificate fingerprint using the SHA1 digest of DER-encoded version of certificate offered by the server. For consistency with other hash values, this value should be formatted as an uppercase hash.

type: keyword

example: `9E393D93138888D288266C2D915214D1D1CCEB2A`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [tls.server.hash.sha1](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-hash-sha1)
| extended | +| $$$field-tls-server-hash-sha256$$$[tls.server.hash.sha256](#field-tls-server-hash-sha256) | Certificate fingerprint using the SHA256 digest of DER-encoded version of certificate offered by the server. For consistency with other hash values, this value should be formatted as an uppercase hash.

type: keyword

example: `0687F666A054EF17A08E2F2162EAB4CBC0D265E1D7875BE74BF3C712CA92DAF0`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [tls.server.hash.sha256](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-hash-sha256)
| extended | +| $$$field-tls-server-issuer$$$[tls.server.issuer](#field-tls-server-issuer) | Subject of the issuer of the x.509 certificate presented by the server.

type: keyword

example: `CN=Example Root CA, OU=Infrastructure Team, DC=example, DC=com`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [tls.server.issuer](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-issuer)
| extended | +| $$$field-tls-server-ja3s$$$[tls.server.ja3s](#field-tls-server-ja3s) | A hash that identifies servers based on how they perform an SSL/TLS handshake.

type: keyword

example: `394441ab65754e2207b1e1b457b3641d`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [tls.server.ja3s](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-ja3s)
| extended | +| $$$field-tls-server-not-after$$$[tls.server.not_after](#field-tls-server-not-after) | Timestamp indicating when server certificate is no longer considered valid.

type: date

example: `2021-01-01T00:00:00.000Z`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [tls.server.not_after](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-not-after)
| extended | +| $$$field-tls-server-not-before$$$[tls.server.not_before](#field-tls-server-not-before) | Timestamp indicating when server certificate is first considered valid.

type: date

example: `1970-01-01T00:00:00.000Z`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [tls.server.not_before](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-not-before)
| extended | +| $$$field-tls-server-subject$$$[tls.server.subject](#field-tls-server-subject) | Subject of the x.509 certificate presented by the server.

type: keyword

example: `CN=www.example.com, OU=Infrastructure Team, DC=example, DC=com`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [tls.server.subject](https://opentelemetry.io/docs/specs/semconv/attributes-registry/tls/#tls-server-subject)
| extended | +| $$$field-tls-version$$$[tls.version](#field-tls-version) | Numeric part of the version parsed from the original string.

type: keyword

example: `1.2`
| extended | +| $$$field-tls-version-protocol$$$[tls.version_protocol](#field-tls-version-protocol) | Normalized lowercase protocol name parsed from original string.

type: keyword

example: `tls`
| extended | + + +## Field reuse [_field_reuse_27] + + +### Field sets that can be nested under TLS [ecs-tls-nestings] + +| Location | Field Set | Description | +| --- | --- | --- | +| `tls.client.x509.*` | [x509](/reference/ecs-x509.md) | These fields contain x509 certificate metadata. | +| `tls.server.x509.*` | [x509](/reference/ecs-x509.md) | These fields contain x509 certificate metadata. | + diff --git a/docs/reference/ecs-tracing.md b/docs/reference/ecs-tracing.md new file mode 100644 index 0000000000..5209c1a2f5 --- /dev/null +++ b/docs/reference/ecs-tracing.md @@ -0,0 +1,23 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-tracing.html +applies_to: + stack: all + serverless: all +--- + +# Tracing fields [ecs-tracing] + +Distributed tracing makes it possible to analyze performance throughout a microservice architecture all in one view. This is accomplished by tracing all of the requests - from the initial web request in the front-end service - to queries made through multiple back-end services. + +Unlike most field sets in ECS, the tracing fields are **not** nested under the field set name. In other words, the correct field name is `trace.id`, not `tracing.trace.id`, and so on. + + +## Tracing field details [_tracing_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-span-id$$$[span.id](#field-span-id) | Unique identifier of the span within the scope of its trace.

A span represents an operation within a transaction, such as a request to another service, or a database query.

type: keyword

example: `3ff9a8981b7ccd5a`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/OTLP-ffdcb2?style=flat "otlp") [span_id](https://github.com/search?q=repo%3Aopen-telemetry%2Fopentelemetry-proto+%22+span_id+%22&type=code)
| extended | +| $$$field-trace-id$$$[trace.id](#field-trace-id) | Unique identifier of the trace.

A trace groups multiple events like transactions that belong together. For example, a user request handled by multiple inter-connected services.

type: keyword

example: `4bf92f3577b34da6a3ce929d0e0e4736`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/OTLP-ffdcb2?style=flat "otlp") [trace_id](https://github.com/search?q=repo%3Aopen-telemetry%2Fopentelemetry-proto+%22+trace_id+%22&type=code)
| extended | +| $$$field-transaction-id$$$[transaction.id](#field-transaction-id) | Unique identifier of the transaction within the scope of its trace.

A transaction is the highest level of work measured within a service, such as a request to a server.

type: keyword

example: `00f067aa0ba902b7`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/n%2Fa-f2f4fb?style=flat "not-applicable") Not applicable.
| extended | + diff --git a/docs/reference/ecs-url.md b/docs/reference/ecs-url.md new file mode 100644 index 0000000000..01229ae8c6 --- /dev/null +++ b/docs/reference/ecs-url.md @@ -0,0 +1,42 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-url.html +applies_to: + stack: all + serverless: all +--- + +# URL fields [ecs-url] + +URL fields provide support for complete or partial URLs, and supports the breaking down into scheme, domain, path, and so on. + + +## URL field details [_url_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-url-domain$$$[url.domain](#field-url-domain) | Domain of the url, such as "www.elastic.co".

In some cases a URL may refer to an IP and/or port directly, without a domain name. In this case, the IP address would go to the `domain` field.

If the URL contains a literal IPv6 address enclosed by `[` and `]` (IETF RFC 2732), the `[` and `]` characters should also be captured in the `domain` field.

type: keyword

example: `www.elastic.co`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [url.domain](https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-domain)
| extended | +| $$$field-url-extension$$$[url.extension](#field-url-extension) | The field contains the file extension from the original request url, excluding the leading dot.

The file extension is only set if it exists, as not every url has a file extension.

The leading period must not be included. For example, the value must be "png", not ".png".

Note that when the file name has multiple extensions (example.tar.gz), only the last one should be captured ("gz", not "tar.gz").

type: keyword

example: `png`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [url.extension](https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-extension)
| extended | +| $$$field-url-fragment$$$[url.fragment](#field-url-fragment) | Portion of the url after the `#`, such as "top".

The `#` is not part of the fragment.

type: keyword

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [url.fragment](https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-fragment)
| extended | +| $$$field-url-full$$$[url.full](#field-url-full) | If full URLs are important to your use case, they should be stored in `url.full`, whether this field is reconstructed or present in the event source.

type: wildcard

Multi-fields:

- url.full.text (type: match_only_text)

example: `https://www.elastic.co:443/search?q=elasticsearch#top`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [url.full](https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-full)
| extended | +| $$$field-url-original$$$[url.original](#field-url-original) | Unmodified original url as seen in the event source.

Note that in network monitoring, the observed URL may be a full URL, whereas in access logs, the URL is often just represented as a path.

This field is meant to represent the URL as it was observed, complete or not.

type: wildcard

Multi-fields:

- url.original.text (type: match_only_text)

example: `https://www.elastic.co:443/search?q=elasticsearch#top or /search?q=elasticsearch`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [url.original](https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-original)
| extended | +| $$$field-url-password$$$[url.password](#field-url-password) | Password of the request.

type: keyword
| extended | +| $$$field-url-path$$$[url.path](#field-url-path) | Path of the request, such as "/search".

type: wildcard

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [url.path](https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-path)
| extended | +| $$$field-url-port$$$[url.port](#field-url-port) | Port of the request, such as 443.

type: long

example: `443`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [url.port](https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-port)
| extended | +| $$$field-url-query$$$[url.query](#field-url-query) | The query field describes the query string of the request, such as "q=elasticsearch".

The `?` is excluded from the query string. If a URL contains no `?`, there is no query field. If there is a `?` but no query, the query field exists with an empty string. The `exists` query can be used to differentiate between the two cases.

type: keyword

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [url.query](https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-query)
| extended | +| $$$field-url-registered-domain$$$[url.registered_domain](#field-url-registered-domain) | The highest registered url domain, stripped of the subdomain.

For example, the registered domain for "foo.example.com" is "example.com".

This value can be determined precisely with a list like the public suffix list ([https://publicsuffix.org](https://publicsuffix.org)). Trying to approximate this by simply taking the last two labels will not work well for TLDs such as "co.uk".

type: keyword

example: `example.com`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [url.registered_domain](https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-registered-domain)
| extended | +| $$$field-url-scheme$$$[url.scheme](#field-url-scheme) | Scheme of the request, such as "https".

Note: The `:` is not part of the scheme.

type: keyword

example: `https`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [url.scheme](https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-scheme)
| extended | +| $$$field-url-subdomain$$$[url.subdomain](#field-url-subdomain) | The subdomain portion of a fully qualified domain name includes all of the names except the host name under the registered_domain. In a partially qualified domain, or if the the qualification level of the full name cannot be determined, subdomain contains all of the names below the registered domain.

For example the subdomain portion of "www.east.mydomain.co.uk" is "east". If the domain has multiple levels of subdomain, such as "sub2.sub1.example.com", the subdomain field should contain "sub2.sub1", with no trailing period.

type: keyword

example: `east`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [url.subdomain](https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-subdomain)
| extended | +| $$$field-url-top-level-domain$$$[url.top_level_domain](#field-url-top-level-domain) | The effective top level domain (eTLD), also known as the domain suffix, is the last part of the domain name. For example, the top level domain for example.com is "com".

This value can be determined precisely with a list like the public suffix list ([https://publicsuffix.org](https://publicsuffix.org)). Trying to approximate this by simply taking the last label will not work well for effective TLDs such as "co.uk".

type: keyword

example: `co.uk`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [url.top_level_domain](https://opentelemetry.io/docs/specs/semconv/attributes-registry/url/#url-top-level-domain)
| extended | +| $$$field-url-username$$$[url.username](#field-url-username) | Username of the request.

type: keyword
| extended | + + +## Field reuse [_field_reuse_28] + +The `url` fields are expected to be nested at: + +* `threat.enrichments.indicator.url` +* `threat.indicator.url` + +Note also that the `url` fields may be used directly at the root of the events. + diff --git a/docs/reference/ecs-user-usage.md b/docs/reference/ecs-user-usage.md new file mode 100644 index 0000000000..7fbe044b20 --- /dev/null +++ b/docs/reference/ecs-user-usage.md @@ -0,0 +1,349 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-user-usage.html +applies_to: + stack: all + serverless: all +--- + +# User fields usage and examples [ecs-user-usage] + +Here are the subjects covered in this page. + +* [Categorization](#ecs-user-usage-categorization) +* [User identifiers](#ecs-user-identifiers) +* [Field reuse](#ecs-user-usage-field-reuse), or all places user fields can appear + + * [User fields at the Root of an Event](#ecs-user-usage-user-at-root) + * [Remote logons](#ecs-user-usage-remote-logons) + * [Privilege changes](#ecs-user-usage-privilege-changes) + * [Identity and access management](#ecs-user-usage-iam) + * [Combining IAM and privilege change](#ecs-user-usage-combining) + * [Subtleties around field reuse](#ecs-user-usage-reuse-subtleties) + +* [Pivoting via related.user](#ecs-user-usage-pivoting) +* [Mapping examples](#ecs-user-usage-mappings) + + +## Categorization [ecs-user-usage-categorization] + +User fields can be present in any kind of event, without affecting the event’s categorization. + +However when the event is about IAM (Identity and Account Management), it should be categorized as follows. In this section we’ll cover specifically `event.category` and `event.type` with regards to IAM activity. Make sure to read the [Categorization section](/reference/ecs-category-field-values-reference.md) to see all allowed values, and read more about `event.kind` and `event.outcome`. + +::::{note} +IAM activity is a bit particular in that events are expected to be assigned 2 event types. One of them indicates the type of activity (creation, deletion, change, etc.), and the other indicates whether a user or a group is the target of the management activity. +:::: + + +Many sections of the examples below are elided, in order to focus on the categorization of the events. + +Creation of group "test-group": + +```JSON +{ + "event": { + "kind": "event", + "category": ["iam"], <1> + "type": ["group", "creation"], <2> + "outcome": "success" + }, + "group": { "name": "test-group", ... }, + "user": { ... }, + "related": { "user": [ ... ] } +} +``` + +1. Category "iam" +2. Both relevant event types to a group creation + + +Adding "test-user" to "test-group": + +```JSON +{ + "event": { + "kind": "event", + "category": ["iam"], <1> + "type": ["user", "change"], <2> + "action": "user added to group", <3> + "outcome": "success" + }, + "user": { + ... + "target": { <4> + "name": "test-user", + "group": { "name": "test-group" } + } + }, + "related": { "user": [ ... ] } +} +``` + +1. Category "iam" +2. Both relevant event types to a user modification +3. `event.action` is not a categorization field, and has no mandated value. It can be populated based on source event details or by a pipeline, to ensure the event captures all subtleties of what’s happening. +4. How to use all possible user fields is detailed below. + + + +## User identifiers [ecs-user-identifiers] + +Different systems use different values for user identifiers. Here are a few pointers to help normalize some simple cases. + +* When a system provides a composite value for the user name (e.g. DOMAINNAME\username), capture the domain name in `user.domain` and the user name (without the domain) in `user.name`. +* When a system uses an email address as the main identifier, populate both `user.id` and `user.email` with it. + + +## Field reuse [ecs-user-usage-field-reuse] + +The user fields can be reused (or appear) in many places across ECS. This makes it possible to capture many users relevant to a single event. + +Here’s the full list of places where the user fields can appear: + +* `user.*` +* `user.effective.*` +* `user.target.*` +* `user.changes.*` +* `source.user.*` +* `destination.user.*` +* `client.user.*` +* `server.user.*` + +Let’s go over the meaning of each. + +The examples below will only populate `user.name` and sometimes `user.id` inside the various `user` nestings, for readability. However in implementations, unless otherwise noted, all `user` fields that can reasonably be populated in each location should be populated. + + +### User fields at the Root of an event [ecs-user-usage-user-at-root] + +The user fields at the root of an event are used to capture the user performing the main action described by the event. This is especially important when there’s more than one user present on the event. `user.*` fields at the root of the event represent the user performing the action. + +In many cases, events that only mention one user should populate the user fields at the root of the event, even if the user is not the one performing the action. + +In cases where a purpose-specific user field such as `url.username` is populated, `user.name` should also be populated with the same user name. + +```json +{ + "url": { "username": "alice" }, <1> + "user": { "name": "alice" }, <2> + "related": { "user": ["alice"] } +} +``` + +1. Purpose-specific username field +2. Username copied to `user.name` to establish `user.name` as a reliable baseline. + + + +### Remote logons [ecs-user-usage-remote-logons] + +When users are crossing host boundaries, the users are captured at `source.user` and `destination.user`. + +Examples of data sources where this is applicable: + +* Remote logons via ssh, kerberos +* Firewalls observing network traffic + +In order to align with ECS' design of having `user` at the root of the event as the user performing the action, all `source.user` fields should be copied to `user` at the root. + +Here’s an example where user "alice" logs on to another host as user "deus": + +```json +{ + "user": { + "name": "alice" + }, + "source": { + "user": { + "name": "alice" + }, + "ip": "10.42.42.42" + }, + "destination": { + "user": { + "name": "deus" + }, + "ip": "10.42.42.43" + }, + "related": { "user": ["alice", "deus"] } +} +``` + +Whenever an event source populates the `client` and `server` fields in addition to `source` and `destination`, the user fields should be copied accordingly as well. You can review [Mapping network events](/reference/ecs-mapping-network-events.md) to learn more about mapping network events. + + +### Privilege changes [ecs-user-usage-privilege-changes] + +The `user.effective` fields are relevant when there’s a privilege escalation or demotion and it’s possible to determine the user requesting/performing the escalation. + +Use the `user` fields at the root to capture who is requesting the privilege change, and `user.effective` to capture the requested privilege level, whether or not the privilege change was successful. + +Here are examples where this is applicable: + +* A user changing identity on a host. + + * Examples: sudo, su, Run as. + +* Running a program as a different user. Examples: + + * A trusted user runs a specific admin command as root via a mechanism such as the Posix setuid/setgid. + * A service manager with administrator privileges starts child processes as limited users, for security purposes (e.g. root runs Apache HTTPD as user "apache") + + +In cases where the event source only gives information about the effective user and not who requested different privileges, the `user` fields at the root of the event should be used instead of `user.effective`. + +Here’s an example of user "alice" running a command as root via sudo: + +```json +{ + "user": { + "name": "alice", + "id": "1001", + "effective": { + "name": "root", + "id": "1" + } + }, + "related": { "user": ["alice", "root"] } +} +``` + +When it’s not possible (or it’s prohibitive) to determine which user is requesting different privilege levels, it’s acceptable to capture the effective user at the root of the event. Typically a privilege change event will already have happened, for example: bob "su" as root; and subsequent events will show the root user performing the actions. + + +### Identity and access management [ecs-user-usage-iam] + +Whenever a user is performing an action that affects another user — typically in IAM scenarios — the user affected by the action is captured at `user.target`. The user performing the IAM activity is captured at the root of the event. + +Examples of IAM activity include: + +* user-a creates or deletes user-b +* user-a modifies user-b + +In the create/delete scenarios, there’s either no prior state (user creation) or no post state (user deletion). In these cases, only `user` at the root and `user.target` must be populated. + +Example where "root" creates user "bob": + +```json +{ + "user": { + "name": "root", + "id": "1", + "target": { + "name": "bob", + "id": "1002", + ... + } + } + "related": { "user": ["bob", "root"] } +} +``` + +When there’s a change of state to an existing user, `user.target` must be used to capture the prior state of the user, and `user.changes` should list only the changes that were performed. + +Example where "root" renames user "bob" to "bob.barker": + +```json +{ + "user": { + "name": "root", + "id": "1", + "target": { + "name": "bob", + "id": "1002" + }, + "changes": { + "name": "bob.barker" + } + }, + "related": { "user": ["bob", "bob.barker", "root"] } +} +``` + +You’ll note in the example above that unmodified attributes like the user ID are not repeated under `user.changes.*`, since they didn’t change. + + +### Combining IAM and privilege change [ecs-user-usage-combining] + +We’ve covered above how `user.target` and `user.changes` can be used at the same time. If privilege escalation is also present in the same IAM event, `user.effective` should of course be used as well. + +Here’s the "rename" example from the IAM section above. In the following example, we know "alice" is escalating privileges to "root", in order to modify user "bob": + +```json +{ + "user": { + "name": "alice", + "id": "1001", + "effective": { + "name": "root", + "id": "1" + }, + "target": { + "name": "bob", + "id": "1002" + }, + "changes": { + "name": "bob.barker" + } + }, + "related": { "user": ["alice", "bob", "bob.barker", "root"] } +} +``` + + +### Subtleties around field reuse [ecs-user-usage-reuse-subtleties] + +Most cases of field reuse in ECS are reusing a field set inside a different field set. Two examples of this are: + +* reusing `group` in `user`, resulting in the `user.group.*` fields, or +* reusing `user` in `destination`, resulting in the `destination.user.*` fields, which also include `destination.user.group.*`. + +The `user` fields can also be reused within `user` as different names, representing the role of each relevant user. Examples are the `user.target.*` or `user.effective.*` fields. + +However, it’s important to note that `user` fields reused within `user` are *not carried around anywhere else*. Let’s illustrate the various permutations of what’s valid and what is not. + +| Field | Validity | Notes | +| --- | --- | --- | +| `user.group.*` | Valid | Normal reuse. | +| `destination.user.group.*` | Valid | The `group` reuse gets carried around when `user` is reused elsewhere.Populate only if relevant to the event. | +| `user.target.group.*`, `user.effective.group.*`, `user.changes.group.*` | Valid | The `group` reuse gets carried around even when `user` is reused within itself.Populate only if relevant to the event. | +| `destination.user.target.*`, `destination.user.effective.*`, `destination.user.changes.*` | **Invalid** | The `user` fields reused within `user` are not carried around anywhere else.The same rule applies when `user` is reused under `source`, `client` and `server`. | + + +## Pivoting via related.user [ecs-user-usage-pivoting] + +In all events in this page, we’ve populated the `related.user` fields. + +Any event that has users in it should always populate the array field `related.user` with all usernames seen in the event; including event names that appear in custom fields. Note that this field is not a nesting of all user fields, it’s a flat array meant to contain user identifiers. + +Taking the example from `user.changes` again, we can see that no matter the role of the each user (before/after privilege escalation, affected user, username after rename), they are all present in `related.user`: + +```json +{ + "user": { + "name": "alice", + "id": "1001", + "effective": { + "name": "root", + "id": "1" + }, + "target": { + "name": "bob", + "id": "1002" + }, + "changes": { + "name": "bob.barker" + } + }, + "related": { "user": ["alice", "root", "bob", "bob.barker"] } +} +``` + +Like the other fields in the [related](/reference/ecs-related.md) field set, `related.user` is meant to facilitate pivoting. For example, if you have a suspicion about user "bob.barker", searching for this name in `related.user` will give you all events related to this user, whether it’s the creation / rename of the user, or events where this user was active in a system. + + +## Mapping examples [ecs-user-usage-mappings] + +For examples of mapping events from various sources, you can look at [RFC 0007 in section Source Data](https://github.com/elastic/ecs/blob/master/rfcs/text/0007-multiple-users.md#source-data). + diff --git a/docs/reference/ecs-user.md b/docs/reference/ecs-user.md new file mode 100644 index 0000000000..2571a5fff6 --- /dev/null +++ b/docs/reference/ecs-user.md @@ -0,0 +1,63 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-user.html +applies_to: + stack: all + serverless: all +--- + +# User fields [ecs-user] + +The user fields describe information about the user that is relevant to the event. + +Fields can have one entry or multiple entries. If a user has more than one id, provide an array that includes all of them. + + +## User field details [_user_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-user-domain$$$[user.domain](#field-user-domain) | Name of the directory the user is a member of.

For example, an LDAP or Active Directory domain name.

type: keyword
| extended | +| $$$field-user-email$$$[user.email](#field-user-email) | User email address.

type: keyword

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [user.email](https://opentelemetry.io/docs/specs/semconv/attributes-registry/user/#user-email)
| extended | +| $$$field-user-full-name$$$[user.full_name](#field-user-full-name) | User’s full name, if available.

type: keyword

Multi-fields:

- user.full_name.text (type: match_only_text)

example: `Albert Einstein`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [user.full_name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/user/#user-full-name)
| extended | +| $$$field-user-hash$$$[user.hash](#field-user-hash) | Unique user hash to correlate information for a user in anonymized form.

Useful if `user.id` or `user.name` contain confidential information and cannot be used.

type: keyword

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [user.hash](https://opentelemetry.io/docs/specs/semconv/attributes-registry/user/#user-hash)
| extended | +| $$$field-user-id$$$[user.id](#field-user-id) | Unique identifier of the user.

type: keyword

example: `S-1-5-21-202424912787-2692429404-2351956786-1000`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [user.id](https://opentelemetry.io/docs/specs/semconv/attributes-registry/user/#user-id)
| core | +| $$$field-user-name$$$[user.name](#field-user-name) | Short name or login of the user.

type: keyword

Multi-fields:

- user.name.text (type: match_only_text)

example: `a.einstein`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [user.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/user/#user-name)
| core | +| $$$field-user-roles$$$[user.roles](#field-user-roles) | Array of user roles at the time of the event.

type: keyword

Note: this field should contain an array of values.

example: `["kibana_admin", "reporting_user"]`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [user.roles](https://opentelemetry.io/docs/specs/semconv/attributes-registry/user/#user-roles)
| extended | + + +## Field reuse [_field_reuse_29] + +The `user` fields are expected to be nested at: + +* `client.user` +* `destination.user` +* `process.attested_user` +* `process.real_user` +* `process.saved_user` +* `process.user` +* `server.user` +* `source.user` +* `user.changes` +* `user.effective` +* `user.target` + +Note also that the `user` fields may be used directly at the root of the events. + + +### Field sets that can be nested under User [ecs-user-nestings] + +| Location | Field Set | Description | +| --- | --- | --- | +| `user.changes.*` | `user` | Captures changes made to a user. | +| `user.effective.*` | `user` | User whose privileges were assumed. | +| `user.group.*` | [group](/reference/ecs-group.md) | User’s group relevant to the event. | +| `user.risk.*` | [risk](/reference/ecs-risk.md) | Fields for describing risk score and level. | +| `user.target.*` | `user` | Targeted user of action taken. | + + +## User field usage [_user_field_usage] + +For usage and examples of the user fields, please see the [User Fields Usage and Examples](/reference/ecs-user-usage.md) section. + + diff --git a/docs/reference/ecs-user_agent.md b/docs/reference/ecs-user_agent.md new file mode 100644 index 0000000000..3d903581ed --- /dev/null +++ b/docs/reference/ecs-user_agent.md @@ -0,0 +1,34 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-user_agent.html +applies_to: + stack: all + serverless: all +--- + +# User agent fields [ecs-user_agent] + +The user_agent fields normally come from a browser request. + +They often show up in web service logs coming from the parsed user agent string. + + +## User agent field details [_user_agent_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-user-agent-device-name$$$[user_agent.device.name](#field-user-agent-device-name) | Name of the device.

type: keyword

example: `iPhone`
| extended | +| $$$field-user-agent-name$$$[user_agent.name](#field-user-agent-name) | Name of the user agent.

type: keyword

example: `Safari`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [user_agent.name](https://opentelemetry.io/docs/specs/semconv/attributes-registry/user_agent/#user-agent-name)
| extended | +| $$$field-user-agent-original$$$[user_agent.original](#field-user-agent-original) | Unparsed user_agent string.

type: keyword

Multi-fields:

- user_agent.original.text (type: match_only_text)

example: `Mozilla/5.0 (iPhone; CPU iPhone OS 12_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.0 Mobile/15E148 Safari/604.1`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [user_agent.original](https://opentelemetry.io/docs/specs/semconv/attributes-registry/user_agent/#user-agent-original)
| extended | +| $$$field-user-agent-version$$$[user_agent.version](#field-user-agent-version) | Version of the user agent.

type: keyword

example: `12.0`

![OTel Badge](https://img.shields.io/badge/OpenTelemetry-4a5ca6?style=flat&logo=opentelemetry "") ![relation](https://img.shields.io/badge/match-93c93e?style=flat "match") [user_agent.version](https://opentelemetry.io/docs/specs/semconv/attributes-registry/user_agent/#user-agent-version)
| extended | + + +## Field reuse [_field_reuse_30] + + +### Field sets that can be nested under user agent [ecs-user_agent-nestings] + +| Location | Field Set | Description | +| --- | --- | --- | +| `user_agent.os.*` | [os](/reference/ecs-os.md) | OS fields contain information about the operating system. | + diff --git a/docs/fields/field-values-usage.asciidoc b/docs/reference/ecs-using-categorization-fields.md similarity index 58% rename from docs/fields/field-values-usage.asciidoc rename to docs/reference/ecs-using-categorization-fields.md index 88905ba3bc..0cc56151ff 100644 --- a/docs/fields/field-values-usage.asciidoc +++ b/docs/reference/ecs-using-categorization-fields.md @@ -1,5 +1,12 @@ -[[ecs-using-the-categorization-fields]] -=== Using the Categorization Fields +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-using-the-categorization-fields.html +applies_to: + stack: all + serverless: all +--- + +# Using the categorization fields [ecs-using-the-categorization-fields] The event categorization fields work together to identify and group similar events from multiple data sources. @@ -7,21 +14,19 @@ These general principles can help guide the categorization process: * Events from multiple data sources that are similar enough to be viewed or analyzed together, should fall into the same `event.category` field. * Both `event.category` and `event.type` are arrays and may be populated with multiple allowed values, if the event can be reasonably classified into more than one category and/or type. -* `event.kind`, `event.category`, `event.type` and `event.outcome` all have allowed values. This is to normalize these fields. Values that aren't in the list of allowed values should not be used. -* Values of `event.outcome` are a very limited set to indicate success or failure. Domain-specific actions, such as deny and allow, that could be considered outcomes are not - captured in the `event.outcome` field, but rather in the `event.type` and/or `event.action` fields. +* `event.kind`, `event.category`, `event.type` and `event.outcome` all have allowed values. This is to normalize these fields. Values that aren’t in the list of allowed values should not be used. +* Values of `event.outcome` are a very limited set to indicate success or failure. Domain-specific actions, such as deny and allow, that could be considered outcomes are not captured in the `event.outcome` field, but rather in the `event.type` and/or `event.action` fields. * Values of `event.category`, `event.type`, and `event.outcome` are consistent across all values of `event.kind`. -* When a specific event doesn't fit into any of the defined allowed categorization values, the field should be left empty. +* When a specific event doesn’t fit into any of the defined allowed categorization values, the field should be left empty. The following examples detail populating the categorization fields and provides some context for the classification decisions. -[float] -==== Firewall blocking a network connection + +### Firewall blocking a network connection [_firewall_blocking_a_network_connection] This event from a firewall describes a successfully blocked network connection: -[source,json] ----- +```json ... { "source": { @@ -53,30 +58,29 @@ This event from a firewall describes a successfully blocked network connection: } } ... ----- +``` + +1. Classifying as an `event`. +2. `event.category` categorizes this event as `network` activity. +3. The event was both an attempted network `connection` and was `denied`. +4. The blocking of this connection is expected. The outcome is a `success` from the perspective of the firewall emitting the event. +5. The firewall classifies this denied connection as `dropped`, and this value is captured in `event.action`. -<1> Classifying as an `event`. -<2> `event.category` categorizes this event as `network` activity. -<3> The event was both an attempted network `connection` and was `denied`. -<4> The blocking of this connection is expected. The outcome is a `success` from the perspective of the firewall emitting the event. -<5> The firewall classifies this denied connection as `dropped`, and this value is captured in `event.action`. A "denied" network connection could fall under different action values: "blocked", "dropped", "quarantined", etc. The `event.action` field captures the action taken as described by the source, and populating `event.type:denied` provides an independent, normalized value. A single query will return all denied network connections which have been normalized with the same categorization values: -[source,sh] ----- +```sh event.category:network AND event.type:denied ----- +``` -[float] -==== Failed attempt to create a user account + +### Failed attempt to create a user account [_failed_attempt_to_create_a_user_account] User `alice` attempts to add a user account, `bob`, into a directory service, but the action fails: -[source,json] ----- +```json { "user": { "name": "alice", @@ -96,20 +100,20 @@ User `alice` attempts to add a user account, `bob`, into a directory service, bu "outcome": "failure" <4> } } ----- +``` + +1. Again classifying as an `event`. +2. Categorized using `iam` for an event user account activity. +3. Both `user` and `creation` +4. The creation of a user account was attempted, but it was not successful. -<1> Again classifying as an `event`. -<2> Categorized using `iam` for an event user account activity. -<3> Both `user` and `creation` -<4> The creation of a user account was attempted, but it was not successful. -[float] -==== Informational listing of a file + +### Informational listing of a file [_informational_listing_of_a_file] A utility, such as a file integrity monitoring (FIM) application, takes inventory of a file but does not access or modify the file: -[source,json] ----- +```json { "file": { "name": "example.png", @@ -127,21 +131,21 @@ A utility, such as a file integrity monitoring (FIM) application, takes inventor ] } } ----- +``` + +1. Classifying as `event`. +2. The event is reporting on a `file`. +3. The `info` type categorizes purely informational events. The target file here was not accessed or modified. + -<1> Classifying as `event`. -<2> The event is reporting on a `file`. -<3> The `info` type categorizes purely informational events. The target file here was not accessed or modified. +The source data didn’t include any context around the event’s outcome, so `event.outcome` should not be populated. -The source data didn't include any context around the event's outcome, so `event.outcome` should not be populated. -[float] -=== Security application failed to block a network connection +## Security application failed to block a network connection [_security_application_failed_to_block_a_network_connection] An intrusion detection system (IDS) attempts to block a connection but fails. The event emitted by the IDS is considered an alert: -[source,json] ----- +```json { "source": { "address": "10.42.42.42", @@ -167,9 +171,11 @@ An intrusion detection system (IDS) attempts to block a connection but fails. Th "outcome": "failure" <4> } } ----- +``` + +1. The IDS emitted this event when a detection rule generated an alert. The `event.kind` is set to `alert`. +2. With the event emitted from a network IDS device, the event is categorized both as `network` and `intrusion_detection`. +3. The alert event is a `connection` that was `denied` by the IDS' configuration. +4. The IDS experienced an issue when attempting to deny the connection. Since the action taken by the IDS failed, the outcome is set as `failure`. + -<1> The IDS emitted this event when a detection rule generated an alert. The `event.kind` is set to `alert`. -<2> With the event emitted from a network IDS device, the event is categorized both as `network` and `intrusion_detection`. -<3> The alert event is a `connection` that was `denied` by the IDS' configuration. -<4> The IDS experienced an issue when attempting to deny the connection. Since the action taken by the IDS failed, the outcome is set as `failure`. diff --git a/docs/reference/ecs-using-ecs.md b/docs/reference/ecs-using-ecs.md new file mode 100644 index 0000000000..a430d5f738 --- /dev/null +++ b/docs/reference/ecs-using-ecs.md @@ -0,0 +1,24 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-using-ecs.html +applies_to: + stack: all + serverless: all +--- + +# Using ECS [ecs-using-ecs] + +ECS fields follow a series of guidelines, to ensure a consistent and predictable feel, across various use cases. + +If you're new to ECS and would like an introduction on implementing and using the schema, check out the [Getting started](/reference/ecs-getting-started.md) guide. + +Whether you're trying to recall a field name, implementing a solution that follows ECS, or proposing a change to the schema, the [Guidelines and best practices](/reference/ecs-guidelines.md) section will help get you there. + +[Design principles](/reference/ecs-principles-design.md) are a set of considerations used to help guide the ECS project. + +If you're wondering how to best capture event details that don't map to existing ECS fields, head over to [Custom fields](/reference/ecs-custom-fields-in-ecs.md). + + + + + diff --git a/docs/reference/ecs-vlan.md b/docs/reference/ecs-vlan.md new file mode 100644 index 0000000000..0b1ba9765b --- /dev/null +++ b/docs/reference/ecs-vlan.md @@ -0,0 +1,38 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-vlan.html +applies_to: + stack: all + serverless: all +--- + +# VLAN fields [ecs-vlan] + +The VLAN fields are used to identify 802.1q tag(s) of a packet, as well as ingress and egress VLAN associations of an observer in relation to a specific packet or connection. + +Network.vlan fields are used to record a single VLAN tag, or the outer tag in the case of q-in-q encapsulations, for a packet or connection as observed, typically provided by a network sensor (e.g. Zeek, Wireshark) passively reporting on traffic. + +Network.inner VLAN fields are used to report inner q-in-q 802.1q tags (multiple 802.1q encapsulations) as observed, typically provided by a network sensor (e.g. Zeek, Wireshark) passively reporting on traffic. Network.inner VLAN fields should only be used in addition to network.vlan fields to indicate q-in-q tagging. + +Observer.ingress and observer.egress VLAN values are used to record observer specific information when observer events contain discrete ingress and egress VLAN information, typically provided by firewalls, routers, or load balancers. + + +## VLAN field details [_vlan_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-vlan-id$$$[vlan.id](#field-vlan-id) | VLAN ID as reported by the observer.

type: keyword

example: `10`
| extended | +| $$$field-vlan-name$$$[vlan.name](#field-vlan-name) | Optional VLAN name as reported by the observer.

type: keyword

example: `outside`
| extended | + + +## Field Reuse [_field_reuse_31] + +The `vlan` fields are expected to be nested at: + +* `network.inner.vlan` +* `network.vlan` +* `observer.egress.vlan` +* `observer.ingress.vlan` + +Note also that the `vlan` fields are not expected to be used directly at the root of the events. + diff --git a/docs/reference/ecs-volume.md b/docs/reference/ecs-volume.md new file mode 100644 index 0000000000..d464e3de69 --- /dev/null +++ b/docs/reference/ecs-volume.md @@ -0,0 +1,39 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-volume.html +applies_to: + stack: all + serverless: all +--- + +# Volume fields [ecs-volume] + +Fields related to storage volume details. + +::::{warning} +These fields are beta and are subject to change. +:::: + + + +## Volume field details [_volume_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-volume-bus-type$$$[volume.bus_type](#field-volume-bus-type) | Bus type of the device, such as `Nvme`, `Usb`, or `FileBackedVirtual`.

type: keyword

example: `FileBackedVirtual`
| extended | +| $$$field-volume-default-access$$$[volume.default_access](#field-volume-default-access) | Describes the default access(es) of the volume.

type: keyword
| extended | +| $$$field-volume-device-name$$$[volume.device_name](#field-volume-device-name) | Full path of the volume device.

Only populate this field for POSIX system volumes.

type: keyword
| extended | +| $$$field-volume-device-type$$$[volume.device_type](#field-volume-device-type) | Volume device type.

The most frequently seen volume device types are `Disk File System` and `CD-ROM File System`.

type: keyword

example: `CD-ROM File System`
| extended | +| $$$field-volume-dos-name$$$[volume.dos_name](#field-volume-dos-name) | The MS-DOS name of a device.

DOS device name is in the format of driver letters, such as `C:`. The field is relevant to Windows systems only.

type: keyword

example: `E:`
| extended | +| $$$field-volume-file-system-type$$$[volume.file_system_type](#field-volume-file-system-type) | Volume device file system type.

The most common volume file system types are `NTFS` and `UDF`.

type: keyword
| extended | +| $$$field-volume-mount-name$$$[volume.mount_name](#field-volume-mount-name) | Mount name of the volume device.

Only populate this field for POSIX system volumes.

type: keyword
| extended | +| $$$field-volume-nt-name$$$[volume.nt_name](#field-volume-nt-name) | The NT device name.

NT device name uses a format of `\Device\HarddiskVolume2`. The field is relevant to Windows systems only.

type: keyword

example: `\Device\Cdrom1`
| extended | +| $$$field-volume-product-id$$$[volume.product_id](#field-volume-product-id) | ProductID of the device.

The vendor provides the ProductID for the volume, if any.

type: keyword
| extended | +| $$$field-volume-product-name$$$[volume.product_name](#field-volume-product-name) | Product name of the volume.

The volume device vendor provides this value.

type: keyword

example: `Virtual DVD-ROM`
| extended | +| $$$field-volume-removable$$$[volume.removable](#field-volume-removable) | Indicates if the volume is removable.

type: boolean
| extended | +| $$$field-volume-serial-number$$$[volume.serial_number](#field-volume-serial-number) | Serial number identifier for the volume device.

The serial number is provided by the vendor of the device, if any.

type: keyword
| extended | +| $$$field-volume-size$$$[volume.size](#field-volume-size) | Size of the volume device in bytes.

type: long
| extended | +| $$$field-volume-vendor-id$$$[volume.vendor_id](#field-volume-vendor-id) | VendorID of the volume device.

The volume device vendor provides this value.

type: keyword
| extended | +| $$$field-volume-vendor-name$$$[volume.vendor_name](#field-volume-vendor-name) | Vendor name of the volume device.

The value is provided by the vendor of the device.

type: keyword

example: `Msft`
| extended | +| $$$field-volume-writable$$$[volume.writable](#field-volume-writable) | Indicates if the volume is writable.

type: boolean
| extended | + diff --git a/docs/reference/ecs-vulnerability.md b/docs/reference/ecs-vulnerability.md new file mode 100644 index 0000000000..ee39d16703 --- /dev/null +++ b/docs/reference/ecs-vulnerability.md @@ -0,0 +1,31 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-vulnerability.html +applies_to: + stack: all + serverless: all +--- + +# Vulnerability fields [ecs-vulnerability] + +The vulnerability fields describe information about a vulnerability that is relevant to an event. + + +## Vulnerability field details [_vulnerability_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-vulnerability-category$$$[vulnerability.category](#field-vulnerability-category) | The type of system or architecture that the vulnerability affects. These may be platform-specific (for example, Debian or SUSE) or general (for example, Database or Firewall). For example ([https://qualysguard.qualys.com/qwebhelp/fo_portal/knowledgebase/vulnerability_categories.htm](https://qualysguard.qualys.com/qwebhelp/fo_portal/knowledgebase/vulnerability_categories.htm))

This field must be an array.

type: keyword

Note: this field should contain an array of values.

example: `["Firewall"]`
| extended | +| $$$field-vulnerability-classification$$$[vulnerability.classification](#field-vulnerability-classification) | The classification of the vulnerability scoring system. For example ([https://www.first.org/cvss/](https://www.first.org/cvss/))

type: keyword

example: `CVSS`
| extended | +| $$$field-vulnerability-description$$$[vulnerability.description](#field-vulnerability-description) | The description of the vulnerability that provides additional context of the vulnerability. For example ([https://cve.mitre.org/about/faqs.html#cve_entry_descriptions_created](https://cve.mitre.org/about/faqs.html#cve_entry_descriptions_created))

type: keyword

Multi-fields:

* vulnerability.description.text (type: match_only_text)

example: `In macOS before 2.12.6, there is a vulnerability in the RPC...`
| extended | +| $$$field-vulnerability-enumeration$$$[vulnerability.enumeration](#field-vulnerability-enumeration) | The type of identifier used for this vulnerability. For example ([https://cve.mitre.org/about/](https://cve.mitre.org/about/))

type: keyword

example: `CVE`
| extended | +| $$$field-vulnerability-id$$$[vulnerability.id](#field-vulnerability-id) | The identification (ID) is the number portion of a vulnerability entry. It includes a unique identification number for the vulnerability. For example ([https://cve.mitre.org/about/faqs.html#what_is_cve_id](https://cve.mitre.org/about/faqs.html#what_is_cve_id))

type: keyword

example: `CVE-2019-00001`
| extended | +| $$$field-vulnerability-reference$$$[vulnerability.reference](#field-vulnerability-reference) | A resource that provides additional information, context, and mitigations for the identified vulnerability.

type: keyword

example: `https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-6111`
| extended | +| $$$field-vulnerability-report-id$$$[vulnerability.report_id](#field-vulnerability-report-id) | The report or scan identification number.

type: keyword

example: `20191018.0001`
| extended | +| $$$field-vulnerability-scanner-vendor$$$[vulnerability.scanner.vendor](#field-vulnerability-scanner-vendor) | The name of the vulnerability scanner vendor.

type: keyword

example: `Tenable`
| extended | +| $$$field-vulnerability-score-base$$$[vulnerability.score.base](#field-vulnerability-score-base) | Scores can range from 0.0 to 10.0, with 10.0 being the most severe.

Base scores cover an assessment for exploitability metrics (attack vector, complexity, privileges, and user interaction), impact metrics (confidentiality, integrity, and availability), and scope. For example ([https://www.first.org/cvss/specification-document](https://www.first.org/cvss/specification-document))

type: float

example: `5.5`
| extended | +| $$$field-vulnerability-score-environmental$$$[vulnerability.score.environmental](#field-vulnerability-score-environmental) | Scores can range from 0.0 to 10.0, with 10.0 being the most severe.

Environmental scores cover an assessment for any modified Base metrics, confidentiality, integrity, and availability requirements. For example ([https://www.first.org/cvss/specification-document](https://www.first.org/cvss/specification-document))

type: float

example: `5.5`
| extended | +| $$$field-vulnerability-score-temporal$$$[vulnerability.score.temporal](#field-vulnerability-score-temporal) | Scores can range from 0.0 to 10.0, with 10.0 being the most severe.

Temporal scores cover an assessment for code maturity, remediation level, and confidence. For example ([https://www.first.org/cvss/specification-document](https://www.first.org/cvss/specification-document))

type: float
| extended | +| $$$field-vulnerability-score-version$$$[vulnerability.score.version](#field-vulnerability-score-version) | The National Vulnerability Database (NVD) provides qualitative severity rankings of "Low", "Medium", and "High" for CVSS v2.0 base score ranges in addition to the severity ratings for CVSS v3.0 as they are defined in the CVSS v3.0 specification.

CVSS is owned and managed by FIRST.Org, Inc. (FIRST), a US-based non-profit organization, whose mission is to help computer security incident response teams across the world. For example ([https://nvd.nist.gov/vuln-metrics/cvss](https://nvd.nist.gov/vuln-metrics/cvss))

type: keyword

example: `2.0`
| extended | +| $$$field-vulnerability-severity$$$[vulnerability.severity](#field-vulnerability-severity) | The severity of the vulnerability can help with metrics and internal prioritization regarding remediation. For example ([https://nvd.nist.gov/vuln-metrics/cvss](https://nvd.nist.gov/vuln-metrics/cvss))

type: keyword

example: `Critical`
| extended | + diff --git a/docs/reference/ecs-x509.md b/docs/reference/ecs-x509.md new file mode 100644 index 0000000000..a55e97bc41 --- /dev/null +++ b/docs/reference/ecs-x509.md @@ -0,0 +1,59 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-x509.html +applies_to: + stack: all + serverless: all +--- + +# x509 certificate fields [ecs-x509] + +This implements the common core fields for x509 certificates. This information is likely logged with TLS sessions, digital signatures found in executable binaries, S/MIME information in email bodies, or analysis of files on disk. + +When the certificate relates to a file, use the fields at `file.x509`. When hashes of the DER-encoded certificate are available, the `hash` data set should be populated as well (e.g. `file.hash.sha256`). + +Events that contain certificate information about network connections, should use the x509 fields under the relevant TLS fields: `tls.server.x509` and/or `tls.client.x509`. + + +## x509 certificate field details [_x509_certificate_field_details] + +| Field | Description | Level | +| --- | --- | --- | +| $$$field-x509-alternative-names$$$[x509.alternative_names](#field-x509-alternative-names) | List of subject alternative names (SAN). Name types vary by certificate authority and certificate type but commonly contain IP addresses, DNS names (and wildcards), and email addresses.

type: keyword

Note: this field should contain an array of values.

example: `*.elastic.co`
| extended | +| $$$field-x509-issuer-common-name$$$[x509.issuer.common_name](#field-x509-issuer-common-name) | List of common name (CN) of issuing certificate authority.

type: keyword

Note: this field should contain an array of values.

example: `Example SHA2 High Assurance Server CA`
| extended | +| $$$field-x509-issuer-country$$$[x509.issuer.country](#field-x509-issuer-country) | List of country (C) codes

type: keyword

Note: this field should contain an array of values.

example: `US`
| extended | +| $$$field-x509-issuer-distinguished-name$$$[x509.issuer.distinguished_name](#field-x509-issuer-distinguished-name) | Distinguished name (DN) of issuing certificate authority.

type: keyword

example: `C=US, O=Example Inc, OU=www.example.com, CN=Example SHA2 High Assurance Server CA`
| extended | +| $$$field-x509-issuer-locality$$$[x509.issuer.locality](#field-x509-issuer-locality) | List of locality names (L)

type: keyword

Note: this field should contain an array of values.

example: `Mountain View`
| extended | +| $$$field-x509-issuer-organization$$$[x509.issuer.organization](#field-x509-issuer-organization) | List of organizations (O) of issuing certificate authority.

type: keyword

Note: this field should contain an array of values.

example: `Example Inc`
| extended | +| $$$field-x509-issuer-organizational-unit$$$[x509.issuer.organizational_unit](#field-x509-issuer-organizational-unit) | List of organizational units (OU) of issuing certificate authority.

type: keyword

Note: this field should contain an array of values.

example: `www.example.com`
| extended | +| $$$field-x509-issuer-state-or-province$$$[x509.issuer.state_or_province](#field-x509-issuer-state-or-province) | List of state or province names (ST, S, or P)

type: keyword

Note: this field should contain an array of values.

example: `California`
| extended | +| $$$field-x509-not-after$$$[x509.not_after](#field-x509-not-after) | Time at which the certificate is no longer considered valid.

type: date

example: `2020-07-16T03:15:39Z`
| extended | +| $$$field-x509-not-before$$$[x509.not_before](#field-x509-not-before) | Time at which the certificate is first considered valid.

type: date

example: `2019-08-16T01:40:25Z`
| extended | +| $$$field-x509-public-key-algorithm$$$[x509.public_key_algorithm](#field-x509-public-key-algorithm) | Algorithm used to generate the public key.

type: keyword

example: `RSA`
| extended | +| $$$field-x509-public-key-curve$$$[x509.public_key_curve](#field-x509-public-key-curve) | The curve used by the elliptic curve public key algorithm. This is algorithm specific.

type: keyword

example: `nistp521`
| extended | +| $$$field-x509-public-key-exponent$$$[x509.public_key_exponent](#field-x509-public-key-exponent) | Exponent used to derive the public key. This is algorithm specific.

type: long

example: `65537`
| extended | +| $$$field-x509-public-key-size$$$[x509.public_key_size](#field-x509-public-key-size) | The size of the public key space in bits.

type: long

example: `2048`
| extended | +| $$$field-x509-serial-number$$$[x509.serial_number](#field-x509-serial-number) | Unique serial number issued by the certificate authority. For consistency, this must be encoded in base 16 and formatted without colons and uppercase characters.

type: keyword

example: `55FBB9C7DEBF09809D12CCAA`
| extended | +| $$$field-x509-signature-algorithm$$$[x509.signature_algorithm](#field-x509-signature-algorithm) | Identifier for certificate signature algorithm. We recommend using names found in Go Lang Crypto library. See [https://github.com/golang/go/blob/go1.14/src/crypto/x509/x509.go#L337-L353](https://github.com/golang/go/blob/go1.14/src/crypto/x509/x509.go#L337-L353).

type: keyword

example: `SHA256-RSA`
| extended | +| $$$field-x509-subject-common-name$$$[x509.subject.common_name](#field-x509-subject-common-name) | List of common names (CN) of subject.

type: keyword

Note: this field should contain an array of values.

example: `shared.global.example.net`
| extended | +| $$$field-x509-subject-country$$$[x509.subject.country](#field-x509-subject-country) | List of country (C) code

type: keyword

Note: this field should contain an array of values.

example: `US`
| extended | +| $$$field-x509-subject-distinguished-name$$$[x509.subject.distinguished_name](#field-x509-subject-distinguished-name) | Distinguished name (DN) of the certificate subject entity.

type: keyword

example: `C=US, ST=California, L=San Francisco, O=Example, Inc., CN=shared.global.example.net`
| extended | +| $$$field-x509-subject-locality$$$[x509.subject.locality](#field-x509-subject-locality) | List of locality names (L)

type: keyword

Note: this field should contain an array of values.

example: `San Francisco`
| extended | +| $$$field-x509-subject-organization$$$[x509.subject.organization](#field-x509-subject-organization) | List of organizations (O) of subject.

type: keyword

Note: this field should contain an array of values.

example: `Example, Inc.`
| extended | +| $$$field-x509-subject-organizational-unit$$$[x509.subject.organizational_unit](#field-x509-subject-organizational-unit) | List of organizational units (OU) of subject.

type: keyword

Note: this field should contain an array of values.
| extended | +| $$$field-x509-subject-state-or-province$$$[x509.subject.state_or_province](#field-x509-subject-state-or-province) | List of state or province names (ST, S, or P)

type: keyword

Note: this field should contain an array of values.

example: `California`
| extended | +| $$$field-x509-version-number$$$[x509.version_number](#field-x509-version-number) | Version of x509 format.

type: keyword

example: `3`
| extended | + + +## Field reuse [_field_reuse_32] + +The `x509` fields are expected to be nested at: + +* `file.x509` +* `threat.enrichments.indicator.x509` +* `threat.indicator.x509` +* `tls.client.x509` +* `tls.server.x509` + +Note also that the `x509` fields are not expected to be used directly at the root of the events. + diff --git a/docs/reference/index.md b/docs/reference/index.md new file mode 100644 index 0000000000..3f24c53989 --- /dev/null +++ b/docs/reference/index.md @@ -0,0 +1,40 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/ecs-reference.html + - https://www.elastic.co/guide/en/ecs/current/index.html +--- + +# ECS reference [ecs-reference] + +This is the documentation of ECS version 9.1.0-dev. + + +## What is ECS? [_what_is_ecs] + +The Elastic Common Schema (ECS) is an open source specification, developed with support from the Elastic user community. ECS defines a common set of fields to be used when storing event data in Elasticsearch, such as logs and metrics. + +ECS specifies field names and Elasticsearch datatypes for each field, and provides descriptions and example usage. ECS also groups fields into ECS levels, which are used to signal how much a field is expected to be present. You can learn more about ECS levels in [Guidelines and Best Practices](/reference/ecs-guidelines.md). Finally, ECS also provides a set of naming guidelines for adding custom fields. + +The goal of ECS is to enable and encourage users of Elasticsearch to normalize their event data, so that they can better analyze, visualize, and correlate the data represented in their events. ECS has been scoped to accommodate a wide variety of events, spanning: + +* **Event sources**: whether the source of your event is an Elastic product, a third- party product, or a custom application built by your organization. +* **Ingestion architectures**: whether the ingestion path for your events includes Beats processors, Logstash, Elasticsearch ingest node, all of the above, or none of the above. +* **Consumers**: whether consumed by API, Kibana queries, dashboards, apps, or other means. + + +## New to ECS? [_new_to_ecs] + +If you’re new to ECS and looking for an introduction to its benefits and examples of the core concepts, [Getting Started](/reference/ecs-getting-started.md) is a great place to start. + + +## My events don’t map with ECS [_my_events_dont_map_with_ecs] + +ECS is a permissive schema. If your events have additional data that cannot be mapped to ECS, you can simply add them to your events, using custom field names. + + +## Maturity [_maturity] + +ECS improvements are released following [Semantic Versioning](https://semver.org/). Major ECS releases are planned to be aligned with major Elastic Stack releases. + +Any feedback on the general structure, missing fields, or existing fields is appreciated. For contributions please read the [Contribution Guidelines](https://github.com/elastic/ecs/blob/master/CONTRIBUTING.md). + diff --git a/docs/reference/migrating-to-ecs.md b/docs/reference/migrating-to-ecs.md new file mode 100644 index 0000000000..8b88fc78d7 --- /dev/null +++ b/docs/reference/migrating-to-ecs.md @@ -0,0 +1,16 @@ +--- +mapped_pages: + - https://www.elastic.co/guide/en/ecs/current/migrating-to-ecs.html +applies_to: + stack: all + serverless: all +--- + +# Migrating to ECS [migrating-to-ecs] + +There are multiple ways to reap the benefit of ECS. The simplest is to use [Products and Solutions that Support ECS](/reference/ecs-products-solutions.md). + +If you have a custom pipeline or application you would like to convert to ECS, please have a look at [Map custom data to ECS](/reference/ecs-converting.md). + + + diff --git a/docs/reference/toc.yml b/docs/reference/toc.yml new file mode 100644 index 0000000000..91b47681b5 --- /dev/null +++ b/docs/reference/toc.yml @@ -0,0 +1,95 @@ +toc: + - file: index.md + - file: ecs-using-ecs.md + children: + - file: ecs-getting-started.md + - file: ecs-guidelines.md + children: + - file: ecs-conventions.md + - file: ecs-principles-implementation.md + - file: ecs-mapping-network-events.md + - file: ecs-principles-design.md + - file: ecs-custom-fields-in-ecs.md + - file: ecs-field-reference.md + children: + - file: ecs-base.md + - file: ecs-agent.md + - file: ecs-as.md + - file: ecs-client.md + - file: ecs-cloud.md + children: + - file: ecs-cloud-usage.md + - file: ecs-code_signature.md + - file: ecs-container.md + - file: ecs-data_stream.md + - file: ecs-destination.md + - file: ecs-device.md + - file: ecs-dll.md + - file: ecs-dns.md + - file: ecs-ecs.md + - file: ecs-elf.md + - file: ecs-email.md + - file: ecs-error.md + - file: ecs-event.md + - file: ecs-faas.md + - file: ecs-file.md + - file: ecs-geo.md + - file: ecs-group.md + - file: ecs-hash.md + - file: ecs-host.md + - file: ecs-http.md + - file: ecs-interface.md + - file: ecs-log.md + - file: ecs-macho.md + - file: ecs-network.md + - file: ecs-observer.md + - file: ecs-orchestrator.md + - file: ecs-organization.md + - file: ecs-os.md + - file: ecs-package.md + - file: ecs-pe.md + - file: ecs-process.md + - file: ecs-registry.md + - file: ecs-related.md + - file: ecs-risk.md + - file: ecs-rule.md + - file: ecs-server.md + - file: ecs-service.md + children: + - file: ecs-service-usage.md + - file: ecs-source.md + - file: ecs-threat.md + children: + - file: ecs-threat-usage.md + - file: ecs-tls.md + - file: ecs-tracing.md + - file: ecs-url.md + - file: ecs-user.md + children: + - file: ecs-user-usage.md + - file: ecs-user_agent.md + - file: ecs-vlan.md + - file: ecs-volume.md + - file: ecs-vulnerability.md + - file: ecs-x509.md + - file: ecs-category-field-values-reference.md + children: + - file: ecs-allowed-values-event-kind.md + - file: ecs-allowed-values-event-category.md + - file: ecs-allowed-values-event-type.md + - file: ecs-allowed-values-event-outcome.md + - file: ecs-using-categorization-fields.md + - file: migrating-to-ecs.md + children: + - file: ecs-products-solutions.md + - file: ecs-converting.md + - file: ecs-opentelemetry.md + children: + - file: ecs-otel-alignment-overview.md + - file: ecs-otel-alignment-details.md + - file: ecs-additional-information.md + children: + - file: ecs-faq.md + - file: ecs-contributing.md + - file: ecs-artifacts.md + - file: ecs-release-notes.md \ No newline at end of file diff --git a/docs/release-notes/8.0.1.asciidoc b/docs/release-notes/8.0.1.asciidoc deleted file mode 100644 index c49c4d0cb6..0000000000 --- a/docs/release-notes/8.0.1.asciidoc +++ /dev/null @@ -1,12 +0,0 @@ -[[ecs-release-notes-8.0.1]] -=== 8.0.1 - -[[tooling-changes-8.0.1]] -[float] -==== Tooling and artifact changes - -[[tooling-bugfixes-8.0.1]] -[float] -===== Bugfixes - -* Pin `markupsafe==2.0.1` to resolve `ImportError` exception. {ecs_pull}1804[#1804] diff --git a/docs/release-notes/8.0.asciidoc b/docs/release-notes/8.0.asciidoc deleted file mode 100644 index 4de5b03034..0000000000 --- a/docs/release-notes/8.0.asciidoc +++ /dev/null @@ -1,61 +0,0 @@ -[[ecs-release-notes-8.0.0]] -=== 8.0.0 - -[[schema-changes-8.0.0]] -[float] -==== Schema changes - -[[schema-breaking-8.0.0]] -[float] -===== Breaking changes - -* Remove `host.user.*` field reuse. {ecs_pull}1439[#1439] -* Remove deprecation notice on `http.request.method`. {ecs_pull}1443[#1443] -* Migrate `log.origin.file.line` from `integer` to `long`. {ecs_pull}1533[#1533] -* Remove `log.original` field. {ecs_pull}1580[#1580] -* Remove `process.ppid` field. {ecs_pull}1596[#1596] - -[[schema-added-8.0.0]] -[float] -===== Added - -* Added `faas.*` field set as beta. {ecs_pull}1628[#1628], {ecs_pull}1755[#1755] - -[[schema-improvements-8.0.0]] -[float] -===== Improvements - -* Wildcard type field migration GA. {ecs_pull}1582[#1582] -* `match_only_text` type field migration GA. {ecs_pull}1584[#1584] -* Threat indicator fields GA from RFC 0008. {ecs_pull}1586[#1586] - -[[tooling-changes-8.0.0]] -[float] -==== Tooling and artifact changes - -[[tooling-breaking-8.0.0]] -[float] -===== Breaking Changes - -* Removing deprecated --oss from generator {ecs_pull}1404[#1404] -* Removing use-cases directory {ecs_pull}1405[#1405] -* Remove Go code generator. {ecs_pull}1567[#1567] -* Remove template generation for ES6. {ecs_pull}1680[#1680] -* Update folder structure for generated ES artifacts. {ecs_pull}1700[#1700], {ecs_pull}1762[#1762] -* Updated support for overridable composable settings template. {ecs_pull}1737[#1737] - -[[tooling-improvements-8.0.0]] -[float] -===== Improvements - -* Align input options for --include and --subset arguments {ecs_pull}1519[#1519] -* Remove remaining Go deps after removing Go code generator. {ecs_pull}1585[#1585] -* Add explicit `default_field: true` for Beats artifacts. {ecs_pull}1633[#1633] -* Reorganize docs directory structure. {ecs_pull}1679[#1679] -* Added support for `analyzer` definitions for text fields. {ecs_pull}1737[#1737] - -[[tooling-bugfixes-8.0.0]] -[float] -===== Bugfixes - -* Fixed the `default_field` flag for root fields in Beats generator. {ecs_pull}1711[#1711] diff --git a/docs/release-notes/8.1.asciidoc b/docs/release-notes/8.1.asciidoc deleted file mode 100644 index df2f07ec8c..0000000000 --- a/docs/release-notes/8.1.asciidoc +++ /dev/null @@ -1,39 +0,0 @@ -[[ecs-release-notes-8.1.0]] -=== 8.1.0 - -[[schema-changes-8.1.0]] -[float] -==== Schema changes - -[[schema-added-8.1.0]] -[float] -===== Added - -* Added two new fields (sha384,tlsh) to hash schema and one field to pe schema (pehash). {ecs_pull}1678[#1678] -* Added `email.*` beta field set. {ecs_pull}1688[#1688], {ecs_pull}1705[#1705] - -[[schema-removed-8.1.0]] -[float] -===== Removed - -* Removing `process.target.*` reuses from experimental schema. {ecs_pull}1666[#1666] -* Removing RFC 0014 `pe.*` fields from experimental schema. {ecs_pull}1670[#1670] - -[[tooling-changes-8.1.0]] -[float] -==== Tooling and artifact changes - -[[tooling-improvements-8.1.0]] -[float] -===== Improvements - -* Update refs from master to main in USAGE.md etc {ecs_pull}1658[#1658] -* Clean up trailing spaces and additional newlines in schemas {ecs_pull}1667[#1667] -* Use higher compression as default in composable index template settings. {ecs_pull}1712[#1712] -* Bump dependencies. {ecs_pull}1782[#1782] - -[[tooling-bugfixes-8.1.0]] -[float] -===== Bugfixes - -* Fix invalid documentation link generation in component templates `_meta`. {ecs_pull}1728[#1728] diff --git a/docs/release-notes/8.10.asciidoc b/docs/release-notes/8.10.asciidoc deleted file mode 100644 index 20c7a80be6..0000000000 --- a/docs/release-notes/8.10.asciidoc +++ /dev/null @@ -1,31 +0,0 @@ -[[ecs-release-notes-8.10.0]] -=== 8.10.0 - -[[schema-changes-8.10.0]] -[float] -==== Schema changes - -[[schema-added-8.10.0]] -[float] -===== Added - -* Added `container.security_context.privileged` to indicated whether a container was started in privileged mode. {ecs_pull}2219[#2219], {ecs_pull}2225[#2225], {ecs_pull}2246[#2246] -* Added `process.thread.capabilities.permitted` to contain the current thread's possible capabilities. {ecs_pull}2245[#2245] -* Added `process.thread.capabilities.effective` to contain the current thread's effective capabilities. {ecs_pull}2245[#2245] - -[[schema-improvements-8.10.0]] -[float] -===== Improvements - -* Permit `ignore_above` if explicitly set on a `flattened` field. {ecs_pull}2248[#2248] - -[[tooling-changes-8.10.0]] -[float] -==== Tooling and artifact changes - -[[tooling-improvements-8.10.0]] -[float] -===== Improvements - -* Improved documentation formatting to better follow the contributing guide. {ecs_pull}2226[#2226] -* Bump `gitpython` dependency from 3.1.30 to 3.1.35 for security fixes. {ecs_pull}2251[#2251], {ecs_pull}2264[#2264], {ecs_pull}2265[#2265] diff --git a/docs/release-notes/8.11.asciidoc b/docs/release-notes/8.11.asciidoc deleted file mode 100644 index 91d35adfc9..0000000000 --- a/docs/release-notes/8.11.asciidoc +++ /dev/null @@ -1,28 +0,0 @@ -[[ecs-release-notes-8.11.0]] -=== 8.11.0 - -[[schema-changes-8.11.0]] -[float] -==== Schema changes - -[[schema-bugfixes-8.11.0]] -[float] -===== Bugfixes - -* Remove `expected_values` from `threat.*.indicator.name` fields. {ecs_pull}2281[#2281] - -[[schema-added-8.11.0]] -[float] -===== Added - -* Added `volume.*` as beta field set. {ecs_pull}2269[#2269] - -[[tooling-changes-8.11.0]] -[float] -==== Tooling and artifact changes - -[[tooling-bugfixes-8.11.0]] -[float] -===== Bugfixes - -* Respect reusable.top_level in Beats generator {ecs_pull}2278[#2278] diff --git a/docs/release-notes/8.16.asciidoc b/docs/release-notes/8.16.asciidoc deleted file mode 100644 index f96da18a24..0000000000 --- a/docs/release-notes/8.16.asciidoc +++ /dev/null @@ -1,38 +0,0 @@ -[[ecs-release-notes-8.16.0]] -=== 8.16.0 - -[[schema-changes-8.16.0]] -[float] -==== Schema changes - -[[schema-bugfixes-8.16.0]] -[float] -===== Bugfixes -* Fix broken link in docs for vulnerability.id. {ecs_pull}2328[#2328] - -[[schema-added-8.16.0]] -[float] -===== Added - -* Added `volume.*` as beta field set. {ecs_pull}2269[#2269] -* Advanced `process.env_vars` to GA. {ecs_pull}2315[#2315] -* Advanced `process.io` and `process.tty` fields to GA. {ecs_pull}2317[#2317] -* Added `threat.indicator.id`. {ecs_pull}2324[#2324] -* Added `process.group` to generated schemas. {ecs_pull}2335[#2335] - - -[[tooling-changes-8.16.0]] -[float] -==== Tooling and artifact changes - -[[tooling-bugfixes-8.16.0]] -[float] -===== Bugfixes -* Fix broken link for vulnerabilty.id {ecs_pull}2328[#2328] - -[[tooling-added-8.16.0]] -[float] -===== Added - -* Documentation in README.md providing instruction on contributions to ECS during the OTel donation {ecs_pull}2325[#2325] - diff --git a/docs/release-notes/8.2.1.asciidoc b/docs/release-notes/8.2.1.asciidoc deleted file mode 100644 index d942f18e67..0000000000 --- a/docs/release-notes/8.2.1.asciidoc +++ /dev/null @@ -1,23 +0,0 @@ -[[ecs-release-notes-8.2.1]] -=== 8.2.1 - -[[schema-changes-8.2.1]] -[float] -==== Schema changes - -[[schema-bugfixes-8.2.1]] -[float] -===== Bugfixes - -* Adding missing process fields for documentation. {ecs_pull}1906[#1906] - -[[tooling-changes-8.2.1]] -[float] -==== Tooling and artifact changes - -[[tooling-improvements-8.2.1]] -[float] -===== Improvements - -* Add type hints to `schema` modules. {ecs_pull}1771[#1771] -* Support `docs_only` param to subset defs. {ecs_pull}1909[#1909] diff --git a/docs/release-notes/8.2.asciidoc b/docs/release-notes/8.2.asciidoc deleted file mode 100644 index 7dc5171e83..0000000000 --- a/docs/release-notes/8.2.asciidoc +++ /dev/null @@ -1,41 +0,0 @@ -[[ecs-release-notes-8.2.0]] -=== 8.2.0 - -[[schema-changes-8.2.0]] -[float] -==== Schema changes - -[[schema-added-8.2.0]] -[float] -===== Added - -* Add beta `container.*` metric fields. {ecs_pull}1789[#1789] -* Add six new syslog fields to `log.syslog.*`. {ecs_pull}1793[#1793] -* Added `faas.id`, `faas.name` and `faas.version` fields as beta. {ecs_pull}1796[#1796] -* Added linux event model beta fields and reuses to support RFC 0030. {ecs_pull}1842[#1842], {ecs_pull}1847[#1847], {ecs_pull}1884[#1884] -* Added `threat.feed.dashboard_id`, `threat.feed.description`, `threat.feed.name`, `threat.feed.reference` fields. {ecs_pull}1844[#1844] - -[[schema-improvements-8.2.0]] -[float] -===== Improvements - -* `email.*` field set now GA. {ecs_pull}1794[#1794], {ecs_pull}1841[#1841] - -[[tooling-changes-8.2.0]] -[float] -==== Tooling and artifact changes - -[[tooling-added-8.2.0]] -[float] -===== Added - -* Adding optional field attribute, `pattern`. {ecs_pull}1834[#1834] -* Added support for re-using a fieldset as an array. {ecs_pull}1838[#1838] -* Added `--force-docs` option to generator. {ecs_pull}1879[#1879] - -[[tooling-improvements-8.2.0]] -[float] -===== Improvements -* Update refs from master to main in USAGE.md etc {ecs_pull}1658[#1658] -* Clean up trailing spaces and additional newlines in schemas {ecs_pull}1667[#1667] -* Use higher compression as default in composable index template settings. {ecs_pull}1712[#1712] \ No newline at end of file diff --git a/docs/release-notes/8.3.1.asciidoc b/docs/release-notes/8.3.1.asciidoc deleted file mode 100644 index 5a5b60c8e5..0000000000 --- a/docs/release-notes/8.3.1.asciidoc +++ /dev/null @@ -1,12 +0,0 @@ -[[ecs-release-notes-8.3.1]] -=== 8.3.1 - -[[schema-changes-8.3.1]] -[float] -==== Schema changes - -[[schema-deprecated-8.3.1]] -[float] -===== Deprecated - -* Deprecate `service.node.role` in favor of upcoming `service.node.roles`. {ecs_pull}1976[#1976] \ No newline at end of file diff --git a/docs/release-notes/8.3.asciidoc b/docs/release-notes/8.3.asciidoc deleted file mode 100644 index 2a6df7151c..0000000000 --- a/docs/release-notes/8.3.asciidoc +++ /dev/null @@ -1,25 +0,0 @@ -[[ecs-release-notes-8.3.0]] -=== 8.3.0 - -[[schema-changes-8.3.0]] -[float] -==== Schema changes - -[[schema-added-8.3.0]] -[float] -===== Added - -* Added `pattern` attribute to `.mac` fields. {ecs_pull}1871[#1871] -* Add `orchestrator.cluster.id`. {ecs_pull}1875[#1875] -* Add `orchestrator.resource.id`. {ecs_pull}1878[#1878] -* Add `orchestrator.resource.parent.type`. {ecs_pull}1889[#1889] -* Add `orchestrator.resource.ip`. {ecs_pull}1889[#1889] -* Add `container.image.hash.all`. {ecs_pull}1889[#1889] -* Add `service.node.role`. {ecs_pull}1916[#1916] -* Advanced `container.*` metric fields to GA. {ecs_pull}1927[#1927] - -[[schema-important-8.3.0]] -[float] -===== Important - -After adding `service.node.role`, it was realized that we intend for this field to have multiple values, and therefore we will be removing `role` and replacing with `roles` at the earliest opportunity. Please do not use `service.node.role`. \ No newline at end of file diff --git a/docs/release-notes/8.4.asciidoc b/docs/release-notes/8.4.asciidoc deleted file mode 100644 index e26cdd06c3..0000000000 --- a/docs/release-notes/8.4.asciidoc +++ /dev/null @@ -1,29 +0,0 @@ -[[ecs-release-notes-8.4.0]] -=== 8.4.0 - -[[schema-changes-8.4.0]] -[float] -==== Schema changes - -[[schema-added-8.4.0]] -[float] -===== Added - -* Initial set of `expected_values`. {ecs_pull}1962[#1962] -* Adding `service.node.roles`. {ecs_pull}1981[#1981] - -[[tooling-changes-8.4.0]] -[float] -==== Tooling and artifact changes - -[[tooling-added-8.4.0]] -[float] -===== Added - -* Introduce `expected_values` attribute. {ecs_pull}1952[#1952] - -[[tooling-improvements-8.4.0]] -[float] -===== Improvements - -* Additional type annotations. # {ecs_pull}1950[#1950] diff --git a/docs/release-notes/8.5.1.asciidoc b/docs/release-notes/8.5.1.asciidoc deleted file mode 100644 index 254f1d9018..0000000000 --- a/docs/release-notes/8.5.1.asciidoc +++ /dev/null @@ -1,12 +0,0 @@ -[[ecs-release-notes-8.5.1]] -=== 8.5.1 - -[[tooling-changes-8.5.1]] -[float] -==== Tooling and artifact changes - -[[tooling-bugfixes-8.5.1]] -[float] -===== Bugfixes - -* Fix type of `normalize` in `process.io.bytes_skipped`. {ecs_pull}2094[#2094] diff --git a/docs/release-notes/8.5.2.asciidoc b/docs/release-notes/8.5.2.asciidoc deleted file mode 100644 index b00717afc4..0000000000 --- a/docs/release-notes/8.5.2.asciidoc +++ /dev/null @@ -1,12 +0,0 @@ -[[ecs-release-notes-8.5.2]] -=== 8.5.2 - -[[schema-changes-8.5.2]] -[float] -==== Schema changes - -[[schema-bugfixes-8.5.2]] -[float] -===== Bugfixes - -* Fixes invalid `number` type on 4 `process.io` subfields. {ecs_pull}2105[#2105] diff --git a/docs/release-notes/8.5.asciidoc b/docs/release-notes/8.5.asciidoc deleted file mode 100644 index 4dd468377c..0000000000 --- a/docs/release-notes/8.5.asciidoc +++ /dev/null @@ -1,36 +0,0 @@ -[[ecs-release-notes-8.5.0]] -=== 8.5.0 - -[[schema-changes-8.5.0]] -[float] -==== Schema changes - -[[schema-added-8.5.0]] -[float] -===== Added - -* Adding `risk.*` fields as experimental. {ecs_pull}1994[#1994], {ecs_pull}2010[#2010] -* Adding `process.io.*` as beta fields. {ecs_pull}1956[#1956], {ecs_pull}2031[#2031] -* Adding `process.tty.rows` and `process.tty.columns` as beta fields. {ecs_pull}2031[#2031] -* Changed `process.env_vars` field type to be an array of keywords. {ecs_pull}2038[#2038] -* `process.attested_user` and `process.attested_groups` as beta fields. {ecs_pull}2050[#2050] -* Added `risk.*` fieldset to beta. {ecs_pull}2051[#2051], {ecs_pull}2058[#2058] -* Moved Linux event model fields to GA. {ecs_pull}2082[#2082] - -[[schema-improvements-8.5.0]] -[float] -===== Improvements - -* Advances `threat.enrichments.indicator` to GA. {ecs_pull}1928[#1928] -* Added `ios` and `android` as valid values for `os.type` {ecs_pull}1999[#1999] - -[[tooling-changes-8.5.0]] -[float] -==== Tooling and artifact changes - -[[tooling-bugfixes-8.5.0]] -[float] -===== Bugfixes - -* Added Deprecation Warning for `misspell` task {ecs_pull}1993[#1993] -* Fix typo in client schema {ecs_pull}2014[#2014] diff --git a/docs/release-notes/8.6.1.asciidoc b/docs/release-notes/8.6.1.asciidoc deleted file mode 100644 index 21870063a5..0000000000 --- a/docs/release-notes/8.6.1.asciidoc +++ /dev/null @@ -1,12 +0,0 @@ -[[ecs-release-notes-8.6.1]] -=== 8.6.1 - -[[schema-changes-8.6.1]] -[float] -==== Schema changes - -[[schema-bugfixes-8.6.1]] -[float] -===== Bugfixes - -* Fixing `tlp_version` and `tlp` field for threat. {ecs_pull}2156[#2156] diff --git a/docs/release-notes/8.6.asciidoc b/docs/release-notes/8.6.asciidoc deleted file mode 100644 index 142ff4e64e..0000000000 --- a/docs/release-notes/8.6.asciidoc +++ /dev/null @@ -1,21 +0,0 @@ -[[ecs-release-notes-8.6.0]] -=== 8.6.0 - -[[schema-changes-8.6.0]] -[float] -==== Schema changes - -[[schema-added-8.6.0]] -[float] -===== Added - -* Adding `vulnerability` option for `event.category`. {ecs_pull}2029[#2029] -* Added `device.*` field set as beta. {ecs_pull}2030[#2030] -* Added `tlp.version` to threat {ecs_pull}2074[#2074] -* Added fields for executable object format metadata for ELF, Mach-O and PE {ecs_pull}2083[#2083] - -[[schema-improvements-8.6.0]] -[float] -===== Improvements - -* Added `CLEAR` and `AMBER+STRICT` as valid values for `threat.indicator.marking.tlp` and `enrichments.indicator.marking.tlp` to accept new [TLP 2.0](https://www.first.org/tlp/) markings {ecs_pull}2022[#2022], {ecs_pull}2074[#2074] diff --git a/docs/release-notes/8.7.asciidoc b/docs/release-notes/8.7.asciidoc deleted file mode 100644 index 44a8ee64fe..0000000000 --- a/docs/release-notes/8.7.asciidoc +++ /dev/null @@ -1,37 +0,0 @@ -[[ecs-release-notes-8.7.0]] -=== 8.7.0 - -[[schema-changes-8.7.0]] -[float] -==== Schema changes - -[[schema-bugfixes-8.7.0]] -[float] -===== Bugfixes - -* Remove duplicated `client.domain` definition {ecs_pull}2120[#2120] - -[[schema-added-8.7.0]] -[float] -===== Added - -* Adding `name` field to `threat.indicator` {ecs_pull}2121[#2121] -* Adding `api` option to `event.category` {ecs_pull}2147[#2147] -* Adding `library` option to `event.category` {ecs_pull}2154[#2154] - -[[schema-improvements-8.7.0]] -[float] -===== Improvements - -* Description for `host.name` definition updated to encourage use of FDQN {ecs_pull}2122[#2122] - -[[tooling-changes-8.7.0]] -[float] -==== Tooling and artifact changes - -[[tooling-improvements-8.7.0]] -[float] -===== Improvements - -* Updated usage docs to include `threat.indicator.url.domain` and changed `indicator.marking.tlp` and `indicator.enrichments.marking.tlp` from "WHITE" to "CLEAR" to align with TLP 2.0. {ecs_pull}2124[#2124] -* Bump `gitpython` from `3.1.27` to `3.1.30` in `/scripts`. {ecs_pull}2139[#2139] diff --git a/docs/release-notes/8.8.asciidoc b/docs/release-notes/8.8.asciidoc deleted file mode 100644 index f94a5f1a84..0000000000 --- a/docs/release-notes/8.8.asciidoc +++ /dev/null @@ -1,25 +0,0 @@ -[[ecs-release-notes-8.8.0]] -=== 8.8.0 - -[[schema-changes-8.8.0]] -[float] -==== Schema changes - -[[schema-added-8.8.0]] -[float] -===== Added - -* Add `event.type: access` as an allowed value for `event.category: file`. {ecs_pull}2174[#2174] -* Add `orchestrator.resource.annotation` and `orchestrator.resource.label`. {ecs_pull}2181[#2181] -* Add `event.kind: asset` as a beta category. {ecs_pull}2191[#2191] - - -[[tooling-changes-8.8.0]] -[float] -==== Tooling and artifact changes - -[[tooling-added-8.8.0]] -[float] -===== Added - -* Add `parameters` property for field definitions, to provide any mapping parameter. {ecs_pull}2084[#2084] diff --git a/docs/release-notes/8.9.asciidoc b/docs/release-notes/8.9.asciidoc deleted file mode 100644 index eafdc36f15..0000000000 --- a/docs/release-notes/8.9.asciidoc +++ /dev/null @@ -1,18 +0,0 @@ -[[ecs-release-notes-8.9.0]] -=== 8.9.0 - -[[schema-changes-8.9.0]] -[float] -==== Schema changes - -[[schema-added-8.9.0]] -[float] -===== Added - -* Added `process.vpid` for namespaced process ids. {ecs_pull}2211[#2211] - -[[schema-deprecated-8.9.0]] -[float] -===== Deprecated - -* Removed `faas.trigger: nested` since we only have one trigger. {ecs_pull}2194[#2194] \ No newline at end of file diff --git a/docs/release-notes/index.asciidoc b/docs/release-notes/index.asciidoc deleted file mode 100644 index 01e792805a..0000000000 --- a/docs/release-notes/index.asciidoc +++ /dev/null @@ -1,49 +0,0 @@ -[[ecs-release-notes]] -== Release Notes - -This section summarizes the changes in each release. - -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> - -// Use these for links to issue and pulls. Note issues and pulls redirect one to -// each other on Github, so don't worry too much on using the right prefix. -:issue: https://github.com/elastic/ecs/issues/ -:pull: https://github.com/elastic/ecs/pull/ - -include::8.16.asciidoc[] -include::8.11.asciidoc[] -include::8.10.asciidoc[] -include::8.9.asciidoc[] -include::8.8.asciidoc[] -include::8.7.asciidoc[] -include::8.6.1.asciidoc[] -include::8.6.asciidoc[] -include::8.5.2.asciidoc[] -include::8.5.1.asciidoc[] -include::8.5.asciidoc[] -include::8.4.asciidoc[] -include::8.3.1.asciidoc[] -include::8.3.asciidoc[] -include::8.2.1.asciidoc[] -include::8.2.asciidoc[] -include::8.1.asciidoc[] -include::8.0.1.asciidoc[] -include::8.0.asciidoc[] diff --git a/docs/using-ecs/custom-fields.asciidoc b/docs/using-ecs/custom-fields.asciidoc deleted file mode 100644 index 897436e05f..0000000000 --- a/docs/using-ecs/custom-fields.asciidoc +++ /dev/null @@ -1,123 +0,0 @@ -[[ecs-custom-fields-in-ecs]] -=== Custom Fields - -ECS defines fields, their datatypes and their usage, and classifies them in -"core" and "extended" levels. - -However, ECS does not define anything about custom fields. By definition, they -are additional fields, exactly as the user or the integration defines them, -independently of ECS. - -Users and integrations are welcome to capture additional -information in their events, as custom fields. -This flexibility is by design, and ensures that no one is ever blocked by -something not being supported by ECS yet. - -ECS is under active development, however. Adding custom fields carries a small -risk of conflicting with a future ECS field. -There are ways of modeling custom fields that will lead to -lower chances of conflict with future versions of ECS. -This section outlines a few of these strategies. - -[float] -==== Modeling to Reduce Chances of Conflict - -[float] -===== The `labels` Field - -Any time a data source has a few extra fields that can be modeled with the -`keyword` data type, the simplest way to capture them is with the ECS field `labels`. - -Example: - -[source,json] -------------- -{ "labels": { "foo_id": "beef42", "env": "production" }, - "message": "...", - "event": { ... } -} -------------- - -If `labels` doesn't work for your use case, here's a few more tips to avoid conflicts. - -[float] -===== Proper Names - -ECS tries to model information by using the name of concepts, -and avoids proper names such as tool names, project names or company names. -By extension, nesting custom fields under a proper name is a relatively safe -approach to adding custom fields. This is the approach taken by Filebeats modules, -for example. - -As an example, an HTTP log from HAProxy will contain typical HTTP information, -as well as proxy details and statistics. The standard HTTP information can be -captured in the ECS field sets `http` and `url`, and the extra details in a custom -`haproxy` section: - -[source,json] -------------- -{ "http": { "request": { "method": "get", ... }, - "response": { "status_code": 200, ... } }, - "url": { "original": "/favicon.ico", ... }, - "haproxy": { "frontend_name": "myfrontend", "backend_name": "mybackend_prod", - "backend_queue": 0, ... } -} -------------- - -[float] -===== Capitalization - -ECS strives for a consistent feel by using nesting to group related concepts, -and underscores to join words. Following these guidelines for custom fields -ensures you preserve the same consistent experience throughout your schemas. - -Note however, that breaking away from these guidelines for your custom fields -can be used to your advantage. It can be a good way to differentiate -between ECS fields and custom fields. Since ECS doesn't -use capitalization for field names, this approach virtually guarantees that -custom fields will not conflict with future ECS fields. - -Common proxy concepts could modelled via a capitalized, but generic concept name. - -HAProxy example: - -[source,json] -------------- -{ "http": { "request": { "method": "get", ... } }, - "url": { "original": "/favicon.ico", ... }, - "Proxy": { "FrontendName": "myfrontend", "BackendName": "mybackend_prod" }, - "event": { "module": "haproxy" } -} -------------- - -NGINX example: - -[source,json] -------------- -{ "http": { "request": { "method": "get", ... } }, - "url": { "original": "/favicon.ico", ... }, - "Proxy": { "FrontendName": "another_frontend", "BackendName": "another_backend_prod" }, - "event": { "module": "nginx" } -} -------------- - -The above demonstrates that using a common concept name in custom fields -can still be beneficial to correlate among multiple sources that populate them. -Using capitalization ensures a future version of ECS that defines a `proxy` -field set will not conflict. - -Here's a sample event, during a migration from the custom field, to using a new -equivalent ECS field set: - -[source,json] -------------- -{ "http": { "request": { "method": "get", ... } }, - "Proxy": { "FrontendName": "myfrontend", "BackendName": "mybackend_prod" }, - "proxy": { "frontend_name": "myfrontend", "backend_name": "mybackend_prod" } -} -------------- - -The above will look strange during the migration. However the ability to start -populating ECS fields while custom fields are still present in your events makes -it possible to decouple the upgrade to a new version of ECS from the time you -adjust your pipelines and analysis content. diff --git a/docs/using-ecs/design.asciidoc b/docs/using-ecs/design.asciidoc deleted file mode 100644 index 9a4dd3a5d6..0000000000 --- a/docs/using-ecs/design.asciidoc +++ /dev/null @@ -1,59 +0,0 @@ -[[ecs-principles-design]] -=== Design Principles - -The considerations here form the basis of ECS design. These principles help guide -making the appropriate decisions for the project. - -[float] -==== A common schema - -A goal of ECS is to maximize interoperability and reuse. When expanding the concepts represented in ECS, -consider how broad or narrow the intended use cases are. - -Defining fields with narrow, lacking, or incorrect definitions limit future use. The best practice is to add -the fewest fields to adequately capture an event. Adding more fields in the future is less complicated than changing or removing -established ones. - -Also, avoid adding fields because a concept exists. For example, a network protocol specification may contain many features, -but some are obscure and used infrequently. Finally, avoid standardization for standardization’s sake. - - -[float] -==== Field sets are namespaces - -Field sets create independent schema sections for understanding a concept in isolation. - -Complex concepts may be better captured using nesting. A field set may contain several sub-components that make up a -larger concept: `dns.question.class`, `dns.question.answer`, `dns.question.type`. - -[float] -==== Naming consistency - -Consistent naming across the schema makes learning and memorizing field names easier. -Do not limit terms with broad meaning to a single case. - -Examples: - -* Many concepts can include a `.name` or `.id` value (for example, `event.id`, `error.id`, `group.id`, `rule.id`, `user.id`). -* Several potential IP addresses in a single event (`source.ip`, `destination.ip`, `host.ip`). - -[float] -==== Reuse - -Introducing extra fields may seem necessary when adding or expanding a concept. But, try to use an existing field -or reuse an existing field set to avoid duplicating fields. Leveraging consistent fields across event sources helps build more straightforward queries and visualizations. - -For example, imagine an app or framework produces a unique ID for each log it emits. Instead of adding a custom `.id` field specific to that app, -consider the `event.id` field. - -Reusing fields simplifies capturing several entities of a type within a single event. One example, the `user.*` field set and the reuse `user.target.*` allow collecting the -same detail about the acting and target users. Redefining the entire `user.*` field set is unnecessary. In limited use, consider an array of field set reuses if multiples of the same reuse need capturing. - -[float] -==== Custom fields are a feature - -Many situations will need custom fields to fully capture the event contents. Users and integrations are -encouraged to add custom fields to capture concepts not defined in ECS. Custom fields give users the flexibility to add fields for their internal use cases, -less common concepts, and experimentation. - -Following the <>, users and integrations can create a path for future migration if a similar concept appears in ECS. diff --git a/docs/using-ecs/getting-started.asciidoc b/docs/using-ecs/getting-started.asciidoc deleted file mode 100644 index 5383093fb2..0000000000 --- a/docs/using-ecs/getting-started.asciidoc +++ /dev/null @@ -1,290 +0,0 @@ -[[ecs-getting-started]] -=== Getting Started - -ECS enables and encourages users to normalize event data in order -to better analyze, visualize, and correlate their events. Collected events -can be normalized at ingest time, consistently searched across indices, -and visualized predictably. - -Note that when adopting an Elastic solution, such as https://www.elastic.co/observability[Observability] -or https://www.elastic.co/security[Security], all events will map to ECS out of the box. Elastic -provides an extensive set of https://www.elastic.co/integrations[integrations] to simplify -ingesting your data sources. - -If you rely on custom data pipelines and/or building content around specific needs, -ECS can still help to alleviate the challenges of searching, analyzing, and visualizing -across your data. Let's see how using a common schema can simplify the search experience, -and then take a look at how an event's contents can be mapped to ECS field sets. - -[float] -==== Simplified Search - -With ECS defining a normalized schema across all of your data sources, querying -against those sources is simplified. Consider searching for a particular source IP -address prior to adopting ECS. All the various data sources and their field mappings -would need to be considered in your query: - -[source,sh] ------------ -src:10.42.42.42 OR client_ip:10.42.42.42 OR apache.access.remote_ip:10.42.42.42 OR -context.user.ip:10.42.42.42 OR src_ip:10.42.42.42 ------------ - -With all sources mapped to ECS, the query becomes much simpler: - -[source,sh] ------------ -source.ip:10.42.42.42 ------------ - -Not only does this simplify writing queries, but saved queries shared with other -users become much more obvious. To gain familiarity with ECS fields, you can also -take a look at the <> section. - -[float] -==== Unified Visualizations - -With normalized data from different data sources, building insightful visualizations -across sources is simple. From a single, centralized dashboard, events from web servers, -IDS/IPS devices, and firewalls can be aggregated and visualized, and enhanced with drill-downs, -and pivoting for delving into deeper investigations. Centralized monitoring of diverse data -sources is straightforward with normalized ECS data. - -[role="screenshot"] -image:images/ecs-getting-started-dashboard.png[Simplify visualization using ECS] - - -[float] -==== Translating Data Sources - -To align events to ECS, some sort of parsing will usually be necessary -to transform the contents of the original event into the relevant ECS fields. Depending on -how you've designed your Elastic Stack data ingestion pipelines, the amount of work to parse -your events will vary. - -For example, an Apache web server log event: - -[source,sh] ------------ -10.42.42.42 - - [15/Jul/2020:20:48:32 +0000] "GET /content HTTP/1.1" 200 2571 "-" -"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko) -Chrome/83.0.4103.106 Safari/537.36" ------------ - -In order to map this event to ECS, the contents of the event is associated with the -appropriate ECS fields. - -[options="header"] -|===== -| Field Name | Value - -// =============================================================== - -| @timestamp -| `2020-07-15T20:48:32.000Z` - -// ============================================================== - -| event.original -| 10.42.42.42 - - [15/Jul/2020:20:48:32 +0000] "GET /content HTTP/1.1" 200 2571 "-" -"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko) -Chrome/83.0.4103.106 Safari/537.36 - -// ============================================================== - -| http.request.method -| GET - -// ============================================================== - -| http.response.body.bytes -| 2571 - -// ============================================================== - -| http.response.status_code -| 200 - -// ============================================================== - -| http.version -| 1.1 - -// ============================================================== - -| message -| GET /content HTTP/1.1" 200 2571 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36 - -// ============================================================== - -| source.address -| 10.42.42.42 - -// ============================================================== - -| source.ip -| 10.42.42.42 - -// ============================================================== - -| url.original -| `/content` - -// ============================================================== - -| user_agent.original -| `Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36` - -|===== - -Beyond extracting values that are present in the original event, we also populate -other fields to provide additional context about the event itself. - -* `ecs.version`: States which version of ECS the ingest pipeline was developed against. -* `event.dataset` and `event.module`: Answers "where is this event from" and are expected to have a - hardcoded value per pipeline, per source. -* `event.kind`, `event.category`, `event.type`, and `event.outcome`: The <> - should also be hardcoded using knowledge of each type of event the source emits. The contents of - these fields are limited to the specifically allowed values detailed in the ECS documentation. - -[options="header"] -|===== -| Field Name | Value - -// =============================================================== - -| ecs.version -| 1.5.0 - -// ============================================================== - -| event.module -| apache - -// =============================================================== - -| event.dataset -| apache.access - -// ============================================================== - -| event.kind -| event - -// =============================================================== - -| event.category -| [ "network", "web" ] - -// ============================================================== - -| event.type -| [ "access" ] - -// ============================================================== - -| event.outcome -| success - -|===== - -Lastly, existing field values can be interpreted or enriched using a processor, with the results -populating additional fields in the final event. - -* The `user_agent` processor extracts details from the original user agent string, `user_agent.original`. -* IP fields like `source.ip` can provide enrichment using the `geoip` processor to add information about the - location and autonomous system number (ASN) associated with an IP address. -* The `registered domain` processor reads a field containing a hostname and writes the registered domain to - another field -* Event collectors, such as {beats-ref}/beats-reference.html[Beats], can enrich - each event with metadata from the machine's hosting provider (cloud) and/or from the host machine (host). - -Here are some examples of additional fields processed by metadata or parser processors. - -[options="header"] -|===== -| Field Name | Value | Processor - -// ============================================================== - -| host.architecture -| x86_64 -| `add_host_metadata` - -// ============================================================== - -| host.hostname -| mbp.example.com -| `add_host_metadata` - -// ============================================================== - -| host.ip -| [ "192.168.1.100" ] -| `add_host_metadata` - -// ============================================================== - -| host.os.family -| darwin -| `add_host_metadata` - -// ============================================================== - -| host.os.kernel -| 19.4.0 -| `add_host_metadata` - -// ============================================================== - -| host.os.name -| Mac OS X -| `add_host_metadata` - -// ============================================================== - -| host.os.version -| 10.15.4 -| `add_host_metadata` - -// ============================================================== - -| user_agent.name -| Chrome -| `user_agent` - -// ============================================================== - -| user_agent.os.full -| Mac OS X 10.15.4 -| `user_agent` - -// ============================================================== - -| user_agent.os.name -| Mac OS X -| `user_agent` - -// ============================================================== - -| user_agent.os.version -| 10.15.4 -| `user_agent` - -// ============================================================== - -| user_agent.version -| 83.0.4103.106 -| `user_agent` - -|===== - -[float] -==== Field Mapping Reference Guides - -We've covered at a high level how to map your events to ECS. Now if you'd like your events to render well in the Elastic -solutions, check out the reference guides below to learn more about each: - -* {observability-guide}/logs-app-fields.html[Log Monitoring Field Reference] -* {observability-guide}/metrics-app-fields.html[Metrics Monitoring Field Reference] -* {security-guide}/siem-field-reference.html[Elastic Security Field Reference] diff --git a/docs/using-ecs/guidelines.asciidoc b/docs/using-ecs/guidelines.asciidoc deleted file mode 100644 index 6b0d3641ef..0000000000 --- a/docs/using-ecs/guidelines.asciidoc +++ /dev/null @@ -1,82 +0,0 @@ -[[ecs-guidelines]] -=== Guidelines and Best Practices - -The {ecs} schema serves best when you follow schema guidelines and best -practices. - -[discrete] -==== ECS Field Levels - -ECS defines "Core" and "Extended" fields. - -* *Core fields.* Fields that are most common across all use cases are defined as *core fields*. -+ -These generalized fields are used by analysis content -(searches, visualizations, dashboards, alerts, machine learning jobs, reports) -across use cases. Analysis content designed to operate on these -fields should work properly on data from any relevant source. -+ -Focus on populating these fields first. - -* *Extended fields.* Any field that is not a core field is defined as an *extended field*. -Extended fields may apply to more narrow use cases, or may be more open -to interpretation depending on the use case. Extended fields are more likely to -change over time. - -Each {ecs} <> in a table is identified as core or extended. - - - -[discrete] -==== General guidelines - -* The document MUST have the `@timestamp` field. -* Use the {ref}/mapping-types.html[data types] - defined for an ECS field. -* Use the `ecs.version` field to define which version of ECS is used. -* Map as many fields as possible to ECS. - - -[discrete] -===== Guidelines for field names - -* *Field names must be lower case* - -* *Combine words using underscore* - -* *No special characters except underscore* - -* *Use present tense* unless field describes historical information. - -* *Use singular and plural names properly* to reflect the field content. -** For example, use `requests_per_sec` rather than `request_per_sec`. - -* *Use prefixes for all fields*, except for the base fields. -** For example, all `host` fields are prefixed with `host.`. Such a grouping is - called a field set. - -* *Nest fields inside a field set* with dots -** The document structure should be nested JSON objects. - If you use Beats or Logstash, the nesting of JSON objects is done for you automatically. - If you're ingesting to Elasticsearch using the API, your fields must be nested - objects, not strings containing dots. -** See <> for more details. - -* *General to specific*. Organise the nesting of field sets from general to specific, - to allow grouping fields into objects with a prefix like `host.*`. - -* *Avoid repetition* or stuttering of words -** If part of the field name is already in the name of the field set, - avoid repeating it. Example: `host.host_ip` should be `host.ip`. -** Exceptions can be made, when changing the name of the field would break a - strong convention in the community. - Example: `host.hostname` is an exception to this rule. - -* *Avoid abbreviations when possible* -** Exceptions can be made, when the name used for the concept is too strongly - in favor of the abbreviation. - Example: `ip` fields, or field sets such as `os`, `geo`. - -include::guidelines/conventions.asciidoc[] -include::guidelines/implementation.asciidoc[] -include::guidelines/mapping-network-events.asciidoc[] diff --git a/docs/using-ecs/guidelines/conventions.asciidoc b/docs/using-ecs/guidelines/conventions.asciidoc deleted file mode 100644 index e0b804debc..0000000000 --- a/docs/using-ecs/guidelines/conventions.asciidoc +++ /dev/null @@ -1,84 +0,0 @@ -[[ecs-conventions]] -==== Conventions - -The implementation of ECS follows a few conventions. Understanding them will -help you understand ECS better. - -[discrete] -===== Datatype for integers - -Unless otherwise noted, the datatype used for integer fields should be `long`. - -[discrete] -===== IDs and most codes are keywords, not integers - -Despite the fact that IDs and codes (such as error codes) are often integers, -this is not always the case. -Since we want to make it possible to map as many systems and data sources -to ECS as possible, we default to using the `keyword` type for IDs and codes. - -Some specific kinds of codes are always integers, like HTTP status codes. -If those have a specific corresponding specific field (as HTTP status does), -its type can safely be an integer type. -But generic fields like `error.code` cannot have this guarantee, and are therefore `keyword`. - -[discrete] -===== Text indexing and multi-fields - -Elasticsearch can index text using datatypes: - -* *`text`* Text indexing allows for full text search, or searching arbitrary words that - are part of the field. - See {ref}/text.html[Text datatype] in the {es} Reference Guide. -* *`keyword`* Keyword indexing offers faster exact match filtering, - prefix search (like autocomplete), - and makes aggregations (like {kib} visualizations) possible. - See the {es} Reference Guide for more information on - {ref}/query-dsl-term-query.html[exact match filtering], - {ref}/query-dsl-prefix-query.html[prefix search], or - {ref}/search-aggregations.html[aggregations]. - -[discrete] -===== Default Elasticsearch convention for indexing text fields - -Unless your index mapping or index template specifies otherwise -(as the <> does), -Elasticsearch indexes a text field as `text` at the canonical field name, -and indexes a second time as `keyword`, nested in a multi-field. - -Default Elasticsearch convention: - -* Canonical field: `myfield` is `text` -* Multi-field: `myfield.keyword` is `keyword` - -[discrete] -===== ECS convention for indexing text fields - -ECS flips the above convention around. - -For monitoring use cases, `keyword` indexing is needed almost exclusively, with -full text search needed on very few fields. -Moreover, indexing for full text search on lots of fields, where it's not expected -to be used is wasteful of resources. - -Given these two premises, ECS defaults -all text indexing to `keyword` datatype (with very few exceptions). -Any use case that requires full text search indexing on additional fields -can add a {ref}/multi-fields.html[multi-field] -for full text search. Doing so does not conflict with ECS, -as the canonical field name will remain `keyword` indexed. - -So the ECS multi-field convention for text is: - -* Canonical field: `myfield` is `keyword` -* Multi-field: `myfield.text` is `text` - -**Exceptions** - -The only exceptions to this convention are fields `message` and `error.message`, -which are indexed for full text search only, with no multi-field. -These two fields don't follow the new convention because they are deemed too big -of a breaking change with these two widely used fields in Beats. - -Any future field that will be indexed for full text search in ECS will however -follow the multi-field convention where `text` indexing is nested in the multi-field. diff --git a/docs/using-ecs/guidelines/implementation.asciidoc b/docs/using-ecs/guidelines/implementation.asciidoc deleted file mode 100644 index 7ea0d1c152..0000000000 --- a/docs/using-ecs/guidelines/implementation.asciidoc +++ /dev/null @@ -1,230 +0,0 @@ -[[ecs-principles-implementation]] -==== Implementation patterns - -Normalizing data provides a more consistent view of events from various data sources. -Following these conventions will help to better describe, discover, identify, and categorize events. - -[discrete] -===== Base fields - -The group of individual fields residing outside any field set at the top-level of ECS -are known as base fields. - -ECS events follow these conventions with the base fields: - -@timestamp:: -All events must populate <> with the event's original timestamp. - -message:: -Most events should populate <>. - -ecs.version:: -The referenced <> used to develop the data mapping or ingest pipeline. -This value helps detect when mappings update or fall behind. -It can also help explain why a particular data source isn't populating the same fields as another. - -tags and labels:: -The <> and <> fields add simple metadata as `keyword` values. - -[discrete] -===== Host - -In ECS, the `host` is the computing instance where the event happened. A `host` can be a physical device, virtual machine, container, or cloud instance. - -The <> field set contains common attributes for different computing instances. -Certain host types have more fields to capture specific details, like -`cloud.*` or `container.*`. - -[discrete] -===== Agent and observer - -An agent is software that collects, observes, measures, or detects the event. -The <> fields capture details about which agent entity captured the event, -including the agent's version. Examples of agents are Beats and Elastic Agent. - -An `observer` is an external monitoring or intermediary device, like a firewall, APM server, or web proxy. -These devices monitor and detect network, security, application events. Capture the details for these device -types in the <> field set. - -[discrete] -===== Timestamps - -ECS requires the `@timestamp` field on every event. Some events also contain extra timestamps to capture. - -@timestamp:: -All events must populate <> with when the event originated. - -event.created:: -The timestamp of when an agent or pipeline saw the event. - -event.ingested:: -The timestamp of when an event arrived in the central data store, like Elasticsearch. - -These three timestamps should typically follow a chronological order: - -[source,sh] ----- -@timestamp < event.created < event.ingested ----- - -event.start:: -This timestamp marks the beginning of the event activity. For example, in a network session, <> -is the timestamp of the first observed packet in the flow. - -event.end:: -This timestamp marks the end of the activity. In a network flow, <> is the timestamp of the last observed packet -in the flow. - -event.duration:: -The difference of `event.end` and `event.start`: - -[source,sh] ----- -event.duration = event.end - event.start ----- - -[discrete] -===== Origin - -Specific `event.*` fields exist to capture where an event originated. - -event.provider:: -Contains the name of the software or operating subsystem that generated the event. - -event.module:: -If the ingest agent or pipeline has a concept of modules or plugins, populate <> with the module or plugin name. - -event.dataset:: -Used to define different types of logs or metrics from an event source. The recommended -convention is `.`. For Apache web server access logs, the -<> value will be `apache.access`. - -[discrete] -===== Categorization - -The event categorization fields group similar events using allowed values for four fields: - -* `event.kind` -* `event.category` -* `event.type` -* `event.outcome` - -<> covers more details on using these four fields together to categorize events. - -[discrete] -===== Enriching events - -A monitoring agent or ingest pipeline can add more details to the original event. -ECS has many fields to hold these enrichment details. - -[discrete] -===== Lookups - -GeoIP:: -Add information about the geographical location of an IPv4 or IPv6 address. Often used to populate the `geo.*` -fields nested under network transaction fields like `source.*`, `destination.*`, `client.*`, and `server.*`. - -[source,json] ----- -{ - "source": { - "address": "8.8.8.8", - "ip": "8.8.8.8", - "geo": { - "continent_name": "North America", - "country_name": "United States", - "country_iso_code": "US", - "location": { "lat": 37.751, "lon": -97.822 } - } - } -} ----- - -Autonomous system number:: -Autonomous System Number (ASN) database lookups determine the ASN associated with an IP address. - -[discrete] -===== Parsing - -User-agent:: -Break the user-agent into individual fields. - -[source,json] ----- -{ - "user_agent": { - "user_agent": { - "name": "Chrome", - "original": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36", - "version": "51.0.2704.103", - "os": { - "name": "Mac OS X", - "version": "10.10.5", - "full": "Mac OS X 10.10.5", - "platform": "darwin", - "type": "macos" - }, - "device" : { - "name" : "Mac" - } - } - } -} ----- - -URL:: -A URL can also break down into its discrete parts. - -[source,json] ----- -{ - "original" : "http://myusername:mypassword@www.example.com:80/foo.gif?key1=val1&key2=val2#fragment", - "url" : { - "path" : "/foo.gif", - "fragment" : "fragment", - "extension" : "gif", - "password" : "mypassword", - "original" : "http://myusername:mypassword@www.example.com:80/foo.gif?key1=val1&key2=val2#fragment", - "scheme" : "http", - "port" : 80, - "user_info" : "myusername:mypassword", - "domain" : "www.example.com", - "query" : "key1=val1&key2=val2", - "username" : "myusername" - } -} ----- - -Domain names:: -Extract the registered domain (also known as the effective top-level domain plus one), -sub-domain, and effective top-level domain from a fully-qualified domain name (FQDN). - -[source,json] ----- -{ - "fqdn": "www.example.ac.uk", - "url": { - "subdomain": "www", - "registered_domain": "example.ac.uk", - "top_level_domain": "ac.uk", - "domain": "www.example.ac.uk" -} ----- - -[discrete] -===== Related fields - -Many events have similar content populating different fields: IP addresses, file hashes, hostnames. -Pivot between these events using the <> fields. - -For example, IP addresses found under the `host.*`, `source.*`, `destination.*`, `client.*`, and -`server.*` fields sets and the `network.forwarded_ip` field. By adding all IP addresses in an event to -the `related.ip` field, there is now a single field to search for a given IP regardless of what field it -appeared: - - -[source,sh] ----- -related.ip: ["10.42.42.42"] ----- diff --git a/docs/using-ecs/guidelines/mapping-network-events.asciidoc b/docs/using-ecs/guidelines/mapping-network-events.asciidoc deleted file mode 100644 index d1df81f478..0000000000 --- a/docs/using-ecs/guidelines/mapping-network-events.asciidoc +++ /dev/null @@ -1,269 +0,0 @@ -[[ecs-mapping-network-events]] -==== Mapping network events - -Network events capture the details of one device communicating with another. The initiator is referred to as the source, and the recipient as the destination. Depending on the data source, a network event can contain details of addresses, protocols, headers, and device roles. - -This guide describes the different field sets available for network-related events in ECS and provides direction on the ECS best practices for mapping to them. - -[discrete] -===== Source and destination baseline - -When an event contains details about the sending and receiving hosts, the baseline for capturing these values will be the <> and <> fields. - -Some events may also indicate each host's role in the exchange: client or server. When this information is available, the <> and <> fields should be used _in addition to_ the `source` and `destination` fields. The fields and values mapped under `source`/`destination` should be copied under `client`/`server`. - -[discrete] -===== Network event mapping example - -Below is a DNS network event. The source device (`192.168.86.222`) makes a DNS query, acting as the client and the DNS server is the destination (`192.168.86.1`). - -Note this event contains additional details that would populate additional fields (such as the <>) if this was a complete mapping example. These additional fields are omitted here to focus on the network details. - -[source,json] ----- -{ - "ts":1599775747.53056, - "uid":"CYqFPH3nOAa0kPxA0d", - "id.orig_h":"192.168.86.222", - "id.orig_p":54162, - "id.resp_h":"192.168.86.1", - "id.resp_p":53, - "proto":"udp", - "trans_id":28899, - "rtt":0.02272200584411621, - "query":"example.com", - "qclass":1, - "qclass_name":"C_INTERNET", - "qtype":1, - "qtype_name":"A", - "rcode":0, - "rcode_name":"NOERROR", - "AA":false, - "TC":false, - "RD":true, - "RA":true, - "Z":0, - "answers":["93.184.216.34"], - "TTLs":[21209.0], - "rejected":false -} ----- - -[discrete] -===== Source and destination fields - -First, the `source.*` and `destination.*` field sets are populated: - -[source,json] ----- - "source": { - "ip": "192.168.86.222", - "port": 54162 - } ----- - -[source,json] ----- - "destination": { - "ip": "192.168.86.1", - "port": 53 - } ----- - -[discrete] -===== Client and server fields - -Looking back at the original event, it shows the source device is the DNS client and the destination device is the DNS server. The values mapped under `source` and `destination` are copied and mapped under `client` and `server`, respectively: - -[source,json] ----- - "client": { - "ip": "192.168.86.222", - "port": 54162 - } ----- - -[source,json] ----- - "server": { - "ip": "192.168.86.1", - "port": 53 - } ----- - -Mapping both pairs of field sets gives query visibility of the same network transaction in two ways. - -* `source.ip:192.168.86.222` returns all events sourced from `192.168.86.222`, regardless its role in a transaction -* `client.ip:192.168.86.222` returns all events with host `192.168.86.222` acting as a client - -The same applies for the `destination` and `server` fields: - -* `destination.ip:192.168.86.1` returns all events destined to `192.168.86.1` -* `server.ip:192.168.86.1` returns all events with `192.168.86.1` acting as the server - -It's important to note that while the values for the `source` and `destination` fields may reverse between events in a single network transaction, the values for `client` and `server` typically will not. The following two tables demonstrate how two DNS transactions involving two clients and one server would map to `source.ip`/`destination.ip` vs. `client.ip`/`server.ip`: - -[options="header"] -.Source/Destination -|===== -| source.ip | destination.ip | event - -// =============================================================== - -| 192.168.86.222 -| 192.168.86.1 -| DNS query request 1 - -// =============================================================== - -| 192.168.86.1 -| 192.168.86.222 -| DNS answer response 1 - -// =============================================================== - -| 192.168.86.42 -| 192.168.86.1 -| DNS answer request 2 - -// =============================================================== - -| 192.168.86.1 -| 192.168.86.42 -| DNS answer request 2 - -|===== - -[options="header"] -.Client/Server -|===== -| client.ip | server.ip | event - -// =============================================================== - -| 192.168.86.222 -| 192.168.86.1 -| DNS query request 1 - -// =============================================================== - -| 192.168.86.222 -| 192.168.86.1 -| DNS answer response 1 - -// =============================================================== - -| 192.168.86.42 -| 192.168.86.1 -| DNS query request 2 - -// =============================================================== - -| 192.168.86.42 -| 192.168.86.1 -| DNS answer response 2 - -|===== - -[discrete] -==== Related fields - -The `related.ip` field captures all the IPs present in the event in a single array: - -[source,json] ----- - "related": { - "ip": [ - "192.168.86.222", - "192.168.86.1", - "93.184.216.34" - ] - } ----- - -The <> are meant to facilitate pivoting. Since these IP addresses can appear in many different fields (`source.ip`, `destination.ip`, `client.ip`, `server.ip`, etc.), you can search for the IP trivially no matter what field it appears using a single query, e.g. `related.ip:192.168.86.222`. - -Network events are not only limited to using `related.ip`. If hostnames or other host identifiers were present in the event, `related.hosts` should be populated too. - -[discrete] -===== Categorization using event fields - -When considering the <>, the `category` and `type` fields are populated using their respective allowed values which best classify the source network event. - -[source,json] ----- - "event": { - "category": [ - "network" - ], - "type": [ - "connection", - "protocol" - ], - "kind": "event" - } ----- - -Most <>/<> ECS pairings are complete on their own. However, the pairing of `event.category:network` and `event.type:protocol` is an exception. When these two fields/value pairs both used to categorize an event, the `network.protocol` field should also be populated: - -[source,json] ----- - "network": { - "protocol": "dns", - "type": "ipv4", - "transport": "udp" - } ----- - -[discrete] -===== Result - -Putting everything together covered so far, we have a final ECS-mapped event: - -[source,json] ----- -{ - "event": { - "category": [ - "network" - ], - "type": [ - "connection", - "protocol" - ], - "kind": "event" - }, - "network": { - "protocol": "dns", - "type": "ipv4", - "transport": "udp" - }, - "source": { - "ip": "192.168.86.222", - "port": 54162 - }, - "destination": { - "ip": "192.168.86.1", - "port": 53 - }, - "client": { - "ip": "192.168.86.222", - "port": 64734 - }, - "server": { - "ip": "192.168.86.1", - "port": 53 - }, - "related": { - "ip": [ - "192.168.86.222", - "192.168.86.1", - "93.184.216.34" - ] - }, - "dns": { ... }, <1> - "zeek": { "ts":1599775747.53056, ... } <2> -} ----- -<1> Again, not diving into the DNS fields here but included for completeness. -<2> Original fields can optionally be kept around as custom fields. diff --git a/docs/using-ecs/index.asciidoc b/docs/using-ecs/index.asciidoc deleted file mode 100644 index f7fc6345a3..0000000000 --- a/docs/using-ecs/index.asciidoc +++ /dev/null @@ -1,22 +0,0 @@ -[[ecs-using-ecs]] -== Using ECS - -ECS fields follow a series of guidelines, to ensure a consistent and predictable -feel, across various use cases. - -If you're new to ECS and would like an introduction on implementing and using -the schema, check out the <> guide. - -Whether you're trying to recall a field name, implementing a solution that -follows ECS, or proposing a change to the schema, the <> section -will help get you there. - -<> are a set of considerations used to help guide the ECS project. - -If you're wondering how to best capture event details that don't map to existing -ECS fields, head over to <>. - -include::getting-started.asciidoc[][] -include::guidelines.asciidoc[] -include::design.asciidoc[] -include::custom-fields.asciidoc[]