Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added question.subdomain field #561

Closed
wants to merge 1 commit into from
Closed

Added question.subdomain field #561

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions code/go/ecs/dns.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

11 changes: 11 additions & 0 deletions docs/field-details.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -890,6 +890,17 @@ example: `google.com`

// ===============================================================

| dns.question.subdomain
| A subdomain is a hostname under it's parent domain.

type: keyword

example: `www`

| extended

// ===============================================================

| dns.question.type
| The type of record being queried.

Expand Down
6 changes: 6 additions & 0 deletions generated/beats/fields.ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -735,6 +735,12 @@
list (http://publicsuffix.org). Trying to approximate this by simply taking
the last two labels will not work well for TLDs such as "co.uk".'
example: google.com
- name: question.subdomain
level: extended
type: keyword
ignore_above: 1024
description: A subdomain is a hostname under it's parent domain.
example: www
- name: question.type
level: extended
type: keyword
Expand Down
1 change: 1 addition & 0 deletions generated/csv/fields.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,7 @@ dns.op_code,keyword,extended,QUERY,1.2.0-dev
dns.question.class,keyword,extended,IN,1.2.0-dev
dns.question.name,keyword,extended,www.google.com,1.2.0-dev
dns.question.registered_domain,keyword,extended,google.com,1.2.0-dev
dns.question.subdomain,keyword,extended,www,1.2.0-dev
dns.question.type,keyword,extended,AAAA,1.2.0-dev
dns.resolved_ip,ip,extended,"['10.10.10.10', '10.10.10.11']",1.2.0-dev
dns.response_code,keyword,extended,NOERROR,1.2.0-dev
Expand Down
24 changes: 17 additions & 7 deletions generated/ecs/ecs_flat.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -869,7 +869,7 @@ dns.answers:
level: extended
name: answers
object_type: keyword
order: 9
order: 10
short: Array of DNS answers.
type: object
dns.answers.class:
Expand All @@ -879,7 +879,7 @@ dns.answers.class:
ignore_above: 1024
level: extended
name: answers.class
order: 12
order: 13
short: The class of DNS data contained in this resource record.
type: keyword
dns.answers.data:
Expand All @@ -891,7 +891,7 @@ dns.answers.data:
ignore_above: 1024
level: extended
name: answers.data
order: 14
order: 15
short: The data describing the resource.
type: keyword
dns.answers.name:
Expand All @@ -905,7 +905,7 @@ dns.answers.name:
ignore_above: 1024
level: extended
name: answers.name
order: 10
order: 11
short: The domain name to which this resource record pertains.
type: keyword
dns.answers.ttl:
Expand All @@ -915,7 +915,7 @@ dns.answers.ttl:
flat_name: dns.answers.ttl
level: extended
name: answers.ttl
order: 13
order: 14
short: The time interval in seconds that this resource record may be cached before
it should be discarded. Zero values mean that the data should not be cached.
type: long
Expand All @@ -926,7 +926,7 @@ dns.answers.type:
ignore_above: 1024
level: extended
name: answers.type
order: 11
order: 12
short: The type of data contained in this resource record.
type: keyword
dns.header_flags:
Expand Down Expand Up @@ -1008,6 +1008,16 @@ dns.question.registered_domain:
order: 8
short: The highest registered domain, stripped of the subdomain.
type: keyword
dns.question.subdomain:
description: A subdomain is a hostname under it's parent domain.
example: www
flat_name: dns.question.subdomain
ignore_above: 1024
level: extended
name: question.subdomain
order: 9
short: The subdomain of the domain.
type: keyword
dns.question.type:
description: The type of record being queried.
example: AAAA
Expand All @@ -1031,7 +1041,7 @@ dns.resolved_ip:
flat_name: dns.resolved_ip
level: extended
name: resolved_ip
order: 15
order: 16
short: Array containing all IPs seen in answers.data
type: ip
dns.response_code:
Expand Down
24 changes: 17 additions & 7 deletions generated/ecs/ecs_nested.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1037,7 +1037,7 @@ dns:
level: extended
name: answers
object_type: keyword
order: 9
order: 10
short: Array of DNS answers.
type: object
answers.class:
Expand All @@ -1047,7 +1047,7 @@ dns:
ignore_above: 1024
level: extended
name: answers.class
order: 12
order: 13
short: The class of DNS data contained in this resource record.
type: keyword
answers.data:
Expand All @@ -1059,7 +1059,7 @@ dns:
ignore_above: 1024
level: extended
name: answers.data
order: 14
order: 15
short: The data describing the resource.
type: keyword
answers.name:
Expand All @@ -1073,7 +1073,7 @@ dns:
ignore_above: 1024
level: extended
name: answers.name
order: 10
order: 11
short: The domain name to which this resource record pertains.
type: keyword
answers.ttl:
Expand All @@ -1084,7 +1084,7 @@ dns:
flat_name: dns.answers.ttl
level: extended
name: answers.ttl
order: 13
order: 14
short: The time interval in seconds that this resource record may be cached
before it should be discarded. Zero values mean that the data should not be
cached.
Expand All @@ -1096,7 +1096,7 @@ dns:
ignore_above: 1024
level: extended
name: answers.type
order: 11
order: 12
short: The type of data contained in this resource record.
type: keyword
header_flags:
Expand Down Expand Up @@ -1179,6 +1179,16 @@ dns:
order: 8
short: The highest registered domain, stripped of the subdomain.
type: keyword
question.subdomain:
description: A subdomain is a hostname under it's parent domain.
example: www
flat_name: dns.question.subdomain
ignore_above: 1024
level: extended
name: question.subdomain
order: 9
short: The subdomain of the domain.
type: keyword
question.type:
description: The type of record being queried.
example: AAAA
Expand All @@ -1202,7 +1212,7 @@ dns:
flat_name: dns.resolved_ip
level: extended
name: resolved_ip
order: 15
order: 16
short: Array containing all IPs seen in answers.data
type: ip
response_code:
Expand Down
4 changes: 4 additions & 0 deletions generated/elasticsearch/6/template.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -452,6 +452,10 @@
"ignore_above": 1024,
"type": "keyword"
},
"subdomain": {
"ignore_above": 1024,
"type": "keyword"
},
"type": {
"ignore_above": 1024,
"type": "keyword"
Expand Down
4 changes: 4 additions & 0 deletions generated/elasticsearch/7/template.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -451,6 +451,10 @@
"ignore_above": 1024,
"type": "keyword"
},
"subdomain": {
"ignore_above": 1024,
"type": "keyword"
},
"type": {
"ignore_above": 1024,
"type": "keyword"
Expand Down
4 changes: 4 additions & 0 deletions generated/legacy/template.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -273,6 +273,10 @@
"ignore_above": 1024,
"type": "keyword"
},
"subdomain": {
"ignore_above": 1024,
"type": "keyword"
},
"type": {
"ignore_above": 1024,
"type": "keyword"
Expand Down
10 changes: 10 additions & 0 deletions schema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -625,6 +625,16 @@
"required": false,
"type": "keyword"
},
"dns.question.subdomain": {
"description": "A subdomain is a hostname under it's parent domain.",
"example": "www",
"footnote": "",
"group": 2,
"level": "extended",
"name": "dns.question.subdomain",
"required": false,
"type": "keyword"
},
"dns.question.type": {
"description": "The type of record being queried.",
"example": "AAAA",
Expand Down
8 changes: 8 additions & 0 deletions schemas/dns.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,14 @@
simply taking the last two labels will not work well for TLDs such as "co.uk".
example: google.com

- name: question.subdomain
level: extended
type: keyword
short: The subdomain of the domain.
description: >
A subdomain is a hostname under it's parent domain.
Copy link
Contributor

@webmat webmat Sep 27, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First sentence is great as is, let's keep it.

However I would like the description to clarify two details (array v string, and trailing period). I know it will be interpreted differently by different people, if we don't specify. Could you add something like this as a second paragraph, please?

If the subdomain has multiple levels, such as "sub2.sub1.example.com", the subdomain field should contain "sub2.sub1", with no trailing period.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am +1 to adding *.subdomain in dns.question.*

Question: Is there any value in adding *.subdomain anywhere else? I am thinking no.

sure - using it to cut up domains in urls would be useful for running the same sorts of analytics as running on DNS info... typically tls connections will start with a reference to the original fqdn of the session in the clear iirc

example: www

- name: answers
level: extended
type: object
Expand Down