Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add architecture and imphash for PE field set #763

Merged
merged 4 commits into from
Apr 6, 2020
Merged

Add architecture and imphash for PE field set #763

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
bd385ec
Add architecture and imphash for PE field set
Feb 26, 2020
0bef404
Add changelog entry
Feb 26, 2020
6b11695
Update schemas/pe.yml
andrewstucki Apr 4, 2020
6ca7236
Merge branch 'master' of github.com:elastic/ecs into imphash
Apr 4, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.next.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ Thanks, you're awesome :-) -->
#### Added

* Added `search.*` fields #729
* Add architecture and imphash for PE field set. (#763)

#### Improvements

Expand Down
11 changes: 11 additions & 0 deletions code/go/ecs/pe.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

28 changes: 28 additions & 0 deletions docs/field-details.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3960,6 +3960,19 @@ These fields contain Windows Portable Executable (PE) metadata.

// ===============================================================

| pe.architecture
| CPU architecture target for the file.

type: keyword



example: `x64`

| extended

// ===============================================================

| pe.company
| Internal company name of the file, provided at compile-time.

Expand Down Expand Up @@ -3999,6 +4012,21 @@ example: `6.3.9600.17415`

// ===============================================================

| pe.imphash
| A hash of the imports in a PE file. An imphash -- or import hash -- can be used to fingerprint binaries even after recompilation or other code-level transformations have occurred, which would change more traditional hash values.

Learn more at https://www.fireeye.com/blog/threat-research/2014/01/tracking-malware-import-hashing.html.

type: keyword



example: `0c6803c4e922103c4dca5963aad36ddf`

| extended

// ===============================================================

| pe.original_file_name
| Internal name of the file, provided at compile-time.

Expand Down
72 changes: 72 additions & 0 deletions generated/beats/fields.ecs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -833,6 +833,13 @@
description: Full file path of the library.
example: C:\Windows\System32\kernel32.dll
default_field: false
- name: pe.architecture
level: extended
type: keyword
ignore_above: 1024
description: CPU architecture target for the file.
example: x64
Copy link

@gabriellandau gabriellandau Feb 26, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we care if we make our own values here or should we use the ones that Microsoft defined? For example, in the sensor outputs x64 but Microsoft uses the nomenclature AMD64 (IMAGE_FILE_MACHINE_AMD64)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My thought was to normalize it like VirusTotal does, but not entirely sure if we'd have to be strict about this.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there's a clear set of instructions we can give on how this should be normalized (e.g. linking to another source) we should do that now.

If there isn't, we can leave this up to the source, and only address later, only if needed.

The thinking: we have to balance the amount of work required by sources to get the normalization right. So I think it's fine to tighten this later, only if needed.

default_field: false
- name: pe.company
level: extended
type: keyword
Expand All @@ -854,6 +861,17 @@
description: Internal version of the file, provided at compile-time.
example: 6.3.9600.17415
default_field: false
- name: pe.imphash
level: extended
type: keyword
ignore_above: 1024
description: 'A hash of the imports in a PE file. An imphash -- or import hash
-- can be used to fingerprint binaries even after recompilation or other code-level
transformations have occurred, which would change more traditional hash values.

Learn more at https://www.fireeye.com/blog/threat-research/2014/01/tracking-malware-import-hashing.html.'
example: 0c6803c4e922103c4dca5963aad36ddf
default_field: false
- name: pe.original_file_name
level: extended
type: keyword
Expand Down Expand Up @@ -1572,6 +1590,13 @@
description: Full path to the file, including the file name. It should include
the drive letter, when appropriate.
example: /home/alice/example.png
- name: pe.architecture
level: extended
type: keyword
ignore_above: 1024
description: CPU architecture target for the file.
example: x64
default_field: false
- name: pe.company
level: extended
type: keyword
Expand All @@ -1593,6 +1618,17 @@
description: Internal version of the file, provided at compile-time.
example: 6.3.9600.17415
default_field: false
- name: pe.imphash
level: extended
type: keyword
ignore_above: 1024
description: 'A hash of the imports in a PE file. An imphash -- or import hash
-- can be used to fingerprint binaries even after recompilation or other code-level
transformations have occurred, which would change more traditional hash values.

Learn more at https://www.fireeye.com/blog/threat-research/2014/01/tracking-malware-import-hashing.html.'
example: 0c6803c4e922103c4dca5963aad36ddf
default_field: false
- name: pe.original_file_name
level: extended
type: keyword
Expand Down Expand Up @@ -2845,6 +2881,13 @@
description: These fields contain Windows Portable Executable (PE) metadata.
type: group
fields:
- name: architecture
level: extended
type: keyword
ignore_above: 1024
description: CPU architecture target for the file.
example: x64
default_field: false
- name: company
level: extended
type: keyword
Expand All @@ -2866,6 +2909,17 @@
description: Internal version of the file, provided at compile-time.
example: 6.3.9600.17415
default_field: false
- name: imphash
level: extended
type: keyword
ignore_above: 1024
description: 'A hash of the imports in a PE file. An imphash -- or import hash
-- can be used to fingerprint binaries even after recompilation or other code-level
transformations have occurred, which would change more traditional hash values.

Learn more at https://www.fireeye.com/blog/threat-research/2014/01/tracking-malware-import-hashing.html.'
example: 0c6803c4e922103c4dca5963aad36ddf
default_field: false
- name: original_file_name
level: extended
type: keyword
Expand Down Expand Up @@ -3258,6 +3312,13 @@
description: The working directory of the process.
example: /home/alice
default_field: false
- name: pe.architecture
level: extended
type: keyword
ignore_above: 1024
description: CPU architecture target for the file.
example: x64
default_field: false
- name: pe.company
level: extended
type: keyword
Expand All @@ -3279,6 +3340,17 @@
description: Internal version of the file, provided at compile-time.
example: 6.3.9600.17415
default_field: false
- name: pe.imphash
level: extended
type: keyword
ignore_above: 1024
description: 'A hash of the imports in a PE file. An imphash -- or import hash
-- can be used to fingerprint binaries even after recompilation or other code-level
transformations have occurred, which would change more traditional hash values.

Learn more at https://www.fireeye.com/blog/threat-research/2014/01/tracking-malware-import-hashing.html.'
example: 0c6803c4e922103c4dca5963aad36ddf
default_field: false
- name: pe.original_file_name
level: extended
type: keyword
Expand Down
8 changes: 8 additions & 0 deletions generated/csv/fields.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -106,9 +106,11 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description
1.6.0-dev,true,dll,dll.hash.sha512,keyword,extended,,,SHA512 hash.
1.6.0-dev,true,dll,dll.name,keyword,core,,kernel32.dll,Name of the library.
1.6.0-dev,true,dll,dll.path,keyword,extended,,C:\Windows\System32\kernel32.dll,Full file path of the library.
1.6.0-dev,true,dll,dll.pe.architecture,keyword,extended,,x64,CPU architecture target for the file.
1.6.0-dev,true,dll,dll.pe.company,keyword,extended,,Microsoft Corporation,"Internal company name of the file, provided at compile-time."
1.6.0-dev,true,dll,dll.pe.description,keyword,extended,,Paint,"Internal description of the file, provided at compile-time."
1.6.0-dev,true,dll,dll.pe.file_version,keyword,extended,,6.3.9600.17415,Process name.
1.6.0-dev,true,dll,dll.pe.imphash,keyword,extended,,0c6803c4e922103c4dca5963aad36ddf,A hash of the imports in a PE file.
1.6.0-dev,true,dll,dll.pe.original_file_name,keyword,extended,,MSPAINT.EXE,"Internal name of the file, provided at compile-time."
1.6.0-dev,true,dll,dll.pe.product,keyword,extended,,Microsoft® Windows® Operating System,"Internal product name of the file, provided at compile-time."
1.6.0-dev,true,dns,dns.answers,object,extended,array,,Array of DNS answers.
Expand Down Expand Up @@ -187,9 +189,11 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description
1.6.0-dev,true,file,file.owner,keyword,extended,,alice,File owner's username.
1.6.0-dev,true,file,file.path,keyword,extended,,/home/alice/example.png,"Full path to the file, including the file name."
1.6.0-dev,true,file,file.path.text,text,extended,,/home/alice/example.png,"Full path to the file, including the file name."
1.6.0-dev,true,file,file.pe.architecture,keyword,extended,,x64,CPU architecture target for the file.
1.6.0-dev,true,file,file.pe.company,keyword,extended,,Microsoft Corporation,"Internal company name of the file, provided at compile-time."
1.6.0-dev,true,file,file.pe.description,keyword,extended,,Paint,"Internal description of the file, provided at compile-time."
1.6.0-dev,true,file,file.pe.file_version,keyword,extended,,6.3.9600.17415,Process name.
1.6.0-dev,true,file,file.pe.imphash,keyword,extended,,0c6803c4e922103c4dca5963aad36ddf,A hash of the imports in a PE file.
1.6.0-dev,true,file,file.pe.original_file_name,keyword,extended,,MSPAINT.EXE,"Internal name of the file, provided at compile-time."
1.6.0-dev,true,file,file.pe.product,keyword,extended,,Microsoft® Windows® Operating System,"Internal product name of the file, provided at compile-time."
1.6.0-dev,true,file,file.size,long,extended,,16384,File size in bytes.
Expand Down Expand Up @@ -355,9 +359,11 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description
1.6.0-dev,true,package,package.size,long,extended,,62231,Package size in bytes.
1.6.0-dev,true,package,package.type,keyword,extended,,rpm,Package type
1.6.0-dev,true,package,package.version,keyword,extended,,1.12.9,Package version
1.6.0-dev,true,pe,pe.architecture,keyword,extended,,x64,CPU architecture target for the file.
1.6.0-dev,true,pe,pe.company,keyword,extended,,Microsoft Corporation,"Internal company name of the file, provided at compile-time."
1.6.0-dev,true,pe,pe.description,keyword,extended,,Paint,"Internal description of the file, provided at compile-time."
1.6.0-dev,true,pe,pe.file_version,keyword,extended,,6.3.9600.17415,Process name.
1.6.0-dev,true,pe,pe.imphash,keyword,extended,,0c6803c4e922103c4dca5963aad36ddf,A hash of the imports in a PE file.
1.6.0-dev,true,pe,pe.original_file_name,keyword,extended,,MSPAINT.EXE,"Internal name of the file, provided at compile-time."
1.6.0-dev,true,pe,pe.product,keyword,extended,,Microsoft® Windows® Operating System,"Internal product name of the file, provided at compile-time."
1.6.0-dev,true,process,process.args,keyword,extended,array,"['/usr/bin/ssh', '-l', 'user', '10.0.0.16']",Array of process arguments.
Expand Down Expand Up @@ -409,9 +415,11 @@ ECS_Version,Indexed,Field_Set,Field,Type,Level,Normalization,Example,Description
1.6.0-dev,true,process,process.parent.uptime,long,extended,,1325,Seconds the process has been up.
1.6.0-dev,true,process,process.parent.working_directory,keyword,extended,,/home/alice,The working directory of the process.
1.6.0-dev,true,process,process.parent.working_directory.text,text,extended,,/home/alice,The working directory of the process.
1.6.0-dev,true,process,process.pe.architecture,keyword,extended,,x64,CPU architecture target for the file.
1.6.0-dev,true,process,process.pe.company,keyword,extended,,Microsoft Corporation,"Internal company name of the file, provided at compile-time."
1.6.0-dev,true,process,process.pe.description,keyword,extended,,Paint,"Internal description of the file, provided at compile-time."
1.6.0-dev,true,process,process.pe.file_version,keyword,extended,,6.3.9600.17415,Process name.
1.6.0-dev,true,process,process.pe.imphash,keyword,extended,,0c6803c4e922103c4dca5963aad36ddf,A hash of the imports in a PE file.
1.6.0-dev,true,process,process.pe.original_file_name,keyword,extended,,MSPAINT.EXE,"Internal name of the file, provided at compile-time."
1.6.0-dev,true,process,process.pe.product,keyword,extended,,Microsoft® Windows® Operating System,"Internal product name of the file, provided at compile-time."
1.6.0-dev,true,process,process.pgid,long,extended,,,Identifier of the group of processes the process belongs to.
Expand Down
Loading