added rule fields #665

dainperkins · 2019-12-02T17:43:36Z

Added rule fields for agent / observer rule based events/actions

Closes #598

Added rule fields for agent / observer rule based events/actions

added PR id

MikePaquette

Where do we add text about field re-use?

schemas/rule.yml

dainperkins · 2019-12-02T21:50:13Z

also added "a link to the vendors documentation regarding the rule" to the reference field (e.g. https://www.snort.org/rule_docs/1-50032)

Should we have multiple examples?

dainperkins · 2019-12-02T21:59:05Z

Where do we add text about field re-use?

unsure @webmat?

Also @MikePaquette & @webmat

should we provide some rule.type examples?
- acl (base ip / port decision)
- url (e.g. blocked url)
- protocol (e.g. malformed or tunneled protocol)
- intrusion (detection/prevention)

updated descriptions per mikes requests, added rule.reference text to to include vendor references

webmat

Thanks @dainperkins!

Here's my preliminary review. A lot of small nitpicks to clean things up, but also a few important questions.

Please run make on subsequent pushes, I'd like to see this in the docs preview as well :-)

schemas/rule.yml

webmat · 2019-12-03T16:20:22Z

schemas/rule.yml

+
+      example: DNS
+
+    - name: ruleset


What's the relationship between ruleset and type? Is there a hierarchy there?

Ruleset is a logical grouping of rules (e.g. a collection of individual rules that are applied to a group for e.g. url filtering or nac policies)

Type is what kind of action does the rule concern itself with, I'm not sure type is necessary across the board.

per ecs meeting - I'm thinking rule.type can probably be dropped for now.... leaving just ruleset which I can see still being useful for easily visualizing alerts / events related to a group of rules

Ok, I'm good with dropping .type for now.

Just to be sure we're on the same page, .ruleset is meant to capture a value according to the source of these events?

For example, using this to map Suricata alerts like this one, we would have this?

rule.category="Attempted Information Leak" (from suricata.eve.alert.category)

Dropped, tho now I am reading the above and thinking maybe we need discrete rulesets (groups of rules/signatures that may apply to e.g. specific networks - e.g. GUEST NET IPS SIGNATURE AND rule.type or rule.category - e.g. "Attempted Information Leak" as they are definitely different things logistically

I think category is more descriptive so I will add that in.

schemas/vlan.yml

removed vlan pr

webmat · 2019-12-03T18:45:14Z

Where do we add text about field re-use?

@MikePaquette What do you mean by that? We agreed that rule would be at the top level, right? Nesting it under agent + observer would mean we can't analyze both sets of rules together.

description and error updates per PR discussion

webmat

Here's another round of comments.

Please don't forget to address the smaller diffs I proposed in my previous review.

schemas/rule.yml

webmat · 2019-12-04T21:02:22Z

schemas/rule.yml

+
+      example: DNS
+
+    - name: ruleset


Ok, I'm good with dropping .type for now.

Just to be sure we're on the same page, .ruleset is meant to capture a value according to the source of these events?

For example, using this to map Suricata alerts like this one, we would have this?

rule.category="Attempted Information Leak" (from suricata.eve.alert.category)

schemas/rule.yml

fixed spelling, cap., and gram errors. Changes rule.type to be rule.category

webmat

Pretty much ready to merge. I'd love to confirm one thing before, and then I think we're good.

What is the relationship between ruleset and category? Is there a hierarchy there? If so which one is higher? Are both vendor-supplied, or do users do these groupings themselves?

webmat · 2019-12-11T13:58:14Z

schemas/rule.yml

+  group: 2
+  short: Fields to capture details about rules used to generate alerts or other events.
+  description: >
+   Rule fields are used to capture the specifics of any observer or agent rules that generate alerts or other events.


Nitpick: "or other notable events"?

I'm good with that, will update today

on the other question - rule.category is, typically, the category of action the rule is alerting on (e.g. "Web Attacks" rulesets would include xss, directory traversal, etc) and you could then apply the "WEB ATTACKS" rule wherever it needed to be applied, instead of enabling each individual rule.

ruleset is a logical configuration which (often times across multiple managed implementations of a solution - e.g. Panorama managing multiple Palo Altos) combines rules into a single logical set (e.g. on the GUEST NETWORK apply the GUEST rule.ruleset which includes [blocks DNS-over_TLS, blocks nsfw/guns/and competitors web sites, limits bandwidth, etc.)

So ruleset is a wider grouping than category? Or are these orthogonal?

orthogonal...

Ruleset is localized configuration/implementation specific (as in apply ruleset MY_RULES to this subnet / vlan / group) where MY_RULES have been defined to include e.g. 10 discrete rules.

Category is a grouping of rules by type (think Mitre technique or vulnerable system type)

schemas/rule.yml

updated rule short & long descriptions

schemas/rule.yml

webmat

LGTM

I'll likely merge this tomorrow. Leaving a little more time for anyone else to jump in with a final look.

Thanks Dain!

andrewstucki · 2019-12-11T20:27:06Z

schemas/rule.yml

+      type: keyword
+      short: Rule description
+      description: The description of the rule generating the event.
+      example: Block requests to public DNS over HTTPS / TLS protocols


I'm assuming the plan was to do a subsequent PR with the corresponding #680 changes that likely ought to be added?

Yeah the default text indexer would be useful here, but we should do another round later.

dainperkins added 2 commits December 2, 2019 12:39

added rule fields

27c7527

Added rule fields for agent / observer rule based events/actions

added PR ID for tracking

f3c656f

added PR id

dainperkins mentioned this pull request Dec 2, 2019

Add Rule Fields for Observers and Agents #598

Closed

7 tasks

MikePaquette reviewed Dec 2, 2019

View reviewed changes

updates from discussion

cbecc35

updated descriptions per mikes requests, added rule.reference text to to include vendor references

webmat reviewed Dec 3, 2019

View reviewed changes

removed vlan pr

61a13c7

removed vlan pr

dainperkins added 2 commits December 3, 2019 16:17

updated descriptions & errors

10ff7e0

description and error updates per PR discussion

make completed per PR discussion

23a07eb

webmat reviewed Dec 4, 2019

View reviewed changes

dainperkins added 2 commits December 9, 2019 15:18

Update rule.yml

3f786f0

Fixed errors & added .category

3b97423

fixed spelling, cap., and gram errors. Changes rule.type to be rule.category

webmat reviewed Dec 11, 2019

View reviewed changes

dainperkins added 2 commits December 11, 2019 13:25

updated rule short & description

42bcb63

updated descriptions

9ebd69a

updated rule short & long descriptions

webmat reviewed Dec 11, 2019

View reviewed changes

schemas/rule.yml Outdated Show resolved Hide resolved

fixed spelling in rule description

aaf0be6

webmat approved these changes Dec 11, 2019

View reviewed changes

andrewstucki reviewed Dec 11, 2019

View reviewed changes

Mathieu Martin added 2 commits December 12, 2019 12:26

Merge branch 'master' into rules

4c44e76

Merge branch 'master' into rules

1cec6bd

webmat merged commit c29bff4 into elastic:master Dec 12, 2019

dcode pushed a commit to dcode/ecs that referenced this pull request Apr 15, 2020

Added rule fields (elastic#665)

c01aefb

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

added rule fields #665

added rule fields #665

dainperkins commented Dec 2, 2019 •

edited by webmat

Loading

MikePaquette left a comment

dainperkins commented Dec 2, 2019

dainperkins commented Dec 2, 2019

webmat left a comment

webmat Dec 3, 2019

dainperkins Dec 3, 2019 •

edited

Loading

dainperkins Dec 4, 2019 •

edited

Loading

webmat Dec 4, 2019

dainperkins Dec 9, 2019

webmat commented Dec 3, 2019

webmat left a comment

webmat Dec 4, 2019

webmat left a comment

webmat Dec 11, 2019

dainperkins Dec 11, 2019

webmat Dec 11, 2019

dainperkins Dec 11, 2019 •

edited

Loading

webmat left a comment

andrewstucki Dec 11, 2019

webmat Dec 11, 2019

added rule fields #665

added rule fields #665

Conversation

dainperkins commented Dec 2, 2019 • edited by webmat Loading

MikePaquette left a comment

Choose a reason for hiding this comment

dainperkins commented Dec 2, 2019

dainperkins commented Dec 2, 2019

webmat left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dainperkins Dec 3, 2019 • edited Loading

Choose a reason for hiding this comment

dainperkins Dec 4, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

webmat commented Dec 3, 2019

webmat left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

webmat left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dainperkins Dec 11, 2019 • edited Loading

Choose a reason for hiding this comment

webmat left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dainperkins commented Dec 2, 2019 •

edited by webmat

Loading

dainperkins Dec 3, 2019 •

edited

Loading

dainperkins Dec 4, 2019 •

edited

Loading

dainperkins Dec 11, 2019 •

edited

Loading