Clarify the semantics of network.direction

[webmat]

I think when these semantics are followed, it will be possible to reliably
do inbound/outbound traffic analysis for both host level and perimeter-level
network activity.

Since a given solution typically does one of the two kinds of monitoring,
analysts should not mix both kinds of monitoring anyway, when doing host-based
analysis vs perimeter-based.

The clarification I'm introducing makes me realise we should likely add another
expected value: "internal". So I've added it as a separate commit. Let me know
if you think this part should be reverted.

[andrewkroh]

The descriptions LGTM.

I'm a little hesitant to add another enum value. Prior to internal I think would have have used unknown for something like traffic on a loopback device. I'm wondering what the implications of the new state will be on analysis tools. Is there some new analysis that can be enabled by this new state? If not then I'd probably say that the three existing states are sufficient.

[webmat]

Is it common for network monitoring tools to say the direction is "unknown" when both sides are private / non-routable addresses, though? I feel like we're losing information that's readily available.

I do think the presence of "internal" traffic is important for some kinds of analysis. For example a database server should have 100% internal traffic. If you see some inbound or outbound traffic, you would want to investigate right away.

I agree that the value for a host-based monitoring solution to tag local traffic as "internal" probably has little use, but do these tools even report local network activity?

[MikePaquette]

I am +1 for this change.

Suggest that we have five recommended values: external internal inbound outbound unknown
because some monitoring systems will provide us with independent indicators whether the source/dest or client/server was local, so we have four possible states if they're populated, and the fifth state in case they're not.

For example, when looking at the Zeek conn.log, we could apply logic as follows:

If Site::local_nets does not exist - network.direction: "unknown"
else {local_orig, local_resp}:
00 - network.direction: "external"
01 - network.direction: "inbound"
10 - network.direction: "outbound"
11 - network.direction: "internal"

We could add a clarifying note to the definition that says: If the network event is "remote," use "external" as the value of this field. If the network event is "local," use "internal" in this field.

[ruflin]

I leave the 👍 or 👎 to @MikePaquette / @robgil / @andrewkroh

[robgil]

I don't really like any of these fields. Specifically because they are very limited in their description and use. inbound, outbound, internal, and external don't give me very much context and seem very ambiguous.

Some samples of where ambiguity rears its head.

internal / external

Internal is very loosely defined. On a network you may have multiple customers such as subsidiaries, remote offices, and so on. Are these internal? No, not really. Most folks would consider them untrusted or define them as a security zone in a firewall.

External is the same thing. Does it mean internet? If so, how do you classify extranets or direct peering (cross connects)?

inbound / outbound

I've also always felt these were ambiguous without the to and from taking in to account. Is inbound always from the internet? or would this apply to extranet providers? The ambiguity here is the fact that network boundaries are not clear.

Proposal

This proposes a revamp of the naming to account for clearer network boundaries when describing direction, internal/external, and inbound/outbound.

For example, we could do something like this.

source.network: internet
destination.network: database

or

source.network: dmz
destination.network: database

This gives the flexibility to define security zones by naming the networks. If a firewall loosely defines something as inbound, they are likely referring to the traffic being inbound in the context of the firewall - not specifically the network itself. DMZs are good examples where there are multiple layers of firewalls (edge and internal for example).

We can further abstract it in to the concept of zones.

source.zone: untrusted
destination.zone: database

or

source.zone: dmz
destination.zone: database

Analyst Impact

With the above proposal, the analyst can easily see if security boundaries have been violated.

If logs come in with something like the following and are unexpected, its a clear red-flag.

source.zone: untrusted
destination.zone: database

Assume the database should not be accessible from untrusted networks

[webmat]

Actually I agree with @MikePaquette that if we're to add "internal" to the list, we should also add "external", bringing the total list to 5:

• unknown
• inbound
• outbound
• internal
• external

I really like @robgil's proposal, and I agree that a single field named network.direction is a simplistic approach. However it does have merit as is, since it's easy for tooling to populate, and it will be possible to leverage it nonetheless. We'll continue exploring Rob's idea in a separate issue, and not for ECS 1.0. Properly implementing this idea would require guidance on the expected values in these fields, to be able to leverage them properly.

[MikePaquette]

LGTM

[robgil]

LGTM for beta/1.0

[webmat]

@andrewkroh Instead of removing "internal", we added "external". Are you good with it anyway?

Are you seeing a problem with the semantics here?

[andrewkroh]

I'm good with the additional enum values.