Add optional pattern attribute

[ebeahan]

Overview

Introduces a new optional field definition attribute: pattern. The pattern attribute holds a regular expression (regex) which expresses the expected constraint on a string field's value.

pattern: '[.*]'

The ECS tooling checks the regex itself to ensure validity. However, The ECS tooling cannot evaluate the regex against provided field values.

Motivation

Allow automated testing to use the regex found in the pattern fields against values populating ECS fields.

Implementation details

Regex validation

If specified, the regex value from pattern goes through a check using re.compile if the --strict flag is present. The generator will only generate a warning if --strict isn't specified.

Generated artifacts

The pattern field appears in the following generated artifacts:

• generated/beats/fields.ecs.yml
• generated/ecs/ecs_flat.yml
• generated/ecs/ecs_nested.yml

Misc refactoring

I took the opportunity to dedup how --strict checks are handled in schema.cleaner.

Example usage

    - name: mac
      level: core
      type: keyword
      short: MAC address of the source.
      example: 00-00-5E-00-53-23
      pattern: '^[A-F0-9]{2}(-[A-F0-9]{2}){5,}$'
      description: >
        MAC address of the source.

        The notation format from RFC 7042 is suggested: Each octet (that is, 8-bit byte) is
        represented by two [uppercase] hexadecimal digits giving the value of
        the octet as an unsigned integer. Successive octets are separated by a
        hyphen.

Resolves #1741

[ebeahan]

@andrewkroh Does the proposed implementation match up with your expectations?

I added regex validation ECS-side, but I know the support of expressions can vary between different languages/engines/implementations. I'm open to removing the validation step if it's best to remove any sort of regexp validation ECS-side.

[andrewkroh]

LGTM. Thanks for adding this to the schema.

[ebeahan]

💚 All backports created successfully

Status	Branch	Result
✅	8.2

Questions ?

Please refer to the Backport tool documentation