Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allowing beats output files to be generated with --include and --subset #814

Merged
merged 2 commits into from
Apr 15, 2020

Conversation

jonathan-buttner
Copy link
Contributor

The beats output files are used to create the fields.yml mapping file for the endpoint package's mapping located here: https://github.com/elastic/package-registry/tree/master/dev/packages/example/endpoint-1.0.0/dataset/events/fields

The endpoint team needs the ability to generate the beats output while using the --include and --subset flags because we specify custom fields for our alerts and also leverage the subset functionality to limit the fields from ecs core.

The typical command I use to generate the fields.yml file is:
python scripts/generator.py --out ../gen --include ../endpoint-app-team/custom_schemas --subset ../endpoint-app-team/custom_subsets/elastic_endpoint/events/* ../endpoint-app-team/custom_subsets/*.yml

Copy link
Contributor

@marshallmain marshallmain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Worth noting that the beats generator is hard coded to look for the base fields so if they're not in the subset it will crash in the beats.generate() call.

Copy link
Contributor

@webmat webmat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jonathan-buttner
Copy link
Contributor Author

Worth noting that the beats generator is hard coded to look for the base fields so if they're not in the subset it will crash in the beats.generate() call.

Just so I remember in the future, I think the reason that the beats generator succeeds is because all of the endpoint events use at least the @timestamp field from base.

@jonathan-buttner jonathan-buttner merged commit 9496824 into elastic:master Apr 15, 2020
@jonathan-buttner jonathan-buttner deleted the allow-beats-output branch April 15, 2020 15:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants