Add code generator for Go

[andrewkroh]

This generates Go code based on the Elastic Common Schema.

I find myself creating structs that match the schema so I thought why not generate some code.

[ruflin]

Very interesting idea. We could do that for other languages too. I assume you plan to import the code directly from ECS so if it gets updated and you go to a new version, you also get the newest fields?

[ruflin]

I really like this. Will make it really easy to look up descriptions during implementations.

[ruflin]

Do you have some plans on how ecs here could be used?

[andrewkroh]

I haven't made use of it yet, but it's there to, at minimum, make it possible to marshal/unmarshal these fields in a programatic manner (if you take the time to read the tag values). I can see a future enhancements to make this better.

[andrewkroh]

@ruflin I updated the PR to include two README files. I also added

• version.go with a const Version
• doc.go to make the godocs better.
• Added a VERSION variable to the Makefile to make updating the version a little easier.

[webmat]

Thanks for sumitting this! I'll look after the Beta 2 release, however ;-)

[ruflin]

We should probably also move to mage on day ;-)

[ph]

@urso I think it would be a good time to show your logger experiment because I believe its kinda linked to this generated code :)

[urso]

Not sure if the format is correct, but If possible we should have this at the top of the file. This way github will also hide the diff by default.

[andrewkroh]

If we move this to the top then the go-licenser is not happy.

code/go/ecs/user_agent.go: is missing the license header

The comment format here follows the format from https://golang.org/s/generatedcode.

[webmat]

The ECS repo contains a whole lot of generated code or markup, and the tooling to generate it is still very basic at this point. So I'd say seeing the diffs for those files is ok. It's actually useful in some cases.

I'm curious if there's another way (perhaps a dotfile?) to instruct GitHub to ignore generated files, though. Could be useful for the files generated by the most solid tooling.

[urso]

It would be nice if we could define a template to set how the output should look like. E.g. in my logging experiments I did come up with:

	log.With(
		ecs.Service.Name("my server"),
		ecs.Hostname("localhost"),
	).Info("start service")

or:

// pass to first subsystem
log = log.With(ecs.Service.Name("my server"))

// second subsystem
log = log.With(ecs.Destination.Port(8080))

// actual go routine handling an http request...

log = log.With(
  	ecs.HTTP.Method("GET"),
	ecs.HTTP.URL("/get_file/file.txt"),
	ecs.Source.Domain("localhost"),
	ecs.Source.IP("127.0.0.1"),
)
...
log.Info("can not read {filename}", "file.txt") // filename becomes an event field {fields.filename: "file.txt"}

One can add a field to an existing namespace and everything is typed + deduplicated when logging. More similar how zap/zerolog and other structured loggers work. The structures I create for this are like:

type serviceNS struct{}

var Service serviceNS

func (serviceNS) Name(name string) Field { return Field{Key: "service.name", Value: name} }

Nesting namespaces goes like this:

type logNS struct {
  File logFileNS
}
type logFileNS struct{}

...

func (logNS) Level(s string) fld.Field        { return ecsString("log.level", s) }
func (logFileNS) Path(s string) fld.Field     { return ecsString("log.file.path", s) }

[webmat]

This is very interesting! Thanks for submitting this.

Are we missing the Base fields, though?

Also two minor things to fix:

• company name in license
• separate the MIT licensed function from the Apache licensed code

[webmat]

The ECS repo contains a whole lot of generated code or markup, and the tooling to generate it is still very basic at this point. So I'd say seeing the diffs for those files is ok. It's actually useful in some cases.

I'm curious if there's another way (perhaps a dotfile?) to instruct GitHub to ignore generated files, though. Could be useful for the files generated by the most solid tooling.

[webmat]

If I understand correctly, this single function is under the MIT license? I think for clarity we should move it to a distinct file, as we have other code immediately below this, whose status now becomes ambiguous.

[webmat]

As discussed elsewhere, the copyright holder should actually be "Elasticsearch B.V.", not "Elastic N.V."

My other two review feedback points still stand, however:

• missing base fields
• separate MIT code from Apache code :-)

[andrewkroh]

@webmat Both points are addressed now. I'm glad you caught that we were missing the base fields.

[webmat]

@andrewkroh Looking good! I wonder why you had device fields, though. What git ref was this based on?

[andrewkroh]

device was there because it existed when I started, but there wasn't anything to delete the generated .go files until now.

[webmat]

Good catch on User_agent