Ingest ZNG-format Zeek logs

[philrz]

In the original vision for opening Zeek logs as captured in #566, it was expected that ZNG would be among the accepted formats. At the moment (Brim commit f837314 talking to zqd tagged v0.12.0) attempting to load such logs into Brim gives the "Unknown file types" error, so it seems we still have work left to do here.

Like we did for Zeek TSV/JSON, we could provide a hyperlink from any ZNG reference, in this case to https://github.com/brimsec/zq/tree/master/zng/docs.

[henridf]

It sounds like this issue would require extending the Brim app's autodetection to recognize zng formatted logs? If so an alternate option might be to add a "output type" flag to zq (that just runs zq's detection mechanism and writes the detected format to stdout) and have Brim run zq with that flag that way to detect the format of input files. The zq work for this would be super easy, and this way Brim wouldn't have to re-implement another detection scheme. Just a 2c idea.

[alfred-landrum]

@henridf : thank you, our plan from discussions today is to make this very simple: Brim will only try to identify pcaps. If it doesn't identify a file as a pcap, it will assume it's some type of log, and send it via the log ingest api (including the json typings), and let zqd figure out what format it is.

[jameskerr]

@mason-fish The functionality of this has been fixed in this PR: #716 Let's just make this ticket about the UI (add zng to the list of supported formats with a link to the zng docs.)

[philrz]

Verified in Brim commit 7e0f417 talking to zqd commit 43d474d. In the video below, I drag the ZNG and TZNG logs from the zq-sample-data into separate tabs of the Brim app and they both import fine.

Verify.mp4.zip

Thanks @mason-fish!