Restrict event references in windowed select queries and add 'emit_empty_windows` setting

[mfelsche]

Pull request

Description

• Forbid references to event or $ in the select target expression (select --> event <-- from ...) if this is a select query with a window, because we cannot access a single event in the select target, in that context we operate on the whole window. A freestanding event reference has always been evaluating to the last event of the window, but we have aggr::win::last(...) for that, and since 0.10 it is possible to have windows without an event (for time based windows), where query execution would error out.
  • We allow those references inside aggregate functions and if they are used in the group by clause.
  • Example:

select {
  "ok": event.foo, // allowed because used in group by
  "ok2": aggr::win::first(event.timestamp), // allowed because in aggregation function
  "bad": event.snot + 2 // not allowed anymore
} from in[my_window]
group by set(event.foo, event.bar)
into out;

• Add the setting emit_empty_windows for windows, which is false if not explicitly set to true. If set to true, time based tumbling windows will emit a window event on every interval if they received an event or not. Some aggregations might return null in this case.

This fixes some unintended edge cases in the window handling changes introduced in 0.10.

Related

• Related Issues: fixes Clarify and codify access event in windows and groupings #823, closed Degradation of handling signals in select statements #799
• Related docs PR

Checklist

• The RFC, if required, has been submitted and approved
• Any user-facing impact of the changes is reflected in docs.tremor.rs
• The code is tested
• Use of unsafe code is reasoned about in a comment
• Update CHANGELOG.md appropriately, recording any changes, bug fixes, or other observable changes in behavior
• The performance impact of the change is measured (see below)

Performance

[Licenser]

Nice improvement :) I left a few inline comments

[coveralls]

Coverage increased (+0.04%) to 81.657% when pulling 065aaf5 on window-select-target into 7b72766 on main.

[Licenser]

we should emit a warning here if emit_empty_windows is set and max_groups is not set

[Licenser]

I love the ast walker :D that's going to be so handy over time! I left a few comments inline most just questions otherwise 👍

[Licenser]

Woooh! Nice job :D