Skip to content

Latest commit

 

History

History
324 lines (209 loc) · 8.13 KB

CHANGELOG.md

File metadata and controls

324 lines (209 loc) · 8.13 KB
Raw

Event Query Language - Changelog

The format is based on Keep a Changelog.

Version 0.9.19

Released 2023-10-31

Added

  • Support for missing events feature used in Elasticsearch sequence queries
  • Added IPv6 support for CidrMatch
  • Removed the regex support for testing CidrMatch in favor of the native ipaddress module testing

Version 0.9.18

Released 2023-09-01

Added

  • Support for sample base query type for Elasticsearch queries

Version 0.9.17

Released 2023-08-02

Fixed

  • Support for runs count field on the first subquery
  • Makefile install command

Version 0.9.16

Released 2023-07-27

Added

  • Support for nested field validation within the arraySearch function

Version 0.9.15

Released 2022-10-11

Fixed

  • Bug where as fields overlapped as keyword

Version 0.9.14

Released 2022-09-16

Added

  • Support for sequence alias syntax if the Endpoint flag is enabled

Version 0.9.13

Released 2022-05-25

Changed

  • update lark-parser dependency to ~=0.12.0

Version 0.9.12

Released 2022-03-15

Added

Version 0.9.11

Released 2021-09-22

Added

Version 0.9.10

Released 2021-08-24

Added

  • Endpoint $variable syntax, behind parser flag

Version 0.9.9

Released 2021-02-22

Added

  • Elasticsearch like/ like~ syntax, behind parser flag
  • Elasticsearch regex / regex~ syntax, behind parser flag
  • Elasticsearch insensitive function syntax function~(...), behind parser flag

Version 0.9.8

Released 2021-01-14

Added

  • Optimization to combine adjacent wildcard() calls over the same field

Version 0.9.7

Released 2020-12-01

Added

  • Support for extending attributes within macros

Version 0.9.6

Released 2020-11-24

Changed

  • Elasticsearch insensitive wildcard : syntax to support lists and wildcards, behind parser flag

Version 0.9.5

Released 2020-10-20

Added

  • Toggle to parse with Elasticsearch syntax

Changed

  • Parser rule for method syntax. No whitespace is allowed between the : and opening (

Version 0.9.4

Released 2020-07-27

Fixed

  • Uncaught exception when macros return a field

Version 0.9.3

Released 2020-07-07

Added

  • Internal toggle to turn off case insensitivity
  • Additional tests to handle case sensitive and insensitive modes

Version 0.9.2

Released 2020-05-29

Fixed

  • Removed invalid index into empty array for UniqueCountPipe.output_schemas with custom walkers

Version 0.9.1

Released 2020-05-28

Removed

  • case_sensitive parameter to between

Version 0.9.0

Released 2020-05-18

Added

  • Support for escaped identifiers in fields using `backtick` syntax
  • Implemented three-value logic for boolean and null handling
  • Migrate python tests to TOML files
  • Optimization rule for inverting comparisons (not (x <= y) ==> x > y)
  • (Internal) AST nodes for IsNull/IsNotNull

Changed

  • (Internal) Changed the type system to use TypeHint/TypeFoldCheck/NodeInfo instead of tuples
  • Treat wildcards the same on the left and right side of == or !=
  • Removed forced float division (now 3 / 2 ==> 1 instead of 1.5)
  • Made <, <=, >=, > perform case-insensitive string comparisons

Fixed

  • Boolean optimizations: x and true and y is correctly converted to x and y
  • DeMorgan logic for expanding not nodes

Removed

  • Deprecated sequence units besides d,h,m,s,ms
  • Fractional sequence intervals
  • Requirements.txt and moved into setup.py

Version 0.8.8

Released 2020-04-24

Changed

  • Regular expressions generated by CidrMatch create non-capturing groups

Version 0.8.7

Released 2020-04-23

Added

  • Function eql.utils.get_query_type
  • Function eql.utils.get_required_event_types
  • Function eql.utils.uses_ancestry

Version 0.8.6

Released 2020-04-06

Changed

  • Made missing where raise a EqlSyntaxError instead of a EqlSemanticError

Version 0.8.5

Released 2020-03-18

Fixed

  • Walker auto-recursion on BaseNode

Version 0.8.4

Released 2020-03-18

Added

  • skip_optimizations toggle to disable optimizations when parsing

Changed

  • Moved optimization methods to Optimizer class.

Version 0.8.3

Released 2020-03-11

Fixed

  • CidrMatch.run signature for cidrMatch optimization

Version 0.8.2

Released 2020-01-13

Fixed

  • Restored missing text from semantic error messages

Version 0.8.1

Released 2020-01-09

Fixed

  • Correctly load definitions/schema with eql.build.get_engine

Version 0.8

Released 2019-11-01

Added

  • Method syntax
  • Mathematical operators +, -, *, /, %
  • Documentation for match() function
  • between() function for extracting the first substring between two strings
  • cidrMatch() function and several helper methods for subnet matching and regex building
  • extract_query_terms to extract the original text for each event in a query

Changed

  • Parser from TatSu to Lark

Fixed

  • Examples for sequences in the Implementation Details page
  • Compatibility for eql shell with Python 2.7

Version 0.7

Released 2019-07-24

Added

  • Badges to README (PyPi, Twitter, ReadTheDocs, Gitter)
  • Multiple values within arrayContains() function
  • arrayCount function for counting the number of matches in an array
  • Interactive shell with tables
  • Validation system that matches types across schema and comparisons
  • SignatureMixin class used to validate input and output arguments to functions, and pipes
  • Better error messages with multiple carets
  • Base class for all EQL errors
  • Test data, queries, and the expected output for unit tests
  • Signature base class
  • Helper Walker classes with better methods and context for AST traversal
  • ParserConfig class with context manager that toggles thread-specific parser settings
  • Additional imports to the root eql module
  • Autogenerated parser with make parser included in python module
  • is_stateful function to eql.utils to determine if a parsed query is stateful
  • match_kv function to eql.utils to autogenerate an AST from a dictionary of fields -> value(s)

Changed

  • Rearranged imports, eql.engines.native is now eql.engine
  • Using walker methods instead of NodeMethods to integrate with engine
  • Moved function call evaluation to custom classes in eql.functions
  • Moved highlighter from sphinx document generation to eql.highlighters
  • Moved PipeCommand subclasses to eql.pipes
  • Exception class names to all have Eql prefix
  • Moved unit tests to eql.tests module and rearranged test
  • Implementation of by for sequences, joins and pipes to perform case-insensitive checks

Fixed

  • Bug where wildcards were only matched on the first line of text in a field

Removed

  • Default EQL schema. Now accepts all input and event types by default

Version 0.6.3

Released 2019-04-17

Added

  • @itsnotapt Made pid and ppid fields configurable

Version 0.6.2

Released 2018-12-13

Fixed

  • Broken implementation of streaming .jsonl files

Version 0.6.1

Released 2019-12-05

Added

  • Support for gzipped files

Version 0.6

Initial Release 2018-11_30

Added (Initial Features)

  • EQL parser with Tatsu
  • Evaluation engine in eql.engines.native
  • Macro and constants for preprocessors
  • eql CLI command for querying against a JSON file
  • Security schema by default (file, network, process, etc.)