diff --git a/grammar/FuncTestCaseLexer.g4 b/grammar/FuncTestCaseLexer.g4 index 30a94b4dd..055b82d23 100644 --- a/grammar/FuncTestCaseLexer.g4 +++ b/grammar/FuncTestCaseLexer.g4 @@ -97,11 +97,9 @@ IntervalDayLiteral ; fragment TimeInterval - : IntegerLiteral HourSuffix (IntegerLiteral MSuffix)? (IntegerLiteral SecondSuffix)? - (IntegerLiteral FractionalSecondSuffix)? - | IntegerLiteral MSuffix (IntegerLiteral SecondSuffix)? (IntegerLiteral FractionalSecondSuffix)? - | IntegerLiteral SecondSuffix (IntegerLiteral FractionalSecondSuffix)? - | IntegerLiteral FractionalSecondSuffix + : IntegerLiteral HourSuffix (IntegerLiteral MSuffix)? (DecimalLiteral SecondSuffix)? + | IntegerLiteral MSuffix (DecimalLiteral SecondSuffix)? + | DecimalLiteral SecondSuffix ; NullLiteral: 'null'; diff --git a/tests/README.md b/tests/README.md new file mode 100644 index 000000000..2223fc38d --- /dev/null +++ b/tests/README.md @@ -0,0 +1,210 @@ +# Substrait Test Format + +This document describes the format for Substrait scalar test files. +A test file consists of the following elements: + +1. Version declaration +2. Optional include statements +3. One or more test groups, each containing one or more test cases + +## Syntax + +### Version Declaration +The version declaration must appear as the first line of the file. It defines the type of tests in the file and test file format version. +Each test file must exclusively contain either scalar tests or aggregate tests. Mixing test types in the same file is not allowed. +The version declaration should follow this format: +```code +### SUBSTRAIT_SCALAR_TEST: V1 +``` +or + +```code +### SUBSTRAIT_AGGREGATE_TEST: V1 +``` + +### Include Statements +Include statements should have at least one include statement. The include statement specifies the path to substrait extension functions. The include statement must be in the following format: +```code +### SUBSTRAIT_INCLUDE: /extensions/functions_aggregate_approx.yaml +``` + +### Test Groups +A test group is a collection of test cases that are logically related. Test groups are purely for categorization purposes and do not affect the execution or meaning of tests. +- **description**: A string describing the test group or case. The description must start with a `#` character. + ```code + # Common Maths + ``` +### Scalar Test Cases +A test case consists of the following elements: + +- **function**: The name of the function being tested. The function name must be an identifier alphanumeric string. +- **arguments**: Comma-separated list of arguments to the function. The arguments must be literals. +- **options**: Optional comma-separated list of options in `key:value` format. The options describe the behavior of the function. The test should be run only on dialects that support the options. If options are not specified, the test should be run for all permutations of the options. +- **result**: The expected result of the function. Either `SUBSTRAIT_ERROR` or a literal value. +- **literal**: In the format `::` +- **description**: A string describing the test case + + ```code + add(126::i8, 1::i8) = 127::i8 # addition of two numbers + ``` + +### Aggregate Test Cases +A test case consists of the following elements: + +- **table definition**: +- **function**: The name of the function being tested. The function name must be an identifier alphanumeric string. +- **arguments**: Comma-separated list of arguments to the function. The arguments can be literals or column references. +- **options**: Optional comma-separated list of options in `key:value` format. The options describe the behavior of the function. The test should be run only on dialects that support the options. If options are not specified, the test should be run for all permutations of the options. +- **result**: The expected result of the function. Either `SUBSTRAIT_ERROR` or a literal value. + +Aggregate test cases support 3 formats: +1. **Single Argument**: The test case for an aggregate function with single argument as a column in a table. The table is defined in the test case. + ```code + sum((1, 2, 3, 4, 5)::i8) = 15::i8 # addition of two numbers + ``` +2. **Multiple Columns Compact**: The test case for an aggregate function with on one or more columns of a table as argument. The table is defined before the function name, in the same line as the testcase. + ```code + ((20, 20), (-3, -3), (1, 1), (10,10), (5,5)) corr(col0::fp32, col1::fp32) = 1::fp64 + ``` +3. **Multiple Columns Verbose**: The test case for an aggregate function with one or more columns of a table as arguments. The table is defined before the function name. The function arguments refer to the columns in a table. + ```code + DEFINE t1(fp32, fp32) = ((20, 20), (-3, -3), (1, 1), (10,10), (5,5)) + corr(t1.col0, t1.col1) = 1::fp64 + ``` + +#### Example: +A testcase with mixed arguments +```code + ((20), (-3), (1), (10)) LIST_AGG(col0::fp32, ','::string) = 1::fp64 +``` + +### Spec + +``` +doc := + ()+ + (()?()+\n)+ +version := ### SUBSTRAIT_SCALAR_TEST: +include := ### SUBSTRAIT_INCLUDE: +test_group := # +test_case := () ([])? = (#)? +description := string +function := string +arguments := , , ... +argument := +literal := :: +result := | +options :=