Add visitors to TESTed-AST to write better code

[BrentBlanckaert]

Each language implements their own generator.py. Almost all of them have functions that look like the following:

def convert_statement(statement: Statement, internal=False, full=False) -> str:
    if isinstance(statement, Identifier):
        return statement
    elif isinstance(statement, FunctionCall):
        return convert_function_call(statement, internal)
    elif isinstance(statement, Value):
        return convert_value(statement)
    elif isinstance(statement, PropertyAssignment):
        return (
            f"{convert_statement(statement.property, True)} = "
            f"{convert_statement(statement.expression, True)}"
        )
    elif isinstance(statement, VariableAssignment):
        if full:
            prefix = "let "
        else:
            prefix = ""
        return (
            f"{prefix}{statement.variable} = "
            f"{convert_statement(statement.expression, True)}"
        )
    raise AssertionError(f"Unknown statement: {statement!r}")

To write this better a visitor could be used. This way all the if-statements will be removed.

[BrentBlanckaert]

I have found that adding visitors to TESTed is significantly more complex than initially anticipated. The main issue lies in the way Union types are defined, for example:

Value = Union[
    NumberType, StringType, BooleanType, SequenceType, ObjectType, NothingType
]

Expression = Identifier | Value | FunctionCall

Assignment = VariableAssignment | PropertyAssignment

Statement = Assignment | Expression

All other types in the TESTed-AST are classes (like assignments or NumberType). To make the accept method work for Statement objects, these Union types would need to be converted into classes.

The biggest challenge, however, arises when the output of a student evaluation is parsed and needs to be converted into a Value object. In parsing.py, three relevant functions are defined:

• parse_json_value: the original function called to process output;
• initialise_converter: a subfunction that initializes a converter for Union types;
• structure_every_union: the implementation of that converter.

Example:
Suppose a student's implementation produces the following (TESTed-serialized) output:

{
    "data": [
        {"data": "Hiccup", "type": "text", "diagnostic": null},
        {"data": "Toothless", "type": "text", "diagnostic": null},
        {"data": "Stoick", "type": "text", "diagnostic": null},
        {"data": "Astrid", "type": "text", "diagnostic": null},
        {"data": "Gobber", "type": "text", "diagnostic": null}
    ],
    "type": "list",
    "diagnostic": null
}

This serialized output is passed as the to_convert parameter in the structure_every_union function. The corresponding type in this case is:

typing.Union[
    tested.serialisation.NumberType,
    tested.serialisation.StringType,
    tested.serialisation.BooleanType,
    tested.serialisation.SequenceType,
    tested.serialisation.ObjectType,
    tested.serialisation.NothingType
]

For my implementation, this type would need to match the Value class. Therefore, we need to find a way to make the parsing and conversion logic work within the class structure. This requires further investigation into the workings of cattrs.