Support for nested python dictionaries for execute()

[vmirz]

I'm experimenting with the execute() command as a light transformation layer over a small collection of python dictionaries.

I'm looking to query nested python dictionaries using execute(), but I haven't been successful with it. Are nested dicts supported in execute()? Does an explicit schema need to be provided?

In BigQuery (as an example), nested structures are queryable via dot . notation, but it appears sqlglot is not able to process the equivalent against python dictionaries.

from sqlglot.executor import execute

tables = {
    "ITEM": [
        {"id": 1, "attributes": { "flavor": "cherry", "taste": "sweet" }},
        {"id": 2, "attributes": { "flavor": "lime", "taste": "sour" }},
        {"id": 3, "attributes": { "flavor": "apple", "taste": None }},
    ]
}

result = execute(
    """
    SELECT i.attributes.flavor FROM `ITEM` i
    """,
    read="bigquery",
    tables=tables
)
print(result)

Traceback (most recent call last):
  File "/home/runner/SQLGlot/.pythonlibs/lib/python3.10/site-packages/sqlglot/executor/python.py", line 40, in execute
    contexts[node] = self.scan(node, context)
  File "/home/runner/SQLGlot/.pythonlibs/lib/python3.10/site-packages/sqlglot/executor/python.py", line 109, in scan
    return self.context({step.name: self._project_and_filter(context, step, table_iter)})
  File "/home/runner/SQLGlot/.pythonlibs/lib/python3.10/site-packages/sqlglot/executor/python.py", line 124, in _project_and_filter
    sink.append(context.eval_tuple(projections))
  File "/home/runner/SQLGlot/.pythonlibs/lib/python3.10/site-packages/sqlglot/executor/context.py", line 38, in eval_tuple
    return tuple(self.eval(code) for code in codes)
  File "/home/runner/SQLGlot/.pythonlibs/lib/python3.10/site-packages/sqlglot/executor/context.py", line 38, in <genexpr>
    return tuple(self.eval(code) for code in codes)
  File "/home/runner/SQLGlot/.pythonlibs/lib/python3.10/site-packages/sqlglot/executor/context.py", line 35, in eval
    return eval(code, self.env)
  File "DOT(scope["i"]["attributes"], "flavor")", line 1, in <module>
NameError: name 'DOT' is not defined

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "main.py", line 11, in <module>
    result = execute(
  File "/home/runner/SQLGlot/.pythonlibs/lib/python3.10/site-packages/sqlglot/executor/__init__.py", line 82, in execute
    result = PythonExecutor(tables=tables_).execute(plan)
  File "/home/runner/SQLGlot/.pythonlibs/lib/python3.10/site-packages/sqlglot/executor/python.py", line 63, in execute
    raise ExecuteError(f"Step '{node.id}' failed: {e}") from e
sqlglot.errors.ExecuteError: Step 'Scan: i (140595004274624)' failed: name 'DOT' is not defined
 

[tobymao]

check out the commit, feel free to make PRs in the future for any of these currently unsupported features