Deserializing lists of dictionary / categorical from parquet

[thobai]

I'm using polars (version 0.13.49 on Python), which is making use of arrow2. I can write columns with lists of type Categorical to parquet, but I cannot read them back. It seems this is not yet implemented on arrow2. Polars fails with the error message: exceptions.ArrowErrorException: NotYetImplemented("Deserializing type Dictionary(UInt32, LargeUtf8, false) from parquet")

The following code reproduces the problem:

import polars as pl
from polars import col
print(pl.version())

df = pl.DataFrame(
    {
        'str': ['A', 'B', 'A', 'B', 'C'],
        'group': [1,1,2,1,2]
    }
).lazy()
df = df.with_column(col('str').cast(pl.Categorical))

df_groups = df.groupby('group').agg([col('str').list().alias('str_list')])
df_groups.collect().write_parquet('test.parquet')
df_groups_import = pl.scan_parquet('test.parquet').collect()

Is there any plan to support this soon?

[jorgecarleitao]

I am working on this :)