Update T4R for ColumnSchema API changes from Merlin Core

Since `is_list` and `is_ragged` have become derived properties computed from the shape, it's no longer possible to directly set them from the constructor. They can be smuggled in through the properties, after which they'll be used to determine an appropriate shape that results in the same `is_list` and `is_ragged` values on the other side.

(This is a first step toward capturing and using more comprehensive shape information, with the goal of putting `Shape` in place while breaking as little as possible. There will be subsequent changes to directly capture more shape information, but this gets us part-way there.)

Depends on NVIDIA-Merlin/core#195

merlin_standard_lib/utils/misc_utils.py

@@ -265,14 +265,14 @@ def _augment_schema(
     for col in sparse_names or []:
         cs = schema[col]
         properties = cs.properties
+        properties["is_list"] = True
+        properties["is_ragged"] = True
         if sparse_max and col in sparse_max:
             properties["value_count"] = {"max": sparse_max[col]}
         schema[col] = ColumnSchema(
             name=cs.name,
             tags=cs.tags,
             dtype=cs.dtype,
-            is_list=True,
-            is_ragged=not sparse_as_dense,
             properties=properties,
         )
 

transformers4rec/torch/model/base.py

@@ -739,19 +739,15 @@ def input_schema(self):
 
             dtype = {0: np.float32, 2: np.int64, 3: np.float32}[column.type]
             tags = column.tags
-            is_list = column.value_count.max > 0
+            value_counts = {"min": column.value_count.min, "max": column.value_count.max}
             int_domain = {"min": column.int_domain.min, "max": column.int_domain.max}
-            properties = {
-                "int_domain": int_domain,
-            }
+            properties = {"int_domain": int_domain, "value_counts": value_counts}
 
             col_schema = ColumnSchema(
                 name,
                 dtype=dtype,
                 tags=tags,
                 properties=properties,
-                is_list=is_list,
-                is_ragged=False,
             )
             core_schema[name] = col_schema
         return core_schema