You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
At present, the columnar data format creates columns for every field and property in the type. For instance, a struct with fields a, b, and c will yield batches that have arrays for a, b, and c as well. However, if no downstream operators require access to field b, it makes no sense to allocate any arrays for it let alone populate them. This change need not actually change the type of the generated columnar format, just that the arrays for unneeded fields don't get allocated or populated.
The text was updated successfully, but these errors were encountered:
At present, the columnar data format creates columns for every field and property in the type. For instance, a struct with fields a, b, and c will yield batches that have arrays for a, b, and c as well. However, if no downstream operators require access to field b, it makes no sense to allocate any arrays for it let alone populate them. This change need not actually change the type of the generated columnar format, just that the arrays for unneeded fields don't get allocated or populated.
The text was updated successfully, but these errors were encountered: