You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The sequential table transformer (#802) is great if later transformations depend on prior ones. Often, however, columns are transformed independently, which gives us opportunity to parallelize.
Desired solution
New class ParallelTableTransformer.
Superclass: InvertibleTableTransformer
Constructor parameters:
transformers: list[TableTransformer].
Raise if multiple transformers are applied to the same column.
fit:
Fit the individual transformers on the given table in parallel. Keep in mind that multithreading is usually not helpful for CPU-bound tasks in Python due to the Global Interpreter lock. Multiprocessing is likely more efficient. Profile your code!
transform:
Transform the table with the individual transformers in parallel (see fit). Gather the results of the transformers and merge them to a single table.
inverse_transform:
Raise an error if any transformer is not invertible
Inverse-transform the table with the individual transformers in parallel (see fit). Gather the results of the transformers and merge them to a single table.
Possible alternatives (optional)
No response
Screenshots (optional)
No response
Additional Context (optional)
No response
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem?
The sequential table transformer (#802) is great if later transformations depend on prior ones. Often, however, columns are transformed independently, which gives us opportunity to parallelize.
Desired solution
New class
ParallelTableTransformer
.InvertibleTableTransformer
transformers: list[TableTransformer]
.fit
:transform
:fit
). Gather the results of the transformers and merge them to a single table.inverse_transform
:fit
). Gather the results of the transformers and merge them to a single table.Possible alternatives (optional)
No response
Screenshots (optional)
No response
Additional Context (optional)
No response
The text was updated successfully, but these errors were encountered: