You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I just realised that the current hook that we use to insert graphchain into dask is delayed_optimize, which only actually applies for "raw" Delayed instances, but doesn't actually apply to e.g. the DataFrame API, which is a very popular part of dask. As described here, these types have their own optimisation hooks that we would need to implement. I will look in to writing one for the DataFrame API. I think we should be able to integrate it fairly easily into the current codebase.
array_optimize
dataframe_optimize
bag_optimize
The text was updated successfully, but these errors were encountered:
Okay so upon investigation, it seems that all the hooks use the same API, so we're all good in that regard, and it was just an unrelated bug that I was hitting.
I just realised that the current hook that we use to insert
graphchain
intodask
isdelayed_optimize
, which only actually applies for "raw"Delayed
instances, but doesn't actually apply to e.g. theDataFrame
API, which is a very popular part ofdask
. As described here, these types have their own optimisation hooks that we would need to implement. I will look in to writing one for theDataFrame
API. I think we should be able to integrate it fairly easily into the current codebase.array_optimize
dataframe_optimize
bag_optimize
The text was updated successfully, but these errors were encountered: