You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Databend generates numerous temporary files when spilling. Currently, we use list_objects to retrieve the complete list and then call delete_objects, which involves many costly list_objects calls.
Therefore, I propose introducing a spill index file to record all temporary files created. This would allow us to remove them by reading this file's contents instead of using list_objects.
The text was updated successfully, but these errors were encountered:
Summary
Databend generates numerous temporary files when spilling. Currently, we use
list_objects
to retrieve the complete list and then calldelete_objects
, which involves many costlylist_objects
calls.Therefore, I propose introducing a spill index file to record all temporary files created. This would allow us to remove them by reading this file's contents instead of using
list_objects
.The text was updated successfully, but these errors were encountered: