Online DDL: cleanup_timestamp is not null, but artifact table still exists

[shlomi-noach]

24 hours after a migration completes/failed, Vitess is expected to delete the artifact tables. In the following scenario:

*************************** 1. row ***************************
                 id: 198
     migration_uuid: 9f61393e_e55e_11eb_94fa_be29b0c9c688
           keyspace: redacted
              shard: -
       mysql_schema: redacted
        mysql_table: redacted
migration_statement: alter table redacted add column ready_at datetime(6) default null
           strategy: gh-ost
            options: 
    added_timestamp: 2021-07-15 11:20:12
requested_timestamp: 0000-00-00 00:00:00
    ready_timestamp: 2021-07-15 11:20:23
  started_timestamp: 2021-07-15 11:20:25
 liveness_timestamp: 2021-07-15 11:20:25
completed_timestamp: 2021-07-15 11:20:34
  cleanup_timestamp: 2021-07-16 11:20:38
   migration_status: complete
           log_path: redacted:/tmp/online-ddl-9f61393e_e55e_11eb_94fa_be29b0c9c688-621330636
          artifacts: _9f61393e_e55e_11eb_94fa_be29b0c9c688_20210715112025_gho,_9f61393e_e55e_11eb_94fa_be29b0c9c688_20210715112025_ghc,_9f61393e_e55e_11eb_94fa_be29b0c9c688_20210715112025_del,_9f61393e_e55e_11eb_94fa_be29b0c9c688_20210715112024_gho,_9f61393e_e55e_11eb_94fa_be29b0c9c688_20210715112024_ghc,_9f61393e_e55e_11eb_94fa_be29b0c9c688_20210715112024_del,
            retries: 0
             tablet: redacted
     tablet_failure: 0
           progress: 100
  migration_context: redacted:d947fc14-a971-4047-975c-fa7626626a58
         ddl_action: alter
            message: executed gh-ost --execute=true, err=<nil>
        eta_seconds: 0
        rows_copied: 75288
         table_rows: 0

The table _9f61393e_e55e_11eb_94fa_be29b0c9c688_20210715112025_del was found to still exist even after cleanup. Note that cleanup_timestamp is not NULL. cleanup_timestamp begins as NULL and is only set to an actual value once all artifact tables are garbage-collected (renamed into lifecycle name). That cleanup_timestamp is not NULL and a listed table still exists is a bug.