[v3-1-test] Fix assets used only as inlets being incorrectly orphaned (#58303) (#58368)#58986

Lee-W merged 1 commit intov3-1-testapache/airflow:v3-1-testfrom

backport-24e682c-v3-1-testapache/airflow:backport-24e682c-v3-1-testCopy head branch name to clipboard