[v23.3.x] miscellaneous idempotency fixes

[vbotbuildovich]

Backport #22687 to branch v23.3.x. Note that this issue was created as a placeholder, since the original PR's commit(s) could not be automatically cherry-picked.
Here are the commands to execute:

git checkout v23.3.x
git checkout -b vbotbuildovich/backport-22687-v23.3.x-706
git cherry-pick -x 26f6150198e043121b5e3e5b5deb6d8b0843e262 7ac5acfb9411bc9acfa7ae7fdd4a948236956e66 2f5b9cbcff2cb9a41290e685046d1ae8939271a6 e2c59b811810fa97b73234b1b8dd5040ee8b68f8 bc3d761b87b42b719ad9fab2172760abd4ac46d0 0ed2a90c0c071b3c6710e9cfacf6895db51b9076 3c36e7c383070b704e1bfbe43ec02fe598459533 1af75574cef77066d5c3f97695c429b4585ea5b9 f8e5c21deba4621c7fc4c8eda40b4a16270574aa 092a2b89e03450d5cf182be938980afe191d6b27

git push origin vbotbuildovich/backport-22687-v23.3.x-706
gh pr create \
  --title "[v23.3.x] miscellaneous idempotency fixes" \
  --base "v23.3.x" \
  --label "kind/backport" \
  --head "vbotbuildovich/backport-22687-v23.3.x-706" \
  --draft \
  --repo "redpanda-data/redpanda" \
  --milestone "v23.3.x-next" \
  --body "Backport of PR https://github.com/redpanda-data/redpanda/pull/22687 "

JIRA Link: [CORE-6805](https://redpandadata.atlassian.net/browse/CORE-6805)