Look at the workflow for pr-review. Restrict it to at least collaborator, in other words remove the automatic running on random prompts (because they're prompts to the agent LLM)
All the rest remains in place, including that it works on label or when a maintainer asks