feat: Enable spark-operator prometheus metrics

[alanty]

What does this PR do?

Enables the prometheus PodMonitor creation in the spark-operator helm chart and updates the prometheus config to pick up monitors across namespaces (kube-prometheus-stack, is the default)

Motivation

as we continue benchmarks/testing having visibility into the operator performance will help us understand the performance

More

• Yes, I have tested the PR using my local account setup (Provide any test evidence report under Additional Notes)
• Mandatory for new blueprints. Yes, I have added a example to support my blueprint PR
• Mandatory for new blueprints. Yes, I have updated the website/docs or website/blog section for this feature
• Yes, I ran pre-commit run -a with this PR. Link for installing pre-commit locally

For Moderators

• E2E Test successfully complete before merge?

Additional Notes