Improve debug logging

[rawwar]

Description

Identify and improve debug logging in core and providers. Debug logs should include all the information used to make the request/invocation, especially when interacting with external systems.

Use case/motivation

It helps identify the data used when interacting with external systems and response data. This can be very helpful in case of failures. Many providers already do this. But, goal of this issue is to identify gaps and fill them

Related issues

Parent Issue: #40975

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[Prab-27]

@omkar-foss, I'd like to work on this issue. I would like to start with the scheduler

[rawwar]

@Prab-27 , feel free to create issues and refer them to this one. We can use this as an Issue to track progress. I'll start tracking them here once you link the issue.

[omkar-foss]

Sure thanks for picking it up @Prab-27, feel free to create a new issue for the Scheduler debug logging improvements and link it to this issue (as @rawwar suggested above).

Also in case you'd like to discuss or brainstorm something quickly, you can drop a message on #sig-improving-debugging channel on Airflow community Slack, link here.