[SPARK-46613][SQL][PYTHON] Log full exception when failed to lookup Python Data Sources

[HyukjinKwon]

What changes were proposed in this pull request?

This PR proposes to log full exception when failed to lookup Python Data Sources

Why are the changes needed?

In my internal testing it logs something like:

...
24/01/05 03:49:49 WARN DataSourceManager: Skipping the lookup of Python Data Sources due to the failure: java.lang.StackOverflowError
24/01/05 03:49:49 WARN DataSourceManager: Skipping the lookup of Python Data Sources due to the failure: java.lang.StackOverflowError
24/01/05 03:49:49 WARN DataSourceManager: Skipping the lookup of Python Data Sources due to the failure: java.lang.StackOverflowError
24/01/05 03:49:49 WARN DataSourceManager: Skipping the lookup of Python Data Sources due to the failure: java.lang.StackOverflowError
24/01/05 03:49:49 WARN PythonWorkerFactory: Failed to open socket to Python daemon:
java.net.ConnectException: Connection refused
	at sun.nio.ch.Net.connect0(Native Method)
...

which is hard to debug. It should show the full error messages so developers can debug.

Does this PR introduce any user-facing change?

No, the main change has not been released out yet.

How was this patch tested?

Manually.

Was this patch authored or co-authored using generative AI tooling?

No.

[HyukjinKwon]

Build https://github.com/HyukjinKwon/spark/actions/runs/7441506339

[HyukjinKwon]

Merged to master.