SNOW-1514764: Support Tracing in SPCS runtime

[sfc-gh-yuwang]

Please answer these questions before submitting your pull requests. Thanks!

1. What GitHub issue is this PR addressing? Make sure that there is an accompanying issue to your PR.

   Fixes #SNOW-1514764

2. Fill out the following pre-review checklist:

   • I am adding a new automated test(s) to verify correctness of my new code
   • I am adding new logging messages
   • I am adding a new telemetry message
   • I am modifying authorization mechanisms
   • I am adding new credentials
   • I am modifying OCSP code
   • I am adding a new dependency

3. Please describe how your code solves the related issue.

   Please write a short description of how your code change solves the related issue.

4. (Optional) PR for stored-proc connector:

[sfc-gh-bdrutu]

Not sure if the log is useful or not...

[sfc-gh-yuwang]

I think it is better than put a pass here

[sfc-gh-yuwang]

@sfc-gh-bdrutu Hi, can you give me a brief explanation why this change has to be in python connector? Is there other ways that we can do this?

[sfc-gh-aling]

this change would impact general python users, some questions in my head:

1. what data is injected into the header?
2. this is on the critical path of query execution, what's the overhead of opentelemetry inject added here, e.g., increase in the payload, potential perf regression? also would try-except cause perf regression? I think we need to test perf of this change, and see the payload change.
3. what kind of telemetry data are sent to snowflake backend, do we need to consult the legal team about the telemetry?
4. will user be aware of this? should we allow customers to opt out open telemetry collection?
5. how would backend react to request with new header, do we need any backend change?

[sfc-gh-yuwang]

@sfc-gh-bdrutu I think some of this question are answered in slack chat, but can you still take a look?

[sfc-gh-aling]

just synced up with @sfc-gh-yuwang , I think once we get perf tested (can be as simple as performing "select 1" for 1 million times) and confirm there is no regression, then let's merge it to unblock SPCS scenario.

[sfc-gh-bdrutu]

what data is injected into the header?

We will add one header "traceparent" which has 55 characters value (always), see https://www.w3.org/TR/trace-context/. So the overhead shouldn't be that significant especially it does not happen if the user doesn't configure opentelemetry. Not sure about performance of try/catch.

what kind of telemetry data are sent to snowflake backend, do we need to consult the legal team about the telemetry?

We are sending the trace/span ids of the notebook operation, again we do this only if the user configures opentelemetry.

will user be aware of this? should we allow customers to opt out open telemetry collection?

User has to import opentelemetry and configure it to have this. This code executes only if the import succeeds which requires user to import and configure opentelemetry.

how would backend react to request with new header, do we need any backend change?

We are taking care of that.