sql: change github links to go through a proxy to gather click telemetry

[jordanlewis]

Currently, many error messages in CockroachDB include a link to an issue on GitHub. We think (and hope) that many people click these links.

We should use a link shortener to proxy these links, so that we can gather click metrics (how many times a particular issue is clicked on). This will help prioritize issues further.

For example, the link could look like https://cockroa.ch/issue/34734, which could redirect to issue #34734 as a random example. Another benefit to this is that, if we ever change issue management technologies, this could be transparent to our users.

cc @bobvawter, @kenliu, @rafiss, @apantel, @awoods187

[awoods187]

Love this idea!

[bobvawter]

I can add pattern-redirects to go.crdb.dev (and certainly support multiple domain frontends).

[jordanlewis]

@bobvawter I like the idea. The main downside to this idea is that we'd be committing to operating the proxy as a production service ~forever.

[bobvawter]

It's not a proxy so much as a redirector. If we ever wanted to turn that service down, we could generate a whole slew of static html pages from the link database and just dump them into a public-facing GCP storage bucket.

[rafiss]

Perhaps for clusters that have telemetry enabled, we could also include a query param in the URL (e.g. ?cid=<uuid unique to cluster>). We could decide to have this be the same as the cluster ID reported to us in telemetry, or make it distinct.

The idea is then we could have additional data on how often each cluster/user runs into an unimplemented issue.

[awoods187]

Have we had any more thoughts here?

[bobvawter]

If we're comfortable using the go/ redirector to capture this, I'll add pattern redirects to it this Friday.

[awoods187]

@apantel this is the issue

[kenliu]

Do we have the redirector behind IAP?

[bobvawter]

The bulk of go/ lives behind IAP. There is a carve-out for go/p/... paths to allow some links to be published to the entire world.

[bobvawter]

Just to be explicit, only links which have been marked as public are actually available on the /p/ prefix.

[bobvawter]

@awoods187 What data do you want to record about an incoming redirect request? Do you want PII like IP address, User-Agent, query params? There's already a trivial click-counting table that I can add more columns to. This data could easily be synched out to Snowflake.

[bobvawter]

Pattern redirects are working now:

https://go.crdb.dev/pr/1234
https://go.crdb.dev/issue/5678

These two path prefixes bypass the IAP proxy and could be used externally. Alternate domain names can be used without significant extra effort.

Let me know what data you want to capture for clicks.

[rafiss]

I think to start we'd just want to capture click counts.

Is it easy to capture more data? It doesn't seem worth spending much time on anything else, since I don't anticipate anyone having the time to dive very deep into the data here.

[bobvawter]

The click data is already being captured.

…

On Wed, Jun 3, 2020 at 1:14 PM Rafi Shamim ***@***.***> wrote: I think to start we'd just want to capture click counts. Is it easy to capture more data? It doesn't seem worth spending much time on anything else, since I don't anticipate anyone having the time to dive very deep into the data here. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#45504 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAI23FGWJKRC7Y5RU7IUBDDRU2AGTANCNFSM4K5AUA5A> .