Simplify Studio onboarding screen and workflow

[shcheklein]

Two things that I found recently trying it:

Related #3434

• I think we should have a single button Get Token (Studio can redirect to the right page, and even if doesn't now I think we should be fine for now). @iterative/studio-frontend can Studio redirect now? What is the complexity of this? To clarify - if I sign up I want it to immediately open the page with my token. We next need to think about also creating a project automatically or semi-automatically since we know the repo URL potentially and can pass it in the studio URL.
• If an ENV variable with a token detect we need to at least say something about it. ENV var in Codespaces is enough to log metrics actually, and some options on this page can contradict with it. E.g. should we show that checkbox enabled it token ENV var is found and if ppl disable it then unset the env?
• After Live metrics config dvc#9433, live sharing will be enabled by default when the studio token is saved to the dvc config, although it can be disabled with studio.offline = true. To comply with this, we need to make the "Share new experiments live" checkbox correspond to the studio.offline config value and stop setting the environment variable.

[mattseddon]

The first item is covered by the last item in #3574.

Drop the sign-in button from Studio setup page (#3574 (comment) - related to https://github.com/iterative/studio/issues/5715)

The Studio issue for frontend redirects is https://github.com/iterative/studio/issues/5715.

We can remove the original action item from #3574 if you like.

[shcheklein]

@mattseddon thanks, yep. Dropped it in that ticket (making it closer to close that one).

[dberenbaum]

I think we should have a single button Get Token (Studio can redirect to the right page, and even if doesn't now I think we should be fine for now).

👍

@iterative/studio-frontend can Studio redirect now? What is the complexity of this? To clarify - if I sign up I want it to immediately open the page with my token. We next need to think about also creating a project automatically or semi-automatically since we know the repo URL potentially and can pass it in the studio URL.

Should we first focus on the part about creating a project? If it's about onboarding, I think first priority needs to be to create the project and token only matters after that.

[dberenbaum]

From #3898:

After iterative/dvc#9433, live sharing will be enabled by default when the studio token is saved to the dvc config, although it can be disabled with studio.offline = true. To comply with this, we need to make the "Share new experiments live" checkbox correspond to the studio.offline config value and stop setting the environment variable.

[mattseddon]

ENV var in Codespaces is enough to log metrics actually, and some options on this page can contradict with it. E.g. should we show that checkbox enabled it token ENV var is found and if ppl disable it then unset the env?

@shcheklein do you mean you can set an environment variable when starting codespaces which will log metrics to Studio if any experiments are run through the extension?

[shcheklein]

@shcheklein do you mean you can set an environment variable when starting codespaces which will log metrics to Studio if any experiments are run through the extension?

Yes, exactly.

[mattseddon]

@shcheklein do you mean you can set an environment variable when starting codespaces which will log metrics to Studio if any experiments are run through the extension?

Yes, exactly.

This seems like a strange/advanced use case now that the token is managed via a user's DVC config(s). I think the smartest thing to do would be to block it off and remove STUDIO_TOKEN DVC_STUDIO_TOKEN from the environment variables we pass to a new process.

[shcheklein]

This seems like a strange/advanced use case now that the token is managed via a user's DVC config(s).

Yes, agreed.

I think the smartest thing to do would be to block it off and remove STUDIO_TOKEN from the environment variables we pass to a new process.

Hmm, sounds like too magical and a bit too intrusive to me. How about we show a message if it's detected (in the Setup screen) or as a notification?

[mattseddon]

Hmm, sounds like too magical and a bit too intrusive to me. How about we show a message if it's detected (in the Setup screen) or as a notification?

There is only a passing reference in the docs to using the token in the way that we are discussing:

• https://dvc.org/doc/studio/user-guide/projects-and-experiments/live-metrics-and-plots#provide-access-token-to-experiment
• https://dvc.org/doc/user-guide/project-structure/configuration#studio

I feel like it is more magical that it actually works and that we don't need to come up with a workaround to cater for this edge case.

[shcheklein]

It's still a valid setup (otherwise we would have deprecated it). I agree though that it's a minor / advanced and we can deprioritize it. (I would not ignore / remove the env variable though, let's user do what they want).

[julieg18]

Studio has implemented a way to get a token with Device Authentication (see https://github.com/iterative/studio/issues/5158#issuecomment-1772610546). I'll start looking into using the method in VSCode.

[julieg18]

Researched how other extensions handled authentication like this and posted findings in Studio (https://github.com/iterative/studio/issues/5158#issuecomment-1783088262). TLDR is that with the current flow, two issues I noticed were that we need to repeatedly poll a studio endpoint for a token and the user doesn't get redirected back to vscode after authentication.

I also created a quick demo of how things would work with the current Studio flow (#4931
):

Screen.Recording.2023-10-30.at.6.30.02.PM.mov

We could either keep working on this and push a first iteration with the current Studio flow, or hold off until we have another iteration on Studio's auth flow. Personally, I think the current flow is good enough to release, but what does everyone else think? cc @shcheklein

Either way, I'll keep working on the pr, since a good chunk of the logic should stay the same.

[julieg18]

With #4931 merged, we now use studio's auth flow in VSCode! What's left:

• Add some kind of sign (probably just a loading screen) to the Studio section when were fetching the token after the user has authorized (Add Studio Token Auth Flow #4931 (comment))
• Test and update once studio#8227 has been merged
• Confirm that authentication flow works in different environments (Codespaces and windows come to mind). Windows work but codespaces is broken (Get token auth flow doesn't work in Codespaces #5050)
• Add a timeout after starting studio authentication, closing the listener after 10-15 minutes.

[julieg18]

Going to close this since all tasks have been done, minus this:

If an ENV variable with a token detect we need to at least say something about it. ENV var in Codespaces is enough to log metrics actually, and some options on this page can contradict with it. E.g. should we show that checkbox enabled it token ENV var is found and if ppl disable it then unset the env?

As discussed synchronously, it is extremely difficult for the extension not detect ENV variables.