RFC/Spec: wrangler init with flag --from-dash

[JacobMGEvans]

Why should we do wrangler init --from-dash?

Furthering the discussion from #1346, we want to provide a bridge from the dashboard where previously it was easy to get up and going with a new Worker. Now we want to encourage the user to an improved developer experience with the CLI and their local developer ecosystem. Since the goal is for the user to develop locally we want to be sure this is a one-time initialization process and not a way to develop Workers simultaneously in the dashboard and local machine.

Solution

The idea of working off the wrangler pull discussion is to leverage work already done with wrangler init creating a flag that can be used --from-dash (that can be bikeshedded). The flag would have an input that would allow targeting a Worker that would utilize the source code for the initialization of the project, eventually including the Configurations of the Worker to be generated as well; I think this should be done in phases, getting the Worker source code project initialization first. The idea is to focus on the "migration" from the dashboard to the local development ecosystem leveraging the DX of Wrangler, meaning there will be safeguards necessary to prevent "synchronizing" behaviors from dash to and from local. Part of the benefit of wrangler init being utilized for this bridging is two-fold

1. We can increase its safeguards around not initializing into existing projects, preventing "sync" or source control-like behaviors.
2. The initialization features currently built-in can be augmented and used to encourage things like TypeScript, GitHub repo creation, etc... furthering the improvement of DX and local development best practices.
3. Messaging either on Dash, Wrangler or both that notify the user they are opting into a particular workflow leveraging Wrangler and "quick edit" on dash will be disabled or no longer supported for that Worker/Service

Notes:

Additional discussion: #1346 (comment)

• This implementation should be agnostic from any source control & storage.
• Encourage augmenting projects, i.e. TypeScript, and source storage (we currently provide GitHub repo creation).
• The flag should only be done in empty projects.

More details: #1623 (reply in thread)

• Free to init the project multiple times if into a new project each time, but cannot init into the same project more than once.
• If not authenticated when performing init from dash, you will be prompted to login; no new design elements needed for this.
• When publishing a migrated worker, it overwrites the original. (i.e. worker-1 migrated to local dev environment and then published will rewrite worker-1)

Concerns:

As previously mentioned avoid any quasi-source control or quasi-source storage patterns or behaviors. This is another primary reason for moving away from wrangler pull and making sure safeguards are in place to make it a one-off usage. Tangentially related corrupting existing projects could be a problem which is an additional reason to make this a one-off command for initialization of a project locally from the dashboard.

Alternatives:

1. wrangler pull was suggested RFC: wrangler pull <name> (and alternatives) #1346 now my main concern with this was source control & storage confusion,
   wrangler pull -> git pull and the dashboard would seem like where the source is "stored" much like GitHub. We would also not have all the benefits of leveraging wrangler init features directly.
2. A dashboard download button was another suggestion that was considered.

Phases of work:

• The Worker source code project initialization
• The configuration auto-generation (i.e. wrangler.toml )
• Messaging either on Dash, Wrangler or both. Solution step 3

[IgorMinar]

Thanks for the spec @JacobMGEvans. I'm in favor of your proposal.

If we are going to split the implementation into two steps, then let's ensure that we don't advertise the feature until it's ready, or clearly mark it as experimental. Thank you

[caass]

Since the goal is for the user to develop locally we want to be sure this is a one-time initialization process and not a way to develop Workers simultaneously in the dashboard and local machine.

👍🏻

I think this looks good! +1

[IgorMinar]

@JacobMGEvans could you please add one more deliverable to this project: let's make this Wrangler feature easily discoverable by modifying the Dash UI to promote this feature when visiting projects that have been deployed only via the Dash.

(We'll likely need to identify these projects first, or maybe we can show the info/prompt only in the dash editor? We can work with our design folks to determine the best way to surface this.)

Thank you

[lrapoport-cf]

@IgorMinar @JacobMGEvans i chatted with design about surfacing this feature as well as the UX/workflow implications for using --from-dash -- they will review and follow up with any initial questions 👍

[lrapoport-cf]

@neynah answers from team discussion:

• free to init the project multiple times if into a new project each time, but cannot init into the same project more than once
• If not authenticated when performing init from dash, you will be prompted to login; no new design elements needed for this
• when publishing a migrated worker, it overwrites the original (i.e. worker-1 migrated to local dev environment and then published will rewrite worker-1)

[neynah]

Got it, I will update the copy in the mocks to reflect the new project limitation. Thanks!

[lrapoport-cf]

thanks @neynah !

[lrapoport-cf]

@JacobMGEvans another question after chatting with @neynah : will the command take a worker name or some sort of identifier as a param? what will that look like? we're thinking of including a snippet on the dash to show what command to run for init-ing the given worker

[JacobMGEvans]

@JacobMGEvans another question after chatting with @neynah : will the command take a worker name or some sort of identifier as a param? what will that look like? we're thinking of including a snippet on the dash to show what command to run for init-ing the given worker

It will look much like this npx wrangler init <optional-name> --from-dash <worker-name> if no optional name is provided the init will use the worker-name to name the project.

[penalosa]

As part of the Dash UI work related to this, I'm wondering if there's any way for the Dash to know via the API that a specific worker has been migrated to wrangler (for e.g. making the code on Dash read-only)?

[penalosa]

A wider question that comes up is whether only the code should be read-only, or whether it should be code+config. My instinct would be that it should be both code and config, since allowing edits to the config on the dashboard that wrangler will overwrite is quite a confusing experience.

This might be an issue with other clients, since if the client doesn't support editing config (not wrangler, of course, but maybe some don't? I'm not sure what types of other clients are out there), users would be left with no way to edit the config. This might be a manufactured concern though.

In terms of API—how about adding dashboard_readonly to the Worker script properties, initially set to false? If a separation of code and config is needed, this could be dashboard_code_readonly and dashboard_config_readonly. They'd be set when the init-from-dash flow is completed in the dashboard UI, and/or set from Wrangler after a wrangler init.

[lrapoport-cf]

@penalosa i agree - i think the config should be read-only as well. you raise a good question of whether there are other clients that don't support editing config (i'm not sure) but we would only be locking the code and config if performing the migration to wrangler, which does support config editing. it's possible someone could migrate via wrangler and then use some other client, but i'm not sure this is an edge case we need to worry about. @IgorMinar @threepointone @JacobMGEvans do you have concerns about locking the config as well as the code after migration?

[JacobMGEvans]

I think this is a great discussion point for Wednesdays meeting. That being said I think there are good points to be made either way when pertaining to locking down or not locking down config for a Service Worker in the Dashboard when using Wrangler.

[penalosa]

Copying over from internal chat:

Another option could be to keep config locked, and repurpose the config UI to retain the UX benefits (searching for bindings etc...) but instead of actually making the change, it would give you a line to add to wrangler.toml. That would have the benefit of there being no possibility of an out-of-sync config.

[penalosa]

We definitely want to lock down

• KV Namespace bindings
• R2 bucket bindings
• Service Bindings
• D1 (not sure if this is possible from the UI, anyway)
• DO Bindings

We currently don’t want to lock down

• Environment variables and secrets, as it would break workflows

Things to consider locking down (things that can be specified in wrangler.toml and might cause mismatches):

• Routes
• usage_model
• Cron triggers
• Service name

[rozenmd]

We want to encourage the user to an improved developer experience with the CLI and their local developer ecosystem

@IgorMinar / @JacobMGEvans / @lrapoport-cf - Have we spoken to customers to see why they use the Cloudflare dash for developing their workers? It's all well and good for us to want them to use the software we're developing, but I wonder if there aren't external factors keeping folks on the dash (like overzealous IT teams blocking npm, for example).

[penalosa]

A couple questions that came up in the meeting today:

• Should this be irreversible? Should someone be able to unlock code & config on the dashboard after they've locked it down?
• Should the UI flow actually lock down the config & code, or should it just be a prompt, after which wrangler publish does the actual lock down?

Thinking about it, making this a toggle would solve the problem, I think—the UI could enable the lockdown toggle, which would reduce the possibility of edits to the code in the dashboard between running wrangler --init-from-dash and running wrangler publish, while also not being too destructive (being a toggle that could be turned off again).

cc @threepointone @JacobMGEvans @IgorMinar

[IgorMinar]

• I think the change should be irreversible — once you pull your Service into a Wrangler managed project and publish it from there, all future changes should be done via Wrangler.
• as for the UI locking things down, I think this is very confusing DX. In the UI we want to offer guidelines for moving a project to Wrangler, but not actually modify any state or lock the Service down to prevent future edits.

I think that a much more natural experience is to get a CLI prompt during the first wrangler publish into an existing Service. Perhaps an experience like this (needs more wordsmithing):

wrangler --init-from-dash my-service
# ... 
wrangler publish
 ⛅️ wrangler 2.0.x (update available 2.0.x)
-------------------------------------------------------
▲ [WARNING] You are about to publish a Workers Service that can be edited via the Cloudflare dashboard.
  These edits will make your local configuration and code diverge from what's deployed.
  Would you like to manage the configuration and code changes exclusively via `wrangler publish` and
  prevent changes via the dashboard? [Y/n] 

If the user selects Y then we lock down the Service. If the user select n then we:
a. publish the Service, don't lock down the dash UI
b. don't publish the Service

Option b. seem a bit harsh for existing Services, but I'd like @rita3ko weigh in on that.

Additionally, for new Services, we don't even prompt the user, and instead lock down the UI.

Lastly, if we are running in a non-interactive shell (usually CI environment), we should not prompt the user and assume n as the answer in order not to break existing workflows for our users/customers.

[penalosa]

Pulling together the various threads on this (and looping in @rita3ko from the internal ticket):

• The UI flow explaining how to migrate to Wrangler should be non-interactive, without any state-changing checkbox
  • The one remaining question here is where this should be surfaced—as a card in settings, or as a dropdown from the Quick Edit button (cc @neynah).
• Locking down the UI: following the flow in @IgorMinar's comment above, with no prompt for new services.
  • How would we determine what is a "new" service? By compatibility date, I suppose, unless that's counter to the intention of compatibility dates?
  • Should option a or option b be taken for existing services?
• Irreversibility: I'd lean towards the irreversibility argument—assuming the CLI flow in wrangler publish, this will be an active decision the user has made, and allowing the lockdown to be reversed will (I think, at least), lead to more confusion. If unlocking the UI is something that users definitely need (although, if the only config being locked down is bindings, I'm not sure how common or useful changing bindings at runtime is? @threepointone mentioned that a significant number of users change env vars at runtime, but do we know if that extends to bindings?), having a flow similar to delete (with the modal requiring typing in the service name) could sufficiently disincentivise it?
  • Honestly, I feel like allowing this to be a toggle is going to lead to headaches though—I can imagine a situation where someone needs to make a quick change urgently, and so goes through the dashboard and unlocks the config, and then another member of the team (or the same person a couple of days later), develops a new feature and runs wrangler publish, overriding their quick fix from the dashboard.
  • Another argument in favour of irreversibility is that as soon as wrangler publish is run, the code that would be visible through Quick Edit after unlocking the UI will be transpiled through esbuild (at least, I think so? I could well be wrong on this), which (depending on settings and the specific code), could be quite different from the code the user has been working on locally.

[rita3ko]

Irreversibility: I'd lean towards the irreversibility argument—assuming the CLI flow in wrangler publish, this will be an active decision the user has made, and allowing the lockdown to be reversed will (I think, at least), lead to more confusion

This assumes that a single user is working on something. Imagine a team, where one engineer set things up, and maybe didn't inform their team or even left. Any action should be reversible, but if there are consequences we should create friction in reversing it and make sure the consequences are clear.

I 100% agree people shouldn't be going back and forth, and it's not the recommended path, and has the potential to create issues. But if we warned someone and they still want to do it, we need to let them.

[penalosa]

@lrapoport-cf and I talked about this earlier, and to summarise some of the conclusions:

• The dashboard UI showing how to migrate to Wrangler will be non interactive
  • Entry points to this UI (dropdown or settings card) still need some thought, but can probably both be implemented (cc @neynah)
• wrangler publish will include the workflow from @IgorMinar's comment, with option (b) for users selecting [n] (for consistency with CI behaviour, which will auto-select [n] and then publish
• wrangler publish will hit the /readonly endpoint to tell the API that the config should be locked if the user selects [y] (and to check whether the config is locked to determine whether to show the prompt) (cc @JacobMGEvans)
  • An enhancement here from the wrangler publish side is to not show the prompt, and default to [y] if the compatibility date in wrangler.toml is past the release date of this feature
• Upon locking, the user won't be able to edit bindings or code within the dashboard, but will be able to view bindings (with a warning making clear that these are reflecting the last wrangler publish
• If a user needs to unlock the dashboard, there will be an option within settings to unlock (coupled with copious warnings). It will probably need a UI flow similar to the Delete action (i.e. confirming the worker name in a modal). wrangler publish will show the prompt to lock settings the next time it's run.
  • I think a global unlock is a clearer solution than allowing one-off edits of bindings—it makes it clear that they're accepting that there will be inconsistencies between wrangler and the dashboard, and will also mean that wrangler publish will give an explicit warning about overwriting settings the next time it's run. @rita3ko I know you were suggesting allowing individual one-off binding edits—would a global unlock solve the same user need?

[IgorMinar]

Thank you @penalosa for the summary. I've been thinking a lot about how to keep things simple because I'm starting to think that we got carried away a bit. This brought me back to the main intentions behind these UI changes, which I think comes down to three things:

1. If a Service was last deployed via the dash, we want to inform the developer to give Wrangler a try (with a list of reasons explaining the benefits, and instructions on how to migrate).

2. Inform the developer about to make changes via the dash to a Service previously published via Wrangler (as well as Terraform or other REST API clients) that these changes might be overwritten by future deployments via the REST API client.

3. Inform the developer about to publish a Service via Wrangler that the previous deploy was via the dashboard and the current deployment will overwrite the dash deployment.

To achieve all of this, we don't need to lock anything in the UI or in EWC, instead we need to keep track of the client responsible for the last deployment (code or config update) and warn/prompt the user when there is a chance of data loss.

We can track if the deployment was done via the dash or not determined by user-agent received by EWC (stored as a boolean property of the last Script/Deployment (or the Service?) — we could call this property deployedFromDash or similar).

Once we have the information about the source of the last deployment, we can decide if the current deployment (config or code change) should warn the user using the following decision matrix:

	new deployment via dash	new deployment via Wrangler
last deployment via dash	no warning in dash	CLI warning / prompt
last deployment via Rest API Client	warning / confirm change	no warning

This will solve the goals #2 and #3. The warning can be a simple prompt in the Dash and CLI prompt in Wrangler.

To achieve the goal #1 we could use the same last deployment property mentioned above to determine when to show a UI element to the user suggesting that they move the project to Wrangler. Let's consult with the design folks to determine the UI element, I suggest we use some lightweight notification bar that opens a modal with Wrangler info and init --from-dash instructions.

Lastly, I assume that we currently don't have information about how the existing Services were deployed, and we don't have a good way to retroactively populate this information. For this reason, we could roll this out incrementally and start populating the deployedFromDash property via EWC and not taking any of the actions mentioned above for Services that don't have this property explicitly set yet. As Services are redeployed we'll populate the datastore with the info about the source of the deployment and this behavior will kick in.

What do you all think?

[JacobMGEvans]

I think this is a great place we can iterate from and makes sense to from a UX perspective.

[lrapoport-cf]

below are summary implementation details for @IgorMinar 's approach after further clarification with the team. note that these details apply for workers of any type (i.e. single or multi file, service or module syntax), and for which the source of last deployment is known. for workers with unknown sources of last deployment, none of what is outlined below should be applied and everything should remain as is.

• on the dash:
  • if the worker was last deployed from dash:
    • surface a suggestion to use wrangler (see point 1 from @IgorMinar 's comment above)
      • the design/experience team has already thought about where this makes most sense to surface, so defer to them for that (cc @neynah)
  • if the worker was last deployed via REST API client:
    • surface warnings that changes deployed via the dash may be overwritten in the future via non-dash deployments (see point 2 from @IgorMinar 's comment above)
      • where these warnings should be surfaced is still an open question. could be in the same place/manner that we surface suggestions to use wrangler. another option is to use small warning icons in close proximity to each location in which modifications can happen, and display warning text in tooltips on hover. @neynah could you create mockups for this latter approach?
      • in the initial implementation, we will not ask for confirmation or acknowledgement of warnings. these will be added later.
• in wrangler:
  • if the worker was last deployed from the dash:
    • surface a warning that deployment may overwrite changes previously deployed via the dash (see point 3 from @IgorMinar 's comment above)
      • ask for confirmation/acknowledgement