Component draft/publish system

[mpeterdev]

It would be beneficial to allow developers to save updated component code without publishing it as the latest version of the component.

Motivation:

• sharing updates with other developers before they are ready for distribution
• persisting WIPs for access across multiple devices
• developing multiple component updates in parallel
  • this would likely require a modification to the Widget component to accept a flag indicating that the staging version of a given component should be loaded

This can be modeled in a familiar way as a draft/publish system. The component editor would provide separate UI for "Save" vs "Publish".

Saving a draft would write component code to a different key in SocialDB. Since this would set a new standard for SocialDB data, it should be discussed.

SocialDB Pattern

Currently, published component code is saved to the empty string key under <account>/widget/<component name>. Here is an example

"Genie": {
    "": "State.init({ selectedQuestion: \"\", ........",
    "metadata": {
      "image": {
        "ipfs_cid": "bafkreic7dxh45ncscqgppb5tfobmhpneiw5fwbe5j6wt5ny5k6ckhb72xy"
      },
      "tags": {
        "app": ""
      }
    }
  },

Some options of where to store the draft

• under a new key draft next to current component code e.g. michaelpeter.near/widget/Genie/draft
  • this keeps draft code neatly colocated with published code, but component fetches would return ~2x the data
• create an additional component set at <account>/draftWidgets which mirrors <account>/widget

These assume that we only wish to save one draft version to SocialDB at a time

[evgenykuzyakov]

under a new key draft next to current component code e.g. michaelpeter.near/widget/Genie/draft
this keeps draft code neatly colocated with published code, but component fetches would return ~2x the data

That's not always the case. Only if you fetch michaelpeter.near/widget/Genie/** or michaelpeter.near/widget/Genie/* you get all subkeys. When you request michaelpeter.near/widget/Genie only the "" key is returned.

Right now the metadata for the widgets are stored under michaelpeter.near/widget/Genie/metadata/... and unless you need it you usually don't fetch it.

As another option we can introduce branching. For example michaelpeter.near/widget/Genie/branch/draft, the only concerns are clean up, but it can be done as well.

[mpeterdev]

As another option we can introduce branching. For example michaelpeter.near/widget/Genie/branch/draft, the only concerns are clean up, but it can be done as well.

I like that a lot. We start to get a rudimentary git-like system by having a history of saves to a branch at specific block heights looking like commits

[mpeterdev]

I wonder if there are any other git-like efforts in the works which this may overlap with though

[evgenykuzyakov]

I don't think we have anything yet in this direction. We should also think where to store the associated commit message.
I'd like to be able to commit multiple items at the same time. To avoid storing commit message for long, we can write it under post/commit similarly to current post/main (for feed posting).

Since the widget code will be committed at the same time as post/commit, you will be able to associate the commit message with the submitted code. Especially, if commit message contains a list of keys for published code.

Example:

{
  "post":{
    "commit": {
      "text": "Initial commit. Adding Genie and Genie.Item",
      "type": "md",
      "keys": ["michaelpeter.near/widget/Genie/branch/draft", "michaelpeter.near/widget/Genie.Item/branch/draft"]
    }
  },
  "widget": {
    "Genie": {
      "branch": {
        "draft": "return \"Hello World\";"
      }
    },
    "Genie.Item": {
      "branch": {
        "draft": "return \"Item\";"
      }
    }
  }
}

Note, the content of post/commit should be stringified JSON. It can also include necessary indexing information, e.g. add this to the feed and notify reviewers.

[AustinBaggio]

Great call-out on commit messages. This should be part of the DevX, and optional.

[marcinbodnar]

I think that a branch-like system has a lot of sense.
Here's how it might look from the frontend point of view and user flow:

Starting with the blank widget:

1. user forks component Genie:

• preparing changes
• clicks Save without publishing
• we show a list of branches and an option to create a new branch
• the user didn't create any branch yet, so he types a new branch name: branch-1 and confirms
• branch-1 is created for this component and branch-1 is added to the list of branches

2. user forks component GenieQuestionList:

• preparing changes
• clicks Save without publishing
• we show a list of branches and an option to create a new branch
• user chooses branch-1 and confirms
• branch-1 is created for this component

3. User finished working, and wants to publish branch-1:

• we show a list of branches, and the user can choose which one to publish
• next, we show two options:
  • does the user wants to overwrite the master branch or save the master branch as a separate branch - if save, he can type the new branch name
  • does the user wants to remove branch-1 after publishing
• based on chosen options we create or/and delete branches

Additional options:

• user should be able to remove the whole branch at any time
• user should be able to remove a chosen component from the branch
• I would also suggest adding an option to create a new component from a branch - for example, a user should be able to create a new component from Genie/branch-1

When it comes to commits, in this flow commit will be created when the user wants to publish the branch.

In this case, the widget can look like this:

{
  "widget": {
    "Genie": {
      "": {
        "": "..."
      },
      "branch-1": {
        "": "..."
      },
      "branch-2": {
        "": "..."
      },
    },
    "GenieQuestionList": {
      "": {
        "": "..."
      },
      "branch-1": {
        "": "..."
      }

    },
  },
  "branches": "[\"branch-1\", \"branch-2\"]"
}

[maxhr]

The most urgent usecase is developing widgets in parallel, without publishing every little thing to chain - just as you wouldn't push every CMD+S to GitHub.

I think there should be a layer to hook into calls to SocialDB to provide a Preview with your WIP, unpublished, files.
Once we have that, it'll also allow development with GitHub repos, even private repos, a perfectly valid requirement if you want to keep your work secret and not expose it to testnet/mainnet until you are ready to go public.

And once we have that we don't need to re-invent the wheel with any git-like systems - just treat testnet+mainnet as your staging+production environment, and work on GitHub as you are used to.

In the future, once we gain enough traction maybe we can think how we migrate development processes on to the blockchain, but today it'll be a blocker for onboarding newcomers - best thing is to keep dev environments that people already know.
That's in my opinion :)

[mpeterdev]

Loading versions of multiple WIP widgets directly from LocalStorage or the filesystem (VS Code) is an overlapping use case with this proposal, but they should likely coexist. I agree that you should not need to use drafts saved on chain when working by yourself and making changes to multiple components at once.

When focused specifically on the online IDE, there is a need to provide some level of versioning/branching without relying on a full git integration to a service like GitHub. The goal should not be to fully re-implement git, but to provide enough basic tooling for persisting and sharing WIP components that the online IDE feels usable.

As a side note: treating testnest as a staging environment only works if all your integration points (i.e. components from other authors) also exist on testnet, which is unlikely in a highly composable system like Near Social

[AustinBaggio]

+1 collaborating across different saved widgets is the main benefit of this feature
A close second are version (contribution) history & reversions. This work can be extended to unblock a PR flow.