refining off-chain indexing #800

jimmychu0807 · 2020-12-11T06:19:20Z

I have:

Combined the topic of off-chain indexing under off-chain workers.
Refined the write-up somewhat (tomek, let me know if all you want to say originally are still there is no mis-interpretation.
I don't think this section is good to have rust code shown up (we are teaching Substrate fundamental concepts here). In fact I think it worth to have another recipe to demonstrating using off-chain indexing. So that part is removed, and a new recipe is queued to be written on this topic.

Let me know what you think. Thanks

tomusdrw

lgtm, although I think we need to point to a code sample to make the example relevant, now it doesn't really help anyone to get started with Off-Chain Indexing (and that was my goal with the original example).

tomusdrw · 2020-12-11T09:40:30Z

docs/knowledgebase/learn-substrate/off-chain-workers.md

+be exactly the same for every node with indexing enabled.
+
+Off-chain database can be read using remote procedure calls (RPC) so it fits the use case of storing
+indefinitely growing data without over-consuming the on-chain storage.


Maybe let's add the code samples to Development Guide as well - right now, since there is no pointers at all it's really hard to go from "I've learned the concept" to "I want to try it out".

I would prefer to follow Jimmy's suggestion and keep the code in the Recipes.

Edit: this definitely implies we need to make these links easy to find and follow 💯

Thanks input from Tomek and Dan.

Tomek, I commit that I will write a sample code, either in recipe or the Runtime section below on off-chain index before Jan 15th. If we put code out, we likely need a section of code and explain to readers the context. So I need to experiment with the API myself first, and this takes some work and time. Will let you guys review it once it is done. Then we will add a link to the code section here.

So I prefer keeping this section to be conceptual and show the code in another section. Let me know if you are okay with this, @tomusdrw

tomusdrw · 2020-12-11T09:44:30Z

docs/knowledgebase/learn-substrate/off-chain-workers.md

@@ -46,3 +46,23 @@ should be implemented to determine what information gets into the chain.

 For more information on how to use off-chain workers in your next runtime development project,
 please refer to our [Development Guide](../runtime/off-chain-workers).
+
+# Off-Chain Indexing


TBH I'm not sure it's fair to bundle Off-Chain Indexing under Off-Chain Workers. These two things can really be though of independently.

I'd suggest renaming the entire article to "Off-Chain Features" or sth, and then having 3 sections that outline 3 independent substrate features:

Off-Chain Database - a node-local database that every substrate node has, can be accessed over RPC (both writes (unsafe) and reads (safe)).

Off-Chain Workers - an idea of running off-chain an arbitrary logic defined within the runtime code (when the node is fully synced). Workers can access Off-Chain DB.

Off-Chain Indexing - a Runtime function to populate Off-Chain DB (opt-in via CLI flag) even during sync.

Thank you both for thinking critically about how we present these concepts to our users 🙏 I would not be opposed to Tomek's suggestion at all, and we could even think about "lifting" this out of the Runtime section and creating a new top-level Off-Chain section or something.

I will follow tomek suggestion of renaming the section as Off-Chain Features and having 3 children sections inside.

danforbes

Thank you both 🙏

docs/knowledgebase/learn-substrate/off-chain-workers.md

danforbes · 2020-12-11T17:55:30Z

docs/knowledgebase/learn-substrate/off-chain-workers.md

@@ -46,3 +46,23 @@ should be implemented to determine what information gets into the chain.

 For more information on how to use off-chain workers in your next runtime development project,
 please refer to our [Development Guide](../runtime/off-chain-workers).
+
+# Off-Chain Indexing


Thank you both for thinking critically about how we present these concepts to our users 🙏 I would not be opposed to Tomek's suggestion at all, and we could even think about "lifting" this out of the Runtime section and creating a new top-level Off-Chain section or something.

danforbes · 2020-12-11T17:56:02Z

docs/knowledgebase/learn-substrate/off-chain-workers.md

+be exactly the same for every node with indexing enabled.
+
+Off-chain database can be read using remote procedure calls (RPC) so it fits the use case of storing
+indefinitely growing data without over-consuming the on-chain storage.


I would prefer to follow Jimmy's suggestion and keep the code in the Recipes.

Edit: this definitely implies we need to make these links easy to find and follow 💯

Co-authored-by: Dan Forbes <dan@parity.io>

jimmychu0807 · 2020-12-12T05:12:04Z

@tomusdrw as you suggested, I have rewritten the section and rename it to Off-Chain Features. Let me know what you think and if they are still technically correct 🙏

Thank you all for the review, tomek and dan.

tomusdrw

❤️ looks great, just small clarification about on-chain logic access to off-chain storage would be good.

docs/knowledgebase/learn-substrate/off-chain-features.md

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

danforbes

Can you guys please review my changes and make sure I didn't mess anything up? f0db1ab This is a great addition...thank you so much, @tomusdrw and @jimmychu0807 🙏🏻

* refining off-chain indexing * Update docs/knowledgebase/learn-substrate/off-chain-workers.md Co-authored-by: Dan Forbes <dan@parity.io> * Update docs/knowledgebase/learn-substrate/off-chain-workers.md Co-authored-by: Dan Forbes <dan@parity.io> * Expanded the section from off-chain-worker to off-chain-features * Update docs/knowledgebase/learn-substrate/off-chain-features.md Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com> * update on content to make it more coherent * Fix link * Links and nitpicks * Prettier Co-authored-by: Dan Forbes <dan@parity.io> Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com> Co-authored-by: Dan Forbes <dan@danforbes.dev>

@joepetrowski

* Add validate address options I have added the 2 main tools that Parity has created to validate ss58 addresses. I also added a list of places for developers to look for some more code examples to use to create their own custom validation tools in various languages. @joepetrowski @danforbes * Fix line length * Add validate address options I have added the 2 main tools that Parity has created to validate ss58 addresses. I also added a list of places for developers to look for some more code examples to use to create their own custom validation tools in various languages. @joepetrowski @danforbes * Nitpicks * Updated Terms of Use per Legal (#803) * Updated Terms of Use per Legal * Prettier `npx prettier --write --print-width 100 ./website/pages/en/terms.js` Co-authored-by: Dan Forbes <dan@danforbes.dev> * refining off-chain indexing (#800) * refining off-chain indexing * Update docs/knowledgebase/learn-substrate/off-chain-workers.md Co-authored-by: Dan Forbes <dan@parity.io> * Update docs/knowledgebase/learn-substrate/off-chain-workers.md Co-authored-by: Dan Forbes <dan@parity.io> * Expanded the section from off-chain-worker to off-chain-features * Update docs/knowledgebase/learn-substrate/off-chain-features.md Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com> * update on content to make it more coherent * Fix link * Links and nitpicks * Prettier Co-authored-by: Dan Forbes <dan@parity.io> Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com> Co-authored-by: Dan Forbes <dan@danforbes.dev> * Bump ini from 1.3.5 to 1.3.7 in /website (#801) Bumps [ini](https://github.com/isaacs/ini) from 1.3.5 to 1.3.7. - [Release notes](https://github.com/isaacs/ini/releases) - [Commits](npm/ini@v1.3.5...v1.3.7) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jimmy Chu <jimmychu0807@gmail.com> * Making most static strings in substrate.dev supporting multi-lingual (#802) * adding <translate> tag for multi-lingual support * Most strings are translatable now * Minor fix of bash syntax in console output * fixing spacing * using relative link so multi-lingual option is kept between navigation * cmd style in console output * Indent all JS to use spacing for indentation * Updated to latest libs * Fix site internal links to respect multi-lingual. * Add validate address options I have added the 2 main tools that Parity has created to validate ss58 addresses. I also added a list of places for developers to look for some more code examples to use to create their own custom validation tools in various languages. @joepetrowski @danforbes * Fix line length * Add validate address options I have added the 2 main tools that Parity has created to validate ss58 addresses. I also added a list of places for developers to look for some more code examples to use to create their own custom validation tools in various languages. @joepetrowski @danforbes * Fix: Comment Edits * Update docs/knowledgebase/advanced/ss58-address-format.md Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com> * Update docs/knowledgebase/advanced/ss58-address-format.md Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com> * Update docs/knowledgebase/advanced/ss58-address-format.md Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com> * Update docs/knowledgebase/advanced/ss58-address-format.md Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com> * Update docs/knowledgebase/advanced/ss58-address-format.md Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com> * Review and edits for SS58 General pruning to make text more succinct. Slight tweak on structure and linking to be more consistent with other KB articles. Added code comments too. * Revert "Review and edits for SS58 " This reverts commit 532d283. * Apply uncontroversial suggestions from code review Co-authored-by: sacha-l <sacha@parity.io> * fix github suggestions error * Update docs/knowledgebase/advanced/ss58-address-format.md Co-authored-by: sacha-l <sacha@parity.io> * fix: wording and formatting * Update docs/knowledgebase/advanced/ss58-address-format.md Co-authored-by: Dan Forbes <dan@danforbes.dev> Co-authored-by: Imad Arain <57454832+imadarai@users.noreply.github.com> Co-authored-by: Jimmy Chu <jimmy@parity.io> Co-authored-by: Dan Forbes <dan@parity.io> Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jimmy Chu <jimmychu0807@gmail.com> Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com> Co-authored-by: Sacha <alexander.lansky@gmail.com> Co-authored-by: sacha-l <sacha@parity.io>

refining off-chain indexing

f9c469f

jimmychu0807 requested review from riusricardo and danforbes December 11, 2020 06:20

tomusdrw approved these changes Dec 11, 2020

View reviewed changes

danforbes reviewed Dec 11, 2020

View reviewed changes

Jimmy Chu and others added 3 commits December 12, 2020 12:28

Update docs/knowledgebase/learn-substrate/off-chain-workers.md

1f51d08

Co-authored-by: Dan Forbes <dan@parity.io>

Update docs/knowledgebase/learn-substrate/off-chain-workers.md

5721585

Co-authored-by: Dan Forbes <dan@parity.io>

Expanded the section from off-chain-worker to off-chain-features

22fc8b4

jimmychu0807 requested a review from danforbes December 12, 2020 05:12

tomusdrw approved these changes Dec 14, 2020

View reviewed changes

docs/knowledgebase/learn-substrate/off-chain-features.md Outdated Show resolved Hide resolved

docs/knowledgebase/learn-substrate/off-chain-features.md Outdated Show resolved Hide resolved

Jimmy Chu and others added 6 commits December 14, 2020 19:50

Update docs/knowledgebase/learn-substrate/off-chain-features.md

7a17e27

Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>

update on content to make it more coherent

2d4d338

Merge remote-tracking branch 'gh-sdh/source' into jc/off-chain-indexing

5195668

Fix link

f4e9ac4

Links and nitpicks

f0db1ab

Prettier

0970cb9

danforbes approved these changes Dec 14, 2020

View reviewed changes

jimmychu0807 merged commit 5fbc893 into source Dec 14, 2020

jimmychu0807 deleted the jc/off-chain-indexing branch December 14, 2020 14:16

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refining off-chain indexing #800

refining off-chain indexing #800

jimmychu0807 commented Dec 11, 2020 •

edited

Loading

tomusdrw left a comment

tomusdrw Dec 11, 2020

danforbes Dec 11, 2020 •

edited

Loading

jimmychu0807 Dec 12, 2020

tomusdrw Dec 11, 2020

danforbes Dec 11, 2020

jimmychu0807 Dec 12, 2020

danforbes left a comment

danforbes Dec 11, 2020

danforbes Dec 11, 2020 •

edited

Loading

jimmychu0807 commented Dec 12, 2020

tomusdrw left a comment

danforbes left a comment

refining off-chain indexing #800

refining off-chain indexing #800

Conversation

jimmychu0807 commented Dec 11, 2020 • edited Loading

tomusdrw left a comment

Choose a reason for hiding this comment

tomusdrw Dec 11, 2020

Choose a reason for hiding this comment

danforbes Dec 11, 2020 • edited Loading

Choose a reason for hiding this comment

jimmychu0807 Dec 12, 2020

Choose a reason for hiding this comment

tomusdrw Dec 11, 2020

Choose a reason for hiding this comment

danforbes Dec 11, 2020

Choose a reason for hiding this comment

jimmychu0807 Dec 12, 2020

Choose a reason for hiding this comment

danforbes left a comment

Choose a reason for hiding this comment

danforbes Dec 11, 2020

Choose a reason for hiding this comment

danforbes Dec 11, 2020 • edited Loading

Choose a reason for hiding this comment

jimmychu0807 commented Dec 12, 2020

tomusdrw left a comment

Choose a reason for hiding this comment

danforbes left a comment

Choose a reason for hiding this comment

jimmychu0807 commented Dec 11, 2020 •

edited

Loading

danforbes Dec 11, 2020 •

edited

Loading

danforbes Dec 11, 2020 •

edited

Loading