Berty's dSocial Builder Journey

[jefft0]

This documents the continuing journey of Berty into Gno.land. This follows from our previous journey to create Gno Native Kit (formerly called GnoMobile).

The goal of dSocial is to create a test bed for building decentralized social media type of applications and experimenting with smart contract integrations around social media features. This test bed is not intended to be a polished end-user app but to experiment and stress test technical issues in using Gno.land related to implementing decentralized social apps.

The first version of dSocial will include the basic Twitter-like features like: post, react, notification, mentions, etc. dSocial will be built for mobile using the Gno Native Kit framework, but we will also include a minimal desktop app and a read-only web version.

Additional R&D issues we will investigate are:

• integrating a DAO moderation feature
• implementing a notification system using the “notifs core” code that is under development by the Gno core team
• studying the scalability of smart contracts and the potential solutions involving Merkel trees

You're invited to follow our Journey where we will document our experience using Gno, writing realm code, and track our issues and R&D results. 🚀

[jefft0]

Contributions to gnodev and gnofaucet

To work on GnoSocial, we have gotten started with the development cycle using gnodev and gnofaucet . We have opened the following PRs with contributions which are making our work smoother and hopefully for others too.

• When Iuri tried to use gnofaucet, it silently failed to start. This little PR adds the code to print the error messages (which was very helpful) chore: HTTP server error handling gno#1494 (Merged) Note: Even though this fixes the gnofaucet in the main repo, we are now using the new faucet at https://github.com/gnolang/faucet .
• We (previously) used gnofaucet with gnodev and restart gnodev, then gnofaucet stops working as explained in this PR that we opened to solve the problem. fix(gnofaucet): fix transactions by querying account number and sequence each time gno#1493 (Merged). Note: gnofaucet is deprecated but until it's removed this will fix the problem.
• gnodev starts a web server and the realm pages have nice help messages with the command to call the realm functions. This little PR updates gnodev to properly display the chain-id and remote address. Now we can copy/paste as intended. chore: In gnodev serveGnoWebServer, set webConfig HelpChainID and HelpRemote gno#1497 (Merged)
• The above PR depends on another little PR so that the web server properly displays the chain-id and remote address. fix: In gnoweb realm_help.html, update to Data.Config.HelpRemote and Data.Config.HelpChainID gno#1496 (Merged)

More to come for sure!

[jefft0]

More contributions to gnodev

In working with gnodev, we ran across an easy and a hard problem.

• The easy problem to solve was getting the correct current time from time.Now(), explained here gnodev time.Now() drifting away from real time gno#1507 and fixed in PR fix(gnodev): time drifting in gnodev gno#1510 (Merged)
• The hard problem is well-known but there was no issue for it yet so we made one gnodev time.Now() values are not preserved on reload gno#1509 . When transactions are "replayed", the result of time.Now() is the time of replay, not the time of the original transaction. This has the effect of erasing all timestamps. It happens in gnodev but also in other places where transactions are replayed such to migrate to a new testnet. You can see that all the timestamps are the same on the message board for testnet3. https://test3.gno.land/r/demo/boards:testboard . There are already plans to fix this in [portal loop] inject sdk.context when replaying transactions. gno#1511 .

[jefft0]

Contributions to r/demo/boards

GnoSocial is a modification of r/demo/boards . As we were developing we found some bugs which we fixed in GnoSocial, so we contributed some PRs to fix the same in r/demo/boards.

• The Render display of a message lets you click on the message creator's name to open the user details. The link was broken which we fixed in fix(r/demo/boards): update the user link in displayAddressMD to correct path gno#1525 (Merged)
• We implemented message reposting in GnoSocial in a similar way to r/demo/boards which was not showing the reposted messages in the Render page. We present one possible solution in fix(r/demo/boards): correctly render reposts gno#1530 (Merged)

[jefft0]

Contribution: Add Run to gnoclient

GnoSocial uses Gno Native Kit which is a thin wrapper on top of the gnoclient API. This already has Call to call a realm function. But for stress testing in GnoSocial we need to use ad-hoc scripts which run on the node and do multiple function calls. This was one of the purposes of the command-line gnokey maketx run. This pull request adds Run to gnoclient so we can do this programmatically. gnolang/gno#1574 (Merged)

[jefft0]

Stress testing app

Thanks to the Gno devs merging the PRs mentioned above, Gno Native Kit can now use the Run command to call realm functions repeatedly. We made a PR to use Run and gnodev to stress test storing thousands of messages. See the PR for preliminary results and near-term recommendations. Eventually, apps like this can be part of the automated testing of the Gno code. gnolang/gno#1583 (1 approval)

Some of the recommendations are done by this gnodev PR. gnolang/gno#1622 (Merged)

[jefft0]

Contribution: Improve error message in calling realm function

One of our devs lost time trying to understand an uninformative error message when calling a realm function with the wrong number of arguments. This will be a common problem for other devs. We made a small PR for the VM to check the number of arguments and give time-saving informative error messages. gnolang/gno#1610 (Merged)

[jefft0]

Contribution: Add Send to gnoclient

The gnoclient API already had support for MsgCall. Above, we mentioned how we added support for MsgRun. Here, we made a PR to add MsgSend to send coins from one account to another. We will be using this in the GnoSocial demo app. gnolang/gno#1639 (Merged)

[jefft0]

Stress testing with on-disk storage

With the help of @gfanton to show where to increase the maximum allowed gas in gnoland, it can be used for stress testing where hundreds of posts are added to the boards realm in each transaction. gnoland stores the data on disk, not in memory like gnodev currently does. So we wanted to stress test adding one million posts. Full details are in the stress test PR. In short, gnoland performed well. Transaction times remained constant and memory usage was reasonable.

[iuricmp]

Exploring the Onboarding Experience

We have explored some approaches to onboard users on the GnoSocial dApp. After some discussions, we've decided to keep using the Faucet Strategy. Even though we know that strategy is not suitable for the Mainet launch, we agreed to keep with it since we don't have any other alternative for now.
You can check out a quick demo about the Faucet Strategy here.

References:
Jae Kwon - gno demo at 25:20 How do you solve the onboarding with gas fees, or even like social media with gas fees?

[jefft0]

Contribution: p/demo/avlhelpers package and ListUsersByPrefix

In GnoSocial, we needed a utility to search the r/demo/users usernames by prefix. This is a useful tool for the avl.Tree, so after some discussion, we decided to make a new package p/demo/avlhelpers with a helper function ListKeysByPrefix. (In general, Gno devs are encouraged to write general packages.) Then the r/demo/users function ListUsersByPrefix simply calls it. gnolang/gno#1708 (Merged)

[jefft0]

Bug report: VM hangs during Render

While developing GnoSocial, we discovered a bug where the GnoVM hangs during the Render function. (This is in line with a goal of the GnoSocial project to stress test Gno.) Here is the PR with a txtar script which reproduces the error:

• test(bug): txtar to reproduce the VM hanging during Render gno#1736 (1 approval)

Here is the bug report (Reviewing). When there is a PR to implement the solution, we'll update this post.

[jefft0]

Progress: Solutions for scalability

As we add features to GnoSocial we're exploring interesting solutions for scalability, as explained below.

• New GnoSocial UI features:
  • Home feed including followed posts
  • Lazy loading/scrolling. There could be thousands of messages, so the mobile app fetches only a few at a time to limit processing and bandwidth.
  • Front end search for username, back end get by prefix. There could be thousands of usernames, so need to return partial match of the search results to limit processing and bandwith.
• GnoSocial persistent home feed. The home feed data structure is large, and the realm can't compute it in one QEval (with no gas). Need to use gas to persist the results, so we do this in a separate "refresh" call which can be done optionally and occasionally.
• Indexer: The mobile app should keep a local cache of some frequently-used data, like user profile pictures. The data could be changed on the blockchain, so need to notify the mobile app to fetch the changes. We set up the tx-indexer on our Berty test node and plan to use it to send these change notifications. We will also explore using the indexer for more flexible searches for usernames and messages. More details soon!

[iuricmp]

Progress and Video: Mobile app interface improvements

• New GnoSocial interface features:

  • Follow / Unfollow
  • Local Cache: Due to the limitation of a blockchain to enrich data (combining data from different AVL Trees), it became necessary to refresh data incrementally. In a real application, this approach would be executed by an indexer.
  • Reply: Allows users to reply to a post.

• Next steps:

  • We want to continue exploring UI/UX features by simplifying the development complexity.
  • GraphQL client on mobile to receive push notifications for updating the local cache of user data.

Here's a short video that shows the UI/UX parts we're working on:

https://www.loom.com/share/621f151459b040b0a2e6a22adf96b371

[costinberty]

Announcement: Renaming of GnoSocial to dSocial

We started the GnoSocial project to experiment with ways to implement social media features in Gno.land. The idea was to see what would be the main challenges to implement social media functionality using this decentralized tech and test various approaches to this.

Some of the basic features are already here: create/delete account, post message and reply to message. And others are on the way: react to post, profile pic and hopefully more complex stuff like notifications or even integrations are cooking in the background.

Recently, to align with the Gno naming policy, we decided to rename the app from GnoSocial to dSocial. So, our new repo is https://github.com/gnolang/dsocial . For the record, we will leave "GnoSocial" in the previous posts and videos of this Journey, but from now on we will use dSocial everywhere.

Oh, and one more thing: we plan to bring the app to Test Flight and Google Play Console very soon. Access will be invite only so if you're interested in playing with it, let us know and we'll set you up with a test account.

[D4ryl00]

Contribution and Video: Indexer services and Docker support

dSocial needs some services available online, e.g. gnodev, faucet, tx-indexer.
To be able to deploy some Gno services on our Berty server, we contribute to add gnodev in the gno's Dockerfile: gnolang/gno#2009 (closed to open a new PR)
You can watch this video that shows how we manage the different services together with Docker: https://www.loom.com/share/5395c039c0b24118bee56ae5edd7a224?sid=30b17f0e-53ea-4f39-8f0e-1b8b3c150eee

[jefft0]

Contribution: Regression test for importing "time" package

While working on dSocial, we discovered that it doesn't load with addpkg. An investigation showed that a recent PR to improve error checking on loading a package produces an error when importing the "time" package. We made a PR with a simple txtar test to show it, gnolang/gno#2111 . This PR was merged as a regression test after the fix in gnolang/gno#2105 was merged.

[iuricmp]

Demo testing available on TestFlight (iOS) and Google Play (Android)

Hi there.
We are happy to share that the dSocial mobile App was released on Apple (TestFlight) and Android Play (Internal Test).
As you may know, this app uses GnoNative npm SDK (@gnolang/gnonative) that makes building on mobile for Gno easy peasy.
We also used the opportunity to create and improve many pieces on the backend side. You can check more details on our journey entries.
If you want to try, please, send a dm with your email to me (Iuri) on Signal. Please say if you have an iPhone or Android phone.

[costinberty]

R&D Report: Scalability tests

Hi everyone,

We consolidated the scalability tests finds around avl.Tree resource consumption, tx time, gas limits and more. Check it out here #67

[costinberty]

"What's on your mind?"

You can now tell us on dSocial. Create Account, Follow, Post, Repost, Reply to a Post features are here (Reply to a post is affected by a bug, but it will work again soon).

We are working to add "React to a post" and "Notifications" features.

Here's how you can test dSocial:

• iOS: follow the instructions of this link. You will need to install Testflight on your iPhone
• Android: send your email address via Signal to our colleague Iuri, using this link: and you will receive install instruction.

Oh, and don't forget that you will need funds to make transactions (posts, cat-posts, replies, etc.) so after you sign-up you'll have to click on "Onboard user" to receive gnots.

After that you'll be able to cat-post

[jefft0]

Contribution: Demos of possible Keybase design issues

Since dSocial (and Gno Native Kit) use the Keybase API, we are interested in how the design of this API enforces bindings between a key address and a username. We identified possible design issues and opened PRs to demonstrate the issues. The PRs are a place to discuss the design and possible solutions.

• fix: In dbKeybase.writeInfo, if replacing a name entry, remove the lookup by the old address gno#2685 (Merged)
• test(bug): In TestExportImport, delete imported key, check if original exists gno#2472 (Reviewing)
• test(bug): In TestDeleteByAddress, delete by address, check deletion of all keys with that address gno#2476 (Reviewing)
• test(bug): In TestAdvancedKeyManagement, check if can get the imported key by address gno#2718 (Reviewing)

[costinberty]

New features are now available on dSocial:

1. ⁠Change your avatar profile image, storing the image on-chain by implementing the Zack's Tinkerer Journey #2 approach of this proposal: r/demo/profile standard for Avatar image gno#2598
2. Account keys and usernames are now synchronised between the local keystore and r/demo/users on the blockchain. Registering usernames in r/demo/users is becoming increasingly important, including for realm code namespaces. And r/demo/users helps maintain account consistency across multiple apps and devices.
3. ⁠We have a new log retrieval tool that will help with the debugging and share logs (looks pretty cool too).
4. Better error catch on Android chore: exposing ErrCode class gnonative#158

If you haven't already, we super encourage you to try dSocial yourselves. All you need to do is

•⁠ ⁠iOS: follow the instructions of this link. You will need to install Testflight on your iPhone
•⁠ ⁠Android: send your email address via Signal to our colleague Iuri, using this link: and you will receive install instructions.