RPC-JSON over WebSockets #847

cschuchardt88 · 2023-11-23T00:14:59Z

Description will be updated soon....

Some things to know

URL(s) - path and query can be anything. (Note: use OnRequest event to get custom paths/headers and query params)
- ws://127.0.0.1:10340/
- ws://127.0.0.1:10340/?MyCustomParam=HelloWorld
- ws://127.0.0.1:10340/MyCustomPath/FakePage.html?say=Hello%20Bob
Uses Neo.Json
Has expandability of API in other plugins.
Emits events that plugins can hook into for more information on clients.
99% uptime (for clients) (unless neo-core fails)
It's up to clients to filter events
Emits block/transaction/contract events to clients
RPC like functionally over WebSocket

Events and their IDs

public enum WebSocketResponseMessageEvent : byte
{
    Block = 0x02,               // Blockchain.Committed
    MemoryPool = 0x03,          // MemoryPool
    Transaction = 0x05,         // Transaction
    ContractNotify = 0x14,      // ApplicationEngine.Notify
    AppLog = 0x15,              // ApplicationEngine.Log
    DebugLog = 0x16,            // Utility.Log
    Error = 0x19,               // Error(s)
    Method = 0x20,              // Result(s) from a method call
}

Request information

{
    "version": "1.0",       // Protocol version
    "requestid": 999,       // For tracking which response goto which request (-1 for system error)
    "method": "getblock",   // the name of the method to invoke
    "params": [             // parameters for the method
        "0"
    ]
}

config.json

{
  "PluginConfiguration": {
    "Network": 860833102,
    "BindAddress": "127.0.0.1",
    "Port": 10340,
    "SslCertFile": "",
    "SslCertPassword": "",
    "TrustedAuthorities": [],
    "EnableBasicAuthentication": false,
    "User": "",
    "Pass": "",
    "AllowOrigins": [],
    "ConcurrentProxyTimeout": 120,
    "MessageSize": 4096,
    "MaxGasInvoke": 2000000000,
    "DebugMode": false
  }
}

Developers API (all static)

Events

WebSocketServerPlugin.OnRequest(HttpContext httpContext)               // Emits after valid WebSocket connection request is made
WebSocketServerPlugin.OnServerStarted()                                // Emits after WebSocket host is up
WebSocketConnection.OnMessageSent(Guid clientId, JToken json)          // Emits after message is successfully sent to client
WebSocketConnection.OnMessageReceived(Guid clientId, JToken json)      // Emits after a message is received and serialized
WebSocketConnection.OnConnect(Guid clientId)                           // Emits after a successful connection is made
WebSocketConnection.OnDisconnect(Guid clientId)                        // Emits after a successful disconnect

Methods

WebSocketServerPlugin.RegisterMethods(object handler)                                      // Registers methods for a given class with attribute WebSocketMethod
WebSocketServerPlugin.SendAllJson(JToken json, int pluginId)                               // Sends to all clients
WebSocketServerPlugin.SendJson(Guid clientId, int requestId byte eventId, JToken json)     // Send to a specified WebSocket by client id, (note: use "OnConnect" event to get client id)

Properties

WebSocketServerPlugin.HasClients        // Whether the server has valid connections to clients

Creating a method in `Code`

[WebSocketMethod]
private JToken Echo(JArray @params) // turns into all lowercase for method name
{
    return @params;
}
// -----OR-----
[WebSocketMethod(Name = "echo")] // name is case sensitive
private async Task<JToken> EchoAsync(JArray @params) // method would be "echoasync`, if not specified by "Name" in attribute
{
    return @params;
}

Video of Syncing (Blockchain)

Recording.2023-11-25.233845.mp4

Screenshots

Block Event (on commit)

Invoke Response (Echo Method)

Contract Response (on syncing/new block)

Transaction Response (on syncing/new block)

Block Response (on syncing/new block)

Change Log

Added WebSocketServer

closes #716

…entProxyTimeout

roman-khimov · 2023-11-23T06:08:34Z

I think community will love two absolutely different websocket interfaces.

cschuchardt88 · 2023-11-23T06:31:19Z

This would make 3? Because neo-cli node has one built-in too (but thats for syncing blockchain). But this is meant to replace #795. Now are you talking about neo-go's? If that is the case maybe we can build the same way in this PR. What my goal is to allow users to have a fully functioning API that does everything that neo-cli and neo-express does. The reason behind this; is to be able to support other platforms and programming languages along with special use cases. The way that neo-core sits right now... it can't be done easily; if not at all. But with the help of this and other people expanding this API. We can tackle web, dotnet and other ecosystems. This also allows others to make plugins to expand the api for custom in house applications that developers or organizations may need for there only business needs. I think the add-on of this new sidechain; it would be time for ALL developers and users to be able to use neo if they choose too. WebSockets would be the easiest way to start. With web3 most people want their own dapp to be able to get blockchain information and full neo-core functionally. Currently you would have to build your own.

cschuchardt88 · 2023-11-25T22:20:26Z

What this feature offers, is MUST to have. My other reasoning is that with neo3-visual-tracker (vscode-extension) uses 3rd party javascript library (neon-js) for accessing the RPCServer of neo-express. Now this is very buggy as we know. Could be the because of old version. With this websocket API we can easily get new blocks and other information that is needed for this extension. I am currently rebuilding this extension, and need this plugin, or else I'll make it just for neo-express, But i thought the features it offers would be a good choice for community and ecosystem.

added uility log added applicationengine.log added connection check

cschuchardt88 · 2023-12-03T08:30:45Z

@roman-khimov @superboyiii We would like to use this feature in neo-express for the updated neo3-visual-tracker to use (vscode extension). Any ETA on or how or if this feature will be added to core?

shargon · 2024-01-02T12:40:58Z

src/WebSocketServer/WebSocketClient.cs

+{
+    internal class WebSocketClient : IDisposable, IEquatable<WebSocketClient>
+    {
+        public WebSocket Socket { get; init; }


required and remove null checks?

It is not null there. Hints the reason why no ? and init on property. We don't want null.

It is not null there. Hints the reason why no ? and init on property. We don't want null.

Yep, but it's used with ? inside the class

Right and that reason, is for if client disconnect and the class becomes disposed. But you should never pass in a null WebSocket.

I marked the project as nullable and I detected it, I was going to fix it, I think that new projects should be nullable

cschuchardt88 · 2024-01-03T07:32:53Z

@superboyiii If this PR can you on your list ASAP for testing. I need this PR to update the UI for neo-express (vscode extension). The extension is pretty bad at the moment. It unworkable. All the project and templates it generates are useless for any new developer; even those that don't know C#.

cschuchardt88

Please fix @shargon

src/WebSocketServer/WebSocketClient.cs

cschuchardt88 · 2024-01-03T09:37:16Z

src/WebSocketServer/WebSocketConnection.cs

@@ -13,10 +13,9 @@

 namespace Neo.Plugins
 {
-    internal class WebSocketConnection<TClient> : IDictionary<Guid, TClient>, IDisposable
-        where TClient : WebSocketClient, new()


Need to change this back. WebSocketClient is a base class. You can extend the class by inheriting it. For example if you want to add more properties/fields or methods to it. Like a tracking number or Client's login information.

I program dynamically and make things reuse able.

Need to change this back. WebSocketClient is a base class. You can extend the class by inheriting it. For example if you want to add more properties/fields or methods to it. Like a tracking number or Client's login information.

We can extend WebSocketClient, we don't need more types of WebSocketClients, sometimes simpler is better

Yes in theory. But this also limits the API or makes more work in the future. If the code is already there or built for reusability, than the code should stay.

If the code is already there or built for reusability, than the code should stay.

I'm not agree, we don't need two WebSocketClients

This reverts commit 074649f.

shargon

Please set the project as nullable, I tried to change it, but it was reverted one of my commits

src/WebSocketServer/WebSocketServerPlugin.cs

src/WebSocketServer/v1/BlockchainMethods.cs

shargon · 2024-01-03T21:08:19Z

src/WebSocketServer/WebSocketClient.cs

+{
+    internal class WebSocketClient : IDisposable, IEquatable<WebSocketClient>
+    {
+        public WebSocket? Socket { get; init; }


We don't need this with ?, only is needed because you want a T type that we don't need

I don't understand. Referring to T type; we do need it! This a BASE CLASS. NOT DOING SO WILL LIMIT AND FLAW EXPANDABLE OF AND POINT OF THIS LIBRARY. If we want to keep limiting the api we can do that. Just look at RpcServer is outdated and cannot be expanded or updated easily. Which is why no one touches it and in the state it's in now.

vncoelho

remainder to check #859, remove websocket

AnnaShaleva

@shargon, @Jim8y, @vncoelho since the work on this PR is continuing, I'd like to pay your attention one more time to the problem mentioned by @roman-khimov in #847 (comment) and to the comment of @cschuchardt88 in #847 (comment).

The case is that NeoGo node already has WS server that handles all standard RPC methods exactly the same way as regular RPC server does (using exactly the same response structures and without handlers code duplication, see the comment below). Also, NeoGo WS server has subscriptions interface and notifications subsystem implemented for more than 3 years (check out the https://github.com/nspcc-dev/neo-go/blob/master/docs/notifications.md and code connected with https://github.com/nspcc-dev/neo-go/blob/d90169761595933febb8a25d1abde2e46154b96d/pkg/services/rpcsrv/server.go#L265). I know that there are community users of the existing NeoGo WS system (and, in particular, WS notification subsystem), and another WS interface implementation would be a problem for our users. We need a single unified and aligned websocket interface. What do you think?

src/WebSocketServer/v1/BlockchainMethods.cs

shargon · 2024-05-21T08:03:34Z

Repository moved to neo, please re-open there

cschuchardt88 added 8 commits November 22, 2023 04:43

init project websocketserver

0f70f8d

Updated json

e2de661

dotnet format

35d7c70

Removed unused code

983d543

Fixed codeql

528e1ca

Added EnableBasicAuthentication, User, Pass, AllowOrigins and Concurr…

cf968be

…entProxyTimeout

Updated SslCert to SslCertFile

5a76fab

fix codeql again

9410d92

revert *.csproj files configuation

665e4f5

cschuchardt88 added 8 commits November 25, 2023 17:55

code clean up

e5a0e29

added uility log added applicationengine.log added connection check

fixed format

8c8f611

format again

8e1c9f7

one more time

e20e84c

one more (project files) dotnet format dont work on them

ec72689

Add new features

8d10b0c

changed function to static

5df1de0

dotnet format

03015d7

cschuchardt88 mentioned this pull request Nov 28, 2023

Bump browserify-sign from 4.2.1 to 4.2.2 in /extentions/neo3-visual-tracker neo-project/neo-express#382

Merged

Merge branch 'master' into WebSocketServer

5acf717

cschuchardt88 added 6 commits December 3, 2023 06:59

Fixed disconnect issue

2f8d05f

Build out basic blockchain methods for getting data

67e0f91

INIT PROJECT

fc67dee

Fixed MERGE

f21c286

Fixed sln file

4bc67df

Added basic protocol methods for the node information.

2597889

cschuchardt88 mentioned this pull request Dec 3, 2023

Milestones 4.0 neo-project/neo-express#411

Closed

7 tasks

finsished the wallet api

1832fd8

cschuchardt88 marked this pull request as ready for review December 30, 2023 19:35

Update WalletSessionManager.cs

8dcb91e

shargon reviewed Jan 2, 2024

View reviewed changes

shargon added 2 commits January 3, 2024 10:32

Remove TClient and null checks

074649f

Prevent problems with date seasons changes

8927853

cschuchardt88 commented Jan 3, 2024

View reviewed changes

cschuchardt88 added 3 commits January 3, 2024 04:47

Revert "Remove TClient and null checks"

32a867e

This reverts commit 074649f.

Fixed WebSocketClient

1ec7644

Fixed error

ecbb806

shargon requested changes Jan 3, 2024

View reviewed changes

Made project nullable

307d263

cschuchardt88 requested a review from shargon January 3, 2024 18:36

shargon reviewed Jan 3, 2024

View reviewed changes

src/WebSocketServer/WebSocketServerPlugin.cs Outdated Show resolved Hide resolved

src/WebSocketServer/v1/BlockchainMethods.cs Outdated Show resolved Hide resolved

shargon reviewed Jan 3, 2024

View reviewed changes

Update BlockchainMethods.cs

23d9a34

vncoelho reviewed Jan 3, 2024

View reviewed changes

AnnaShaleva reviewed Jan 10, 2024

View reviewed changes

src/WebSocketServer/v1/BlockchainMethods.cs Show resolved Hide resolved

cschuchardt88 marked this pull request as draft January 11, 2024 15:04

cschuchardt88 force-pushed the WebSocketServer branch from 2edeeaf to 23d9a34 Compare January 12, 2024 03:57

cschuchardt88 added 4 commits January 11, 2024 22:58

Merge branch 'master' into WebSocketServer

ac6526f

format

580b864

RPC-JSON added still needs programming

7d8f492

Added event system

35675e2

cschuchardt88 changed the title ~~Web Socket Server~~ RPC-JSON over WebSockets Jan 14, 2024

Jim8y added the Need Active Pr will be closed after one week if no new activity. label Feb 12, 2024

shargon closed this May 21, 2024

Jim8y mentioned this pull request Jul 26, 2024

【Aug.15】Core Dev Regular Meeting Proposal Discussion neo-project/neo#3440

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

RPC-JSON over WebSockets #847

RPC-JSON over WebSockets #847

cschuchardt88 commented Nov 23, 2023 •

edited

Loading

roman-khimov commented Nov 23, 2023

cschuchardt88 commented Nov 23, 2023 •

edited

Loading

cschuchardt88 commented Nov 25, 2023 •

edited

Loading

cschuchardt88 commented Dec 3, 2023

shargon Jan 2, 2024

cschuchardt88 Jan 2, 2024

shargon Jan 3, 2024

cschuchardt88 Jan 3, 2024 •

edited

Loading

shargon Jan 3, 2024

cschuchardt88 commented Jan 3, 2024

cschuchardt88 left a comment •

edited

Loading

cschuchardt88 Jan 3, 2024

cschuchardt88 Jan 3, 2024 •

edited

Loading

shargon Jan 3, 2024

cschuchardt88 Jan 3, 2024

shargon Jan 3, 2024

shargon left a comment •

edited

Loading

shargon Jan 3, 2024

cschuchardt88 Jan 3, 2024 •

edited

Loading

vncoelho left a comment •

edited

Loading

AnnaShaleva left a comment

shargon commented May 21, 2024

RPC-JSON over WebSockets #847

RPC-JSON over WebSockets #847

Conversation

cschuchardt88 commented Nov 23, 2023 • edited Loading

Some things to know

Developers API (all static)

Events

Methods

Properties

Creating a method in Code

Video of Syncing (Blockchain)

Screenshots

roman-khimov commented Nov 23, 2023

cschuchardt88 commented Nov 23, 2023 • edited Loading

cschuchardt88 commented Nov 25, 2023 • edited Loading

cschuchardt88 commented Dec 3, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cschuchardt88 Jan 3, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cschuchardt88 commented Jan 3, 2024

cschuchardt88 left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cschuchardt88 Jan 3, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shargon left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cschuchardt88 Jan 3, 2024 • edited Loading

Choose a reason for hiding this comment

vncoelho left a comment • edited Loading

Choose a reason for hiding this comment

AnnaShaleva left a comment

Choose a reason for hiding this comment

shargon commented May 21, 2024

cschuchardt88 commented Nov 23, 2023 •

edited

Loading

Creating a method in `Code`

cschuchardt88 commented Nov 23, 2023 •

edited

Loading

cschuchardt88 commented Nov 25, 2023 •

edited

Loading

cschuchardt88 Jan 3, 2024 •

edited

Loading

cschuchardt88 left a comment •

edited

Loading

cschuchardt88 Jan 3, 2024 •

edited

Loading

shargon left a comment •

edited

Loading

cschuchardt88 Jan 3, 2024 •

edited

Loading

vncoelho left a comment •

edited

Loading