ApplicationLog: New storage implementation for expandability

[cschuchardt88]

With this new implementation of "ApplicationLog" you will be easier to expand the "ApplicationLog" for use in neo-node-cli and RPC. Tested with my xunit tests locally.

Note that you need to resync the whole blockchain for this new version of "ApplicationLog"

Change Log

• New Storage implementation
• Added new console commands to neo-cli
• Added lookups by block, transaction and contract in neo-cli
• Added Sort methods for by event name

Storage Size comparison to blockchain

Sizes	BlockChain	Application
2.30GB	Testnet T5	Blockchain
2.41GB	Testnet T5	ApplicationLog

Now you can use ApplicationLog in neo-cli here are the commands

Output of Command Console for ApplicationLog

[cschuchardt88]

@dotnet-policy-service agree

[Jim8y]

what vm in your command means?

[cschuchardt88]

@Liaojinghui means execution of the vm and event and/or notifylog

[Jim8y]

vm is not an intuitive name, how about log. And please update the format with dotnet format

[cschuchardt88]

@Liaojinghui ok made the changes you requested. I'm not good with names, so thank you.

Edit: if i were to add "GetContractLog" or update "GetApplicationLog" to allow scripthashes for RPC Methods. Is that a bad thing to do? reason being @roman-khimov told me not to change schema. also if its ok to do that, would i have to update "RpcClient" and create the tests? Or is that someone else's job?

[Jim8y]

Change existing interface should be something to avoid as much as we can, it might cause inconsistancy and break applications that rely on it. and you can update rpcclient directly, no need to wait for others.

[shargon]

@superboyiii could you test it?

[cschuchardt88]

Should issue #665 be added to this PR?

[roman-khimov]

It's a very different problem and problems better be solved one by one.

BTW, getnotificationlog worries me here. I'm OK with just about any internal storage changes for existing APIs, but a new API needs to be discussed additionally. Not that it's bad, I'm just not sure it's very useful (we've worked for a long long time without it) and it requires some additional storage.

[cschuchardt88]

I figured it would be nice addition to add since we have the information. The reason for me adding it is, some people including me just want to get events on the blockchain for a certain contract and we don't know the block hash or index or even a transaction hash. I have jobs running that look for certain events on my contracts to fire than an action to do something i.e. an external application. I can remove it if need be. But instead of adding other plugins with similar storage data requirements. I added it in here. Most plugins don't have external API besides RPC which is limited. No sense in storing the same data again.

edit: plus i want to make the cli command line not be limited in anyway and require users to use a blockchain explorer or have to use Rcp (bad for non-technical users).

[superboyiii]

Very great feature. I will test it on testnet first!

[Jim8y]

Very great feature. I will test it on testnet first!

@superboyiii Any update?

[superboyiii]

@cschuchardt88 How about supporting log block <blockhashOrIndex> [eventName=null], this could be more convenient?

[cschuchardt88]

Currently the store i created doesn't not store Indexes, But i guess i could query hash than use the hash for my store. I'll look into it.

[superboyiii]

Currently the store i created doesn't not store Indexes, But i guess i could query hash than use the hash for my store. I'll look into it.

Maybe just like

neo-modules/src/RpcServer/RpcServer.Blockchain.cs

Line 82 in 2d7fb27

return NativeContract.Ledger.GetBlockHash(snapshot, height).ToString();

to get BlockHash?

[Jim8y]

@shargon @roman-khimov

[roman-khimov]

@Liaojinghui, #807 (comment) And we've only got worse wrt this. Each logical change should be tracked and evaluated separately. Right now we have too many of them here to me. I'd start with storage schema change (that is strictly something not affecting any of external interfaces, pure optimization), then any addons (and CLI better be split from RPC ones). It'd be much easier to review, track and communicate to the outside world then (docs/changelogs).

[cschuchardt88]

@superboyiii for #807 (comment) is commenting the code out ok? or you want it removed completely?

Edit:

Nevermind i removed the code. Ill create new PR for RPC Methods after this PR is merged.

[cschuchardt88]

LogEventArgs are now working

From Testnet T5

{
    "jsonrpc": "2.0",
    "id": 1,
    "result": {
        "txid": "0x49e493cee3798bd0e6f2e745ac75291534b290fa4033ac36bed9204d429ead4c",
        "executions": [
            {
                "trigger": "Application",
                "vmstate": "HALT",
                "exception": null,
                "gasconsumed": "264528",
                "stack": [
                    {
                        "type": "ByteString",
                        "value": "SGVsbG8sIGJvYg=="
                    }
                ],
                "notifications": [],
                "logs": [
                    {
                        "contract": "0xa6cd0a163167fc37b24cc3b226342474e853a30a",
                        "message": "Hello, bob"
                    }
                ]
            }
        ]
    }
}

[cschuchardt88]

@superboyiii @Liaojinghui maybe you can help me with a problem. Trying to get all LogEventArgs for a transaction. But I get duplicates because the node is sending that transaction and than processing the transaction again once processed by the blockchain. Inturn emits LogEventArgs twice. Maybe an Issue to add LogEventArgs to Blockchain.ApplicationExecuted?

[Jim8y]

I have considered this issue, but solution would change the core, so i suggest to keep it this way for a while. We can fix it after the monorrpo merge.

[cschuchardt88]

Let me know when the new release of neo-core is here in this repo so i can update

Now changed to public static byte[] Serialize(StackItem item, ExecutionEngineLimits limits) in neo-project/neo#2948

[superboyiii]

Tested. It's good for me now.

[Jim8y]

@shargon

[superboyiii]

Could you have a look? @shargon

[shargon]

Could you have a look? @shargon

Yes, but I need more time

[cschuchardt88]

Any update @shargon on this?

[shargon]

Deserialize content always come from a Serialize output?

[cschuchardt88]

I'm overriding the base class of NotifyLogState in turn inherits ISerialize

[cschuchardt88]

We initially create it 1st than serialize it, Hints the Create static method, above. But to answer question yes.

[cschuchardt88]

@shargon any news?

[shargon]

I need more time sorry