Development plan for the neo core (C#)

[Jim8y]

Core Development (coredev):

1. Set up a blockchain technical committee - The technical committee responsible for the technical direction of the NEO blockchain.
2. NEO Specification (Yellow Paper) - Standardization of the NEO blockchain through the NEO specification.
3. New Incentive Plan for Core Developers - An incentive plan to encourage core developers to contribute to the NEO blockchain.
4. New Core Developers Working Protocol (Mandatory Vote, Exit Mechanism) - A working protocol to ensure the quality of the NEO blockchain.
5. Hiring Full-time Core Developers - Full-time core developers responsible for the development of the NEO blockchain.
6. Reconstruct NEO core into a single solution (single binary to run everything) - A single solution to simplify the development and deployment of the NEO blockchain.
7. Add State Root to the Header - Introducing a state root in the header to simplify state verification of the NEO blockchain.
8. Native Contract Update - Updating native contracts to enhance the performance of the NEO blockchain.
9. Resync Issue - Addressing resync issues to enhance the performance of the NEO blockchain.
10. Add Benchmark to the NEO VM - Incorporating benchmark testing to measure the performance of the NEO blockchain.
11. Add Unit Test Cases for Edge Situations - Ensuring the quality of the NEO blockchain with unit test cases for edge situations.
12. Integrated Tests - Conducting integrated tests to ensure the quality of the NEO blockchain.
13. Add Sample Contracts, Documents, and Tutorials - Providing sample contracts, documentation, and tutorials to assist developers in creating smart contracts on the NEO blockchain.
14. User Experience Improvements (from lock9):
    • UInt160 Revision: Standardize the UInt160 representation across all platforms by defining a clear encoding and endianness guideline to avoid ambiguity, with the proposal to use addresses for simplicity.
    • Review of Older Issues and PRs: Go through over 300 open issues and PRs across different repositories to address the feeling of abandonment and ensure that important feedback is not missed.
    • Facilitate Testnet Smart-Contract Deployments: Improve the faucet system to allow developers more frequent test deployments, as the current limit hampers testing capabilities.
    • Standard Interface for CLIs and SDKs: Create and define a standard interface for Command Line Interfaces and Software Development Kits to enable the creation of consistent and comprehensive documentation.
15. Checkpoint - Establishing checkpoints to enhance the performance of the NEO blockchain.
16. Docker and Docker Compose - Utilizing Docker and Docker Compose to simplify the deployment of the NEO blockchain.
17. HSM Support - Improving the security of the NEO blockchain with Hardware Security Module (HSM) support.
18. TEE Support - Enhancing the security of the NEO blockchain through Trusted Execution Environment (TEE) support.
19. Notary Service - Improving the performance of the NEO blockchain with notary services.
20. Abstract Account - Enhancing the performance of the NEO blockchain with abstract accounts.

Service Layer (service layer):

1. Service Layer Specification - Defining and standardizing the functionalities and protocols of the NEO service layer.
2. Notary Service - Providing a service to ensure the fairness and legality of transactions on the NEO blockchain.
3. Oracle Service - Offering various types of external data sources for the NEO blockchain, enabling smart contracts to utilize real-world information.
4. Event Service - Providing an event trigger service for NEO smart contracts to automate execution and interaction.
5. Full Node Service - Supplying full node operation and state information services for the NEO blockchain to ensure network stability and data accessibility.
6. Fair Ordering Service - Ensuring fair ordering of transactions on the NEO network to prevent manipulation of transaction sequences.
7. Abstract Account Service - Providing an abstract account mechanism for more flexible and advanced identity and permission management.
8. Random Number Service - Offering a secure random number generation mechanism for applications such as smart contracts to ensure security and unpredictability.

[lock9]

Hello,

I'm not sure this is the best place to put this because the title is aimed at core development only. However, I have some stuff here that we may want to include:

User Experience Improvements:

• UInt160 Revision: Standardize the UInt160 representation across all platforms by defining a clear encoding and endianness guideline to avoid ambiguity. The easiest solution is to use addresses.
• Review older issues and PRs. There are over 300 open issues and prs across different repositories. This creates a feeling of abandonment.
• Facilitate testnet smart-contract deployments. It demands a faucet, and the amount is only enough to deploy the contract 4 times every 24 hours.
• Create/define a standard interface for CLIs and SDKs. The lack of consistency prevents us from creating 'decent' documentation.

[shargon]

I think we should add only tasks that have already been agreed upon, NotaryService, ServiceLayer for example are still under discussion

[roman-khimov]

Set up a blockchain technical committee

Technical Steering Committee, OK. https://governance.neo.org is a good starting point for it. Where is it discussed, what are the options, who is invited?

NEO Specification (Yellow Paper)

OK

New Incentive Plan for Core Developers

Where is it discussed, what are the options, who's making the decision?

New Core Developers Working Protocol (Mandatory Vote, Exit Mechanism)

Where is it discussed, what are the options, who's making the decision?

Hiring Full-time Core Developers

We need more details here, "development of the NEO blockchain" is a very broad topic. NeoFS is a part of the Neo blockchain, so whole NSPCC does that full-time.

Reconstruct NEO core into a single solution (single binary to run everything)

#2944, OK.

Add State Root to the Header

#1526 with block version 1, yay.

Native Contract Update

OK, we're working on it already. #2942 and related.

Resync Issue

In fact it's directly tied to state root problem. Once we're to have state root in headers we'll have to keep compatibility with them and resyncs won't be needed.

Add Benchmark to the NEO VM

1. Neo VM itself works fine, it's just not a bottleneck.
2. If it is, throw away the GC part of it and it won't be that again.
3. We have https://github.com/nspcc-dev/neo-bench/ for the whole system and it doesn't care which node implementation it tests.
4. So I'd say there is not a lot to do here (except throwing away VM GC)

Add Unit Test Cases for Edge Situations

Sure and it's related to specification.

Integrated Tests

We need to discuss this, we have a lot of tests in NeoGo (many of which use our testing framework). It'd be nice to have more generic tests that can be reused by different implementation, like we have for VM.

Add Sample Contracts, Documents, and Tutorials

More specifics is needed here, it's not like we have nothing.

Checkpoint

Implemented in NeoGo as ability to reset to some older state, can be reimplemented in C# (requires complete state history though).

Docker and Docker Compose

Not sure what you mean here, please create a more detailed issue.

HSM Support

I doubt it's a priority. To me it's more a niche thing.

TEE Support

Also seems to be a niche thing. Not that it's bad, but we have more important problems likely.

Notary Service

No doubt about it. Let's have a link to #1573 and remember that with #2905 it should be possible to process a notary-enabled chain (like NeoFS sidechain) with C# node.

Abstract Account

#2917 and associated, yes.

Service Layer Specification

To me service is a role, roles can be extended, easy as that. Then each service has some specifics.

Notary Service

Sure.

Oracle Service

We have some already.

Event Service

#2817 and all associated. OK.

Full Node Service

What's that?

Fair Ordering Service

Not a priority to me. Neo has a governance model where bad nodes can be voted out, so while this can be an improvement, it's not something that Neo desperately needs.

Abstract Account Service

Same #2917 thing.

Random Number Service

OK

I think the list is a bit more abstract than I'd like it to be. And there are very different things there, some are organizational, some are technical, but need more research, some just need to be implemented.

From my perspective, there are very important (affecting whole ecosystem) things missing there like neo-project/proposals#160 or neo-project/proposals#156. And problems like #1891 or #2346, they're not big, but they're still deficiencies of the protocol. A lot of other smaller ones, of course, but this can be discussed.

[Jim8y]

I think we should add only tasks that have already been agreed upon, NotaryService, ServiceLayer for example are still under discussion

@shargon Firstly we should respect the vote from the communities. Then, if something is not suitable, we can further discuss, but now, we need a plan, otherwise, there will be endless debate like past.

[Jim8y]

Hello,

I'm not sure this is the best place to put this because the title is aimed at core development only. However, I have some stuff here that we may want to include:

User Experience Improvements:

• UInt160 Revision: Standardize the UInt160 representation across all platforms by defining a clear encoding and endianness guideline to avoid ambiguity. The easiest solution is to use addresses.
• Review older issues and PRs. There are over 300 open issues and prs across different repositories. This creates a feeling of abandonment.
• Facilitate testnet smart-contract deployments. It demands a faucet, and the amount is only enough to deploy the contract 4 times every 24 hours.
• Create/define a standard interface for CLIs and SDKs. The lack of consistency prevents us from creating 'decent' documentation.

Definately, will update the draft to make it more detailed.

[Jim8y]

@roman-khimov items in the list has no priorities, some voted, some mentioned, some might just be there to discuss.

• HSM and TEE is for node security, attack happend to neo because of lacking protection on private key.
• oracle service contains more types service than existing oracle, data feed, data stream, low latency data, extra.
• Fair ordering is anti-MEV service.
• docker and docker compose, well, just like neogo and other projects, docker images and docker compose to setup the network.
• vm benchmark is necessary since we dont know its performance, we dont know how much overhead our patch will introduce, we just update it again and again withtout knowing whether it will cause OOM or DOS.
• Integrated Tests as is required by the community, may copy from neogo, but we need it for C#.
• New Incentive Plan for Core Developers. Not up to us, we just know there will be a new one, but not an open discussion (yet?).

Most importantly, I know neogo has it, so I put it here to wish C# has it. Dont worry, I will copy everything from neogo.

[ixje]

New Core Developers Working Protocol (Mandatory Vote, Exit Mechanism) - A working protocol to ensure the quality of the NEO blockchain.

Is cleaning up the list of people in the organisation part of this? Because there are a handful of people listed there that have not been active for at least 2 years or more.

[Jim8y]

New Core Developers Working Protocol (Mandatory Vote, Exit Mechanism) - A working protocol to ensure the quality of the NEO blockchain.

Is cleaning up the list of people in the organisation part of this? Because there are a handful of people listed there that have not been active for at least 2 years or more.

@ixje This will be decided by NGD @steven1227 , we only focus on maintaining the core here.

[cschuchardt88]

Bug fixes should be number one priority over new features. Also they should come out once every two weeks to a month. HotFixes once a week to two weeks. I been waiting for #2899 #2890 to be fixed and release for over 4 months now and still waiting. Not to mention this bug should be a hotfix and have priority over new features that are still being built, than ignored. Thx @shargon for fixing 😄. I can't imagine how many peoples apps broke. This bug was with NativeContract.ListContracts crashing on a special contract.

So to recap Bug Fixes should be getting released as soon as possible depending upon the severity.

[Jim8y]

Bug fixes should be number one priority over new features. Also they should come out once every two weeks to a month. HotFixes once a week to two weeks. I been waiting for #2899 #2890 to be fixed and release for over 4 months now and still waiting. Not to mention this bug should be a hotfix and have priority over new features that are still being built, than ignored. Thx @shargon for fixing 😄. I can't imagine how many peoples apps broke. This bug was with NativeContract.ListContracts crashing on a special contract.

So to recap Bug Fixes should be getting released as soon as possible depending upon the severity.

We have decided to freeze adding new features but focus on fixing existing issues and bugs. There was a reason why things took so long to be fixed, as no one works full time for neo to maintaining the core, we are all part time, but we have agreed to change.