Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Blockchain updates for L2 extension #1430

Open
wants to merge 71 commits into
base: master
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
63b159c
Initial draft
esuwu Jun 14, 2024
69e9dbd
Changed a comment
esuwu Jun 14, 2024
b35bea1
moved a file
esuwu Jun 14, 2024
27494ad
rename a folder
esuwu Jun 14, 2024
983babc
Sent the updates channel into block appplier
esuwu Jun 20, 2024
5272899
Added some design comments
esuwu Jun 20, 2024
4fef6e3
Context in the updates function
esuwu Jun 20, 2024
dcd18f3
Merge branch 'master' into node-updates-plugin-l2
esuwu Jul 1, 2024
3cc00d5
Added protobuf
esuwu Jul 1, 2024
6752f8a
Moved proto, added conversion
esuwu Jul 16, 2024
bdc8854
Added state comparison
esuwu Jul 19, 2024
19cc302
Added a draft for retrieving history entries
esuwu Jul 22, 2024
b6fb33c
Renamed a function
esuwu Jul 22, 2024
dca31ee
Fixed most of the linter issues
esuwu Aug 1, 2024
f524fe9
Merge branch 'master' into node-updates-plugin-l2
esuwu Aug 1, 2024
18eab8c
Added paging
esuwu Aug 9, 2024
e2da71e
Added changes detector
esuwu Aug 28, 2024
da48829
Fixed a bug
esuwu Sep 12, 2024
b9318c2
Renamed functions
esuwu Sep 12, 2024
1e91dac
Merged from main
esuwu Sep 12, 2024
4dd8e5a
Merged from master
esuwu Sep 12, 2024
59f75f3
Started writing the filter
esuwu Sep 17, 2024
3a0ce55
Merged from master
esuwu Sep 17, 2024
7b77774
Commented an unused function
esuwu Sep 17, 2024
4d9419f
Added a basic filter
esuwu Oct 3, 2024
6338578
Fixed filtering
esuwu Oct 3, 2024
3d2bf74
Added tests
esuwu Oct 16, 2024
9fe8ed9
Added tests and parameters for nats subscriber
esuwu Oct 16, 2024
e057d2a
Fixed tests and linter errors
esuwu Oct 24, 2024
b2a461c
Merged from master
esuwu Oct 24, 2024
860b830
fixed gosec issues
esuwu Oct 24, 2024
ec15106
fixed one more gosec issue
esuwu Oct 24, 2024
8f70624
Mod clean.
nickeskov Nov 6, 2024
c0652df
Merge branch 'master' into node-updates-plugin-l2
nickeskov Nov 6, 2024
02f82ef
Fix lint for 'runNode' function.
nickeskov Nov 6, 2024
bb3bd64
Fix gosec issues.
nickeskov Nov 6, 2024
47320cf
Deleted stale go protobuf generated code.
nickeskov Nov 7, 2024
fb56f6b
Make some types and functions package private.
nickeskov Nov 7, 2024
89ffaf2
Update 'go.mod' go version.
nickeskov Nov 7, 2024
28086dc
Change location of 'blockchaininfo_test.go'.
nickeskov Nov 7, 2024
2f8c3c0
Add new method 'PartialBlockHeader' for 'ProtobufConverter'.
nickeskov Nov 7, 2024
5f1010c
Refactor 'BUpdatesInfoFromProto' function with unused functions removal.
nickeskov Nov 7, 2024
7dc0748
Refactored a bit 'L2ContractDataEntriesFromProto'.
nickeskov Nov 7, 2024
42bf0ff
Merge branch 'master' into node-updates-plugin-l2
nickeskov Nov 11, 2024
3f9bb83
Added a parse function
esuwu Nov 12, 2024
5d37195
Merge branch 'master' into node-updates-plugin-l2
nickeskov Nov 18, 2024
cf23bdf
Set 'NoSigs' option for nats sever to true.
nickeskov Nov 18, 2024
d7dcd5e
Fix 'runPublisher' in case when updates channel is closed.
nickeskov Nov 18, 2024
d13b940
Add nats sever shutdown.
nickeskov Nov 18, 2024
4dc94d5
Fix deadlock and refactoring.
nickeskov Nov 18, 2024
214bd46
Merge branch 'master' into node-updates-plugin-l2
nickeskov Nov 18, 2024
8fbe2b4
Fixed err not found handling
esuwu Nov 20, 2024
bb23dbc
Fixed a toolchain not found issue (#1545)
esuwu Nov 20, 2024
b65c6d9
Bump github.com/stretchr/testify from 1.9.0 to 1.10.0 (#1546)
dependabot[bot] Nov 25, 2024
68db8b3
Handle blockchain import errors properly (#1548)
nickeskov Nov 26, 2024
d66423a
Bump github.com/elliotchance/orderedmap/v2 from 2.4.0 to 2.5.0 (#1549)
dependabot[bot] Nov 28, 2024
c598d67
Check on leasing state added. (#1550)
alexeykiselev Nov 28, 2024
bd889f5
Prepare timers for go1.23. (#1552)
nickeskov Dec 2, 2024
c24cb57
Peer address parsing refactoring (#1551)
nickeskov Dec 3, 2024
6814098
Merge branch 'master' into node-updates-plugin-l2
esuwu Dec 4, 2024
b87f8ad
Left a todo
esuwu Dec 24, 2024
cdb6083
Merged from master
esuwu Dec 24, 2024
7960470
Pulled the contract updates from snapshots, removed by block filterin…
esuwu Jan 9, 2025
a8fedad
merged from master
esuwu Jan 9, 2025
673f800
Added nats requests functionality
esuwu Jan 12, 2025
a873866
When restarted, the previous state must be nil
esuwu Jan 13, 2025
45523aa
Formatted a line correctly
esuwu Jan 13, 2025
52377b4
Merged from master
esuwu Jan 13, 2025
fb0e42f
Fixed some review issues
esuwu Jan 13, 2025
c750bae
Added an unmarshal function for blockMeta
esuwu Jan 15, 2025
6fd4d59
Write a test for a function that is not used, so the linter doesn't a…
esuwu Jan 15, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Moved proto, added conversion
  • Loading branch information
esuwu committed Jul 16, 2024

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
commit 6752f8a113e3e9b62abfa9ff5d3d71eb6dd9de04
4 changes: 2 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
@@ -226,8 +226,8 @@ proto:
@protoc --proto_path=pkg/grpc/protobuf-schemas/proto/ --go_out=./ --go_opt=module=$(MODULE) --go-vtproto_out=./ --go-vtproto_opt=features=marshal_strict+unmarshal+size --go-vtproto_opt=module=$(MODULE) pkg/grpc/protobuf-schemas/proto/waves/lang/*.proto
@protoc --proto_path=pkg/grpc/protobuf-schemas/proto/ --go_out=./ --go_opt=module=$(MODULE) --go-vtproto_out=./ --go-vtproto_opt=features=marshal_strict+unmarshal+size --go-vtproto_opt=module=$(MODULE) pkg/grpc/protobuf-schemas/proto/waves/events/*.proto
@protoc --proto_path=pkg/grpc/protobuf-schemas/proto/ --go_out=./ --go_opt=module=$(MODULE) --go-grpc_out=./ --go-grpc_opt=require_unimplemented_servers=false --go-grpc_opt=module=$(MODULE) pkg/grpc/protobuf-schemas/proto/waves/events/grpc/*.proto
proto1:
@protoc --proto_path=pkg/grpc/protobuf-schemas/proto/ --go_out=./ --go_opt=module=$(MODULE) --go-vtproto_out=./ --go-vtproto_opt=features=marshal_strict+unmarshal+size --go-vtproto_opt=module=$(MODULE) pkg/grpc/protobuf-schemas/proto/waves/bupdates/*.proto
proto-l2:
@protoc --proto_path=pkg/grpc/protobuf-schemas/proto/ --proto_path=pkg/grpc/l2/blockchain_info/ --go_out=./ --go_opt=module=$(MODULE) --go-vtproto_out=./ --go-vtproto_opt=features=marshal_strict+unmarshal+size --go-vtproto_opt=module=$(MODULE) pkg/grpc/l2/blockchain_info/*.proto

build-node-mainnet-amd64-deb-package: release-node
@mkdir -p build/dist
81 changes: 81 additions & 0 deletions cmd/blockchaininfo/nuts_subscriber.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
package main

import (
"encoding/json"
"fmt"
"github.com/nats-io/nats.go"
"github.com/wavesplatform/gowaves/pkg/blockchaininfo"
g "github.com/wavesplatform/gowaves/pkg/grpc/l2/blockchain_info"
"log"
)

func printBlockInfo(blockInfoProto *g.BlockInfo) error {
blockInfo, err := blockchaininfo.BUpdatesInfoFromProto(blockInfoProto)
if err != nil {
return err
}
blockInfoJson, err := json.Marshal(blockInfo)
fmt.Println(string(blockInfoJson))
return nil
}

func printContractInfo(contractInfoProto *g.L2ContractDataEntries) error {
contractInfo, err := blockchaininfo.L2ContractDataEntriesFromProto(contractInfoProto)
if err != nil {
return err
}
contractInfoJson, err := json.Marshal(contractInfo)
fmt.Println(string(contractInfoJson))
return nil
}

func main() {
// Connect to a NATS server
nc, err := nats.Connect(nats.DefaultURL)
if err != nil {
log.Fatal(err)
}
defer nc.Close()

topic := blockchaininfo.BlockUpdates
//for _, topic := range blockchaininfo.Topics {
_, err = nc.Subscribe(topic, func(msg *nats.Msg) {

//log.Printf("Received on %s: %s\n", msg.Subject, string(msg.Data))
if topic == blockchaininfo.BlockUpdates {
blockUpdatesInfo := new(g.BlockInfo)
err := blockUpdatesInfo.UnmarshalVT(msg.Data)
if err != nil {
return
}

err = printBlockInfo(blockUpdatesInfo)
if err != nil {
return
}
log.Printf("Received on %s:\n", msg.Subject)
}

if topic == blockchaininfo.ContractUpdates {
contractUpdatesInfo := new(g.L2ContractDataEntries)
err := contractUpdatesInfo.UnmarshalVT(msg.Data)
if err != nil {
return
}
log.Printf("Received on %s:\n", msg.Subject)

err = printContractInfo(contractUpdatesInfo)
if err != nil {
return
}

}

})
if err != nil {
log.Fatal(err)
}
//}
// Block main goroutine so the server keeps running
select {}
}
27 changes: 0 additions & 27 deletions cmd/blockchainupdates/nuts_subscriber.go

This file was deleted.

10 changes: 5 additions & 5 deletions cmd/node/node.go
Original file line number Diff line number Diff line change
@@ -6,7 +6,7 @@ import (
"errors"
"flag"
"fmt"
"github.com/wavesplatform/gowaves/pkg/blockchainupdates"
"github.com/wavesplatform/gowaves/pkg/blockchaininfo"
"math"
"math/big"
"net/http"
@@ -241,7 +241,7 @@ func (c *config) parse() {
flag.BoolVar(&c.enableLightMode, "enable-light-mode", false,
"Start node in light mode")

flag.BoolVar(&c.enableBlockchainUpdatesPlugin, "enable-blockchain-updates", false,
flag.BoolVar(&c.enableBlockchainUpdatesPlugin, "enable-blockchain-info", false,
"Turn on blockchain updates plugin")
flag.Parse()
c.logLevel = *l
@@ -393,8 +393,8 @@ func main() {

var bUpdatesExtension *state.BlockchainUpdatesExtension
if nc.enableBlockchainUpdatesPlugin {
updatesChannel := make(chan blockchainupdates.BUpdatesInfo)
bUpdatesExtensionState := blockchainupdates.NewBUpdatesExtensionState(2000)
updatesChannel := make(chan blockchaininfo.BUpdatesInfo)
bUpdatesExtensionState := blockchaininfo.NewBUpdatesExtensionState(2000)

l2address, err := proto.NewAddressFromString("3MsqKJ6o1ABE37676cHHBxJRs6huYTt72ch")
if err != nil {
@@ -407,7 +407,7 @@ func main() {
L2ContractAddress: l2address,
}

go bUpdatesExtensionState.RunBlockchainUpdatesPublisher(ctx, updatesChannel)
go bUpdatesExtensionState.RunBlockchainUpdatesPublisher(ctx, updatesChannel, cfg.AddressSchemeCharacter)
}

// Send updatesChannel into BlockchainSettings. Write updates into this channel
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package blockchainupdates
package blockchaininfo

import (
"context"
@@ -9,8 +9,6 @@ import (
"log"
)

// protoc --go_out=. --go_opt=paths=source_relative blockchainupdates.proto

type BUpdatesInfo struct {
Height uint64
VRF proto.B58Bytes
@@ -33,7 +31,42 @@ func (bu *BUpdatesExtensionState) hasStateChanged() {

}

func (bu *BUpdatesExtensionState) RunBlockchainUpdatesPublisher(ctx context.Context, updatesChannel <-chan BUpdatesInfo) {
func (bu *BUpdatesExtensionState) publishUpdates(updates BUpdatesInfo, nc *nats.Conn, scheme proto.Scheme) error {
/* first publish block related info */
blockInfo, err := BUpdatesInfoToProto(updates, scheme)
if err != nil {
return err
}
blockInfoProtobuf, err := blockInfo.MarshalVTStrict()
if err != nil {
return err
}
err = nc.Publish(BlockUpdates, blockInfoProtobuf)
if err != nil {
log.Printf("failed to publish message on topic %s", BlockUpdates)
return err
}
fmt.Printf("Published on topic: %s\n", BlockUpdates)

/* second publish contract data entries */
dataEntries := L2ContractDataEntriesToProto(updates.AllDataEntries)
dataEntriesProtobuf, err := dataEntries.MarshalVTStrict()
if err != nil {
return err
}
if dataEntries.DataEntries != nil {
err = nc.Publish(ContractUpdates, dataEntriesProtobuf)
if err != nil {
log.Printf("failed to publish message on topic %s", ContractUpdates)
return err
}
fmt.Printf("Published on topic: %s\n", ContractUpdates)
}

return nil
}

func (bu *BUpdatesExtensionState) RunBlockchainUpdatesPublisher(ctx context.Context, updatesChannel <-chan BUpdatesInfo, scheme proto.Scheme) {
opts := &server.Options{
Host: "127.0.0.1",
Port: 4222,
@@ -67,18 +100,11 @@ func (bu *BUpdatesExtensionState) RunBlockchainUpdatesPublisher(ctx context.Cont
// compare the current state to the previous state

// if there is any diff, send the update

fmt.Println(updates.Height)
var msg string
msg = "hello"
// Publish blockchain updates
topic := block_updates
err := nc.Publish(topic, []byte(msg))
err := bu.publishUpdates(updates, nc, scheme)
log.Printf("published")
if err != nil {
log.Printf("failed to publish message %s on topic %s", msg, topic)
log.Printf("failed to publish updates")
}
fmt.Printf("Published: %s\n", msg)

case <-ctx.Done():
return
}
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package blockchainupdates
package blockchaininfo

import (
"github.com/nats-io/nats-server/v2/server"
82 changes: 82 additions & 0 deletions pkg/blockchaininfo/proto_converters.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
package blockchaininfo

import (
"errors"
"github.com/wavesplatform/gowaves/pkg/grpc/generated/waves"
g "github.com/wavesplatform/gowaves/pkg/grpc/l2/blockchain_info"
"github.com/wavesplatform/gowaves/pkg/proto"
)

func BUpdatesInfoToProto(blockInfo BUpdatesInfo, scheme proto.Scheme) (*g.BlockInfo, error) {
blockHeader, err := blockInfo.BlockHeader.HeaderToProtobufHeader(scheme)
if err != nil {
return nil, err
}
return &g.BlockInfo{
Height: blockInfo.Height,
VRF: blockInfo.VRF,
BlockID: blockInfo.BlockID.Bytes(),
BlockHeader: blockHeader,
}, nil
}

func BUpdatesInfoFromProto(blockInfoProto *g.BlockInfo) (BUpdatesInfo, error) {
blockID, err := proto.NewBlockIDFromBytes(blockInfoProto.BlockID)
if err != nil {
return BUpdatesInfo{}, err
}
blockHeader, err := proto.ProtobufHeaderToBlockHeader(blockInfoProto.BlockHeader)
if err != nil {
return BUpdatesInfo{}, err
}
return BUpdatesInfo{
Height: blockInfoProto.Height,
VRF: blockInfoProto.VRF,
BlockID: blockID,
BlockHeader: blockHeader,
AllDataEntries: nil,
}, nil
}

func L2ContractDataEntriesToProto(dataEntries []proto.DataEntry) *g.L2ContractDataEntries {
var protobufDataEntries []*waves.DataEntry
for _, entry := range dataEntries {
entryProto := entry.ToProtobuf()
protobufDataEntries = append(protobufDataEntries, entryProto)
}
return &g.L2ContractDataEntries{
DataEntries: protobufDataEntries,
}
}

func L2ContractDataEntriesFromProto(protoDataEntries *g.L2ContractDataEntries) (L2ContractDataEntries, error) {
var dataEntries []proto.DataEntry
for _, protoEntry := range protoDataEntries.DataEntries {
dataEntryType := proto.DataEntryType{Type: protoEntry.Key}
entry, err := proto.GuessDataEntryType(dataEntryType)
if err != nil {
return L2ContractDataEntries{}, err
}
entry.SetKey(protoEntry.Key)
switch e := entry.(type) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use ProtobufConverter.Entry method from pkg/proto/protobuf_converters.go

case *proto.IntegerDataEntry:
e.Value = protoEntry.GetIntValue()
entry = e
case *proto.BooleanDataEntry:
e.Value = protoEntry.GetBoolValue()
entry = e
case *proto.BinaryDataEntry:
e.Value = protoEntry.GetBinaryValue()
entry = e
case *proto.StringDataEntry:
e.Value = protoEntry.GetStringValue()
entry = e
case *proto.DeleteDataEntry:
default:
return L2ContractDataEntries{}, errors.New("failed to convert proto data entries into data entries")
}
dataEntries = append(dataEntries, entry)
}

return L2ContractDataEntries{AllDataEntries: dataEntries}, nil
}
27 changes: 27 additions & 0 deletions pkg/blockchaininfo/topics.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
package blockchaininfo

import "github.com/wavesplatform/gowaves/pkg/proto"

/* Topics */
const (
BlockUpdates = "block_topic"
microblockUpdates = "microblock_topic"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unused

ContractUpdates = "contract_topi"
)

var Topics = []string{BlockUpdates, microblockUpdates, ContractUpdates}

// block updates

type BlockUpdatesInfo struct {
Height uint64 `json:"height"`
VRF proto.B58Bytes `json:"vrf"`
BlockID proto.BlockID `json:"block_id"`
BlockHeader *proto.BlockHeader `json:"block_header"`
}

// l2 contract data entries

type L2ContractDataEntries struct {
AllDataEntries []proto.DataEntry `json:"all_data_entries"`
}
10 changes: 0 additions & 10 deletions pkg/blockchainupdates/topics.go

This file was deleted.

Loading
Loading