Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Merge v0.9.0 into master #2278

Merged
merged 16 commits into from
Jun 16, 2022
Merged

Merge v0.9.0 into master #2278

merged 16 commits into from
Jun 16, 2022

Conversation

jonastheis
Copy link
Contributor

v0.9.0 - 2022-06-16

This major release introduces concurrency throughout the node with asynchronous events and parallelized booking. This is achieved with serix, a generic deterministic serialization library, and through generic thread-safe data models. The ledger and conflictDAG and output processing (introduction of UTXO VM) have been completely rewritten. The Blockfactory received a total revamp as well and tip selection should show better behavior in resolving conflicts. In addition, a bunch of unused modules and code have been removed rendering the project overall more readable. The short identifier of a nodeID has been adjusted to show the first 8 characters of the full identity.

The snapshot has been changed and thus the ledger state is fully reset.

jonastheis and others added 16 commits May 22, 2022 20:25
@jonastheis
Update marker docs (#2146)
69de038
@jonastheis
GoReleaser dryrun on PRs (#2213)
3af49d7 
* Add GoReleaser dry run

* Remove unit tests on push

* Use --snapshot option

* Remove snapshot download

* Create fake config.json and snapshot.bin

* Remove unnecessary go setup step

* Remove -race from unit test execution

* Add --rm-dist
@karimodm
Force static linking of final binary in builtin script (#2216)
e6743c1
@georgysavva
Fix serializer package dependency (#2219)
ebc066f
@karimodm
Reviewdog version bump (#2223)
61d556c
@jonastheis @hmoog @karimodm
Multithreaded booking and new data flow with solidification of transa…
42f7cf9 
…ctions (#2151)

* Refactor: started to comment utils

* Feat: started refactoring models

* Feat: commented ChildBranch

* Fix: fixed bug with default options

* Refactor: refactored Conflict

* Refactor: removed unused code

* Feat: finished branchdag refactor

* Refactor: refactored some code

* Add multithreaded test for solidification and propagation (#2147)

* Feat: commented Ledger (models missing)

* Add tests for edge cases

* remove unused code

* Refactor: refactored code

* Add tests for childbranches when asserting branch DAG references

* Refactor: refactored code

* Fix: fixed some refactor bugs

* Fix: fixed refactor errors in devnetvm

* Feat: go mod tidy

* Feat: commented Outputs

* Feat: commented more code

* Feat: commented models

* Refactor: reviewed the code

* Refactor: changed more code

* Add comments to testframework

* Add locks to testframework

* Feat: added comments for properties in TestFramework

* Start implementing new tangle data flow

* Fix: fixed some random things

* Feat: started implementing error handling

* Fix: fixed some random stuff

* Feat: added comment

* Feat: added Shutdown calls

* Feat: added context to all final events Booked/Invalid

* change to improved data flow and embed payloadbooker into booker

* Rename ledgerstate to ledgerstateOLD in preparation to get rid of it. Add new ledger to Tangle

* WIP

* WIP

* fix more errors

* Feat: added UndefinedBranchID

* fix more errors

* Feat: refactored comment

* Feat: use EventLoop for gossip manager

* Feat: fixed some bugs

* Fix: fixed some random code

* fix more stuff

* Feat: more generic events

* Feat: started implementing indexer

* Feat: more generic events

* Feat: more generic events

* Feat: more generic events

* Feat: more generic events

* Feat: added an Indexer for the devnetvm

* Fix: fixed more random things

* Fix: fixed approvalweightmanager

* fix more and more stuff

* Feat: more generic events

* Fix: fixed more stuff

* Fix: fixed more bugs

* Feat: more generic events

* Feat: upgrade hive.go

* Fix: change workerpool in tangle_test

* Fix: fixed workerpool

* Feat: more generic events

* Feat: fixed wallet + client

* Fix: fixed some stuff in otv test

* Fix: fixed more stuff in otv.test

* Feat: more generic events

* Fix: remove replace in go.mod

* Fix: fixed otv_test

* Fix: fixed more stuff

* Fix: fixed more stuff

* Fix: fixed more ledgerstate references

* Fix: removed all references to ledgerstate package

* Feat: started adding snapshots

* Fix more errors

* Fix more stuff

* Fix more compile errors, only snapshot left now

* Feat: Implemented TransactionConfirmed event (#2168)

* Feat: Implemented TransactionConfirmed event

* Feat: started implementing mana snapshit

* Feat: node builds

* Refactor: simplified code by removing some wrappers

* Fix: fixed refactor related bugs

* Refactor: removed Transaction wrapping

* Fix: fixed broken mocks

* Update to latest hive.go version and fix out of order event error on startup

* Fix permissions in Docker file when mounting volume

* Feat: started reworking snapshot logic

* Feat: added OutputMetadata snapshot

* Feat: added more snapshot stuff

* Feat: fixed create snapshot tool

* Fix snapshit generation

* Fix uninitialized events

* Fix: fixed problem in booker

* Use eventloop instead of goroutines

* Add snapshot to Docker network and index outputs from snapshot when loading it

* Fix: fixed waiting in tests

* Feat: debug

* Fix: fixed some stuff

* Fix: removed unused code

* debug tests and stuff

* Feat: added debug outputs

* Fix one issue with out of order test

* Hook forking events

* Fix: fixed stuff

* Fix: fixed some stuff

* Feat: refactored solidifier

* Fix: fix bug

* Fix: booker_test TestScenario_1

* Fix: fixed test

* Fix: fixed test

* Fix: fixed tests

* Fix: fixed otv test

* Fix: fixed broken plugins

* Fix: fixed go.mod

* Fix: fixed some minor things

* Fix: fixed go.mod

* Fix: fixed broken test

* Feat: TSA remove past-cone checks and unconfirmed weak references

* Feat: upgraded to latest hive.go

* Feat: implemented commitment and proof systems for state commitments

* Fix: fixed order in booking

* tangle booker test

* Fix TestMultiThreadedBookingAndForking

* Fix: debug.Enabled race condition

* Fix: make tests more solid

* Feat: generic branchdag (#2204)

* Bump hive version

* Remove debugging prints

* Bump hive.go to fix serializer/v2 nested module

* Fix: TestMultiThreadedBookingAndForking test

* Refactor: renamed BranchDAG to ConflictDAG

* Refactor: started cleaning up stuff

* Fix: code warnings

* Feat: removed MasterBranch

* Feat: TestMultiThreadedBookingAndForkingNested

* Fix: BranchDAG -> ConflictDAG renaming

* Faster TestMultiThreadedBookingAndForkingNested

* Fix snapshot generation with new snapshotcreator tool

* Fix: fixed snapshit

* Feat: feed outputs to the messagelayer Indexer

* Fix: make sure ConflictIDs are empty sets

* Fix snapshot generation for integration tests

* Fix common integration tests

* Fix autopeering integration test

* Remove diagnostics integration tests

* Remove drng integration tests

* Fix: dashboard

* Fix: fixed some tests

* Fix: dashboard again

* Fix: fixed some stuff

* Fix: fixed test

* Fix: fixed stuff

* Fix: some unit tests

* Fix mana integration tests

* Fix faucet integration tests

* Feat: renamed branchdag

* Refactor: moved conflictdag package

* Refactor: refactored more conflictdag models

* Fix faucet integration tests

* Fix value integration tests

* Fix nil pointer in conflicts dashboard

* Fix reference creation for shallow dislike and TestConflictSpamAndMergeToMaster integration test

* Fix consensus integration tests

* Run integration tests for multithreaded tangle (#2217)

* Adjust runTests.sh

* Add missing context import

* Fix: dashboard

* Refactor: refactored code

* Fix: critical - return strongparent as-is if branch liked

* Refactor: cleaned up stuff

* Merge develop

* Revert "Fix: critical - return strongparent as-is if branch liked"

This reverts commit 74288b2.

* Fix issue where liked branch is not returned when LikedConflictMember is called with the liked branch of the conflict set (#2174)

* Add Feature network identities in snapshot creation tool

* WIP

* Make code compile

* Fix output unmarshallimg

* Fix: do not evaluate time difference if message approves Genesis

* Fix: Faucet properly attach async events

* Feat: cleaned up first model

* Adjust models in conflict dag

* Fix: fixed conflictdag

* Fix: fixed some more stuff

* fixed test

* Ledger tests pass

* Ledger tests and devnetvm tests pass again

* Fix indexer

* Fix objectstorage interface for some objects

* Use serix for model.Model serialization

* Fix more stuff

* Fix: fixed stuff

* Refactor: removed unnecessary helper

* Feat: cleaned up more code

* Refactor: changed to new models

* Fix more stuff

* Fix snapshit again

* Fix: fixed bug with models

* Bump hive.go

* Few fixes and prints

* Feat: added test for outputcommitment

* Add manual serialization to Ouput types to enable correct interface (de)serialization

* go mod tidy

* Fix value test hopefully

* Fix unsanitized user input print

* Fix: removed unused identifier

* Update CHANGELOG.md

* Update pkger

Co-authored-by: Hans Moog <3293976+hmoog@users.noreply.github.com>
Co-authored-by: Andrea V <1577639+karimodm@users.noreply.github.com>
@dependabot
Build(deps): bump eventsource in /plugins/dashboard/frontend (#2232)
123d1e2 
Bumps [eventsource](https://github.com/EventSource/eventsource) from 1.0.7 to 1.1.1.
- [Release notes](https://github.com/EventSource/eventsource/releases)
- [Changelog](https://github.com/EventSource/eventsource/blob/master/HISTORY.md)
- [Commits](EventSource/eventsource@v1.0.7...v1.1.1)

---
updated-dependencies:
- dependency-name: eventsource
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@dependabot
Build(deps): bump eventsource in /plugins/analysis/dashboard/frontend (…
c68c78f 
…#2233)

Bumps [eventsource](https://github.com/EventSource/eventsource) from 1.0.7 to 1.1.1.
- [Release notes](https://github.com/EventSource/eventsource/releases)
- [Changelog](https://github.com/EventSource/eventsource/blob/master/HISTORY.md)
- [Commits](EventSource/eventsource@v1.0.7...v1.1.1)

---
updated-dependencies:
- dependency-name: eventsource
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@jonastheis @piotrm50 @hmoog
Refactor models to use generic base models (#2231)
9ed63ea 
* Update MessageMetadata

* Update Attachment

* refactor stuff

* Rework marker models (#2230)

* Feat: Started reworking marker models

* Feat: updated hive.go everywhere

* Fix: upgrade hive.go to use constraints

* Fix: more hive.go fixes

* Refactor: fixed models

* Fix: fixed markersmanager

* Fix: fixed some bugs

* Feat: added Bytes to Marker

* More refactors.

* Fix: fixed markers

* Fix things

* Fix tests

* Remove unnecessary code

* Fix: fixed more models

* Fix: fixed more models

* Fix: fixed more models

* Fix approval weight manager

* Update MessageID to use types.Identifier

* Refactor mana structs

* Fix: fixed more marker models

* Fix: fixed more model

* Fix: fixed thresholdmap serialization

* Fix: fixed missing serix tag

* Fix: fixed missing serix tag

* Fix: fixed missing serix tags

* Fix: fixed bug?

* Fix: fixed model

* Fix: added debug outputs

* Fix: fixed stuff

* Refactor: less debug outputs

* Refactor and fix stuff

* Fix: Fixed test debug output

* Fix: fixe some unnecessary code

* Refactor: reverted test

* Feat: upgrade hive.go

* Feat: changed marker to be non-pointer

* Refactor: cleaned up code

* Update Message to use storable model

* Do not deserialize payload within message by default

Co-authored-by: Piotr Macek <piotr.macek@iota.org>
Co-authored-by: Hans Moog <3293976+hmoog@users.noreply.github.com>
@dependabot
Build(deps): bump eventsource in /plugins/dagsvisualizer/frontend (#2235
0d2c69c 
)

Bumps [eventsource](https://github.com/EventSource/eventsource) from 1.1.0 to 1.1.1.
- [Release notes](https://github.com/EventSource/eventsource/releases)
- [Changelog](https://github.com/EventSource/eventsource/blob/master/HISTORY.md)
- [Commits](EventSource/eventsource@v1.1.0...v1.1.1)

---
updated-dependencies:
- dependency-name: eventsource
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@jonastheis @hmoog
Change identity.ID short representation (#2238)
a68848a 
* Change identity.ID short representation

* Update plugins/peer/plugin.go

Co-authored-by: Hans Moog <3293976+hmoog@users.noreply.github.com>
@jonastheis @karimodm
Delete unused modules (#2234)
2b8d1e9 
Co-authored-by: Andrea V <1577639+karimodm@users.noreply.github.com>
@piotrm50
Remove future markers (#2241)
1d08132 
* Remove future markers

* Update hive.go version

* Remove future markers from dashboard

* Update dashboard

* Update hive.go in go.mod files

* Fix lock

* Update hive.go version

* Fix go.mod

* fix go.sum
@piotrm50 @jonastheis @hmoog
Refactor models to be thread-safe with serix (#2248)
ffa0dca 
* Start refactoring models

* Continue refactoring models WIP

* Continue refactor

* Fix ExtendedLockedOutput

* WIP debugging MarkerIndexBranchIDMapping

* Fix MarkerIndexBranchIDMapping: due to the embedded type, serix did call Decode on a nil value without initializing it before with reflect.New because it was seeing the outer type markerIndexBranchIDMap

* Update code to latest models

* Update code to latest models

* Update code to latest hive.go

* Add tests with -race to github action

* Fix go.mod

* Fix unit tests

* Fix integration tests

* Skip long unit test in -race test

* Turn off more test for race test

* Refactor more models

* Fix integration tests

* Update go.mod

* Run unit tests with -race in separate job and increase timeout

* Update packages/tangle/payload/data.go

* Fix: fixed bug

Co-authored-by: jonastheis <4181434+jonastheis@users.noreply.github.com>
Co-authored-by: Hans Moog <3293976+hmoog@users.noreply.github.com>
@hmoog @jonastheis
Refactor MessageFactory, introduce ReferenceProvider and OrphanageMan…
04f2020 
…ager (#2240)

* WIP refactor reference creation

* Feat: started fixing TSA

* Wire up and fix reference provider

* clean up issue payload function

* Improve way we select tips

* Debugging WIP

* Fix parents count when issuing a message

* Fix stuff

* Feat: refactored sth

* Feat: updated hive.go

* Fix: fixed some stuff

* Fix unit tests

* Remove replace in go.mod

* Go mod tidy

* Fix: fixed race condition in test

* Fixed race condition

* Feat: push stuff

* Update go.mod

* Feat: implemented new logic for references

* Walk future cone of disliked conflictSetMembers to find all conflicts that are excluded

* Refactor: cleaned up code

* Feat: started removing Shallow Dislike

* Fix: fixed more tests

* Fix: fixed more tests

* Remove dislike from frontend stuff and update pkger

* Do not add subjectively invald blocks to the tipmanager

* Fix: fix otv tests

* Fix: fixed some stuff

* Add conflict/branch inclusion state to dashboard and API

* Fix IsBranchConfirmed

* Disable tipsForLastBranch functionality

* Adjust forking of individually mapped messages according to logic from forking markers (always propagate parent branches forward)

* Do not exit earlier to always determine subjectively invalid messages correctly

* Fix unit tests

* Fix duplicate weak and strong parent when referencing it directly weakly and also via strong parent. Fixes integration test as well.

* Refactor: cleaned up some code

* Feat: added tracking of missing conflicts

* Feat: added error message in tests

* Fix: fixed nil references

* Fix: fixed missing setup

* Fix: removed empty printline

* Do not exit earlier to always determine subjectively invalid messages correctly

Co-authored-by: jonastheis <4181434+jonastheis@users.noreply.github.com>
@jonastheis
Release v0.9.0 (#2277)
7d2cd7a 
* Bump versions

* Add changelog
@jonastheis jonastheis changed the title v0.9.0 Merge v0.9.0 into master Jun 16, 2022
@karimodm karimodm merged commit 3eec0b9 into master Jun 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@karimodm karimodm karimodm approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@jonastheis @karimodm @georgysavva @piotrm50 @hmoog