Merge branch 'develop' into mardizzone/bump-versions

.github/ISSUE_TEMPLATE/bug.md

@@ -6,7 +6,7 @@ labels: 'type:bug'
 assignees: ''
 ---
 
-Our support team has aggregated some common issues and their solutions from past which are faced while running or interacting with a bor client. In order to prevent redundant efforts, we would encourage you to have a look at the [FAQ's section](https://wiki.polygon.technology/docs/faq/technical-faqs/) of our documentation mentioning the same, before filing an issue here. In case of additional support, you can also join our [discord](https://discord.com/invite/zdwkdvMNY2) server
+Our support team has aggregated some common issues and their solutions from past which are faced while running or interacting with a bor client. In order to prevent redundant efforts, we would encourage you to have a look at the [FAQ's section](https://wiki.polygon.technology/docs/faq/technical-faqs/) of our documentation mentioning the same, before filing an issue here. In case of additional support, you can also join our [discord](https://discord.com/invite/0xPolygonDevs) server
 
 <!--
 NOTE: Please make sure to check of any addresses / private keys / rpc url's / IP's before sharing the logs or anything from the additional information section (start.sh or heimdall config).

.github/ISSUE_TEMPLATE/question.md

@@ -8,4 +8,4 @@ assignees: ''
 
 This should only be used in very rare cases e.g. if you are not 100% sure if something is a bug or asking a question that leads to improving the documentation. 
 
-For general questions please join our [discord](https://discord.com/invite/zdwkdvMNY2) server.
+For general questions please join our [discord](https://discord.com/invite/0xPolygonDevs) server.

README.md

@@ -1,111 +1,71 @@
 # Bor Overview
-Bor is the Official Golang implementation of the Matic protocol. It is a fork of Go Ethereum - https://github.com/ethereum/go-ethereum and EVM compatible.
+Bor is the Official Golang implementation of the Polygon PoS blockchain. It is a fork of [geth](https://github.com/ethereum/go-ethereum) and is EVM compatible (upto London fork).
 
-![Forks](https://img.shields.io/github/forks/maticnetwork/bor?style=social)
-![Stars](https://img.shields.io/github/stars/maticnetwork/bor?style=social)
-![Languages](https://img.shields.io/github/languages/count/maticnetwork/bor)
-![Issues](https://img.shields.io/github/issues/maticnetwork/bor)
-![PRs](https://img.shields.io/github/issues-pr-raw/maticnetwork/bor)
+[![API Reference](
+https://camo.githubusercontent.com/915b7be44ada53c290eb157634330494ebe3e30a/68747470733a2f2f676f646f632e6f72672f6769746875622e636f6d2f676f6c616e672f6764646f3f7374617475732e737667
+)](https://pkg.go.dev/github.com/maticnetwork/bor)
+[![Go Report Card](https://goreportcard.com/badge/github.com/maticnetwork/bor)](https://goreportcard.com/report/github.com/maticnetwork/bor)
 ![MIT License](https://img.shields.io/github/license/maticnetwork/bor)
-![contributors](https://img.shields.io/github/contributors-anon/maticnetwork/bor)
-![size](https://img.shields.io/github/languages/code-size/maticnetwork/bor)
-![lines](https://img.shields.io/tokei/lines/github/maticnetwork/bor)
-[![Discord](https://img.shields.io/discord/714888181740339261?color=1C1CE1&label=Polygon%20%7C%20Discord%20%F0%9F%91%8B%20&style=flat-square)](https://discord.gg/zdwkdvMNY2)
+[![Discord](https://img.shields.io/discord/714888181740339261?color=1C1CE1&label=Polygon%20%7C%20Discord%20%F0%9F%91%8B%20&style=flat-square)](https://discord.com/invite/0xPolygonDevs)
 [![Twitter Follow](https://img.shields.io/twitter/follow/0xPolygon.svg?style=social)](https://twitter.com/0xPolygon)
 
-## How to contribute
+### Installing bor using packaging
 
-### Contribution Guidelines
-We believe one of the things that makes Polygon special is its coherent design and we seek to retain this defining characteristic. From the outset we defined some guidelines to ensure new contributions only ever enhance the project:
+The easiest way to get started with bor is to install the packages using the command below. Refer to the [releases](https://github.com/maticnetwork/bor/releases) section to find the latest stable version of bor.
+
+ curl -L https://raw.githubusercontent.com/maticnetwork/install/main/bor.sh | bash -s -- v0.4.0 <network> <node_type>
 
-* Quality: Code in the Polygon project should meet the style guidelines, with sufficient test-cases, descriptive commit messages, evidence that the contribution does not break any compatibility commitments or cause adverse feature interactions, and evidence of high-quality peer-review
-* Size: The Polygon project’s culture is one of small pull-requests, regularly submitted. The larger a pull-request, the more likely it is that you will be asked to resubmit as a series of self-contained and individually reviewable smaller PRs
-* Maintainability: If the feature will require ongoing maintenance (eg support for a particular brand of database), we may ask you to accept responsibility for maintaining this feature
-### Submit an issue
+The network accepts `mainnet` or `mumbai` and the node type accepts `validator` or `sentry` or `archive`. The installation script does the following things:
+- Create a new user named `bor`.
+- Install the bor binary at `/usr/bin/bor`.
+- Dump the suitable config file (based on the network and node type provided) at `/var/lib/bor` and uses it as the home dir.
+- Create a systemd service named `bor` at `/lib/systemd/system/bor.service` which starts bor using the config file as `bor` user.
 
-- Create a [new issue](https://github.com/maticnetwork/bor/issues/new/choose)
-- Comment on the issue (if you'd like to be assigned to it) - that way [our team can assign the issue to you](https://github.blog/2019-06-25-assign-issues-to-issue-commenters/).
-- If you do not have a specific contribution in mind, you can also browse the issues labelled as `help wanted`
-- Issues that additionally have the `good first issue` label are considered ideal for first-timers
+The releases supports both the networks i.e. Polygon Mainnet and Mumbai (Testnet) unless explicitly specified. Before the stable release for mainnet, pre-releases will be available marked with `beta` tag for deploying on Mumbai (testnet). On sufficient testing, stable release for mainnet will be announced with a forum post.
 
-### Fork the repository (repo)
-
-- If you're not sure, here's how to [fork the repo](https://help.github.com/en/articles/fork-a-repo)
-
-- If this is your first time forking our repo, this is all you need to do for this step:
+### Building from source
 
+- Install Go (version 1.19 or later) and a C compiler.
+- Clone the repository and build the binary using the following commands:
+ ```shell
+ make bor
  ```
- $ git clone git@github.com:[your_github_handle]/bor
+- Start bor using the ideal config files for validator and sentry provided in the `packaging` folder.
+ ```shell
+ ./build/bin/bor server --config ./packaging/templates/mainnet-v1/sentry/sentry/bor/config.toml
  ```
-
-- If you've already forked the repo, you'll want to ensure your fork is configured and that it's up to date. This will save you the headache of potential merge conflicts.
-
-- To [configure your fork](https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/configuring-a-remote-for-a-fork):
-
+- To build full set of utilities, run:
+ ```shell
+ make all
  ```
- $ git remote add upstream https://github.com/maticnetwork/bor
+- Run unit and integration tests
+ ```shell
+ make test && make test-integration
  ```
 
-- To [sync your fork with the latest changes](https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/syncing-a-fork):
+#### Using the new cli
 
- ```
- $ git checkout master
- $ git fetch upstream
- $ git merge upstream/master
- ```
+Post `v0.3.0` release, bor uses a new command line interface (cli). The new-cli (located at `internal/cli`) has been built with keeping the flag usage similar to old-cli (located at `cmd/geth`) with a few notable changes. Please refer to [docs](./docs) section for flag usage guide and example.
 
-### Building the source
+### Documentation
 
-- Building `bor` requires both a Go (version 1.19 or later) and a C compiler. You can install
-them using your favourite package manager. Once the dependencies are installed, run
+- The official documentation for the Polygon PoS chain can be found [here](https://wiki.polygon.technology/docs/pos/getting-started/). It contains all the conceptual and architectural details of the chain along with operational guide for users running the nodes.
+- New release announcements and discussions can be found on our [forum page](https://forum.polygon.technology/).
+- Polygon improvement proposals can be found [here](https://github.com/maticnetwork/Polygon-Improvement-Proposals/)
 
-  ```shell
-  $ make bor
-  ```
+### Contribution guidelines
 
-### Make awesome changes!
+Thank you for considering helping out with the source code! We welcome contributions from anyone on the internet, and are grateful for even the smallest of fixes! If you'd like to contribute to bor, please fork, fix, commit and send a pull request for the maintainers to review and merge into the main code base. 
 
-1. Create new branch for your changes
-
- ```
- $ git checkout -b new_branch_name
- ```
-
-2. Commit and prepare for pull request (PR). In your PR commit message, reference the issue it resolves (see [how to link a commit message to an issue using a keyword](https://docs.github.com/en/free-pro-team@latest/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword).
-
-
- Checkout our [Git-Rules](https://wiki.polygon.technology/docs/contribute/orientation/#git-rules)
-
- ```
- $ git commit -m "brief description of changes [Fixes #1234]"
- ```
-
-3. Push to your GitHub account
-
- ```
- $ git push
- ```
-
-### Submit your PR
-
-- After your changes are committed to your GitHub fork, submit a pull request (PR) to the `master` branch of the `maticnetwork/bor` repo
-- In your PR description, reference the issue it resolves (see [linking a pull request to an issue using a keyword](https://docs.github.com/en/free-pro-team@latest/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword))
- - ex. `Updates out of date content [Fixes #1234]`
-- Why not say hi and draw attention to your PR in [our discord server](https://discord.gg/0xpolygon)?
-
-### Wait for review
-
-- The team reviews every PR
-- Acceptable PRs will be approved & merged into the `master` branch
-
-<hr style="margin-top: 3em; margin-bottom: 3em;">
-
-## Release
-
-- You can [view the history of releases](https://github.com/maticnetwork/bor/releases), which include PR highlights
-
-<hr style="margin-top: 3em; margin-bottom: 3em;">
+From the outset we defined some guidelines to ensure new contributions only ever enhance the project:
 
+* Quality: Code in the Polygon project should meet the style guidelines, with sufficient test-cases, descriptive commit messages, evidence that the contribution does not break any compatibility commitments or cause adverse feature interactions, and evidence of high-quality peer-review. Code must adhere to the official Go [formatting](https://golang.org/doc/effective_go.html#formatting) guidelines (i.e. uses [gofmt](https://golang.org/cmd/gofmt/)).
+* Testing: Please ensure that the updated code passes all the tests locally before submitting a pull request. In order to run unit tests, run `make test` and to run integration tests, run `make test-integration`.
+* Size: The Polygon project’s culture is one of small pull-requests, regularly submitted. The larger a pull-request, the more likely it is that you will be asked to resubmit as a series of self-contained and individually reviewable smaller PRs.
+* Maintainability: If the feature will require ongoing maintenance (e.g. support for a particular brand of database), we may ask you to accept responsibility for maintaining this feature
+* Pull requests need to be based on and opened against the `develop` branch.
+* PR title should be prefixed with package(s) they modify.
+ * E.g. "eth, rpc: make trace configs optional"
 
 ## License
 
@@ -117,8 +77,6 @@ The go-ethereum binaries (i.e. all code inside of the `cmd` directory) are licen
 [GNU General Public License v3.0](https://www.gnu.org/licenses/gpl-3.0.en.html), also
 included in our repository in the `COPYING` file.
 
-<hr style="margin-top: 3em; margin-bottom: 3em;">
-
 ## Join our Discord server
 
-Join Polygon community – share your ideas or just say hi over [on Discord](https://discord.gg/zdwkdvMNY2).
+Join Polygon community – share your ideas or just say hi over [on Discord](https://discord.com/invite/0xPolygonDevs).

builder/files/config.toml

@@ -38,9 +38,9 @@ syncmode = "full"
  # nodekeyhex = ""
  [p2p.discovery]
  # v5disc = false
- bootnodes = ["enode://0cb82b395094ee4a2915e9714894627de9ed8498fb881cec6db7c65e8b9a5bd7f2f25cc84e71e89d0947e51c76e85d0847de848c7782b13c0255247a6758178c@44.232.55.71:30303", "enode://88116f4295f5a31538ae409e4d44ad40d22e44ee9342869e7d68bdec55b0f83c1530355ce8b41fbec0928a7d75a5745d528450d30aec92066ab6ba1ee351d710@159.203.9.164:30303"]
+ bootnodes = ["enode://76316d1cb93c8ed407d3332d595233401250d48f8fbb1d9c65bd18c0495eca1b43ec38ee0ea1c257c0abb7d1f25d649d359cdfe5a805842159cfe36c5f66b7e8@52.78.36.216:30303", "enode://b8f1cc9c5d4403703fbf377116469667d2b1823c0daf16b7250aa576bacf399e42c3930ccfcb02c5df6879565a2b8931335565f0e8d3f8e72385ecf4a4bf160a@3.36.224.80:30303", "enode://8729e0c825f3d9cad382555f3e46dcff21af323e89025a0e6312df541f4a9e73abfa562d64906f5e59c51fe6f0501b3e61b07979606c56329c020ed739910759@54.194.245.5:30303", "enode://681ebac58d8dd2d8a6eef15329dfbad0ab960561524cf2dfde40ad646736fe5c244020f20b87e7c1520820bc625cfb487dd71d63a3a3bf0baea2dbb8ec7c79f1@34.240.245.39:30303"]
  # Uncomment below `bootnodes` field for Mumbai bootnode
- # bootnodes = ["enode://095c4465fe509bd7107bbf421aea0d3ad4d4bfc3ff8f9fdc86f4f950892ae3bbc3e5c715343c4cf60c1c06e088e621d6f1b43ab9130ae56c2cacfd356a284ee4@18.213.200.99:30303"]
+ # bootnodes = ["enode://bdcd4786a616a853b8a041f53496d853c68d99d54ff305615cd91c03cd56895e0a7f6e9f35dbf89131044e2114a9a782b792b5661e3aff07faf125a98606a071@43.200.206.40:30303", "enode://209aaf7ed549cf4a5700fd833da25413f80a1248bd3aa7fe2a87203e3f7b236dd729579e5c8df61c97bf508281bae4969d6de76a7393bcbd04a0af70270333b3@54.216.248.9:30303"]
  # bootnodesv4 = []
  # bootnodesv5 = []
  # static-nodes = []

cmd/geth/chaincmd.go

@@ -195,7 +195,7 @@ func initGenesis(ctx *cli.Context) error {
  defer stack.Close()
 
  for _, name := range []string{"chaindata", "lightchaindata"} {
- chaindb, err := stack.OpenDatabaseWithFreezer(name, 0, 0, ctx.String(utils.AncientFlag.Name), "", false)
+ chaindb, err := stack.OpenDatabaseWithFreezer(name, 0, 0, ctx.String(utils.AncientFlag.Name), "", false, rawdb.ExtraDBConfig{})
  if err != nil {
  utils.Fatalf("Failed to open database: %v", err)
  }
@@ -229,7 +229,7 @@ func dumpGenesis(ctx *cli.Context) error {
  // dump whatever already exists in the datadir
  stack, _ := makeConfigNode(ctx)
  for _, name := range []string{"chaindata", "lightchaindata"} {
- db, err := stack.OpenDatabase(name, 0, 0, "", true)
+ db, err := stack.OpenDatabase(name, 0, 0, "", true, rawdb.ExtraDBConfig{})
 
  if err != nil {
  if !os.IsNotExist(err) {

cmd/geth/dao_test.go

@@ -123,7 +123,7 @@ func testDAOForkBlockNewChain(t *testing.T, test int, genesis string, expectBloc
  // Retrieve the DAO config flag from the database
  path := filepath.Join(datadir, "geth", "chaindata")
 
- db, err := rawdb.NewLevelDBDatabase(path, 0, 0, "", false)
+ db, err := rawdb.NewLevelDBDatabase(path, 0, 0, "", false, rawdb.ExtraDBConfig{})
  if err != nil {
  t.Fatalf("test %d: failed to open test database: %v", test, err)
  }

cmd/utils/flags.go

@@ -461,6 +461,31 @@ var (
  Value: 50,
  Category: flags.PerfCategory,
  }
+
+ LevelDbCompactionTableSizeFlag = &cli.Uint64Flag{
+ Name: "leveldb.compaction.table.size",
+ Usage: "LevelDB SSTable/file size in mebibytes",
+ Category: flags.PerfCategory,
+ }
+
+ LevelDbCompactionTableSizeMultiplierFlag = &cli.Float64Flag{
+ Name: "leveldb.compaction.table.size.multiplier",
+ Usage: "Multiplier on LevelDB SSTable/file size. Size for a level is determined by: `leveldb.compaction.table.size * (leveldb.compaction.table.size.multiplier ^ Level)`",
+ Category: flags.PerfCategory,
+ }
+
+ LevelDbCompactionTotalSizeFlag = &cli.Uint64Flag{
+ Name: "leveldb.compaction.total.size",
+ Usage: "Total size in mebibytes of SSTables in a given LevelDB level. Size for a level is determined by: `leveldb.compaction.total.size * (leveldb.compaction.total.size.multiplier ^ Level)`",
+ Category: flags.PerfCategory,
+ }
+
+ LevelDbCompactionTotalSizeMultiplierFlag = &cli.Float64Flag{
+ Name: "leveldb.compaction.total.size.multiplier",
+ Usage: "Multiplier on level size on LevelDB levels. Size for a level is determined by: `leveldb.compaction.total.size * (leveldb.compaction.total.size.multiplier ^ Level)`",
+ Category: flags.PerfCategory,
+ }
+
  CacheTrieFlag = &cli.IntFlag{
  Name: "cache.trie",
  Usage: "Percentage of cache memory allowance to use for trie caching (default = 15% full mode, 30% archive mode)",
@@ -2170,7 +2195,7 @@ func RegisterFilterAPI(stack *node.Node, backend ethapi.Backend, ethcfg *ethconf
  LogCacheSize: ethcfg.FilterLogCacheSize,
  })
 
- filterAPI := filters.NewFilterAPI(filterSystem, isLightClient, ethconfig.Defaults.BorLogs)
+ filterAPI := filters.NewFilterAPI(filterSystem, isLightClient, ethcfg.BorLogs)
  stack.RegisterAPIs([]rpc.API{{
  Namespace: "eth",
  Service: filterAPI,
@@ -2291,6 +2316,8 @@ func MakeChainDatabase(ctx *cli.Context, stack *node.Node, readonly bool) ethdb.
 
  err error
  chainDb ethdb.Database
+
+ dbOptions = resolveExtraDBConfig(ctx)
  )
 
  switch {
@@ -2304,9 +2331,9 @@ func MakeChainDatabase(ctx *cli.Context, stack *node.Node, readonly bool) ethdb.
 
  chainDb = remotedb.New(client)
  case ctx.String(SyncModeFlag.Name) == "light":
- chainDb, err = stack.OpenDatabase("lightchaindata", cache, handles, "", readonly)
+ chainDb, err = stack.OpenDatabase("lightchaindata", cache, handles, "", readonly, dbOptions)
  default:
- chainDb, err = stack.OpenDatabaseWithFreezer("chaindata", cache, handles, ctx.String(AncientFlag.Name), "", readonly)
+ chainDb, err = stack.OpenDatabaseWithFreezer("chaindata", cache, handles, ctx.String(AncientFlag.Name), "", readonly, dbOptions)
  }
 
  if err != nil {
@@ -2316,6 +2343,15 @@ func MakeChainDatabase(ctx *cli.Context, stack *node.Node, readonly bool) ethdb.
  return chainDb
 }
 
+func resolveExtraDBConfig(ctx *cli.Context) rawdb.ExtraDBConfig {
+ return rawdb.ExtraDBConfig{
+ LevelDBCompactionTableSize: ctx.Uint64(LevelDbCompactionTableSizeFlag.Name),
+ LevelDBCompactionTableSizeMultiplier: ctx.Float64(LevelDbCompactionTableSizeMultiplierFlag.Name),
+ LevelDBCompactionTotalSize: ctx.Uint64(LevelDbCompactionTotalSizeFlag.Name),
+ LevelDBCompactionTotalSizeMultiplier: ctx.Float64(LevelDbCompactionTotalSizeMultiplierFlag.Name),
+ }
+}
+
 func IsNetworkPreset(ctx *cli.Context) bool {
  for _, flag := range NetworkFlags {
  bFlag, _ := flag.(*cli.BoolFlag)

common/types.go

@@ -28,8 +28,9 @@ import (
  "reflect"
  "strings"
 
- "github.com/ethereum/go-ethereum/common/hexutil"
  "golang.org/x/crypto/sha3"
+
+ "github.com/ethereum/go-ethereum/common/hexutil"
 )
 
 // Lengths of hashes and addresses in bytes.
@@ -66,6 +67,12 @@ func BigToHash(b *big.Int) Hash { return BytesToHash(b.Bytes()) }
 // If b is larger than len(h), b will be cropped from the left.
 func HexToHash(s string) Hash { return BytesToHash(FromHex(s)) }
 
+func HexToRefHash(s string) *Hash {
+ v := BytesToHash(FromHex(s))
+
+ return &v
+}
+
 // Bytes gets the byte representation of the underlying hash.
 func (h Hash) Bytes() []byte { return h[:] }
 

consensus/bor/api.go

@@ -341,10 +341,6 @@ func (api *API) GetRootHash(start uint64, end uint64) (string, error) {
  return root, nil
 }
 
-func (api *API) GetVoteOnHash(starBlockNr uint64, endBlockNr uint64, hash string, milestoneId string) (bool, error) {
- return false, nil
-}
-
 func (api *API) initializeRootHashCache() error {
  var err error
  if api.rootHashCache == nil {