From 29e91eda40ae165b5a810178ab201f7a237afc58 Mon Sep 17 00:00:00 2001 From: Atul Mahamuni Date: Tue, 27 Aug 2019 00:30:39 +0000 Subject: [PATCH 1/5] Updated introduction in the README file --- README.md | 33 +++++++++++++++++++++++++++++++-- 1 file changed, 31 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 017614101e1..cc263421df8 100644 --- a/README.md +++ b/README.md @@ -5,9 +5,38 @@ This BetaMirrorNode implementation supports CryptoService, FileService and SmartContractService through a proxy. It can also parse the RecordStream and balance files generated by Hedera nodes. -## Description +## Overview + +### Full mirror node +Here is how the full Hedera mirror node works: +- Hedera mirror nodes receive exactly the same information as the mainnet nodes through gossip protocol +- Mirror nodes run exactly the same algorithm to construct the Hashgraph (gossip about gossip) and to decide on consensus order & timestamps (virtual voting) as the mainnet nodes. +- Mirror nodes also participate in the gossip protocol and provide information to each other. +- As a result, the mirror nodes arrive at exactly the same decisions (fast, fair and secure consensus order & timestamp) as the mainnet nodes. + +In addition, +- Mirror nodes can provide services (e.g. querying of historical data, transaction analytics, visibility services, security threat modeling, state proofs, data monetization). +- They can also run additional business logic to provide an application built on top of the Hedera mainnet + +### Beta mirror node +To make the initial deployments easier, the beta mirror node strives to take away the burden of running a full Hedera node through creation of periodic files that contain processed information (such as account balances or transaction records), and have the full trust of the Hedera mainnet nodes. +Here's how it works: -Beta mirror node is a temporary mirror node implementation until the Hedera Mirrornet and associated Mirror Nodes receiving gossip about gossip are available. +- All mainnet nodes periodically create and cryptographically sign three types of files: + - Balances for all accounts + - Transaction records + - All events (gossip information) + +- Each Hedera mainnet node pushes these files to collectors +- Beta mirror node software receives these files, validates them, parses the information in these files, puts data in a database, and offers a REST API to query the information + +####Advantages of beta mirror node +- Lower compute, bandwidth requirement +- It allows users to only save what they care about, and discard what they don’t (lower storage requirement) +- Easy searchable database so the users can add value quickly +- Easy to consume REST APIs to make integrations faster + +## Description The Beta mirror node works as follows: From 4add0243b28d462433709e5364b706c72b8bd8a4 Mon Sep 17 00:00:00 2001 From: Atul Mahamuni Date: Tue, 27 Aug 2019 01:39:45 +0000 Subject: [PATCH 2/5] Updated based on review comments --- README.md | 18 ++++++------------ 1 file changed, 6 insertions(+), 12 deletions(-) diff --git a/README.md b/README.md index cc263421df8..0c50034957d 100644 --- a/README.md +++ b/README.md @@ -7,19 +7,13 @@ It can also parse the RecordStream and balance files generated by Hedera nodes. ## Overview -### Full mirror node -Here is how the full Hedera mirror node works: -- Hedera mirror nodes receive exactly the same information as the mainnet nodes through gossip protocol -- Mirror nodes run exactly the same algorithm to construct the Hashgraph (gossip about gossip) and to decide on consensus order & timestamps (virtual voting) as the mainnet nodes. -- Mirror nodes also participate in the gossip protocol and provide information to each other. -- As a result, the mirror nodes arrive at exactly the same decisions (fast, fair and secure consensus order & timestamp) as the mainnet nodes. +Hedera mirror nodes receive the information from the mainnet nodes and they provide value-added services such as providing audit support, access to historical data, transaction analytics, visibility services, security threat modeling, state proofs, data monetization services, etc. Mirror nodes can also run additional business logic to support applications built using Hedera mainnet. -In addition, -- Mirror nodes can provide services (e.g. querying of historical data, transaction analytics, visibility services, security threat modeling, state proofs, data monetization). -- They can also run additional business logic to provide an application built on top of the Hedera mainnet +While mirror nodes receive information from the mainnet nodes, they do not contribute to consensus on the mainnet, and their votes are not counted. Only the votes from the mainnet nodes are counted for determining consensus. The trust of Hedera mainnet is derived based on the the consensus reached by the mainnet nodes. That trust is transferred to the mirror nodes using signatures, chain of hashes and state proofs. ### Beta mirror node -To make the initial deployments easier, the beta mirror node strives to take away the burden of running a full Hedera node through creation of periodic files that contain processed information (such as account balances or transaction records), and have the full trust of the Hedera mainnet nodes. +Eventually, the mirror nodes can run the same code as the Hedera mainnet nodes so that they can see the transactions in real time. To make the initial deployments easier, the beta mirror node strives to take away the burden of running a full Hedera node through creation of periodic files that contain processed information (such as account balances or transaction records), and have the full trust of the Hedera mainnet nodes. + Here's how it works: - All mainnet nodes periodically create and cryptographically sign three types of files: @@ -28,9 +22,9 @@ Here's how it works: - All events (gossip information) - Each Hedera mainnet node pushes these files to collectors -- Beta mirror node software receives these files, validates them, parses the information in these files, puts data in a database, and offers a REST API to query the information +- Beta mirror node software receives these files, validates them (ensure authenticity by verifying signatures of a supermajority of mainnet nodes, verifying chain of hashes on files), parses the information in these files, puts data in a database, and offers a REST API to query the information -####Advantages of beta mirror node +###Advantages of beta mirror node - Lower compute, bandwidth requirement - It allows users to only save what they care about, and discard what they don’t (lower storage requirement) - Easy searchable database so the users can add value quickly From 5d8feaa95c6098e5d8361a19da519a03f8ef0a97 Mon Sep 17 00:00:00 2001 From: Atul Mahamuni Date: Tue, 27 Aug 2019 04:51:49 +0000 Subject: [PATCH 3/5] Updated per review comments from Greg and Steven --- README.md | 15 +++------------ 1 file changed, 3 insertions(+), 12 deletions(-) diff --git a/README.md b/README.md index 0c50034957d..2e445849f6a 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,7 @@ # Beta Mirror Node This BetaMirrorNode implementation supports CryptoService, FileService and SmartContractService through a proxy. -It can also parse the RecordStream and balance files generated by Hedera nodes. +It can also parse the RecordStream, balance, and events files generated by Hedera mainnet nodes. ## Overview @@ -12,19 +12,10 @@ Hedera mirror nodes receive the information from the mainnet nodes and they prov While mirror nodes receive information from the mainnet nodes, they do not contribute to consensus on the mainnet, and their votes are not counted. Only the votes from the mainnet nodes are counted for determining consensus. The trust of Hedera mainnet is derived based on the the consensus reached by the mainnet nodes. That trust is transferred to the mirror nodes using signatures, chain of hashes and state proofs. ### Beta mirror node -Eventually, the mirror nodes can run the same code as the Hedera mainnet nodes so that they can see the transactions in real time. To make the initial deployments easier, the beta mirror node strives to take away the burden of running a full Hedera node through creation of periodic files that contain processed information (such as account balances or transaction records), and have the full trust of the Hedera mainnet nodes. +Eventually, the mirror nodes can run the same code as the Hedera mainnet nodes so that they can see the transactions in real time. To make the initial deployments easier, the beta mirror node strives to take away the burden of running a full Hedera node through creation of periodic files that contain processed information (such as account balances or transaction records), and have the full trust of the Hedera mainnet nodes. The beta mirror node software reduces the processing burden by receiving pre-constructed files from the mainnet, validating those, populating a database and providing REST APIs. -Here's how it works: -- All mainnet nodes periodically create and cryptographically sign three types of files: - - Balances for all accounts - - Transaction records - - All events (gossip information) - -- Each Hedera mainnet node pushes these files to collectors -- Beta mirror node software receives these files, validates them (ensure authenticity by verifying signatures of a supermajority of mainnet nodes, verifying chain of hashes on files), parses the information in these files, puts data in a database, and offers a REST API to query the information - -###Advantages of beta mirror node +#### Advantages of beta mirror node - Lower compute, bandwidth requirement - It allows users to only save what they care about, and discard what they don’t (lower storage requirement) - Easy searchable database so the users can add value quickly From 682f27c63c663aac2be59a2943cf47a51d15dcf2 Mon Sep 17 00:00:00 2001 From: Greg Scullard Date: Tue, 27 Aug 2019 10:49:19 +0200 Subject: [PATCH 4/5] Update README.md --- README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 2e445849f6a..7d5ce42da9e 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,7 @@ # Beta Mirror Node This BetaMirrorNode implementation supports CryptoService, FileService and SmartContractService through a proxy. -It can also parse the RecordStream, balance, and events files generated by Hedera mainnet nodes. +It can also parse the RecordStream, balance, and events files generated by Hedera mainnet and testnet nodes. ## Overview @@ -12,10 +12,12 @@ Hedera mirror nodes receive the information from the mainnet nodes and they prov While mirror nodes receive information from the mainnet nodes, they do not contribute to consensus on the mainnet, and their votes are not counted. Only the votes from the mainnet nodes are counted for determining consensus. The trust of Hedera mainnet is derived based on the the consensus reached by the mainnet nodes. That trust is transferred to the mirror nodes using signatures, chain of hashes and state proofs. ### Beta mirror node + Eventually, the mirror nodes can run the same code as the Hedera mainnet nodes so that they can see the transactions in real time. To make the initial deployments easier, the beta mirror node strives to take away the burden of running a full Hedera node through creation of periodic files that contain processed information (such as account balances or transaction records), and have the full trust of the Hedera mainnet nodes. The beta mirror node software reduces the processing burden by receiving pre-constructed files from the mainnet, validating those, populating a database and providing REST APIs. #### Advantages of beta mirror node + - Lower compute, bandwidth requirement - It allows users to only save what they care about, and discard what they don’t (lower storage requirement) - Easy searchable database so the users can add value quickly From 7b6f828ba9821facade5310592b79a293d639bd4 Mon Sep 17 00:00:00 2001 From: Greg Scullard Date: Tue, 27 Aug 2019 10:51:36 +0200 Subject: [PATCH 5/5] Update README.md --- README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 7d5ce42da9e..13e04221902 100644 --- a/README.md +++ b/README.md @@ -28,7 +28,7 @@ Eventually, the mirror nodes can run the same code as the Hedera mainnet nodes s The Beta mirror node works as follows: - When a transaction reaches consensus, Hedera nodes add the transaction and its associated record to a record file. -- The file is closed on a regular cadence and a new file is created for the next batch of transactions and records. The interval is currently set to 1 minute but may vary between networks. +- The file is closed on a regular cadence and a new file is created for the next batch of transactions and records. The interval is currently set to 5 seconds but may vary between networks. - Once the file is closed, nodes generate a signature file which contains the signature generated by the node for the record file. - Record files also contain the hash of the previous record file, thus creating an unbreakable validation chain. @@ -39,6 +39,8 @@ The Beta mirror node works as follows: - For each valid signature file, the corresponding record file is then downloaded from the cloud. - Record files can then be processed and transactions and records processed for long term storage. +- Event files are handled in exactly the same manner. + - In addition, nodes regularly generate a balance file which contains the list of Hedera accounts and their corresponding balance which is also uploaded to S3 and Google File Storage. - The files are also signed by the nodes. - This mirror node software can download the balance files, validate 2/3rd of nodes have signed and then process the balance files for long term storage.