Skip to content

Transaction

Jump to bottom
Gennady Laventman edited this page Sep 23, 2020 · 4 revisions

Transaction flow and structure

Transaction Flow

Transaction creation/execution

Transaction creation/execution done in Client SDK and includes following steps

  • Creation Transaction Context during db.Begin() call
  • Sending statement(s) from Client to Client SDK
  • Collecting RSets and MVCC data for each txCtx.Get() call
  • Calculating WSets for each txCtx.Put() call
  • Composing all this data to transaction, including signature of current user
  • Revalidate MVCC, to fail Tx early, see Transaction Isolation
  • Passing result Tx to Sequencer

Transaction Isolation algorithm is part of Transaction creation/execution process

  • MVCC version for each key is composed from ledger height (block number) and Tx number, same as Fabric
  • During creation Tx context - db.Begin() call, ledger height (last known block number) is stored as part of context
  • During each txCtx.Get() call inside Transaction, RSet version validated again ledger height stored in context
    • If block number in version is bigger that stored in context, Tx marked as invalid and will fail during call to txCtx.Commit()
    • We will not fail during after txCtx.Get() call not to mess with client side logic
    • In diagram it marked as Early Fail Tx (1)

When a Sequencer receives a transaction envelope, it needs to check whether the signer is a member of the blockchain DB. When the envelope is signed by the valid signer, the transaction would be added to a queue.

Transaction Serialization

  • At the block timeout, reaching size limit or receiving configuration change (admin API call), the leader node would pick the next batch of transactions to be included in the next block. The leader can pick using one the following option provided by the user:
    1. FIFO_ORDER -- fair to all clients
    2. REORDER_FOR_CONFLICT_REDUCTION -- trading fairness for the performance
    3. MAXIMIZE_PARALLEL_VALIDATION -- trading fairness for the performance

For more details about transaction serialization algorithm, see Transaction Serialization

Important remark - to reduce amount of inter-block conflicts, block transactions RSets re-checked before block creation (before serialization) and block pipelining will be disabled (at least, for PoC), i.e. new block will start to form in leader only after previous block was applied to local database. Transaction failure during this check marked as Early Fail Tx (2)

Important remark 2 - configuration change will be stored in block as well.

Consensus and block distribution

Once transaction serialization done, a block created in leader node, consensus would be executed across nodes. Once the majority of the nodes agree on the created block (as per the employed consensus algorithm), the block would be added to the block queue in each node.

From this point all operations executed sequentially on each network node, not only on Sequencer

Appending the Block to Block Store

  • As the block store is the source of truth, we maintain it in the file system similar to what Fabric v0.6 and v2.0 do. We can copy the block store code from the Fabric v2.0.
  • Although Sequencer remove transaction conflicts, at least for REORDER_FOR_CONFLICT_REDUCTION and MAXIMIZE_PARALLEL_VALIDATION, so transaction should not fail during validation of RWSets and MVCC, but in byzantine environment we can have malicious leader and we need to store transaction status (OK/FAIL) after performing transaction execution
    • This is true only if block pipelining disabled, because of possibility of inter-block conflicts.
    • For FIFO_ORDER, we should do it in CFT as well.

Transactions scheduling

  • Transaction scheduler tried to block transactions for parallel validation/execution
    • For MAXIMIZE_PARALLEL_VALIDATION, transactions already arranged in blocks for parallel execution
    • For REORDER_FOR_CONFLICT_REDUCTION, rw dependency graph for transaction in block is build and transaction executed based on BFS order
    • For FIFO_ORDER, because rw dependency graph may contain cycles, much simpler algorithm used
      • Tx in block passed one by one, checking for access to same data entries
      • Once suspect to any conflict appear, already accumulated (passed) Tx are sent to Executor and new Tx accumulation is started

Provenance data creation and data integrity proofs

At PoC phase, we will store provenance data in KV Store, as rest of the blockchaindb data

Provenance data structure:

  • After Tx validation/execution, Provenance data Tx created from its RWSet.
    • This is done to add transaction execution status to provenance data.
  • For each entry in WSet, tuple {Key, newValue, BlockTime, BlockNumber, TxTime, TxId, ClientId, IsDelete, IsOk} added to database
    • Key will be combination of Key || TxId
    • This data should be indexed by Key || BlockTime and Key || TxTime, to search by Key and time
  • In addition, we will create reverse index that maps TxId to BlockNumber
    • To easy find block that contains specific transaction

For proof generation and example, see Transaction Proof document

Transaction validation/commit

Basic assumption is no conflicting transactions send to be validated/committed in parallel

  • We revalidate Tx RSet correctness, by reading same values again and checking MVCC values
  • For database that support WSet apply, we just apply WSet
  • For database that not support WSet apply, we just execute Put statements in transaction, one by one

Important note - if block contains configuration change (admin API call), it will be executed after all transactions validation/commit.

Handling responses

We have multiple network nodes that commit transactions. Some ot the nodes may fail, become byzantine, so how we should handle node responses to transaction

  • Each node that sends response to SDK, should sign this response.
    • Response send using pub/sub primitive, explained in Client SDK APIs and Server APIs documents.
      • In addition, regeneration of response possible, see same documents
  • SDK collect required number of responses, defined in TxOptions - per transaction or DBOptions - for database and only after that pass result of transaction execution to client
    • For KV Store, only OK/FAIL responses are possible

We define multiple policies to handle responses

  • MAJORITY - more that 2/3*N nodes responded with same result
  • MIN_BFT - more that 1/3*N + 1 nodes responded with same result
  • MIN - just one node responded

Transaction structure

Transaction structure inspired by Transaction envelop structure from Fabric.

  • Envelop
    • Signature
    • Payload
      • Header
      • Transaction
        • Statement(s)
          • Operation
          • Argument(s)
        • KVRWSet
          • KVRead(s)
            • key
            • version
          • KVWrite(s)
            • key
            • is_delete
            • value
syntax = "proto3";

package transaction;


// Envelope wraps a Payload with a signature so that the message may be authenticated
message Envelope {
    // A marshaled Payload
    bytes payload = 1;

    // A signature by the creator specified in the Payload header
    bytes signature = 2;
}

// Payload is the message contents (and header to allow for signing)
message Payload {

    // Header is included to provide identity and prevent replay
    Header header = 1;

    // Encoded transaction
    bytes transaction = 2;
}

message Header {
    // Creator of the message, a marshaled msp.SerializedIdentity
    bytes creator = 1;

    // Arbitrary number that may only be used once. Can be used to detect replay attacks.
    bytes nonce = 2;
}

message Transaction {
    enum DataModel {
        KV = 0;
    }

    DataModel datamodel = 1;
    repeated Statement statements = 2;
    bytes rwset = 3;
}

message KVRWSet {
    repeated KVRead rset = 1;
    repeated KVWrite wset = 2;
}

message KVRead {
    string key = 1;
    bytes version = 2;
}

message KVWrite {
    string key = 1;
    bool is_delete = 2;
    bytes value = 3;
}

message Statement{
    string operation = 1;
    repeated bytes arguments = 2;
}
Clone this wiki locally