NEP-541: Transaction priority fee

neps/nep-0541.md

@@ -0,0 +1,196 @@
+---
+NEP: 541
+Title: Transaction priority fee
+Authors: Bowen Wang <bowen@near.org>, Jakob Meier <inbox@jakobmeier.ch>
+Status: New
+DiscussionsTo: https://github.com/near/NEPs/pull/541
+Type: Protocol
+Version: 1.0.0
+Created: 2024-04-02
+LastUpdated: 2024-04-03
+---
+
+## Summary
+
+This NEP proposes transaction priority fee, which is an optional fee that a user can attach to get their transactions and subsequent receipts prioritized during processing.
+Prioritizing transactions is useful when the network is congested and users are willing to pay more to get their transactions processed first. It also deters spamming from arbitrage bots under normal circumstances. When a priority fee is attached, a percentage is burnt and the rest is rewarded to the chunk producer who includes the transaction.
+
+## Motivation
+
+Currently there is no way in the protocol for a user to prioritize their transaction against other transactions. As short-term congestion becomes more prevalent, many users complain
+that there is no way to get their transaction through even if they would like to pay more.
+This is especially frustrating for defi users who care less about transaction fee and more about latency. The lack of prioritization mechanism also makes it difficult for liquidation bots and price oracles to work during congestion.
+With congestion becoming more regular, it is important that for users who are willing to pay more, they can still use the network with a decent user experience.
+
+## Specification
+
+We add a `priority_fee` field to `Transaction`:
+```rust
+pub struct Transaction {
+    /// An account on which behalf transaction is signed
+    pub signer_id: AccountId,
+    /// A public key of the access key which was used to sign an account.
+    /// Access key holds permissions for calling certain kinds of actions.
+    pub public_key: PublicKey,
+    /// Nonce is used to determine order of transaction in the pool.
+    /// It increments for a combination of `signer_id` and `public_key`
+    pub nonce: Nonce,
+    /// Receiver account for this transaction
+    pub receiver_id: AccountId,
+    /// The hash of the block in the blockchain on top of which the given transaction is valid
+    pub block_hash: CryptoHash,
+    /// A list of actions to be applied
+    pub actions: Vec<Action>,
+    /// Optional priority fee (unit is 10^12 yotcoNEAR)
+    pub priority_fee: u64
+}
+```
+
+Here, `priority_fee * 10^12` is the amount of yotcoNEAR attached. We do not use `u128` here because there is no need to have that level of granularity and also save 8 bytes per transaction.
+
+We define the priority of a transaction to be `priority_fee / attached_gas`, where attached gas is the sum of all action costs and attached gas to function call actions.
+Receipts that spawn from a transaction have the same priority as the transaction itself.
+
+Regardless of priority fee, a transaction still needs to pay the base cost, which is determined by `minimum_gas_price`. The current dynamic gas price mechanism will be removed since it does not make sense for the congestion of one shard to affect the gas price of transaction in a completely different shard, even if there is no congestion there.
+
+When a chunk is produced, the transactions should be sorted by priority in descending order. Otherwise the chunk is invalid.
+However, when the priority of two transactions are the same, no specific order is enforced by the protocol.
+
+When a transaction is processed and converted to a receipt, X percent (X is TBD) of the priority fee is burnt and the rest is rewarded to the chunk producer.
+During receipt processing, receipts with highest priorites (from incoming and delayed receipts combined) will get processed first, but up to a fixed amount of gas (the threshold is TBD). The rest of the gas limit is devoted to processing all receipts, regardless of priority, in a FIFO order. This is to ensure that the system does not deadlock in light of [NEP-539](https://github.com/near/NEPs/pull/539).
+
+## Reference Implementation
+
+We discuss the implementation of two important pieces of the proposal, receipt processing and validator reward, in this section.
+
+### Receipt Processing
+
+A max heap is created in the state for storing priorities of delayed receipts. More specifically, we add a trie column `RECEIPTS_PRIORITY_QUEUE: u8 = 13`. Each value in this heap is of the form
+```rust
+struct ReceiptPriority {
+    /// index of the corresponding receipt in the delayed receipts queue. Use u32 here to save some space in the state
+    receipt_index: u32
+    /// priority of the receipt
+    priority: u64
+}
+```
+Since each value is quite small and we don't expect the delayed receipt queue to be extremely long (even if there are 100k delayed receipts, it is only 1.2MB in size), and it only needs to be read and write once per chunk, this heap can be stored as a single value in the state.
+
+The receipt processing order is roughly as follows:
+
+```python
+# Delayed receipts is a max heap and incoming receipts is sorted by priority in descending order
+# gas limit refers to the gas limit assigned to process receipts with priority
+def process_priority_receipts(delayed_receipts, incoming_receipts, gas_limit):
+    if delayed_receipts.empty() and incoming_receipts.empty():
+        return
+    gas_used = 0
+    while gas_used < gas_limit:
+        delayed_receipt_head = if delayed_receipts.empty() {-Inf} else {delayed_receipts.top() }
+        incoming_receipt_head = if incoming_receipts.empty() {-Inf} else {incoming_receipts.top() }
+        receipt = None
+        if delayed_receipt_head.priority > incoming_receipts_head.priority: 
+            delayed_receipts.pop()
+            receipt = get_receipt(delayed_receipt_head.index) # get_receipt removes receipt from delayed receipt queue
+        else:
+            receipt = incoming_receipts.pop()
+        gas_used += process_receipt(receipt).gas_used
+```
+The regular order of receipt processing (same as today) resumes after `process_priority_receipts` is finished.
+
+Note that to efficiently delete from the max heap as part of FIFO receipt processing, a delayed receipt should also store its position in the max heap. This would allow for a O(logn) deletion. In other words, the process of persisting a delayed receipt is as follows:
+```python
+
+def store_delayed_receipt(receipt):
+    # index in the delayed receipt queue
+    next_available_index = get_delayed_receipt_index()
+    # get the index of the receipt priority structure in the heap
+    priority_index = store_receipt_priority(ReceiptPriority(receipt.priority, next_available_index))
+    # store the actual receipt with index in the heap
+    store_receipt(DelayedReceipt(receipt, priority_index))
+```
+
+### Validator Reward
+
+Even though conceptually, a chunk producer is rewarded a portion of the priority fee for each transaction included in the chunk. We cannot implement this naively due to two reasons:
+- the chunk producer account may be on a different shard;
+- the staking logic dictates that validator reward be applied to validator accounts at the beginning of an epoch.
+
+To make this computation easy, we add a new field `total_priority_fee: u64` to chunk header that stores the sum of all priority fees of transactions included in this chunk. Then, at the end of an epoch, we tally the cumulative priority fees for each chunk producer (in implementation it can be updated on each block) and then add it to their staking reward.
+If a chunk producer does not meet the online requirement for an epoch, they are not eligible for reward based on priority fees for the entire epoch.
+From the perspective of total supply of the protocol, the priority fee is first all burnt when a transaction is processed and then at the beginning of the next epoch, some tokens are minted to reward chunk producers for including those transactions.
+
+One thing to note here is that since `total_priority_fee` should be within `u64` range, we should cap the maximum of `priority_fee` for each transaction at ~1800N (this generously assumes that a chunk may contain up to 10,000 transactions).
+
+
+## Security Implications
+
+[Explicitly outline any security concerns in relation to the NEP, and potential ways to resolve or mitigate them. At the very least, well-known relevant threats must be covered, e.g. person-in-the-middle, double-spend, XSS, CSRF, etc.]
+
+## Alternatives
+
+One clear alternative is to not have FIFO processing at all and make all receipt processing purely based on priority. This, however, would create the problem that some receipts without priority can be indefinitely delayed when receipts with higher priority keeps coming in.
+Then there is the question of whether those receipts should pay for its storage in the state since there is no bound on when they may get processed.
+Furthermore, since there is no way to cancel a receipt after it is created, even if a user notices that their receipt is stuck in the queue for a long time and would like to cancel,
+they won't be able to do it.
+
+## Future possibilities
+
+This NEP should be combined with [NEP-513](https://github.com/near/NEPs/pull/539). They together redefine how congestion is handled and how users can still send transactions during congestion. It is possible to explore more complex mechanism on priority fees when there is congestion. For example, the protocol could require that transactions to a congested shard must attach a priority fee and even place a minimum on the priority fee based on previous chunk's priority fees.
+
+## Consequences
+
+[This section describes the consequences, after applying the decision. All consequences should be summarized here, not just the "positive" ones. Record any concerns raised throughout the NEP discussion.]
+
+### Positive
+
+* p1
+
+### Neutral
+
+* n1
+
+### Negative
+
+* n1
+
+### Backwards Compatibility
+
+[All NEPs that introduce backwards incompatibilities must include a section describing these incompatibilities and their severity. Author must explain a proposes to deal with these incompatibilities. Submissions without a sufficient backwards compatibility treatise may be rejected outright.]
+
+## Unresolved Issues (Optional)
+
+[Explain any issues that warrant further discussion. Considerations
+
+* What parts of the design do you expect to resolve through the NEP process before this gets merged?
+* What parts of the design do you expect to resolve through the implementation of this feature before stabilization?
+* What related issues do you consider out of scope for this NEP that could be addressed in the future independently of the solution that comes out of this NEP?]
+
+## Changelog
+
+[The changelog section provides historical context for how the NEP developed over time. Initial NEP submission should start with version 1.0.0, and all subsequent NEP extensions must follow [Semantic Versioning](https://semver.org/). Every version should have the benefits and concerns raised during the review. The author does not need to fill out this section for the initial draft. Instead, the assigned reviewers (Subject Matter Experts) should create the first version during the first technical review. After the final public call, the author should then finalize the last version of the decision context.]
+
+### 1.0.0 - Initial Version
+
+> Placeholder for the context about when and who approved this NEP version.
+
+#### Benefits
+
+> List of benefits filled by the Subject Matter Experts while reviewing this version:
+
+* Benefit 1
+* Benefit 2
+
+#### Concerns
+
+> Template for Subject Matter Experts review for this version:
+> Status: New | Ongoing | Resolved
+
+|   # | Concern | Resolution | Status |
+| --: | :------ | :--------- | -----: |
+|   1 |         |            |        |
+|   2 |         |            |        |
+
+## Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).