Deferred calls: Autonomous smart contracts with guaranteed execution

[damip]

Intro

The current ASC model allows for rich wakeup conditions, execution ranges and so on.. but does not ensure 100% execution probability under congestion. We introduce Deferred Calls to complement existing ASCs. Deferred calls work this way:

• the smart contract asks for a quote on reserving a certain amount of gas at a target slot and gets one of two possible answers:
  • impossible to reserve that amount of gas a the target slot
  • it is possible to reserve that amount of gas at the target slot, here is how much it would cost
• when the smart contract does the reservation, it pays the quoted amount
• once reserved, the deferred call is 100% guaranteed to execute exactly at that moment

General considerations

• target slot needs to be exact. No ranges can be provided
• contrary to ASCs, no enable conditions can be provided
• we dedicate the current async slot gas at the beginning of each slot only to deferred calls
• we move the existing async pool execution at the end of each slot and make it use all the unused slot gas

ABIs

• deferred_call_quote(target_slot, max_gas) -> Option<Amount>: computes the coin cost of a deferred call. The retun value indicates whether the slot can be used (Some(X)) or cannot fit that max_gas anymore (None). Returns None if the slot is in the past, is the current slot (too late!), or is too far in the future by more than deferred_call_max_future_slots slots.
• deferred_call_register(target_slot, target_addr, target_func, params, coins, max_gas) -> CallID: registers a deferred call and returns its ID. This function spends coins + deferred_call_quote(target_slot, max_gas).unwrap() from the caller, fails if the balance is insufficient or if the quote would return None.
• deferred_call_exists(CallID) -> bool: returns true only if the deferred call is still exists and was not cancelled
• deferred_call_cancel(CallID): cancels a deferred call. Can only be called by the creator of the deferred call. Reimburses coins to the sender but not the deferred call fee to avoid spam. Cancelled items are not removed from storage to avoid manipulation, just ignored when it is their turn to be executed.

Emission

When deferred_call_register is called:

• fail if deferred_call_quote(target_slot, max_gas) == None
• debit deferred_call_quote(target_slot, max_gas).unwrap() + coins from the address on top of the call stack, fail if it can't be debited
• add the deferred call to the state db

Execution

Execution sequence of a slot s:

• For each unexecuted slot s since restart checkpoint (if any):
  • For each deferred call D targeting slot s in index order within the slot:
  • if D is not marked as cancelled:
    • execute D (transfer coins to target + run the function)
    • if the execution fails, cancel the effects of the previous step and transfer coins back to the sender
  • remove D from the db
  • if there is a block at slot s, execute the block
  • execute async messages as long as there is remaining gas in the slot (counting both unused max_async_gas and max_block_gas, and the latter can be used in full in case of block miss)

Storage

We need to limit how far in the future calls can be programmed according to the formula:

deferred_call_max_future_slots < floor(max_storage_space / (max_deferred_call_size * ceil(total_slot_gas / deferred_call_abi_gas)))

where:

• max_storage_space is a protocol constant on how much storage space we want to allocate to deferred calls
• max_deferred_call_size is a protocol constant about the max size of a deferred call item in storage
• total_slot_gas is the total gas of a slot (protocol constant)
• deferred_call_abi_gas is a protocol constant defining the gas used by the defer_call ABI.

Deferred call ID

Deferred calls need to be:

• listed in arrival order for a given target slot for execution and removal when executed
• an arbitrary deferred call needs to be directly accessed by ID for existence check or for cancellation

So let's define the deferred call ID as:
binary_id = [version as u64.to_be_bytes][target_slot as Slot.to_bytes_key][index_in_target_slot as u64.to_be_bytes][trail_hash]
text_id = 'D' + bs58check(binary_id)

The binary ID is used directly in DB for sorting and access.

Fee computation and usage

// Get the price it would cost to reserve "gas" at target slot "slot".
fn deferred_call_quote(slot, gas) -> Result<Amount> {
    // check if the slot is valid
    if(slot <= current_slot || slot > current_slot + deferred_call_max_future_slots) {
        return Err("Invalid target slot");
    }

    // check if there is enough gas available at the target slot
    if(gas > max_async_gas - slot.async_gas_booked) {
        return Err("No enough gas available at target slot");
    }
    
    // We perform Dynamic Pricing of slot gas booking using a Proportional-Integral controller (https://en.wikipedia.org/wiki/Proportional–integral–derivative_controller).
    // It regulates the average slot async gas usage towards `target_async_gas` by adjusting fees.
    
    // Constant part of the fee: directly depends on the base async gas cost for the target slot.
    // This is the "Integral" part of the Proportional-Integral controller.
    // When a new slot `S` is made available for booking, the `S.base_async_gas_cost` is increased or decreased compared to `(S-1).base_async_gas_cost` depending on the average gas usage over the `deferred_call_max_future_slots` slots before `S`.  
    let integral_fee = slot.base_deferred_call_gas_cost * gas;
    
    // The integral fee is not enough to respond to quick demand surges within the long booking period `deferred_call_max_future_slots`. Proportional regulation is also necessary.
    
    // A fee that linearly depends on the total load over `deferred_call_max_future_slots` slots but only when the average load is above `target_async_gas` to not penalize normal use. Booking all the gas from all slots within the booking period requires using the whole initial coin supply.
    let proportional_fee = compute_overbooking_fee(
        deferred_call_max_future_slots * max_async_gas,  // total available async gas during the booking period
        deferred_call_max_future_slots * max_async_gas / 2,  // target a 50% async gas usage over the booking period
        get_current_total_booked_async_gas(),  // total amount of async gas currently booked in the booking period
        gas, // amount of gas to book
        total_initial_coin_supply  // fully booking all slots of the booking period requires spending the whole initial supply of coins
    );
    
    // Finally, a per-slot proportional fee is also added to prevent attackers from denying significant ranges of consecutive slots within the long booking period. 
    let proportional_slot_fee = compute_overbooking_fee(
        max_async_gas,  // total available async gas during the target slot
        max_async_gas / 2,  // target a 50% async gas usage during the target slot
        slot.async_gas_booked,  // total amount of async gas currently booked in the target slot
        gas, // amount of gas to book in the target slot
        total_initial_coin_supply/10000  // fully booking 10000 consecutive slots (~1h40 of slots) requires spending the whole initial supply of coins
    );
    
    return Ok(integral_fee + proportional_fee + proportional_slot_fee);
}

// This function assumes that we have a resource with a total supply `resource_supply`.
// Below a certain target occupancy `target_occupancy` of that resource, the cost of a unit of the resource is zero.
// Above the target, the resource unit cost grows linearly with occupancy.
// The linear fee growth is chosen so that if someone occupies all the resource, they incur a `max_penalty` cost.
fn compute_overbooking_fee(resource_supply, target_occupancy, current_occupancy, resource_request, max_penalty) {
    let before = max(current_occupancy, target_occupancy) - target_occupancy;
    let after = max(current_occupancy + resource_request, target_occupancy) - target_occupancy;
    let denominator = resource_supply - target_occupancy;
    return max_penalty * (after*after - before*before) / (denominator*denominator)
}

fn when_slot_starts(S) {
    // At the very beginning of a slot execution, we make a new slot `S = current_slot + ASYNC_BOOKING_SLOT_COUNT` available for booking.
    // The deferred calls of the current slot are allowed to book new deferred calls up to that new slot in the future.
    // This algorithm is inspired by https://eips.ethereum.org/EIPS/eip-1559
    
    const BASE_FEE_MAX_CHANGE_DENOMINATOR = 8;
    const MIN_GAS_INCREMENT = 1 nano-massa;
    const MIN_GAS_COST = 10 nano-massa
    
    let TARGET_BOOKING = MAX_ASYNC_GAS/2;
    let total_booked_gas = get_current_total_booked_async_gas();
    let avg_booked_gas = total_booked_gas/ASYNC_BOOKING_SLOT_COUNT;

    if (avg_booked_gas == TARGET_BOOKING) {
	    S.base_fee_per_gas = (S-1).base_async_gas_cost;
	} else if (avg_booked_gas > TARGET_BOOKING) {
	    gas_used_delta = avg_booked_gas - TARGET_BOOKING;
	    S.base_fee_per_gas = (S-1).base_async_gas_cost + ((S-1).base_async_gas_cost * gas_used_delta / TARGET_BOOKING / BASE_FEE_MAX_CHANGE_DENOMINATOR, MIN_GAS_INCREMENT)
    } else {
	    gas_used_delta = TARGET_BOOKING - total_booked_gas
	    S.base_fee_per_gas = max((S-1).base_async_gas_cost - (S-1).base_async_gas_cost * gas_used_delta / TARGET_BOOKING / BASE_FEE_MAX_CHANGE_DENOMINATOR, MIN_GAS_COST)
    }
    
    set_current_total_booked_async_gas(total_booked_gas - current_slot.async_gas_booked);
}

fn deferred_call_register(target_slot, target_addr, target_func, params, coins, max_gas) -> Result<AsyncCallID> {
    let total_booked_gas = get_current_total_booked_async_gas();

    const CONST_ASYNC_GAS: u64 = XXXXX; // TODO calibrate: const_gas is the gas used even when gas=0 in order to process the item
    let effective_gas = CONST_ASYNC_GAS + max_gas;
    let fee = get_price(target_slot, effective_gas)?;
    
    // TODO: make sender pay `coins + fee`
    
    let call = /* ... */;
    let id = /* ... */ ;
    
    target_slot.booked_calls.push(callID, call);
    target_slot.async_gas_booked += effective_gas;
    
    set_current_total_booked_async_gas(total_booked_gas + effective_gas)
}

[peterjah]

To go further on future usage specification i would like to add:

• new ABIS, should be callable from JSONRPC and AS sdk (and GRPC ?)
• From a user perspective, it would be needed get quotes from a slot range, so i propose to implement deferred_call_quote with a Slot range as argument

[damip]

Here is an idea:

• ABIS (AS + WASMV1):
  • deferred_call_quote
  • deferred_call_register
  • deferred_call_exists
  • deferred_call_cancel
  • For ranges, the underlying implementation would anyway be querying slots one by one, so might as well keep the L1 simple and put SC range queries in as-sdk
• APIS (JSONRPC + GRPC): no need for it for now, maybe as a followup:
  • deferred_call_quote
  • deferred_call_get_info(id) to integrate in QueryState for GRPC or as standalone for jsonrpc
  • list_deferred_calls(slot)
  • Note: those are not very important except maybe for explorer purposes so for now, no need to implement them, they can be discussed as followup

[peterjah]

Yes! lets split in 2 different milestones!
But i have the feeling that getting the asc slot booking price without actually deploying/executing a SC will be needed

[damip]

What use case do you imagine that will need the deferred_call_quote API call ?

[peterjah]

I think as soon as you have to pay for something (to scheduele a call) its always good to know the price before actually doing it.
But yeah its maybe not the top priority and there is most likely workarounds to this

[damip]

Those reservations are made by smart contracts themselves, and they should be the ones probing the price atomically. Any API quote can be quickly outdated

[modship]

It might be a good idea to add a function that retrieves the first available slot bookable based on the gas ?
This would save the user from having to test each slot

[damip]

This raises 2 questions:

• "bookable" is very relative: maybe a slot is bookable but will cost millions to book
• the way it would be implemented would essentially be exactly what you said: check slots one by one and return the best one. So might as well do this within the smart contract itself (maybe through a function in the as-sdk that hides it from the dev) so that we don't make the L1 more complicated than necessary

[peterjah]

Given that a significant use case involves scheduling recurring calls outside of the one-week bookable window, the proposed solution of using a deferred call to book another future call presents a major challenge: How can we ensure that the future booking will succeed?

• Failure could occur if the calling contract lacks sufficient coins (for storage or booking fees).
• The deferred call serves two purposes: executing the scheduled task and booking the next call. If the task fails, the booking of the next call will also fail, which is likely an unintended side effect.
  The root issue lies in combining the scheduling mechanism with the execution of the smart contract task. This introduces considerable complexity for smart contracts and may compromise the fully decentralized execution model that this feature aims to achieve.

To address this, I propose adding an additional feature: a recursive_deferred_call that accepts a period as an argument. The recursive scheduling would be managed by the node itself, outside of the runtime environment. This approach would separate the scheduling functionality from the execution of the task, ensuring that even if the task fails, the next scheduled call is still triggered.

Of course, this does not resolve issues related to insufficient coins for future calls.

I didn't though about how the bookable fee of a such feature could be computed, but i think the idea worth to be considered.