(async backing) parachain-system: track limitations for unincluded blocks

[slumber]

Resolves #1866

[slumber]

Also #1866 mentions

UMP messages processed

Was it supposed to be dmp?

[slumber]

And do we want it in master?

[rphmeier]

And do we want it in master?

Only if it's fully backwards-compatible

Was it supposed to be dmp?

Yes, just an oversight. I've edited the original issue to fix this as well. Thanks

[slumber]

Do we want to account for max_ancestry_len here?

[rphmeier]

Do we want to account for max_ancestry_len here?

This comment doesn't point to any specific code, not sure what you mean by this.

[slumber]

Do we want to account for max_ancestry_len here?

This comment doesn't point to any specific code, not sure what you mean by this.

It's a general question, we have max unincluded length which is mapped to max_candidate_depth

But async backing parameters also have allowed_ancestry_len, which we're not accounting for in this PR

[rphmeier]

But async backing parameters also have allowed_ancestry_len, which we're not accounting for in this PR

We could account for it in the runtime, but allowed_ancestry_len is a parameter of the relay chain block where the parachain block gets backed, whereas candidate_depth is a property of the relay chain block that is set as the relay-parent. We don't know ahead of time at one point a fresh parablock will get posted to the relay chain, so I'd imagined that the consensus/authoring layer that chooses relay parents would account for that instead.

[slumber]

I believe there's no other way to get this value for bandwidth limits besides putting it into storage.
Second option is to introduce migration and save its len along with ProcessedDownwardMessages and rename it into something like DownwardQueueSize.

[rphmeier]

If it's cleared in on_finalize then it never hits storage in practice, only the in-memory overlay.

[slumber]

It's inserted in set_validation_data, fetched in on_finalize, cleared in on_initialize

[rphmeier]

two things stand out to me:

• only the length of the list is used, not the messages themselves
• it could be cleared in on_finalize too, right?

[rphmeier]

it also doesn't seem well-defined to me what pending_downward_messages is in this context.

i.e. is it all pending downward messages that were present in the relay-parent state? If so, then we need to update process_inbound_downward_messages to avoid processing them in duplicate.

or is it only the pending downward messages which haven't been processed? In that case, we shouldn't have to make any changes to the processing logic.

For efficiency's sake, I'd favor the latter - we shouldn't include the same downward messages into parachain blocks multiple times.

horizontal logic probably needs the same, no?

[slumber]

I followed the pattern of cleaning everything in on_initialize

As for the messages, yes, I suggested in the first comments to either put only len or

Second option is to introduce migration and save its len along with ProcessedDownwardMessages and rename it into something like DownwardQueueSize.

[slumber]

i.e. is it all pending downward messages that were present in the relay-parent state? If so, then we need to update process_inbound_downward_messages to avoid processing them in duplicate.

These are all messages from the downward queue. They're not used for processing in the current code, so it's not an issue. We should probably just use len.
But the thing is, according to the code each candidate exhaust the dm queue:

let dm_count = downward_messages.len() as u32;
// ...
ProcessedDownwardMessages::<T>::put(dm_count);

Do we even need to track this limitation at all?

[rphmeier]

If we assume that ParachainInherentData only contains messages which have not been posted to any prior parachain block (which is the way it should be to avoid redundancy), I don't know that that current bandwidth limits approach works.

The bandwidth limits for DMP/HRMP-in should be based on the total size of the incoming queues at the relay-parent, whereas some (or most) of the messages may have already been processed.

[rphmeier]

according to the code each candidate exhaust the dm queue

Yes, this looks currently guaranteed. We could avoid tracking this property as a result, and reintroduce the concept if we get to the point where we implement things more piecemeal.

[rphmeier]

Yes, it does - parachains manually adjusting MaxUnincludedLen would potentially brick their chain. This is fine as long as there is a warning in the storage item doc comment.

We will need good ways for parachains to adjust this.

[slumber]

I basically referenced #2438 (comment), need to link to a github issue too before the merge

[rphmeier]

I just realized something - I don't think that we can actually access the relay-chain state proof within the on_initialize area of code. It only gets handled later, in the set_inherent_data inherent. The logic we currently have in on_initialize needs to go there, actually.

[slumber]

I just realized something - I don't think that we can actually access the relay-chain state proof within the on_initialize area of code. It only gets handled later, in the set_inherent_data inherent.

Data put into storage by set_validation_data (= create_inherent) is available in on_finalize and dropped in on_initialize, why isn't state proof accessible?

[rphmeier]

I think the answer here, now that we've moved this to run in the inherent, is "it doesn't".

on_initialize always runs before inherents, so any pallet setting the MaxUnincludedLen in code that executes in the on_initialize of any pallet would run before this, so it is safe.

[slumber]

What about scenario when there're 4 blocks in the unincluded segment, and the len is reduced to e.g. 1?

[rphmeier]

That is fine. For example, assume that there is some unconditional code that sets the maximum length to 1 in on_initialize at a certain block height. For more dynamically controlled systems, the reasoning is the same but slightly more complicated.

Let's say we are trying to build on a parent parachain block B, which has unincluded segment length 4. As long as the unincluded segment has length > 1, we can't author a new block. But new relay-parents will "drain" the unincluded segment from the back, so eventually* we will have a (relay-parent, B) pair where the unincluded segment will be short enough to push a new block B' onto. This is the desired behavior of shrinking the maximum segment length.

*eventually assuming B and its ancestors actually get backed and included in the relay chain

[slumber]

I see, failing create_inherent is ok, while on_initialize isn't. Thanks for explanation.

[rphmeier]

It's rather that we are creating a block validity rule based on the relay-parent. It'll be impossible to create valid blocks on top of some (para parent, relay parent) pairs, and that failure will manifest in the inherent that handles the storage proof. Anything that runs in on_initialize could alter the block validity condition (e.g. altering the maximum unincluded segment length). We will probably also want hooks for people to write dynamic block validity rules based on the relay-parent in question. These dynamic rules would need the relay parent and storage proof as a parameter, so they could only run in the inherent, once those things have been supplied.

In principle, it's OK to fail in on_initialize as well, but there are footguns like changing the unincluded segment length that could lead the chain to brick.

[rphmeier]

A comment for the benefit of other reviewers: the "node-side" here could also include the ProvideInherent implementation. That is probably the best way to ensure this in practice. Just trimming some MQCs.

[rphmeier]

Looks good!