Refactor - Delay visibility of program un-/re-/deployment #29654

Lichtso · 2023-01-11T20:00:35Z

Problem

Currently some changes to programs made by un-/re-/deployment are visible immediately, even within the same transaction. These inconsistency between top-level instructions and CPI should be removed. This is a preparation for the replacement of the current executor cache by one that supports a bitemporal model. Once that is done, the new executor cache will further delay the visibility from the end of the transaction to a slot in the future, with a constant offset from the slot the action occurred in.

Summary of Changes

Use three separate HashMaps instead of the enum TxBankExecutorCacheDiff
Replaces all places which deploy programs by a macro
Load previous version into the cache before deploying the new one
Adjusts tests for un-/re-/deployment in same transaction
Adds a feature gate

Old vs New Behavior (inside the same transaction)

Initial Deployment: Was visible to CPI but not transaction level, will become invisible to both
Upgrade / Redeployment: Was visible to both, will become unavailable to both
Close / Undeployment: Was invisible to both, will become visible to both

In other words the new consistent behavior is, that whenever a program is un-/re-/deployed,
it becomes unavailable for the rest of the transaction.

Feature Gate Issue: #30085

jackcmay · 2023-02-07T06:29:43Z

program-runtime/src/executor_cache.rs

+                self.executors.insert(key, Some(executor.clone()));
+            } else {
+                // Do not overwrite an existing entry
+                self.executors.entry(key).or_insert(None);


Why insert None if not found?

That is essentially an explicit invalid entry similar to a tombstone. So that if a program was not deployed before the transaction begins, and it is deployed during the transaction, that the rest of the transaction still sees it as undeployed. In other words it is necessary to get case 1 (deployment invisibility) right.

tao-stones

Catching up on the issue, asked few questions. Will deep dive again.

program-runtime/src/executor_cache.rs

runtime/src/bank/tests.rs

program-runtime/src/executor_cache.rs

programs/bpf_loader/src/lib.rs

jackcmay · 2023-02-09T06:18:09Z

programs/bpf_loader/src/lib.rs

+            // Make sure the old version is in the cache so that we don't
+            // load the new version from the database while the old one is still active
+            let mut get_or_create_executor_time = Measure::start("get_or_create_executor_time");
+            let result = create_executor_from_account(


deploy might jit compile a program and place it in the cache even if it might never be called by this transaction?

Yes, unfortunately that is necessary because we overwrite the original program in its account. So if it was called later there would be no way to load the old version, which is necessary so that the change stays invisible during the transaction.

Or we fail any attempts to do so. Aka, policy is you cannot call a program that you have previously upgraded in the same transaction

Yes that would also be an option. This PR is only leading up to the replacement of the executor cache. The new one will extend the delay of visibility of changes way beyond the end of the transaction, to multiple slots.

But I like the idea, let me try to change it so, that recently redeployed programs are essentially not deployed, which causes the transaction to fail.

runtime/src/bank.rs

program-runtime/src/executor_cache.rs

runtime/src/bank.rs

jackcmay · 2023-02-09T06:39:14Z

programs/bpf_loader/src/lib.rs

+            // Executor exists and is cached, use it
+            Some(Some(executor)) => return Ok((executor, None)),
+            // We cached that the Executor does not exist, abort
+            Some(None) => return Err(InstructionError::InvalidAccountData),


This looks a little scary since it is a new error condition that is not directly featurized. Afaict this case can only occur once the delay feature is activated, but that is a bit distant and could potentially lead to confusion or a missed code path

Hm yes, I added a comment to point out that this case only exists when the feature is active.
The thing is, if we made a condition here that would check the feature, it would still result in an error being returned.
Not much gained IMO.

Yeah, comment would help

program-runtime/src/executor_cache.rs

tao-stones

lgtm, the deploy program macro made tests easier to read too 👍🏼

tao-stones · 2023-02-10T20:29:45Z

program-runtime/src/executor_cache.rs

+        if upgrade {
+            if delay_visibility_of_program_deployment {
+                // Place a tombstone in the cache so that
+                // we don't load the new version from the database as it should remain invisible


## Summary solana-labs/solana#29654 introduced a delay between the slot a program is deployed and when it can be called. Let's build this into the loader now so as to remove a footgun from folks trying to call programs immediately after deploying them. ## Test Plan ``` pnpm test:live-with-test-validator ```

Lichtso added work in progress This isn't quite right yet feature-gate Pull Request adds or modifies a runtime feature gate labels Jan 11, 2023

Lichtso force-pushed the refactor/delay_visibility_of_program_deployment branch 5 times, most recently from 689c831 to d0f1e9c Compare January 12, 2023 17:30

Lichtso mentioned this pull request Jan 14, 2023

Revert "Cap accounts data a transaction can load by its requested limit" #29373

Merged

Lichtso force-pushed the refactor/delay_visibility_of_program_deployment branch 3 times, most recently from 7c56ecf to d631faf Compare January 24, 2023 09:04

Lichtso mentioned this pull request Jan 24, 2023

Program-Runtime v2 - Road Map #28755

Open

Lichtso force-pushed the refactor/delay_visibility_of_program_deployment branch 5 times, most recently from b579aa5 to 115644a Compare January 30, 2023 18:07

Lichtso mentioned this pull request Jan 31, 2023

LoadedPrograms #29803

Closed

Lichtso force-pushed the refactor/delay_visibility_of_program_deployment branch from 115644a to a729199 Compare February 2, 2023 11:18

Lichtso mentioned this pull request Feb 2, 2023

Feature Gate: Delay visibility of program deployment #30085

Open

Lichtso force-pushed the refactor/delay_visibility_of_program_deployment branch from a729199 to 76cc1e7 Compare February 2, 2023 11:19

Lichtso removed the work in progress This isn't quite right yet label Feb 2, 2023

Lichtso requested review from tao-stones and jackcmay February 2, 2023 14:34

Lichtso force-pushed the refactor/delay_visibility_of_program_deployment branch 2 times, most recently from a6fa38a to 90850fb Compare February 6, 2023 22:57

jackcmay reviewed Feb 7, 2023

View reviewed changes

tao-stones reviewed Feb 7, 2023

View reviewed changes

program-runtime/src/executor_cache.rs Outdated Show resolved Hide resolved

program-runtime/src/executor_cache.rs Show resolved Hide resolved

runtime/src/bank/tests.rs Outdated Show resolved Hide resolved

program-runtime/src/executor_cache.rs Outdated Show resolved Hide resolved

Lichtso force-pushed the refactor/delay_visibility_of_program_deployment branch from 90850fb to 28f3ed2 Compare February 8, 2023 16:40

jackcmay reviewed Feb 9, 2023

View reviewed changes

programs/bpf_loader/src/lib.rs Outdated Show resolved Hide resolved

jackcmay reviewed Feb 9, 2023

View reviewed changes

runtime/src/bank.rs Outdated Show resolved Hide resolved

jackcmay reviewed Feb 9, 2023

View reviewed changes

program-runtime/src/executor_cache.rs Outdated Show resolved Hide resolved

jackcmay reviewed Feb 9, 2023

View reviewed changes

runtime/src/bank.rs Outdated Show resolved Hide resolved

jackcmay reviewed Feb 9, 2023

View reviewed changes

program-runtime/src/executor_cache.rs Outdated Show resolved Hide resolved

Lichtso force-pushed the refactor/delay_visibility_of_program_deployment branch 4 times, most recently from b3d1b70 to 473f9d5 Compare February 9, 2023 19:34

jackcmay previously approved these changes Feb 9, 2023

View reviewed changes

Lichtso dismissed jackcmay’s stale review via 3bc4d2f February 9, 2023 22:29

Lichtso force-pushed the refactor/delay_visibility_of_program_deployment branch 2 times, most recently from 3bc4d2f to b18c0da Compare February 9, 2023 23:48

Lichtso added 4 commits February 10, 2023 11:31

Use three separate HashMaps instead of the enum TxBankExecutorCacheDiff.

90bbc6e

Replaces all places which deploy programs by a macro.

d22fe11

Adds a feature gate.

5c8c6c5

Adjust tests.

cbe0cb2

Lichtso force-pushed the refactor/delay_visibility_of_program_deployment branch from b18c0da to 9a469fe Compare February 10, 2023 11:01

Makes undeployment visible immediately.

640ef50

Lichtso force-pushed the refactor/delay_visibility_of_program_deployment branch from 9a469fe to 640ef50 Compare February 10, 2023 12:30

Lichtso changed the title ~~Refactor - Delay visibility of program deployment~~ Refactor - Delay visibility of program un-/re-/deployment Feb 10, 2023

tao-stones approved these changes Feb 10, 2023

View reviewed changes

Lichtso merged commit 6558c8f into solana-labs:master Feb 11, 2023

Lichtso deleted the refactor/delay_visibility_of_program_deployment branch February 11, 2023 10:18

steveluscher mentioned this pull request May 24, 2023

fix: wait until the slot after a program is deployed solana-labs/solana-web3.js#1309

Merged

Lichtso mentioned this pull request Nov 14, 2023

SIMD XXXX: Enable Core BPF Programs buffalojoec/solana-improvement-documents#2

Closed

topointon-jump mentioned this pull request May 29, 2024

flamenco: bpf loader rewrite firedancer-io/firedancer#1905

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor - Delay visibility of program un-/re-/deployment #29654

Refactor - Delay visibility of program un-/re-/deployment #29654

Lichtso commented Jan 11, 2023 •

edited

Loading

jackcmay Feb 7, 2023

Lichtso Feb 7, 2023

tao-stones left a comment

jackcmay Feb 9, 2023 •

edited

Loading

Lichtso Feb 9, 2023

jackcmay Feb 9, 2023

Lichtso Feb 9, 2023 •

edited

Loading

jackcmay Feb 9, 2023 •

edited

Loading

Lichtso Feb 9, 2023

jackcmay Feb 9, 2023

tao-stones left a comment

tao-stones Feb 10, 2023

Refactor - Delay visibility of program un-/re-/deployment #29654

Refactor - Delay visibility of program un-/re-/deployment #29654

Conversation

Lichtso commented Jan 11, 2023 • edited Loading

Problem

Summary of Changes

Old vs New Behavior (inside the same transaction)

jackcmay Feb 7, 2023

Choose a reason for hiding this comment

Lichtso Feb 7, 2023

Choose a reason for hiding this comment

tao-stones left a comment

Choose a reason for hiding this comment

jackcmay Feb 9, 2023 • edited Loading

Choose a reason for hiding this comment

Lichtso Feb 9, 2023

Choose a reason for hiding this comment

jackcmay Feb 9, 2023

Choose a reason for hiding this comment

Lichtso Feb 9, 2023 • edited Loading

Choose a reason for hiding this comment

jackcmay Feb 9, 2023 • edited Loading

Choose a reason for hiding this comment

Lichtso Feb 9, 2023

Choose a reason for hiding this comment

jackcmay Feb 9, 2023

Choose a reason for hiding this comment

tao-stones left a comment

Choose a reason for hiding this comment

tao-stones Feb 10, 2023

Choose a reason for hiding this comment

Lichtso commented Jan 11, 2023 •

edited

Loading

jackcmay Feb 9, 2023 •

edited

Loading

Lichtso Feb 9, 2023 •

edited

Loading

jackcmay Feb 9, 2023 •

edited

Loading