Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(zk_toolbox): add fees integration test to toolbox #2898

Merged
merged 56 commits into from
Sep 30, 2024
Merged
Changes from 1 commit
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
51381cf
feat: add fees integration test to toolbox
manuelmauro Sep 17, 2024
19b0976
ci: add CI step for fees integration test
manuelmauro Sep 17, 2024
533655b
Merge branch 'main' into manuel-add-ts-integration-fees-test-to-toolbox
manuelmauro Sep 17, 2024
4f586d5
ci: add CI step for fees integration test
manuelmauro Sep 17, 2024
0d01aa6
refactor: use a single constant for TS_INTEGRATION_PATH
manuelmauro Sep 17, 2024
f934384
style: lint
manuelmauro Sep 17, 2024
3028221
refactor: simplify code
manuelmauro Sep 17, 2024
ff58c82
Merge branch 'main' into manuel-add-ts-integration-fees-test-to-toolbox
manuelmauro Sep 17, 2024
4379a32
fix: restore contracts
manuelmauro Sep 17, 2024
732ab9d
feat: test fees on all chains
manuelmauro Sep 17, 2024
5b816e1
refactor: remove unused arg
manuelmauro Sep 18, 2024
c364c8e
ci: parallelize fees testing
manuelmauro Sep 18, 2024
c67a0f1
style: remove unused import
manuelmauro Sep 18, 2024
aeb9c50
fix: wrong args type
manuelmauro Sep 18, 2024
9b363d6
feat: add FEES_LOGS_DIR to GITHUB_ENV
manuelmauro Sep 18, 2024
8c87f51
fix: truly run fees testing
manuelmauro Sep 18, 2024
b270415
revert: temporarily revert to sequential fees testing
manuelmauro Sep 18, 2024
0b29ee4
fix: try to re-run servers before fees testing
manuelmauro Sep 18, 2024
2860b8f
revert: do not re-run servers
manuelmauro Sep 19, 2024
ebbe4c1
style: remove optional double dash
manuelmauro Sep 19, 2024
66c4f5f
refactor: use managed start/kill node
manuelmauro Sep 19, 2024
ffda3f7
style: format
manuelmauro Sep 19, 2024
6b7233b
fix: update config on the fly
manuelmauro Sep 20, 2024
7ab0b1c
feat: add L2 node PID to TestEnvironment
manuelmauro Sep 20, 2024
7f62481
fix: set right properties on the fly
manuelmauro Sep 20, 2024
539517b
style: lint
manuelmauro Sep 20, 2024
eacd4de
fix: kill server before running fees.test.ts
manuelmauro Sep 20, 2024
b36f2d1
fix: logs folder
manuelmauro Sep 20, 2024
72d92ce
Merge branch 'main' into manuel-add-ts-integration-fees-test-to-toolbox
manuelmauro Sep 20, 2024
e3e33ff
test: temporarily run fees tests only on era chain
manuelmauro Sep 20, 2024
7e3889b
style: use const case for env variable in CI
manuelmauro Sep 20, 2024
87a3d26
style: format code
manuelmauro Sep 20, 2024
1699dd4
fix: bring forward server start
manuelmauro Sep 20, 2024
2ffcd92
style: format
manuelmauro Sep 20, 2024
5ba3f36
fix: read test environment before killing the node
manuelmauro Sep 20, 2024
a12bf21
fix: fix operations in beforeAll step
manuelmauro Sep 20, 2024
285289d
Revert "style: format code"
manuelmauro Sep 20, 2024
fb75f04
fix: remove extra node restart
manuelmauro Sep 20, 2024
a936b04
refactor: remove unused code from tester
manuelmauro Sep 20, 2024
a5d2f8f
fix: unref node processes
manuelmauro Sep 22, 2024
19138a7
fix: unref only when needed
manuelmauro Sep 22, 2024
44f743b
refactor: simplify waiting
manuelmauro Sep 22, 2024
dd71070
Merge branch 'main' into manuel-add-ts-integration-fees-test-to-toolbox
Deniallugo Sep 24, 2024
1ce733b
Fix queries
Deniallugo Sep 24, 2024
790e1a5
Fix
Deniallugo Sep 24, 2024
f5cfc0d
fix
Deniallugo Sep 24, 2024
f70e7d8
Correct kill the process
Deniallugo Sep 25, 2024
df0a451
Reset pid
Deniallugo Sep 25, 2024
74e2951
Fix lints and increase timings
Deniallugo Sep 26, 2024
f02b261
Fix nits
Deniallugo Sep 27, 2024
fb31b2d
Fix logs place
Deniallugo Sep 27, 2024
1cec206
Run all fee tests
Deniallugo Sep 27, 2024
77e8814
Fix multiple chains for fees
Deniallugo Sep 27, 2024
1a47104
Merge branch 'main' into manuel-add-ts-integration-fees-test-to-toolbox
Deniallugo Sep 27, 2024
4028d35
Add no deps
Deniallugo Sep 27, 2024
c89b351
add logs
Deniallugo Sep 30, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix: update config on the fly
manuelmauro committed Sep 20, 2024

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
commit 6b7233bada6e61cf537a636d5b187e5c2264730b
56 changes: 36 additions & 20 deletions core/tests/ts-integration/src/utils.ts
Original file line number Diff line number Diff line change
@@ -6,6 +6,11 @@ import { killPidWithAllChilds } from 'utils/build/kill';
import * as utils from 'utils';
import fs from 'node:fs/promises';
import * as zksync from 'zksync-ethers';
import {
setInternalL1PricingMultiplier,
setInternalPubdataPricingMultiplier,
setTransactionSlots
} from '../tests/utils';

// executes a command in background and returns a child process handle
// by default pipes data to parent's stdio but this can be overridden
@@ -50,8 +55,6 @@ export function runServerInBackground({
cwd,
env,
useZkInception,
newL1GasPrice,
newPubdataPrice,
chain
}: {
components?: string[];
@@ -71,22 +74,6 @@ export function runServerInBackground({
}
} else {
command = 'zk server';
command = `DATABASE_MERKLE_TREE_MODE=full ${command}`;

if (newPubdataPrice) {
command = `ETH_SENDER_GAS_ADJUSTER_INTERNAL_ENFORCED_PUBDATA_PRICE=${newPubdataPrice} ${command}`;
}

if (newL1GasPrice) {
// We need to ensure that each transaction gets into its own batch for more fair comparison.
command = `ETH_SENDER_GAS_ADJUSTER_INTERNAL_ENFORCED_L1_GAS_PRICE=${newL1GasPrice} ${command}`;
}

const testMode = newPubdataPrice || newL1GasPrice;
if (testMode) {
// We need to ensure that each transaction gets into its own batch for more fair comparison.
command = `CHAIN_STATE_KEEPER_TRANSACTION_SLOTS=1 ${command}`;
}
}
return runInBackground({ command, components, stdio, cwd, env });
}
@@ -171,6 +158,37 @@ export class NodeSpawner {
const env = this.env ?? process.env;
const { fileConfig, pathToHome, options, logs } = this;

const testMode = newPubdataPrice || newL1GasPrice;

console.log('New L1 Gas Price: ', newL1GasPrice);
console.log('New Pubdata Price: ', newPubdataPrice);

if (fileConfig.loadFromFile) {
setTransactionSlots(pathToHome, fileConfig, testMode ? 1 : 8192);
setInternalL1PricingMultiplier(pathToHome, fileConfig, newL1GasPrice ? parseFloat(newL1GasPrice) : 0.8);
setInternalPubdataPricingMultiplier(
pathToHome,
fileConfig,
newPubdataPrice ? parseFloat(newPubdataPrice) : 1.0
);
} else {
env['DATABASE_MERKLE_TREE_MODE'] = 'full';

if (newPubdataPrice) {
env['ETH_SENDER_GAS_ADJUSTER_INTERNAL_ENFORCED_PUBDATA_PRICE'] = newPubdataPrice;
}

if (newL1GasPrice) {
// We need to ensure that each transaction gets into its own batch for more fair comparison.
env['ETH_SENDER_GAS_ADJUSTER_INTERNAL_ENFORCED_L1_GAS_PRICE'] = newL1GasPrice;
}

if (testMode) {
// We need to ensure that each transaction gets into its own batch for more fair comparison.
env['CHAIN_STATE_KEEPER_TRANSACTION_SLOTS'] = '1';
}
}

let components = 'api,tree,eth,state_keeper,da_dispatcher,vm_runner_protective_reads';
if (options.enableConsensus) {
components += ',consensus';
@@ -184,8 +202,6 @@ export class NodeSpawner {
cwd: pathToHome,
env: env,
useZkInception: fileConfig.loadFromFile,
newL1GasPrice: newL1GasPrice,
newPubdataPrice: newPubdataPrice,
chain: fileConfig.chain
});

6 changes: 6 additions & 0 deletions core/tests/ts-integration/tests/fees.test.ts
Original file line number Diff line number Diff line change
@@ -21,6 +21,7 @@ import { loadConfig, shouldLoadConfigFromFile } from 'utils/build/file-configs';
import { logsTestPath } from 'utils/build/logs';
import path from 'path';
import { NodeSpawner, Node, NodeType } from '../src/utils';
import { setInternalL1PricingMultiplier, setInternalPubdataPricingMultiplier, setTransactionSlots } from './utils';

const UINT32_MAX = 2n ** 32n - 1n;
const MAX_GAS_PER_PUBDATA = 50_000n;
@@ -274,6 +275,11 @@ testFees('Test fees', function () {
await mainNode.killAndWaitForShutdown();
mainNode = await mainNodeSpawner.spawnMainNode();

// Restore defaults
setTransactionSlots(pathToHome, fileConfig, 8192);
setInternalL1PricingMultiplier(pathToHome, fileConfig, 0.8);
setInternalPubdataPricingMultiplier(pathToHome, fileConfig, 1.0);

await testMaster.deinitialize();
});
});
29 changes: 29 additions & 0 deletions core/tests/ts-integration/tests/utils.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
import * as fs from 'fs';
import { getConfigPath } from 'utils/build/file-configs';

export function setInternalPubdataPricingMultiplier(pathToHome: string, fileConfig: any, value: number) {
setPropertyInGeneralConfig(pathToHome, fileConfig, 'internal_pubdata_pricing_multiplier', value);
}

export function setInternalL1PricingMultiplier(pathToHome: string, fileConfig: any, value: number) {
setPropertyInGeneralConfig(pathToHome, fileConfig, 'internal_l1_pricing_multiplier', value);
}

export function setTransactionSlots(pathToHome: string, fileConfig: any, value: number) {
setPropertyInGeneralConfig(pathToHome, fileConfig, 'transaction_slots', value);
}

function setPropertyInGeneralConfig(pathToHome: string, fileConfig: any, property: string, value: number) {
const generalConfigPath = getConfigPath({
pathToHome,
chain: fileConfig.chain,
configsFolder: 'configs',
config: 'general.yaml'
});
const generalConfig = fs.readFileSync(generalConfigPath, 'utf8');

const regex = new RegExp(`${property}:\\s*\\d+(\\.\\d+)?`, 'g');
const newGeneralConfig = generalConfig.replace(regex, `${property}: ${value}`);

fs.writeFileSync(generalConfigPath, newGeneralConfig, 'utf8');
}