Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(dashmate): check for DKG before stopping node #1683

Merged
merged 27 commits into from
May 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
4fec12e
feat(dashmate): draft DKG window check
pshenmic Jan 24, 2024
3ef382e
feat(dashmate): add DKG check prompt
pshenmic Jan 30, 2024
1e480db
feat(dashmate): skip DKG check for local networks
pshenmic Jan 30, 2024
ba87047
feat(dashmate): lint fix
pshenmic Jan 30, 2024
4aae4fa
Merge branch 'v1.0-dev' into feat/dashmate-dkg-check
pshenmic Mar 14, 2024
6eee4a7
feat(dashmate): replace call with dkginfo
pshenmic Mar 14, 2024
ac5f284
feat(dashmate): check active and next dkgs before stop
pshenmic Mar 14, 2024
f3ddf50
feat(dashmate): replace skip with enable
pshenmic Mar 14, 2024
182009c
feat(dashmate): lint fix
pshenmic Mar 14, 2024
35dbff1
Merge branch 'v1.0-dev' into feat/dashmate-dkg-check
pshenmic Apr 22, 2024
a2422f4
Merge branch 'v1.0-dev' into feat/dashmate-dkg-check
pshenmic May 2, 2024
e66a3f2
feat(dashmate): implement --safe and waitForDKGWindowPass
pshenmic May 5, 2024
09ceb39
feat(dashmate): fix wait func
pshenmic May 5, 2024
b26caf8
feat(dashmate): fix error string
pshenmic May 5, 2024
d661b95
feat(dashmate): add masternode check
pshenmic May 5, 2024
f74e885
feat(dashmate): remove active dkgs check
pshenmic May 5, 2024
89bd966
feat(dashmate): add safe flag for groups
pshenmic May 5, 2024
2af204d
Merge branch 'v1.0-dev' into feat/dashmate-dkg-check
pshenmic May 5, 2024
f92cd56
chore(dashmate): fix lint
pshenmic May 6, 2024
3ed9e05
chore(dashmate): add safe flag to restart command
pshenmic May 6, 2024
8147ac8
fix(dashmate): add --safe flag in the e2e tests
pshenmic May 6, 2024
05d80c3
fix(dashmate): code review fixes
pshenmic May 6, 2024
ad045eb
Merge branch 'v1.0-dev' into feat/dashmate-dkg-check
pshenmic May 6, 2024
00a6814
feat(dashmate): turn off check for fullnodes
pshenmic May 6, 2024
7336c94
fix(dashmate): lint fix
pshenmic May 7, 2024
f7bc2af
Merge branch 'v1.0-dev' into feat/dashmate-dkg-check
pshenmic May 8, 2024
42d6039
fix(dashmate): refactor to do-while
pshenmic May 8, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 21 additions & 9 deletions packages/dashmate/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,8 @@ USAGE
$ dashmate start [-v] [--config <value>] [-w]

FLAGS
-f, --force force start even if any services are already running
-p, --platform start only platform
-v, --verbose use verbose mode for output
-w, --wait-for-readiness wait for nodes to be ready
--config=<value> configuration name to use
Expand All @@ -193,12 +195,15 @@ The `stop` command is used to stop a running node.

```
USAGE
$ dashmate stop [-v] [--config <value>] [-f]
$ dashmate stop [--config <value>] [-v] [-f] [-p] [-s]

FLAGS
-f, --force force stop even if any is running
-f, --force force stop even if any service is running
-p, --platform stop only platform
-s, --safe wait for dkg before stop
-v, --verbose use verbose mode for output
--config=<value> configuration name to use

```

To stop a node:
Expand All @@ -212,11 +217,13 @@ The `restart` command is used to restart a node with the default or specified co

```
USAGE
$ dashmate restart [-v] [--config <value>]
$ dashmate restart [--config <value>] [-v] [-p] [-s]

FLAGS
-v, --verbose use verbose mode for output
--config=<value> configuration name to use
-p, --platform restart only platform
-s, --safe wait for dkg before stop
-v, --verbose use verbose mode for output
--config=<value> configuration name to use
```

### Show node status
Expand Down Expand Up @@ -391,7 +398,8 @@ USAGE
$ dashmate group stop [-v] [--group <value>] [-f]

FLAGS
-f, --force force stop even if any service is running
-f, --force force stop even if any is running
-s, --safe wait for dkg before stop
-v, --verbose use verbose mode for output
--group=<value> group name to use
```
Expand All @@ -402,11 +410,15 @@ The `group restart` command is used to restart group nodes belonging to the defa

```
USAGE
$ dashmate group restart [-v] [--group <value>]
$ dashmate group restart [--group <value>] [-v] [-s]

FLAGS
-v, --verbose use verbose mode for output
--group=<value> group name to use
-s, --safe wait for dkg before stop
-v, --verbose use verbose mode for output
--group=<value> group name to use

DESCRIPTION
Restart group nodes
```

#### Show group status
Expand Down
11 changes: 9 additions & 2 deletions packages/dashmate/src/commands/group/restart.js
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,11 @@ export default class GroupRestartCommand extends GroupBaseCommand {

static flags = {
...GroupBaseCommand.flags,
safe: {
char: 's',
description: 'wait for dkg before stop',
default: false,
},
};

/**
Expand All @@ -21,6 +26,7 @@ export default class GroupRestartCommand extends GroupBaseCommand {
async runWithDependencies(
args,
{
safe: isSafe,
verbose: isVerbose,
},
dockerCompose,
Expand All @@ -38,8 +44,8 @@ export default class GroupRestartCommand extends GroupBaseCommand {
{
title: 'Stop nodes',
task: () => (
// So we stop the miner first, as there's a chance that MNs will get banned
// if the miner is still running when stopping them
// So we stop the miner first, as there's a chance that MNs will get banned
// if the miner is still running when stopping them
new Listr(configGroup.reverse().map((config) => ({
task: () => stopNodeTask(config),
})))
Expand All @@ -66,6 +72,7 @@ export default class GroupRestartCommand extends GroupBaseCommand {
try {
await tasks.run({
isVerbose,
isSafe,
});
} catch (e) {
throw new MuteOneLineError(e);
Expand Down
7 changes: 7 additions & 0 deletions packages/dashmate/src/commands/group/stop.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,11 @@ export default class GroupStopCommand extends GroupBaseCommand {
description: 'force stop even if any is running',
default: false,
}),
safe: Flags.boolean({
char: 's',
description: 'wait for dkg before stop',
default: false,
}),
};

/**
Expand All @@ -27,6 +32,7 @@ export default class GroupStopCommand extends GroupBaseCommand {
args,
{
force: isForce,
safe: isSafe,
verbose: isVerbose,
},
dockerCompose,
Expand Down Expand Up @@ -63,6 +69,7 @@ export default class GroupStopCommand extends GroupBaseCommand {
await tasks.run({
isVerbose,
isForce,
isSafe,
});
} catch (e) {
throw new MuteOneLineError(e);
Expand Down
3 changes: 3 additions & 0 deletions packages/dashmate/src/commands/restart.js
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ export default class RestartCommand extends ConfigBaseCommand {
static flags = {
...ConfigBaseCommand.flags,
platform: Flags.boolean({ char: 'p', description: 'restart only platform', default: false }),
safe: Flags.boolean({ char: 's', description: 'wait for dkg before stop', default: false }),
};

/**
Expand All @@ -25,6 +26,7 @@ export default class RestartCommand extends ConfigBaseCommand {
{
verbose: isVerbose,
platform: platformOnly,
safe: isSafe,
},
dockerCompose,
restartNodeTask,
Expand All @@ -51,6 +53,7 @@ export default class RestartCommand extends ConfigBaseCommand {
try {
await tasks.run({
isVerbose,
isSafe,
platformOnly: platformOnly === true,
});
} catch (e) {
Expand Down
7 changes: 7 additions & 0 deletions packages/dashmate/src/commands/stop.js
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,11 @@ export default class StopCommand extends ConfigBaseCommand {
description: 'stop only platform',
default: false,
}),
safe: Flags.boolean({
char: 's',
description: 'wait for dkg before stop',
default: false,
}),
};

/**
Expand All @@ -32,6 +37,7 @@ export default class StopCommand extends ConfigBaseCommand {
args,
{
force: isForce,
safe: isSafe,
verbose: isVerbose,
platform: platformOnly,
},
Expand Down Expand Up @@ -59,6 +65,7 @@ export default class StopCommand extends ConfigBaseCommand {
await tasks.run({
isForce,
isVerbose,
isSafe,
platformOnly: platformOnly === true,
});
} catch (e) {
Expand Down
3 changes: 3 additions & 0 deletions packages/dashmate/src/constants.js
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,9 @@ export const QUORUM_TYPES = {
export const MASTERNODE_COLLATERAL_AMOUNT = 1000;
export const HPMN_COLLATERAL_AMOUNT = 4000;

// number of blocks to wait before core DKG exchange session
export const MIN_BLOCKS_BEFORE_DKG = 6;
shumkov marked this conversation as resolved.
Show resolved Hide resolved

export const PACKAGE_ROOT_DIR = path.join(url.fileURLToPath(import.meta.url), '../..');
export const TEMPLATES_DIR = path.join(PACKAGE_ROOT_DIR, 'templates');

Expand Down
40 changes: 40 additions & 0 deletions packages/dashmate/src/core/quorum/waitForDKGWindowPass.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
import { MIN_BLOCKS_BEFORE_DKG } from '../../constants.js';
import wait from '../../util/wait.js';

/**
* @param {RpcClient} rpcClient
* @return {Promise<void>}
*/
export default async function waitForDKGWindowPass(rpcClient) {
let startBlockCount;
let startNextDkg;

let isInDKG = true;

do {
const [currentBlockCount, currentDkgInfo] = await Promise
.all([rpcClient.getBlockCount(), rpcClient.quorum('dkginfo')]);

const { result: blockCount } = currentBlockCount;
const { result: dkgInfo } = currentDkgInfo;

const { next_dkg: nextDkg } = dkgInfo;

if (!startBlockCount) {
startBlockCount = blockCount;
}

if (!startNextDkg) {
startNextDkg = nextDkg;
}

isInDKG = nextDkg <= MIN_BLOCKS_BEFORE_DKG;

if (isInDKG && blockCount > startBlockCount + startNextDkg + 1) {
throw new Error(`waitForDKGWindowPass deadline exceeded: dkg did not happen for ${startBlockCount + nextDkg + 1} ${startNextDkg + 1} blocks`);
}

await wait(10000);
}
while (isInDKG);
}
34 changes: 33 additions & 1 deletion packages/dashmate/src/listr/tasks/stopNodeTaskFactory.js
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
/* eslint-disable no-console */
import { Listr } from 'listr2';
import { MIN_BLOCKS_BEFORE_DKG } from '../../constants.js';
import waitForDKGWindowPass from '../../core/quorum/waitForDKGWindowPass.js';

/**
* @param {DockerCompose} dockerCompose
Expand Down Expand Up @@ -34,6 +37,36 @@ export default function stopNodeTaskFactory(
}
},
},
{
title: 'Check node is participating in DKG',
enabled: (ctx) => config.get('core.masternode.enable') && !ctx.isForce && !ctx.isSafe,
task: async () => {
const rpcClient = createRpcClient({
port: config.get('core.rpc.port'),
user: config.get('core.rpc.user'),
pass: config.get('core.rpc.password'),
host: await getConnectionHost(config, 'core', 'core.rpc.host'),
});

const { result: dkgInfo } = await rpcClient.quorum('dkginfo');
const { next_dkg: nextDkg } = dkgInfo;

if (nextDkg <= MIN_BLOCKS_BEFORE_DKG) {
throw new Error('Your node is currently participating in DKG exchange session and '
+ 'stopping it right now may result in PoSE ban. Try again later, or continue with --force or --safe flags');
}
},
},
{
title: 'Wait for DKG window to pass',
enabled: (ctx) => config.get('core.masternode.enable') && !ctx.isForce && ctx.isSafe,
task: async () => waitForDKGWindowPass(createRpcClient({
port: config.get('core.rpc.port'),
user: config.get('core.rpc.user'),
pass: config.get('core.rpc.password'),
host: await getConnectionHost(config, 'core', 'core.rpc.host'),
})),
},
{
title: 'Save core node time',
enabled: () => config.get('group') === 'local',
Expand All @@ -58,7 +91,6 @@ export default function stopNodeTaskFactory(
if (ctx.platformOnly) {
profiles.push('platform');
}

await dockerCompose.stop(config, { profiles });
},
},
Expand Down
2 changes: 2 additions & 0 deletions packages/dashmate/test/e2e/testnetEvonode.spec.js
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,7 @@ describe('Testnet Evonode', function main() {

await task.run({
isVerbose: true,
isSafe: true,
});

// TODO: Assert all services are running
Expand Down Expand Up @@ -230,6 +231,7 @@ describe('Testnet Evonode', function main() {

await task.run({
isVerbose: true,
isSafe: true,
});

// TODO: Assert all services are running
Expand Down
Loading