Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore: add script to pull dashboards from remote API #5525

Merged
merged 1 commit into from
May 24, 2023
Merged

chore: add script to pull dashboards from remote API #5525

merged 1 commit into from
May 24, 2023

Conversation

dapplion
Copy link
Contributor

@dapplion dapplion commented May 24, 2023 •
edited
Loading

Motivation

Our DX to update dashboards is not good, since it requires to manually download files and perform some changes to minimize diff. We want to promote faster experimentation with dashboards to ensure they are up to date and persisted in git.

Download script

With one command run it lists all dashboards in the API, downloads then, cleans them up ready to be commited. This allows to very fast iteration of dashboards:

  • Log into Grafana UI
  • Visually edit a dashboard
  • Run download script and commit

To use the script:

Create a file .secrets.env with envs

GRAFANA_API_KEY=$token
GRAFANA_URL=https://yourgrafanaapi.io

Run

node scripts/download_dashboards.mjs

Check git diff of resulting files, commit, push and open PR

New linter rules

To ensure the download command is deterministic we need more linter rules. Using the HTTP API does not allow to set "Export for sharing externally" to true. So the fields starting with __ must be set manually, see https://community.grafana.com/t/export-dashboard-for-external-use-via-http-api/50716

  • Force all panels to be expanded. Toggling expand is huge source of big diffs. By forcing panels to always be expanded there's less opportunity for diff and gives full visibility to panels. If a dashboard now feels to long, it should be split into two, not collapsed.
  • Insert fixed __inputs field and remove __requirements

@dapplion dapplion requested a review from a team as a code owner May 24, 2023 06:54
@dapplion dapplion changed the title Add script to pull dashboards from remote API chore: add script to pull dashboards from remote API May 24, 2023
@dapplion dapplion marked this pull request as draft May 24, 2023 06:59
@dapplion dapplion mentioned this pull request May 24, 2023
Merged
@dapplion
Copy link
Contributor Author

Opened a second PR with only dashboard diff to allow easier review of the download scripts here #5526

@github-actions
Copy link
Contributor

github-actions bot commented May 24, 2023 •
edited
Loading

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: f680769 Previous: b5ab113 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 606.66 us/op 838.27 us/op 0.72
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 49.631 us/op 55.522 us/op 0.89
BLS verify - blst-native 1.2837 ms/op 1.3080 ms/op 0.98
BLS verifyMultipleSignatures 3 - blst-native 2.6149 ms/op 2.6287 ms/op 0.99
BLS verifyMultipleSignatures 8 - blst-native 5.6311 ms/op 5.6297 ms/op 1.00
BLS verifyMultipleSignatures 32 - blst-native 20.147 ms/op 20.414 ms/op 0.99
BLS aggregatePubkeys 32 - blst-native 28.381 us/op 27.649 us/op 1.03
BLS aggregatePubkeys 128 - blst-native 108.38 us/op 107.77 us/op 1.01
getAttestationsForBlock 58.993 ms/op 69.009 ms/op 0.85
isKnown best case - 1 super set check 285.00 ns/op 278.00 ns/op 1.03
isKnown normal case - 2 super set checks 294.00 ns/op 272.00 ns/op 1.08
isKnown worse case - 16 super set checks 290.00 ns/op 267.00 ns/op 1.09
CheckpointStateCache - add get delete 6.3860 us/op 6.1270 us/op 1.04
validate gossip signedAggregateAndProof - struct 3.2420 ms/op 3.0049 ms/op 1.08
validate gossip attestation - struct 1.5631 ms/op 1.3900 ms/op 1.12
pickEth1Vote - no votes 1.5537 ms/op 1.4651 ms/op 1.06
pickEth1Vote - max votes 11.855 ms/op 10.700 ms/op 1.11
pickEth1Vote - Eth1Data hashTreeRoot value x2048 11.559 ms/op 9.8265 ms/op 1.18
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 18.862 ms/op 17.508 ms/op 1.08
pickEth1Vote - Eth1Data fastSerialize value x2048 850.38 us/op 775.27 us/op 1.10
pickEth1Vote - Eth1Data fastSerialize tree x2048 7.8592 ms/op 5.6307 ms/op 1.40
bytes32 toHexString 900.00 ns/op 576.00 ns/op 1.56
bytes32 Buffer.toString(hex) 513.00 ns/op 379.00 ns/op 1.35
bytes32 Buffer.toString(hex) from Uint8Array 799.00 ns/op 577.00 ns/op 1.38
bytes32 Buffer.toString(hex) + 0x 523.00 ns/op 371.00 ns/op 1.41
Object access 1 prop 0.23700 ns/op 0.18600 ns/op 1.27
Map access 1 prop 0.20300 ns/op 0.17800 ns/op 1.14
Object get x1000 7.5970 ns/op 7.0270 ns/op 1.08
Map get x1000 0.70800 ns/op 0.58100 ns/op 1.22
Object set x1000 106.34 ns/op 72.951 ns/op 1.46
Map set x1000 85.551 ns/op 59.374 ns/op 1.44
Return object 10000 times 0.38590 ns/op 0.27250 ns/op 1.42
Throw Error 10000 times 6.0491 us/op 4.5480 us/op 1.33
fastMsgIdFn sha256 / 200 bytes 4.4430 us/op 3.6960 us/op 1.20
fastMsgIdFn h32 xxhash / 200 bytes 406.00 ns/op 324.00 ns/op 1.25
fastMsgIdFn h64 xxhash / 200 bytes 612.00 ns/op 495.00 ns/op 1.24
fastMsgIdFn sha256 / 1000 bytes 14.912 us/op 12.299 us/op 1.21
fastMsgIdFn h32 xxhash / 1000 bytes 572.00 ns/op 469.00 ns/op 1.22
fastMsgIdFn h64 xxhash / 1000 bytes 690.00 ns/op 593.00 ns/op 1.16
fastMsgIdFn sha256 / 10000 bytes 128.38 us/op 112.00 us/op 1.15
fastMsgIdFn h32 xxhash / 10000 bytes 2.2010 us/op 2.1150 us/op 1.04
fastMsgIdFn h64 xxhash / 10000 bytes 1.5960 us/op 1.5510 us/op 1.03
enrSubnets - fastDeserialize 64 bits 1.7790 us/op 1.9310 us/op 0.92
enrSubnets - ssz BitVector 64 bits 689.00 ns/op 664.00 ns/op 1.04
enrSubnets - fastDeserialize 4 bits 237.00 ns/op 219.00 ns/op 1.08
enrSubnets - ssz BitVector 4 bits 684.00 ns/op 654.00 ns/op 1.05
prioritizePeers score -10:0 att 32-0.1 sync 2-0 131.75 us/op 137.83 us/op 0.96
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 166.10 us/op 172.29 us/op 0.96
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 208.27 us/op 215.34 us/op 0.97
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 429.89 us/op 389.15 us/op 1.10
prioritizePeers score 0:0 att 64-1 sync 4-1 582.46 us/op 442.65 us/op 1.32
array of 16000 items push then shift 2.0589 us/op 1.7495 us/op 1.18
LinkedList of 16000 items push then shift 13.123 ns/op 9.7210 ns/op 1.35
array of 16000 items push then pop 169.12 ns/op 130.69 ns/op 1.29
LinkedList of 16000 items push then pop 11.309 ns/op 10.065 ns/op 1.12
array of 24000 items push then shift 2.8664 us/op 2.5357 us/op 1.13
LinkedList of 24000 items push then shift 11.805 ns/op 10.271 ns/op 1.15
array of 24000 items push then pop 113.31 ns/op 89.016 ns/op 1.27
LinkedList of 24000 items push then pop 11.327 ns/op 9.5700 ns/op 1.18
intersect bitArray bitLen 8 15.856 ns/op 14.726 ns/op 1.08
intersect array and set length 8 143.79 ns/op 98.322 ns/op 1.46
intersect bitArray bitLen 128 52.797 ns/op 47.546 ns/op 1.11
intersect array and set length 128 1.5657 us/op 1.3615 us/op 1.15
Buffer.concat 32 items 5.8850 us/op 3.3660 us/op 1.75
Uint8Array.set 32 items 3.4900 us/op 2.7290 us/op 1.28
pass gossip attestations to forkchoice per slot 2.7814 ms/op 2.5155 ms/op 1.11
computeDeltas 3.3874 ms/op 3.5805 ms/op 0.95
computeProposerBoostScoreFromBalances 2.0025 ms/op 1.8937 ms/op 1.06
altair processAttestation - 250000 vs - 7PWei normalcase 3.6152 ms/op 3.5347 ms/op 1.02
altair processAttestation - 250000 vs - 7PWei worstcase 4.9410 ms/op 4.9543 ms/op 1.00
altair processAttestation - setStatus - 1/6 committees join 156.25 us/op 155.78 us/op 1.00
altair processAttestation - setStatus - 1/3 committees join 306.47 us/op 299.31 us/op 1.02
altair processAttestation - setStatus - 1/2 committees join 426.81 us/op 397.35 us/op 1.07
altair processAttestation - setStatus - 2/3 committees join 541.42 us/op 497.09 us/op 1.09
altair processAttestation - setStatus - 4/5 committees join 716.02 us/op 734.89 us/op 0.97
altair processAttestation - setStatus - 100% committees join 842.33 us/op 847.90 us/op 0.99
altair processBlock - 250000 vs - 7PWei normalcase 25.045 ms/op 21.018 ms/op 1.19
altair processBlock - 250000 vs - 7PWei normalcase hashState 33.278 ms/op 32.709 ms/op 1.02
altair processBlock - 250000 vs - 7PWei worstcase 65.129 ms/op 61.362 ms/op 1.06
altair processBlock - 250000 vs - 7PWei worstcase hashState 91.265 ms/op 78.021 ms/op 1.17
phase0 processBlock - 250000 vs - 7PWei normalcase 2.7334 ms/op 2.8337 ms/op 0.96
phase0 processBlock - 250000 vs - 7PWei worstcase 41.095 ms/op 33.682 ms/op 1.22
altair processEth1Data - 250000 vs - 7PWei normalcase 724.86 us/op 730.08 us/op 0.99
vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15 16.332 us/op 14.457 us/op 1.13
vc - 250000 eb 0.95 eth1 0.1 we 0.05 wn 0 - smpl 219 35.046 us/op 42.799 us/op 0.82
vc - 250000 eb 0.95 eth1 0.3 we 0.05 wn 0 - smpl 42 16.063 us/op 17.784 us/op 0.90
vc - 250000 eb 0.95 eth1 0.7 we 0.05 wn 0 - smpl 18 12.475 us/op 14.923 us/op 0.84
vc - 250000 eb 0.1 eth1 0.1 we 0 wn 0 - smpl 1020 105.19 us/op 158.94 us/op 0.66
vc - 250000 eb 0.03 eth1 0.03 we 0 wn 0 - smpl 11777 876.56 us/op 1.0059 ms/op 0.87
vc - 250000 eb 0.01 eth1 0.01 we 0 wn 0 - smpl 16384 1.3650 ms/op 1.1740 ms/op 1.16
vc - 250000 eb 0 eth1 0 we 0 wn 0 - smpl 16384 1.0295 ms/op 1.1054 ms/op 0.93
vc - 250000 eb 0 eth1 0 we 0 wn 0 nocache - smpl 16384 3.0185 ms/op 2.7523 ms/op 1.10
vc - 250000 eb 0 eth1 1 we 0 wn 0 - smpl 16384 1.6853 ms/op 1.9211 ms/op 0.88
vc - 250000 eb 0 eth1 1 we 0 wn 0 nocache - smpl 16384 7.4809 ms/op 4.5681 ms/op 1.64
Tree 40 250000 create 527.11 ms/op 499.56 ms/op 1.06
Tree 40 250000 get(125000) 228.64 ns/op 202.63 ns/op 1.13
Tree 40 250000 set(125000) 2.1602 us/op 1.0496 us/op 2.06
Tree 40 250000 toArray() 37.178 ms/op 27.138 ms/op 1.37
Tree 40 250000 iterate all - toArray() + loop 38.572 ms/op 29.613 ms/op 1.30
Tree 40 250000 iterate all - get(i) 101.05 ms/op 78.063 ms/op 1.29
MutableVector 250000 create 17.541 ms/op 13.384 ms/op 1.31
MutableVector 250000 get(125000) 8.3070 ns/op 6.9120 ns/op 1.20
MutableVector 250000 set(125000) 754.86 ns/op 318.43 ns/op 2.37
MutableVector 250000 toArray() 7.3471 ms/op 3.7571 ms/op 1.96
MutableVector 250000 iterate all - toArray() + loop 7.2362 ms/op 3.8516 ms/op 1.88
MutableVector 250000 iterate all - get(i) 1.8243 ms/op 1.6003 ms/op 1.14
Array 250000 create 5.6907 ms/op 3.9292 ms/op 1.45
Array 250000 clone - spread 4.2609 ms/op 1.2865 ms/op 3.31
Array 250000 get(125000) 1.6700 ns/op 0.66800 ns/op 2.50
Array 250000 set(125000) 1.8590 ns/op 0.76600 ns/op 2.43
Array 250000 iterate all - loop 138.23 us/op 97.036 us/op 1.42
effectiveBalanceIncrements clone Uint8Array 300000 92.504 us/op 51.362 us/op 1.80
effectiveBalanceIncrements clone MutableVector 300000 611.00 ns/op 558.00 ns/op 1.09
effectiveBalanceIncrements rw all Uint8Array 300000 192.20 us/op 217.62 us/op 0.88
effectiveBalanceIncrements rw all MutableVector 300000 203.37 ms/op 156.59 ms/op 1.30
phase0 afterProcessEpoch - 250000 vs - 7PWei 145.76 ms/op 127.30 ms/op 1.15
phase0 beforeProcessEpoch - 250000 vs - 7PWei 95.171 ms/op 51.920 ms/op 1.83
altair processEpoch - mainnet_e81889 494.50 ms/op 375.45 ms/op 1.32
mainnet_e81889 - altair beforeProcessEpoch 148.47 ms/op 75.618 ms/op 1.96
mainnet_e81889 - altair processJustificationAndFinalization 43.862 us/op 31.702 us/op 1.38
mainnet_e81889 - altair processInactivityUpdates 11.502 ms/op 7.4011 ms/op 1.55
mainnet_e81889 - altair processRewardsAndPenalties 85.148 ms/op 64.739 ms/op 1.32
mainnet_e81889 - altair processRegistryUpdates 7.8400 us/op 4.1780 us/op 1.88
mainnet_e81889 - altair processSlashings 1.7850 us/op 680.00 ns/op 2.63
mainnet_e81889 - altair processEth1DataReset 1.8770 us/op 972.00 ns/op 1.93
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.6217 ms/op 1.2908 ms/op 1.26
mainnet_e81889 - altair processSlashingsReset 12.268 us/op 5.4830 us/op 2.24
mainnet_e81889 - altair processRandaoMixesReset 11.867 us/op 4.8400 us/op 2.45
mainnet_e81889 - altair processHistoricalRootsUpdate 2.0820 us/op 1.2030 us/op 1.73
mainnet_e81889 - altair processParticipationFlagUpdates 5.6010 us/op 3.6530 us/op 1.53
mainnet_e81889 - altair processSyncCommitteeUpdates 1.6590 us/op 772.00 ns/op 2.15
mainnet_e81889 - altair afterProcessEpoch 144.47 ms/op 135.41 ms/op 1.07
phase0 processEpoch - mainnet_e58758 670.01 ms/op 400.20 ms/op 1.67
mainnet_e58758 - phase0 beforeProcessEpoch 315.33 ms/op 173.39 ms/op 1.82
mainnet_e58758 - phase0 processJustificationAndFinalization 48.550 us/op 28.941 us/op 1.68
mainnet_e58758 - phase0 processRewardsAndPenalties 120.74 ms/op 73.924 ms/op 1.63
mainnet_e58758 - phase0 processRegistryUpdates 10.931 us/op 14.472 us/op 0.76
mainnet_e58758 - phase0 processSlashings 1.6260 us/op 1.1360 us/op 1.43
mainnet_e58758 - phase0 processEth1DataReset 1.2800 us/op 1.1890 us/op 1.08
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 3.7950 ms/op 1.6753 ms/op 2.27
mainnet_e58758 - phase0 processSlashingsReset 5.9930 us/op 5.8820 us/op 1.02
mainnet_e58758 - phase0 processRandaoMixesReset 12.702 us/op 10.374 us/op 1.22
mainnet_e58758 - phase0 processHistoricalRootsUpdate 1.5370 us/op 1.3760 us/op 1.12
mainnet_e58758 - phase0 processParticipationRecordUpdates 9.1210 us/op 9.4400 us/op 0.97
mainnet_e58758 - phase0 afterProcessEpoch 117.82 ms/op 110.99 ms/op 1.06
phase0 processEffectiveBalanceUpdates - 250000 normalcase 2.5104 ms/op 1.3588 ms/op 1.85
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 3.1460 ms/op 2.0278 ms/op 1.55
altair processInactivityUpdates - 250000 normalcase 37.364 ms/op 34.756 ms/op 1.08
altair processInactivityUpdates - 250000 worstcase 42.003 ms/op 32.749 ms/op 1.28
phase0 processRegistryUpdates - 250000 normalcase 14.939 us/op 11.244 us/op 1.33
phase0 processRegistryUpdates - 250000 badcase_full_deposits 555.87 us/op 371.33 us/op 1.50
phase0 processRegistryUpdates - 250000 worstcase 0.5 254.47 ms/op 154.18 ms/op 1.65
altair processRewardsAndPenalties - 250000 normalcase 124.49 ms/op 75.229 ms/op 1.65
altair processRewardsAndPenalties - 250000 worstcase 129.03 ms/op 76.978 ms/op 1.68
phase0 getAttestationDeltas - 250000 normalcase 15.146 ms/op 9.3866 ms/op 1.61
phase0 getAttestationDeltas - 250000 worstcase 13.053 ms/op 8.2403 ms/op 1.58
phase0 processSlashings - 250000 worstcase 5.5289 ms/op 4.0727 ms/op 1.36
altair processSyncCommitteeUpdates - 250000 226.11 ms/op 197.12 ms/op 1.15
BeaconState.hashTreeRoot - No change 335.00 ns/op 380.00 ns/op 0.88
BeaconState.hashTreeRoot - 1 full validator 65.544 us/op 56.784 us/op 1.15
BeaconState.hashTreeRoot - 32 full validator 724.23 us/op 599.09 us/op 1.21
BeaconState.hashTreeRoot - 512 full validator 8.4463 ms/op 6.1354 ms/op 1.38
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 81.834 us/op 67.164 us/op 1.22
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.1163 ms/op 1.0195 ms/op 1.09
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 15.756 ms/op 14.138 ms/op 1.11
BeaconState.hashTreeRoot - 1 balances 67.168 us/op 55.302 us/op 1.21
BeaconState.hashTreeRoot - 32 balances 625.39 us/op 551.95 us/op 1.13
BeaconState.hashTreeRoot - 512 balances 6.1669 ms/op 5.1164 ms/op 1.21
BeaconState.hashTreeRoot - 250000 balances 96.899 ms/op 85.922 ms/op 1.13
aggregationBits - 2048 els - zipIndexesInBitList 29.851 us/op 20.767 us/op 1.44
regular array get 100000 times 42.074 us/op 36.231 us/op 1.16
wrappedArray get 100000 times 47.042 us/op 35.878 us/op 1.31
arrayWithProxy get 100000 times 19.325 ms/op 17.790 ms/op 1.09
ssz.Root.equals 696.00 ns/op 604.00 ns/op 1.15
byteArrayEquals 675.00 ns/op 599.00 ns/op 1.13
shuffle list - 16384 els 8.0938 ms/op 7.4108 ms/op 1.09
shuffle list - 250000 els 125.38 ms/op 117.34 ms/op 1.07
processSlot - 1 slots 17.023 us/op 12.467 us/op 1.37
processSlot - 32 slots 1.9070 ms/op 1.4823 ms/op 1.29
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 49.073 ms/op 39.266 ms/op 1.25
getCommitteeAssignments - req 1 vs - 250000 vc 4.0621 ms/op 3.1265 ms/op 1.30
getCommitteeAssignments - req 100 vs - 250000 vc 5.8556 ms/op 4.4348 ms/op 1.32
getCommitteeAssignments - req 1000 vs - 250000 vc 6.4601 ms/op 4.8400 ms/op 1.33
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 6.4900 ns/op 5.4100 ns/op 1.20
state getBlockRootAtSlot - 250000 vs - 7PWei 1.3393 us/op 737.10 ns/op 1.82
computeProposers - vc 250000 14.378 ms/op 11.400 ms/op 1.26
computeEpochShuffling - vc 250000 129.38 ms/op 111.89 ms/op 1.16
getNextSyncCommittee - vc 250000 220.73 ms/op 178.54 ms/op 1.24
computeSigningRoot for AttestationData 18.585 us/op 14.773 us/op 1.26
hash AttestationData serialized data then Buffer.toString(base64) 3.2468 us/op 2.6651 us/op 1.22
toHexString serialized data 1.7530 us/op 1.3505 us/op 1.30
Buffer.toString(base64) 440.59 ns/op 377.78 ns/op 1.17

by benchmarkbot/action

@dapplion dapplion force-pushed the dapplion/download-dashboards branch from bee9d04 to 06d9c40 Compare May 24, 2023 09:55
@dapplion dapplion marked this pull request as ready for review May 24, 2023 09:55
wemeetagain
wemeetagain approved these changes May 24, 2023
View reviewed changes
Copy link
Member

@wemeetagain wemeetagain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

amazing, love this

@wemeetagain wemeetagain merged commit 7b53ebd into unstable May 24, 2023
@wemeetagain wemeetagain deleted the dapplion/download-dashboards branch May 24, 2023 15:57
@nflaig
Copy link
Member

nflaig commented May 24, 2023

Might wanna update CONTRIBUTING.md to reflect new workflow in another PR.

@wemeetagain wemeetagain mentioned this pull request May 24, 2023
Closed
@twoeths twoeths mentioned this pull request May 25, 2023
Merged
This was referenced May 30, 2023
Merged
Merged
@nflaig nflaig mentioned this pull request Jun 20, 2023
Merged
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.9.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wemeetagain wemeetagain wemeetagain approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@dapplion @nflaig @wemeetagain