Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add build if changed script #3543

Merged
merged 2 commits into from
Dec 21, 2021
Merged

Add build if changed script #3543

merged 2 commits into from
Dec 21, 2021

Conversation

dapplion
Copy link
Contributor

@dapplion dapplion commented Dec 21, 2021

Motivation

In some setups I'm building in VPS the build command is run many times when no changes have been done.

The current yarn build command blindly nukes the build artifacts and does a build from scratch, which take ~2-3 min

Description

  • Persist the last time a specific package src was modified
  • Check if there has been modifications between that time and now

If there has been no modification running the script over all packages takes 0.5 sec. find is very fast.

yarn build:ifchanged

Caveats

  • May only work on some UNIX, Linux systems

@codeclimate
Copy link

codeclimate bot commented Dec 21, 2021

Code Climate has analyzed commit d769f30 and detected 0 issues on this pull request.

View more on Code Climate.

@codecov
Copy link

codecov bot commented Dec 21, 2021

Codecov Report

Merging #3543 (d769f30) into master (89dd289) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #3543   +/-   ##
=======================================
  Coverage   37.56%   37.56%           
=======================================
  Files         310      310           
  Lines        8253     8253           
  Branches     1273     1273           
=======================================
  Hits         3100     3100           
  Misses       5005     5005           
  Partials      148      148           

@github-actions
Copy link
Contributor

github-actions bot commented Dec 21, 2021

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 02af1a2 Previous: 89dd289 Ratio
BeaconState.hashTreeRoot - No change 640.00 ns/op 660.00 ns/op 0.97
BeaconState.hashTreeRoot - 1 full validator 174.91 us/op 164.98 us/op 1.06
BeaconState.hashTreeRoot - 32 full validator 2.6380 ms/op 2.2908 ms/op 1.15
BeaconState.hashTreeRoot - 512 full validator 33.396 ms/op 30.112 ms/op 1.11
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 170.27 us/op 144.43 us/op 1.18
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 2.6644 ms/op 2.4145 ms/op 1.10
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 37.193 ms/op 32.953 ms/op 1.13
BeaconState.hashTreeRoot - 1 balances 125.03 us/op 105.34 us/op 1.19
BeaconState.hashTreeRoot - 32 balances 1.0130 ms/op 914.46 us/op 1.11
BeaconState.hashTreeRoot - 512 balances 9.3422 ms/op 8.6955 ms/op 1.07
BeaconState.hashTreeRoot - 250000 balances 177.29 ms/op 155.99 ms/op 1.14
processSlot - 1 slots 72.289 us/op 55.421 us/op 1.30
processSlot - 32 slots 4.2042 ms/op 3.3490 ms/op 1.26
getCommitteeAssignments - req 1 vs - 250000 vc 5.6806 ms/op 6.1925 ms/op 0.92
getCommitteeAssignments - req 100 vs - 250000 vc 8.3626 ms/op 8.7442 ms/op 0.96
getCommitteeAssignments - req 1000 vs - 250000 vc 8.7501 ms/op 9.0352 ms/op 0.97
computeProposers - vc 250000 27.781 ms/op 24.437 ms/op 1.14
computeEpochShuffling - vc 250000 213.42 ms/op 220.61 ms/op 0.97
getNextSyncCommittee - vc 250000 436.83 ms/op 404.58 ms/op 1.08
altair processAttestation - 250000 vs - 7PWei normalcase 56.963 ms/op 48.699 ms/op 1.17
altair processAttestation - 250000 vs - 7PWei worstcase 75.836 ms/op 51.021 ms/op 1.49
altair processAttestation - setStatus - 1/6 committees join 16.643 ms/op 12.375 ms/op 1.34
altair processAttestation - setStatus - 1/3 committees join 32.656 ms/op 26.207 ms/op 1.25
altair processAttestation - setStatus - 1/2 committees join 47.310 ms/op 40.710 ms/op 1.16
altair processAttestation - setStatus - 2/3 committees join 61.747 ms/op 52.809 ms/op 1.17
altair processAttestation - setStatus - 4/5 committees join 76.247 ms/op 63.144 ms/op 1.21
altair processAttestation - setStatus - 100% committees join 99.859 ms/op 79.594 ms/op 1.25
altair processAttestation - updateEpochParticipants - 1/6 committees join 17.086 ms/op 13.387 ms/op 1.28
altair processAttestation - updateEpochParticipants - 1/3 committees join 35.219 ms/op 27.956 ms/op 1.26
altair processAttestation - updateEpochParticipants - 1/2 committees join 33.995 ms/op 51.244 ms/op 0.66
altair processAttestation - updateEpochParticipants - 2/3 committees join 34.976 ms/op 28.436 ms/op 1.23
altair processAttestation - updateEpochParticipants - 4/5 committees join 44.085 ms/op 30.582 ms/op 1.44
altair processAttestation - updateEpochParticipants - 100% committees join 38.352 ms/op 33.071 ms/op 1.16
altair processAttestation - updateAllStatus 27.829 ms/op 24.207 ms/op 1.15
altair processBlock - 250000 vs - 7PWei normalcase 59.317 ms/op 50.092 ms/op 1.18
altair processBlock - 250000 vs - 7PWei worstcase 156.84 ms/op 133.80 ms/op 1.17
altair processEpoch - mainnet_e81889 1.2758 s/op 1.2333 s/op 1.03
mainnet_e81889 - altair beforeProcessEpoch 317.43 ms/op 307.71 ms/op 1.03
mainnet_e81889 - altair processJustificationAndFinalization 124.14 us/op 81.609 us/op 1.52
mainnet_e81889 - altair processInactivityUpdates 22.347 ms/op 21.292 ms/op 1.05
mainnet_e81889 - altair processRewardsAndPenalties 264.14 ms/op 267.20 ms/op 0.99
mainnet_e81889 - altair processRegistryUpdates 20.812 us/op 10.406 us/op 2.00
mainnet_e81889 - altair processSlashings 6.2120 us/op 3.6400 us/op 1.71
mainnet_e81889 - altair processEth1DataReset 6.3500 us/op 2.8000 us/op 2.27
mainnet_e81889 - altair processEffectiveBalanceUpdates 14.958 ms/op 13.863 ms/op 1.08
mainnet_e81889 - altair processSlashingsReset 36.784 us/op 16.717 us/op 2.20
mainnet_e81889 - altair processRandaoMixesReset 46.007 us/op 20.823 us/op 2.21
mainnet_e81889 - altair processHistoricalRootsUpdate 9.2940 us/op 4.0500 us/op 2.29
mainnet_e81889 - altair processParticipationFlagUpdates 118.77 ms/op 118.95 ms/op 1.00
mainnet_e81889 - altair processSyncCommitteeUpdates 5.3630 us/op 3.1240 us/op 1.72
mainnet_e81889 - altair afterProcessEpoch 248.68 ms/op 258.55 ms/op 0.96
altair processInactivityUpdates - 250000 normalcase 92.353 ms/op 86.307 ms/op 1.07
altair processInactivityUpdates - 250000 worstcase 91.463 ms/op 86.239 ms/op 1.06
altair processParticipationFlagUpdates - 250000 anycase 113.96 ms/op 105.81 ms/op 1.08
altair processRewardsAndPenalties - 250000 normalcase 138.13 ms/op 144.56 ms/op 0.96
altair processRewardsAndPenalties - 250000 worstcase 139.04 ms/op 145.76 ms/op 0.95
altair processSyncCommitteeUpdates - 250000 461.06 ms/op 416.45 ms/op 1.11
Tree 40 250000 create 900.27 ms/op 788.53 ms/op 1.14
Tree 40 250000 get(125000) 363.17 ns/op 394.71 ns/op 0.92
Tree 40 250000 set(125000) 3.0844 us/op 2.5355 us/op 1.22
Tree 40 250000 toArray() 50.612 ms/op 51.836 ms/op 0.98
Tree 40 250000 iterate all - toArray() + loop 50.432 ms/op 44.050 ms/op 1.14
Tree 40 250000 iterate all - get(i) 139.21 ms/op 136.99 ms/op 1.02
MutableVector 250000 create 29.565 ms/op 29.061 ms/op 1.02
MutableVector 250000 get(125000) 16.100 ns/op 16.796 ns/op 0.96
MutableVector 250000 set(125000) 741.61 ns/op 595.97 ns/op 1.24
MutableVector 250000 toArray() 10.276 ms/op 10.363 ms/op 0.99
MutableVector 250000 iterate all - toArray() + loop 10.029 ms/op 10.486 ms/op 0.96
MutableVector 250000 iterate all - get(i) 3.8198 ms/op 3.7440 ms/op 1.02
Array 250000 create 6.2133 ms/op 6.7760 ms/op 0.92
Array 250000 clone - spread 2.5472 ms/op 2.6633 ms/op 0.96
Array 250000 get(125000) 1.2080 ns/op 1.2110 ns/op 1.00
Array 250000 set(125000) 1.1730 ns/op 1.2440 ns/op 0.94
Array 250000 iterate all - loop 154.01 us/op 196.39 us/op 0.78
aggregationBits - 2048 els - readonlyValues 255.28 us/op 276.07 us/op 0.92
aggregationBits - 2048 els - zipIndexesInBitList 46.365 us/op 48.795 us/op 0.95
regular array get 100000 times 60.237 us/op 79.990 us/op 0.75
wrappedArray get 100000 times 61.490 us/op 79.363 us/op 0.77
arrayWithProxy get 100000 times 37.170 ms/op 37.436 ms/op 0.99
ssz.Root.equals 1.3680 us/op 1.3760 us/op 0.99
ssz.Root.equals with valueOf() 1.7990 us/op 1.5890 us/op 1.13
byteArrayEquals with valueOf() 1.6530 us/op 1.5310 us/op 1.08
phase0 processBlock - 250000 vs - 7PWei normalcase 14.135 ms/op 12.273 ms/op 1.15
phase0 processBlock - 250000 vs - 7PWei worstcase 102.20 ms/op 89.822 ms/op 1.14
phase0 afterProcessEpoch - 250000 vs - 7PWei 231.76 ms/op 243.59 ms/op 0.95
phase0 beforeProcessEpoch - 250000 vs - 7PWei 768.38 ms/op 666.66 ms/op 1.15
phase0 processEpoch - mainnet_e58758 981.36 ms/op 912.09 ms/op 1.08
mainnet_e58758 - phase0 beforeProcessEpoch 623.20 ms/op 562.46 ms/op 1.11
mainnet_e58758 - phase0 processJustificationAndFinalization 117.88 us/op 70.339 us/op 1.68
mainnet_e58758 - phase0 processRewardsAndPenalties 192.17 ms/op 114.53 ms/op 1.68
mainnet_e58758 - phase0 processRegistryUpdates 83.053 us/op 51.545 us/op 1.61
mainnet_e58758 - phase0 processSlashings 6.6240 us/op 3.7490 us/op 1.77
mainnet_e58758 - phase0 processEth1DataReset 5.3070 us/op 2.6650 us/op 1.99
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 11.998 ms/op 10.691 ms/op 1.12
mainnet_e58758 - phase0 processSlashingsReset 30.734 us/op 14.099 us/op 2.18
mainnet_e58758 - phase0 processRandaoMixesReset 39.495 us/op 21.300 us/op 1.85
mainnet_e58758 - phase0 processHistoricalRootsUpdate 7.2600 us/op 3.8330 us/op 1.89
mainnet_e58758 - phase0 processParticipationRecordUpdates 28.856 us/op 14.574 us/op 1.98
mainnet_e58758 - phase0 afterProcessEpoch 211.86 ms/op 248.66 ms/op 0.85
phase0 processEffectiveBalanceUpdates - 250000 normalcase 13.546 ms/op 12.301 ms/op 1.10
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.8481 s/op 1.4594 s/op 1.27
phase0 processRegistryUpdates - 250000 normalcase 91.517 us/op 49.018 us/op 1.87
phase0 processRegistryUpdates - 250000 badcase_full_deposits 3.8908 ms/op 3.3602 ms/op 1.16
phase0 processRegistryUpdates - 250000 worstcase 0.5 2.6171 s/op 2.0014 s/op 1.31
phase0 getAttestationDeltas - 250000 normalcase 39.142 ms/op 42.383 ms/op 0.92
phase0 getAttestationDeltas - 250000 worstcase 39.481 ms/op 41.965 ms/op 0.94
phase0 processSlashings - 250000 worstcase 46.578 ms/op 45.232 ms/op 1.03
shuffle list - 16384 els 14.753 ms/op 14.731 ms/op 1.00
shuffle list - 250000 els 202.09 ms/op 213.21 ms/op 0.95
getEffectiveBalances - 250000 vs - 7PWei 12.585 ms/op 14.111 ms/op 0.89
computeDeltas 4.4339 ms/op 4.2646 ms/op 1.04
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 2.7560 ms/op 2.4548 ms/op 1.12
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 834.45 us/op 750.17 us/op 1.11
BLS verify - blst-native 2.4032 ms/op 2.1685 ms/op 1.11
BLS verifyMultipleSignatures 3 - blst-native 4.7751 ms/op 4.4958 ms/op 1.06
BLS verifyMultipleSignatures 8 - blst-native 10.381 ms/op 9.6921 ms/op 1.07
BLS verifyMultipleSignatures 32 - blst-native 37.453 ms/op 34.880 ms/op 1.07
BLS aggregatePubkeys 32 - blst-native 50.145 us/op 46.885 us/op 1.07
BLS aggregatePubkeys 128 - blst-native 200.29 us/op 183.81 us/op 1.09
getAttestationsForBlock 96.706 ms/op 87.039 ms/op 1.11
CheckpointStateCache - add get delete 19.049 us/op 18.281 us/op 1.04
validate gossip signedAggregateAndProof - struct 5.7136 ms/op 5.2288 ms/op 1.09
validate gossip signedAggregateAndProof - treeBacked 5.6098 ms/op 5.1914 ms/op 1.08
validate gossip attestation - struct 2.6587 ms/op 2.4606 ms/op 1.08
validate gossip attestation - treeBacked 2.6729 ms/op 2.4892 ms/op 1.07
Object access 1 prop 0.43100 ns/op 0.39400 ns/op 1.09
Map access 1 prop 0.34100 ns/op 0.33000 ns/op 1.03
Object get x1000 17.025 ns/op 20.759 ns/op 0.82
Map get x1000 1.1070 ns/op 1.1280 ns/op 0.98
Object set x1000 119.06 ns/op 128.86 ns/op 0.92
Map set x1000 84.296 ns/op 80.234 ns/op 1.05
Return object 10000 times 0.45480 ns/op 0.43070 ns/op 1.06
Throw Error 10000 times 7.3290 us/op 6.8269 us/op 1.07
RateTracker 1000000 limit, 1 obj count per request 208.85 ns/op 213.56 ns/op 0.98
RateTracker 1000000 limit, 2 obj count per request 154.29 ns/op 162.20 ns/op 0.95
RateTracker 1000000 limit, 4 obj count per request 134.19 ns/op 133.22 ns/op 1.01
RateTracker 1000000 limit, 8 obj count per request 124.32 ns/op 120.36 ns/op 1.03
RateTracker with prune 5.0120 us/op 4.7760 us/op 1.05

by benchmarkbot/action

@dapplion dapplion merged commit 148ac6f into master Dec 21, 2021
@dapplion dapplion deleted the dapplion/build_if_changed branch December 21, 2021 10:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants