Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: make it work with large airdrops #37

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

bowd
Copy link

@bowd bowd commented Sep 19, 2024

Update scripts in order for merkle tree generation to work with very large CSVs

@bowd bowd changed the title feat: make it work via sqlite feat: make it work with large airdrops Sep 19, 2024
Copy link

New and removed dependencies detected. Learn more about Socket for GitHub ↗︎

Package New capabilities Transitives Size Publisher
npm/@fastify/busboy@2.1.1 None 0 80.2 kB gurgunday
npm/@gar/promisify@1.1.3 None 0 4.2 kB gar
npm/@nomicfoundation/edr-darwin-arm64@0.5.2 None 0 12.2 MB nomic-foundation-publisher
npm/@nomicfoundation/edr-darwin-x64@0.5.2 None 0 12.8 MB nomic-foundation-publisher
npm/@nomicfoundation/edr-linux-arm64-gnu@0.5.2 None 0 17.8 MB nomic-foundation-publisher
npm/@nomicfoundation/edr-linux-arm64-musl@0.5.2 None 0 17.8 MB nomic-foundation-publisher
npm/@nomicfoundation/edr-linux-x64-gnu@0.5.2 None 0 19.7 MB nomic-foundation-publisher
npm/@nomicfoundation/edr-linux-x64-musl@0.5.2 None 0 19.7 MB nomic-foundation-publisher
npm/@nomicfoundation/edr-win32-x64-msvc@0.5.2 None 0 13.9 MB nomic-foundation-publisher
npm/@nomicfoundation/edr@0.5.2 filesystem, shell 0 129 kB nomic-foundation-publisher
npm/@nomicfoundation/ethereumjs-common@4.0.4 None 0 649 kB fvictorio
npm/@nomicfoundation/ethereumjs-rlp@5.0.4 None 0 125 kB fvictorio
npm/@nomicfoundation/ethereumjs-tx@5.0.4 None 0 647 kB fvictorio
npm/@nomicfoundation/ethereumjs-util@9.0.4 network 0 545 kB fvictorio
npm/@nomicfoundation/solidity-analyzer-darwin-arm64@0.1.2 None 0 471 kB alcuadrado
npm/@nomicfoundation/solidity-analyzer-darwin-x64@0.1.2 None 0 485 kB alcuadrado
npm/@nomicfoundation/solidity-analyzer-linux-arm64-gnu@0.1.2 None 0 2 MB alcuadrado
npm/@nomicfoundation/solidity-analyzer-linux-arm64-musl@0.1.2 None 0 1.97 MB alcuadrado
npm/@nomicfoundation/solidity-analyzer-linux-x64-gnu@0.1.2 None 0 1.94 MB alcuadrado
npm/@nomicfoundation/solidity-analyzer-linux-x64-musl@0.1.2 None 0 1.93 MB alcuadrado
npm/@nomicfoundation/solidity-analyzer-win32-x64-msvc@0.1.2 None 0 386 kB alcuadrado
npm/@nomicfoundation/solidity-analyzer@0.1.2 filesystem, shell 0 42.1 kB alcuadrado
npm/@nomiclabs/hardhat-ethers@2.2.3 None 0 80.9 kB fvictorio
npm/@nomiclabs/hardhat-waffle@2.0.6 None 0 49.3 kB fvictorio
npm/@npmcli/fs@1.1.1 filesystem 0 41.9 kB nlf
npm/@npmcli/move-file@1.1.2 filesystem 0 7.89 kB nlf
npm/@protobufjs/aspromise@1.1.2 None 0 9.05 kB dcode
npm/@protobufjs/base64@1.1.2 None 0 9.22 kB dcode
npm/@protobufjs/codegen@2.0.4 None 0 9.14 kB dcode
npm/@protobufjs/eventemitter@1.1.0 None 0 7.75 kB dcode
npm/@protobufjs/fetch@1.1.0 network 0 8.76 kB dcode
npm/@protobufjs/float@1.0.2 None 0 27 kB dcode
npm/@protobufjs/inquire@1.1.0 None 0 4.29 kB dcode
npm/@protobufjs/path@1.1.2 None 0 7.77 kB dcode
npm/@protobufjs/pool@1.1.0 None 0 6.25 kB dcode
npm/@protobufjs/utf8@1.1.0 None 0 23.5 kB dcode
npm/@tootallnate/once@1.1.2 None 0 4.08 kB tootallnate
npm/@types/big-json@3.2.4 None 0 3.76 kB types
npm/@types/bn.js@4.11.6 None 0 13.9 kB types
npm/abbrev@1.1.1 None 0 4.78 kB isaacs
npm/agentkeepalive@4.5.0 network 0 43.7 kB fengmk2
npm/ansi-align@3.0.1 None 0 7.68 kB nexdrew
npm/aproba@2.0.0 None 0 8.05 kB iarna
npm/are-we-there-yet@3.0.1 None 0 14.3 kB lukekarrys
npm/big-json@3.2.0 None 0 13.5 kB donutespresso
npm/boxen@5.1.2 None 0 18.4 kB sindresorhus
npm/cacache@15.3.0 filesystem 0 76.3 kB nlf
npm/cli-boxes@2.2.1 None 0 6.14 kB sindresorhus
npm/console-control-strings@1.1.0 None 0 12.7 kB iarna
npm/csv-parse@5.5.6 None 0 1.41 MB david
npm/deep-extend@0.6.0 None 0 9.19 kB unclechu
npm/delegates@1.0.0 None 0 7.46 kB tjholowaychuk
npm/detect-libc@2.0.3 filesystem, shell 0 23.6 kB lovell
npm/diff@5.2.0 None 0 429 kB explodingcabbage
npm/encoding@0.1.13 None 0 7.12 kB andris
npm/err-code@2.0.3 None 0 12.3 kB achingbrain
npm/expand-template@2.0.3 None 0 5.41 kB ralphtheninja
npm/from2@2.3.0 None 0 9.35 kB mafintosh
npm/gauge@4.0.4 None 0 43.2 kB gar
npm/github-from-package@0.0.0 None 0 4.61 kB substack
npm/hardhat@2.22.10 environment, filesystem, network, shell 0 3.09 MB kanej
npm/has-unicode@2.0.1 environment 0 3.44 kB iarna
npm/http-proxy-agent@4.0.1 network 0 17.1 kB tootallnate
npm/humanize-ms@1.2.1 None 0 3.66 kB dead_horse
npm/imurmurhash@0.1.4 None 0 11.9 kB jensyt
npm/infer-owner@1.0.4 filesystem 0 4.29 kB isaacs
npm/into-stream@5.1.1 None 0 6.47 kB sindresorhus
npm/ip-address@9.0.5 None 0 177 kB beaugunderson
npm/is-lambda@1.0.1 None 0 2.94 kB watson
npm/isarray@1.0.0 None 0 3.89 kB juliangruber
npm/json-stream-stringify@2.0.4 None 0 203 kB faleij
npm/jsonparse@1.3.1 None 0 36.8 kB creationix
npm/jsonstream@1.3.5 None 0 0 B
npm/long@5.2.3 None 0 119 kB dcode
npm/make-fetch-happen@9.1.0 environment, network 0 57.6 kB gar
npm/minipass-collect@1.0.2 None 0 4.87 kB isaacs
npm/minipass-fetch@1.4.1 environment, network 0 43.2 kB isaacs
npm/minipass-flush@1.0.5 None 0 3.77 kB isaacs
npm/minipass-pipeline@1.2.4 None 0 7 kB isaacs
npm/minipass-sized@1.0.3 None 0 124 kB isaacs
npm/mkdirp-classic@0.5.3 filesystem 0 4.5 kB mafintosh
npm/napi-build-utils@1.0.2 None 0 15.5 kB inspiredware
npm/negotiator@0.6.3 None 0 27.4 kB dougwilson
npm/node-abi@3.67.0 None 0 16.2 kB electron-cfa
npm/node-addon-api@7.1.1 None 0 396 kB legendecas
npm/node-gyp@8.4.1 environment, shell 0 1.98 MB rvagg
npm/nopt@5.0.0 environment 0 25.8 kB isaacs
npm/npmlog@6.0.2 None 0 17.1 kB lukekarrys
npm/p-is-promise@3.0.0 None 0 3.31 kB sindresorhus
npm/prebuild-install@7.1.2 environment, filesystem 0 33.9 kB lovell
npm/promise-inflight@1.0.1 None 0 3.04 kB iarna
npm/promise-retry@2.0.1 None 0 15.6 kB achingbrain
npm/protobufjs@7.4.0 filesystem, network 0 2.77 MB google-wombot
npm/rc@1.2.8 environment, filesystem 0 17.3 kB dominictarr
npm/readable-stream@2.3.7 environment 0 87.7 kB matteo.collina
npm/retry@0.12.0 None 0 32.2 kB tim-kos
npm/serialize-javascript@6.0.2 None 0 16.9 kB redonkulus
npm/signal-exit@3.0.7 None 0 9.96 kB isaacs
npm/smart-buffer@4.2.0 None 0 138 kB joshglazebrook
npm/socks-proxy-agent@6.2.1 network 0 22.9 kB kikobeats
npm/socks@2.8.3 network 0 156 kB joshglazebrook
npm/sprintf-js@1.1.3 None 0 39.9 kB alexei
npm/sqlite3@5.1.7 None 0 3.35 MB daniellockyer
npm/ssri@8.0.1 None 0 46.9 kB nlf
npm/string_decoder@1.1.1 None 0 15.3 kB matteo.collina
npm/tar-fs@2.1.1 filesystem 0 28 kB mafintosh
npm/through2@3.0.2 None 0 9.51 kB rvagg
npm/undici@5.28.4 environment, network, unsafe 0 1.17 MB matteo.collina
npm/unique-filename@1.1.1 None 0 41.4 kB iarna
npm/unique-slug@2.0.2 None 0 2.68 kB zkat
npm/widest-line@3.1.0 None 0 3.73 kB sindresorhus
npm/workerpool@6.5.1 eval 0 374 kB josdejong

🚮 Removed packages: npm/@ethereumjs/block@3.6.3, npm/@ethereumjs/blockchain@5.5.3, npm/@ethereumjs/common@2.6.5, npm/@ethereumjs/ethash@1.1.0, npm/@ethereumjs/tx@3.5.2, npm/@ethereumjs/vm@5.9.3, npm/@nomiclabs/hardhat-ethers@2.0.6, npm/@nomiclabs/hardhat-waffle@2.0.3, npm/@solidity-parser/parser@0.14.2, npm/@types/abstract-leveldown@7.2.0, npm/@types/bn.js@5.1.0, npm/@types/level-errors@3.0.0, npm/@types/levelup@4.3.3, npm/@types/sinon-chai@3.2.8, npm/@types/sinon@10.0.12, npm/@types/sinonjs__fake-timers@8.1.2, npm/@types/underscore@1.11.4, npm/@types/web3@1.0.19, npm/@ungap/promise-all-settled@1.1.2, npm/abort-controller@3.0.0, npm/antlr4ts@0.5.0-alpha.4, npm/crc-32@1.2.2, npm/diff@5.0.0, npm/event-target-shim@5.0.1, npm/hardhat@2.9.9, npm/isarray@0.0.1, npm/mcl-wasm@0.7.9, npm/nanoid@3.3.1, npm/readable-stream@1.0.34, npm/semaphore-async-await@1.5.1, npm/serialize-javascript@6.0.0, npm/string_decoder@0.10.31, npm/through2@2.0.5, npm/true-case-path@2.2.1, npm/undici@5.5.1, npm/workerpool@6.2.0

View full report↗︎

Copy link

🚨 Potential security issues detected. Learn more about Socket for GitHub ↗︎

To accept the risk, merge this PR and you will not be notified again.

Alert Package NoteSourceCI
Native code npm/sqlite3@5.1.7 ⚠︎
Install scripts npm/protobufjs@7.4.0
  • Install script: postinstall
  • Source: node scripts/postinstall
⚠︎

View full report↗︎

Next steps

What's wrong with native code?

Contains native code which could be a vector to obscure malicious code, and generally decrease the likelihood of reproducible or reliable installs.

Ensure that native code bindings are expected. Consumers may consider pure JS and functionally similar alternatives to avoid the challenges and risks associated with native code bindings.

What is an install script?

Install scripts are run when the package is installed. The majority of malware in npm is hidden in install scripts.

Packages should not be running non-essential scripts during install and there are often solutions to problems people solve with install scripts that can be run at publish time instead.

Take a deeper look at the dependency

Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support [AT] socket [DOT] dev.

Remove the package

If you happen to install a dependency that Socket reports as Known Malware you should immediately remove it and select a different dependency. For other alert types, you may may wish to investigate alternative packages or consider if there are other ways to mitigate the specific risk posed by the dependency.

Mark a package as acceptable risk

To ignore an alert, reply with a comment starting with @SocketSecurity ignore followed by a space separated list of ecosystem/package-name@version specifiers. e.g. @SocketSecurity ignore npm/foo@1.0.0 or ignore all packages with @SocketSecurity ignore-all

  • @SocketSecurity ignore npm/sqlite3@5.1.7
  • @SocketSecurity ignore npm/protobufjs@7.4.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant