perf(fs): realpath native support

[sambacha]

Subsantiave changes are documented below:

- import path from "path";
+ import path from 'node:path';



- : files.map((f) => fs.realpathSync(f));
+ : files.map((f) => fs.realpathSync.native(f));


- const configFile = fs.realpathSync(userConfigPath);
+ const configFile = fs.realpathSync.native(userConfigPath);

- fs.realpathSync(resolvedPackageJson) === fs.realpathSync(thisPackageJson)
+ fs.realpathSync.native(resolvedPackageJson) === fs.realpathSync.native(thisPackageJson)



+  // TODO: 
+ fs.mkdtempSync(path.join(os.tmpdir(), `hardhat-tests-${nameHint}`));
- const tmpDir = path.join(tmpDirContainer, `hardhat-tests-${nameHint}`);


+ [path.basename(s)]: { content: fs.realpathSync.native(s, "utf8") },
- [path.basename(s)]: { content: fs.readFileSync(s, "utf8") },


+ projectRoot = fs.realpathSync.native(".");
- projectRoot = fs.realpathSync(".");


+ const projectRoot = fs.realpathSync.native(".");
- const projectRoot = fs.realpathSync(".");


@@ packages/hardhat-core/test/internal/solidity/resolver.ts
+ const realpathSync = fs.realpathSync.native ?? fs.realpathSync;



- fs.realpathSync(path.join("contracts", "fromContracts.sol"))
+ fs.realpathSync.native(path.join("contracts", "fromContracts.sol"))

[changeset-bot]

⚠️ No Changeset found

Latest commit: 9cc752b

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[github-actions]

This issue is also being tracked on Linear.

We use Linear to manage our development process, but we keep the conversations on Github.

LINEAR-ID: 1332152d-044f-4f8f-9880-9b2b935cdc71

[sambacha]

Note this requires NodeJS 14, and typescript 4.6.3

[sambacha]

the build was previously failing, fixed that now

[sambacha]

Workflow changes

• Changes made to workflows because this is a breaking change in typescript versioning and besides I do not think yall officially support nodejs 12 anymore.

• by specifying --prefer-offline, yarn will check the cache first, and only try to download the package from the registry if it cannot find it in the cache.

• updated the nodejs action to v3

Dependency changes

• typescript@^4.6.3 (4.7.1 is latest)
• I added turbo by accident , will remove that ofc.

Notes

Completely unsolicited ideas, I think some might be ez wins others, well YMMV - would like to know your thoughts ofc!

0. Include certain compiler versions (e.g. 0.6.12) or pre-compiled contracts (OZ, Uni, etc) with hardhat potentially. For reference solc-typed-ast compiler does include a cache env for including compilers, see https://github.com/ConsenSys/solc-typed-ast/blob/master/docker/download.sh

1. Nx Distributed Task Execution (DTE) Example/Benchmark, https://github.com/vsavkin/interstellar

2. tree diffing when restoring files from the cache https://github.com/vsavkin/large-monorepo#why-is-nx-faster-than-turbo

3. core_d.js launches a server so that your services invoke method is called with the same arguments. Later calls to invoke will be executed on the same instance.

3 may be useful for packages/hardhat-core/src/internal/util/scripts-runner.ts

4. potentially invoking nodejs directly e.g.

node --expose-gc node_modules/.bin/jest --coverage --globals "{\"coverage\":true}" src/

5. Utilize a more performant cache system like https://github.com/next-boost/hybrid-disk-cache - see why sqlite3 is faster than the filesystem (below certain file sizes ref: https://www.sqlite.org/fasterthanfs.html

Questions

How to best benchmark this? What sort of test would actually show any statistically significant differences? I would think maybe measuring access time or something as opposed to measuring compilation time.

[alcuadrado]

Lots to digest here. Thanks @sambacha! What's the part that requires node 14?

Also, are the node: imports faster?

[github-actions]

This PR was marked as stale because it didn't have any activity in the last 30 days. Please excuse us if we didn't have enough time to review it and get it merged. If you are still interested in getting these changes applied, please leave a comment indicating so. Otherwise, it will be closed in 7 days.

[github-actions]

This PR was closed because it has been stalled for 7 days with no activity.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated
hardhat	❌ Failed (Inspect)		Jul 11, 2022 at 11:31AM (UTC)
hardhat-storybook	❌ Failed (Inspect)		Jul 11, 2022 at 11:31AM (UTC)

[alcuadrado]

This took some time, but we finally got to optimize the compilation process a lot. You can take a look at #3048 to #3053.

Thanks for these ideas <3

[alcuadrado]

@gitpoap-bot @sambacha deserves a gitpoap, or a hundred actually

[alcuadrado]

@gitpoap-bot @sambacha deserves a gitpoap

(second try)

[gitpoap-bot]

Congrats, @sambacha ! You've earned a GitPOAP for your contribution!

GitPOAP: 2022 Hardhat Contributor:

Head to gitpoap.io & connect your GitHub account to mint!

Learn more about GitPOAPs here.