PERF: Consider using xxhash

[LifeIsStrange]

Clear and concise description of the problem

Hi @yyx990803

xxhash3 is the most popular next generation hashing algorithm although there are many other contenders.
A significantly faster hashing algorithm could reduce bottlenecks?

Suggested solution

Webpack is switching to xxhash64 by default
webpack/webpack#14306
Esbuild switched to xxhash64
evanw/esbuild#1099
evanw/esbuild#1107
(which is slower than xxhash3)

No idea about what rollup use.

Since this can lead to potentially significant performance win, I suggest Vite to migrate to xxhash64 or the faster xxhash3.

BTW xxhash60 (no idea about xxhash3) would make the hashing FIPS compliant
webpack/webpack#13572

Question:
Does vite hashes needs to be aligned with the same algorithm than esbuild/rollup? No idea
The length of hashes should no be altered Iengthened guess? It would be a visual regression for debugging/readability if hashes were longer.

Alternative

xxhash is great but alternatives could be considered, especially if xxhash3 is not enough to eliminate all hash related bottlenecks.

Wyhash is significantly faster then xxhash3 in general.
https://github.com/tkaitchuck/aHash is significantly faster than wyhash and is probably the fastest useful hashing function.

Additional context

.

Validations

• Follow our Code of Conduct
• Read the Contributing Guidelines.
• Read the docs.
• Check that there isn't already an issue that request the same feature to avoid creating a duplicate.

[LifeIsStrange]

Btw I wonder whether vite has a riche caching system similar to 'contenthash' on webpack
https://webpack.js.org/guides/caching/

[patak-dev]

Interesting idea, I think it would be good to see a PR and some benchmarks after this change 👍🏼

[LifeIsStrange]

I will not submit a PR myself unfortunately :)

[wangyi-fudan]

Clear and concise description of the problem

Hi @yyx990803

xxhash3 is the most popular next generation hashing algorithm although there are many other contenders. A significantly faster hashing algorithm could reduce bottlenecks?

Suggested solution

Webpack is switching to xxhash64 by default webpack/webpack#14306 Esbuild switched to xxhash64 evanw/esbuild#1099 evanw/esbuild#1107 (which is slower than xxhash3)

No idea about what rollup use.

Since this can lead to potentially significant performance win, I suggest Vite to migrate to xxhash64 or the faster xxhash3.

BTW xxhash60 (no idea about xxhash3) would make the hashing FIPS compliant webpack/webpack#13572

Question: Does vite hashes needs to be aligned with the same algorithm than esbuild/rollup? No idea The length of hashes should no be altered Iengthened guess? It would be a visual regression for debugging/readability if hashes were longer.

Alternative

xxhash is great but alternatives could be considered, especially if xxhash3 is not enough to eliminate all hash related bottlenecks.

Wyhash is significantly faster then xxhash3 in general. https://github.com/tkaitchuck/aHash is significantly faster than wyhash and is probably the fastest useful hashing function.

Additional context

.

Validations

• Follow our Code of Conduct
• Read the Contributing Guidelines.
• Read the docs.
• Check that there isn't already an issue that request the same feature to avoid creating a duplicate.

ahash is slower than wyhash despite ahash's own claim in rust. smhasher has both benchmarks.

[LifeIsStrange]

Yeah you're probably right
https://github.com/rurban/smhasher

[bluwy]

The only hashing library I see Vite is using is hash-sum, and that's for plugin-vue only. Otherwise Vite itself implements a simple hash function like so:

vite/packages/vite/src/node/optimizer/index.ts

Lines 823 to 825 in 42c15f6

	export function getHash(text: string): string {
	return createHash('sha256').update(text).digest('hex').substring(0, 8)
	}

And IMO I'm not sure bringing in a new library here is worth the effort, considering it would also increase the bundle size.

[patak-dev]

Closing as #7975 has been merged