GH-41015: [JS] [Benchmarking] allow JS benchmarks to run more portably

[austin3dickey]

Fixes #41015. This PR allows users to run yarn perf to run the JS benchmarks on more machines; in particular, machines that do not support env -S like Amazon Linux.

• GitHub Issue: [JS] [Benchmarking] JavaScript benchmarks don't work on Amazon Linux #41015

[github-actions]

⚠️ GitHub issue #41015 has been automatically assigned in GitHub to PR creator.

[austin3dickey]

@ursabot please benchmark lang=JavaScript

[ursabot]

Benchmark runs are scheduled for commit e5f6c93. Watch https://buildkite.com/apache-arrow and https://conbench.ursa.dev for updates. A comment will be posted here when the runs are complete.

[domoritz]

Hmm, now we have the same command (node --no-warnings --loader ts-node/esm/transpile-only) in two places (the package.json and the ts file). Where else do we call perf from? Could we just change all the calls to run yarn perf / npm run perf?

[austin3dickey]

Where else do we call perf from? Could we just change all the calls to run yarn perf / npm run perf?

As far as I know, everything that runs benchmarks just uses yarn perf.

Now we have the same command in two places

If this is a problem, perhaps we should remove the shebang then? The trouble with the shebang as written is that using /usr/bin/env -S is non-portable. So if we're using yarn perf everywhere, it should be fine to remove the shebang.

[domoritz]

Sounds good to me. We have a few other places that use the same shebang and I think we should update them as well.

Rather than yarn, maybe we should use npm run since it's more portable. But that can be a separate change.

[austin3dickey]

We have a few other places that use the same shebang and I think we should update them as well.

Just looked into this briefly and found places in archery that might fail if we remove the shebangs:

arrow/dev/archery/archery/integration/tester_js.py

Lines 27 to 30 in 83359d6

	_VALIDATE = os.path.join(_EXE_PATH, 'integration.ts')
	_JSON_TO_ARROW = os.path.join(_EXE_PATH, 'json-to-arrow.ts')
	_STREAM_TO_FILE = os.path.join(_EXE_PATH, 'stream-to-file.ts')
	_FILE_TO_STREAM = os.path.join(_EXE_PATH, 'file-to-stream.ts')

So we could go through and fix all these, but it might be more trouble than it's worth. Would you mind if I keep this PR scoped to just fixing the benchmarks' portability?

(For context, I'm trying to move the orchestration of the Arrow merge-commit JS benchmarks off a machine in someone's basement to an EC2 instance. This PR unblocks that work.)

[domoritz]

So we could go through and fix all these, but it might be more trouble than it's worth. Would you mind if I keep this PR scoped to just fixing the benchmarks' portability?

Agreed. So to unblock you, can you run node --no-warnings --loader ts-node/esm/transpile-only perf/index.ts from the perf script? Otherwise, sure, let's merge this as is. I approve either way.

[conbench-apache-arrow]

Thanks for your patience. Conbench analyzed the 1 benchmarking run that has been run so far on PR commit e5f6c93.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details.

[austin3dickey]

I think that might be slightly difficult on the orchestration side. Cool, thanks for the review. 💜 would you mind merging? I don't have write access.

[conbench-apache-arrow]

After merging your PR, Conbench analyzed the 7 benchmarking runs that have been run so far on merge-commit 02bc653.

There was 1 benchmark result indicating a performance regression:

• Commit Run on ursa-i9-9960x at 2024-04-05 21:57:32Z
  • tpch (R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-02, scale_factor=1

The full Conbench report has more details. It also includes information about 3 possible false positives for unstable benchmarks that are known to sometimes produce them.