Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix simdjson update script and update to v3.6.1 #50986

Closed
wants to merge 2 commits into from
Closed

Fix simdjson update script and update to v3.6.1 #50986

wants to merge 2 commits into from

Conversation

anonrig
Copy link
Member

@anonrig anonrig commented Dec 1, 2023

There was a bug in update-simdjson.sh preventing us from updating to latest version. I've fixed it and updated simdjson to latest version.

@anonrig anonrig added fast-track PRs that do not need to wait for 48 hours to land. commit-queue-rebase Add this label to allow the Commit Queue to land a PR in several commits. labels Dec 1, 2023
@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/security-wg

@nodejs-github-bot nodejs-github-bot added dependencies Pull requests that update a dependency file. needs-ci PRs that need a full CI run. tools Issues and PRs related to the tools directory. labels Dec 1, 2023
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Dec 1, 2023

Fast-track has been requested by @anonrig. Please 👍 to approve.

@anonrig anonrig added the request-ci Add this label to start a Jenkins CI on a PR. label Dec 1, 2023
@anonrig anonrig force-pushed the fix-simdjson-update-script branch from 43ed739 to 07bc0be Compare December 1, 2023 00:33
@anonrig anonrig added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Dec 1, 2023
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Dec 1, 2023
@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/56024/

@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/56026/

@StefanStojanovic
Copy link
Contributor

Hello @anonrig while checking the status of the CI, I noticed that in both runs this PR had, all of the Windows ARM64 tests failed. From what I saw, the error always seems to be Error: Invalid package config and is usually triggered when calling modulesBinding.readPackageJSON.

cc @lemire

@lemire
Copy link
Member

lemire commented Dec 1, 2023

@StefanStojanovic @anonrig The issue is verified. There has been a regression. Working a patch to fix this regression on ARM64 Windows.

@lemire
Copy link
Member

lemire commented Dec 1, 2023 •
edited
Loading

@StefanStojanovic @anonrig The regression is with version 3.6. Version 3.5 did not have an issue.

Update. The regression is with 3.6.1. Version 3.6.0 is fine.

@lemire
Copy link
Member

lemire commented Dec 1, 2023

Ok. Version 3.6.2 has the fix, and it is out:

https://github.com/simdjson/simdjson/releases/tag/v3.6.2

@StefanStojanovic
Copy link
Contributor

Ok. Version 3.6.2 has the fix, and it is out:

https://github.com/simdjson/simdjson/releases/tag/v3.6.2

Thanks for addressing this quickly!

@lemire
Copy link
Member

lemire commented Dec 1, 2023 •
edited
Loading

@StefanStojanovic I bought an ARM-based developer kit for this very purpose.

It is precisely because I tried to fine tune the support that I got in trouble.

I am still unhappy about the code generation. I think Microsoft needs to generate better ARM binaries from Visual Studio. It is not at all at the level of LLVM.

If possible, I'd recommend building the Windows binaries with ClangCL. It would assuredly be faster.

@StefanStojanovic
Copy link
Contributor

@StefanStojanovic I bought an ARM-based developer kit for this very purpose.

It is precisely because I tried to fine tune the support that I got in trouble.

I am still unhappy about the code generation. I think Microsoft needs to generate better ARM binaries from Visual Studio. It is not at all at the level of LLVM.

If possible, I'd recommend building the Windows binaries with ClangCL. It would assuredly be faster.

I understand your concerns @lemire but currently, ClangCL cannot be used for building on Windows. One thing that comes to mind is for simdjson to use #ifdef to separate Clang and MSVC compilation on Windows. That way you'd be able to get better code generation with Clang, and MSVC would not have the regression.

@lemire
Copy link
Member

lemire commented Dec 5, 2023

@StefanStojanovic The problem is not supporting Visual Studio, nor is it that it impacts other compilers (it does not, we already use macros).

The problem is that when you stare at the generated code from Visual Studio, you know that you are leaving performance on the table for no good reason.

The problem is that Visual Studio builds slowly and generate slow code compared to LLVM (clang):

https://lemire.me/blog/2023/02/27/visual-studio-versus-clangcl/

Google builds Chrome under Windows LLVM:

https://blog.llvm.org/2018/03/clang-is-now-used-to-build-chrome-for.html

Firefox is built using LLVM under Windows:

https://www.phoronix.com/news/Firefox-Clang-LTO-All-Platforms

@targos
Copy link
Member

targos commented Dec 5, 2023

I tried to build Node.js with ClangCL but there are many roadblocks. See #35433 (help appreciated!).

@lemire
Copy link
Member

lemire commented Dec 5, 2023

@targos There you go!

@anonrig anonrig force-pushed the fix-simdjson-update-script branch from 07bc0be to 190a019 Compare December 5, 2023 17:45
@anonrig anonrig added the request-ci Add this label to start a Jenkins CI on a PR. label Dec 5, 2023
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Dec 5, 2023
@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/56106/

@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/56115/

@anonrig anonrig added the commit-queue Add this label to land a pull request using GitHub Actions. label Dec 6, 2023
@nodejs-github-bot nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Dec 6, 2023
@nodejs-github-bot
Copy link
Collaborator

Landed in c97322a...ac9e594

nodejs-github-bot pushed a commit that referenced this pull request Dec 6, 2023
@anonrig @nodejs-github-bot
tools: fix simdjson updater
28b05cf 
PR-URL: #50986
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
nodejs-github-bot pushed a commit that referenced this pull request Dec 6, 2023
@anonrig @nodejs-github-bot
deps: update simdjson to v3.6.2
ac9e594 
PR-URL: #50986
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
RafaelGSS pushed a commit that referenced this pull request Dec 15, 2023
@anonrig @RafaelGSS
tools: fix simdjson updater
4705023 
PR-URL: #50986
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
RafaelGSS pushed a commit that referenced this pull request Dec 15, 2023
@anonrig @RafaelGSS
deps: update simdjson to v3.6.2
429fbb3 
PR-URL: #50986
Reviewed-By: Geoffrey Booth <webadmin@geoffreybooth.com>
Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com>
@RafaelGSS RafaelGSS mentioned this pull request Dec 15, 2023
Merged
@marco-ippolito marco-ippolito added the backport-blocked-v20.x PRs that should land on the v20.x-staging branch but are blocked by another PR's pending backport. label May 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@marco-ippolito marco-ippolito marco-ippolito approved these changes

@GeoffreyBooth GeoffreyBooth GeoffreyBooth approved these changes

@lemire lemire lemire approved these changes

Assignees
No one assigned
Labels
author ready PRs that have at least one approval, no pending requests for changes, and a CI started. backport-blocked-v20.x PRs that should land on the v20.x-staging branch but are blocked by another PR's pending backport. commit-queue-rebase Add this label to allow the Commit Queue to land a PR in several commits. dependencies Pull requests that update a dependency file. fast-track PRs that do not need to wait for 48 hours to land. needs-ci PRs that need a full CI run. tools Issues and PRs related to the tools directory.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@anonrig @nodejs-github-bot @StefanStojanovic @lemire @targos @GeoffreyBooth @marco-ippolito