Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Windows: Rename operation throwing exception #3751

Closed
andipabst opened this issue Sep 13, 2022 · 6 comments · Fixed by #3769
Closed

Windows: Rename operation throwing exception #3751

andipabst opened this issue Sep 13, 2022 · 6 comments · Fixed by #3769
Assignees
Labels
bug This issue is a bug. module/kernel Issues affecting the `jsii-kernel` module os/windows Related specifically to Windows behavior p1

Comments

@andipabst
Copy link

Describe the bug

With JSII 1.67.0 the bug from #992 resurfaces again, the rename operation is throwing an exception.

Expected Behavior

JSII should run without errors

Current Behavior

jsii.errors.JavaScriptError: 
  Error: EPERM: operation not permitted, rename 'C:\<usertemp>\jsii-runtime-untar-4ppdjT' -> 'C:\<usertemp>\jsii-kernel-fxOoDQ\node_modules\aws-cdk-lib'
      at renameSync (fs.js:772:3)
      at exports.Kernel._load (C:\<usertemp>\tmprc5y26mb\lib\program.js:5356:37)
      at C:\<usertemp>\tmprc5y26mb\lib\program.js:5329:52
      at exports.Kernel._debugTime (C:\<usertemp>\tmprc5y26mb\lib\program.js:5863:28)
      at exports.Kernel.load (C:\<usertemp>\tmprc5y26mb\lib\program.js:5329:29)
      at exports.KernelHost.processRequest (C:\<usertemp>\tmprc5y26mb\lib\program.js:6618:36)
      at exports.KernelHost.run (C:\<usertemp>\tmprc5y26mb\lib\program.js:6592:48)
      at Immediate._onImmediate (C:\<usertemp>\tmprc5y26mb\lib\program.js:6593:46)
      at processImmediate (internal/timers.js:461:21)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "app.py", line 5, in <module>
    import aws_cdk as cdk
  File "C:\<project>\.venv\lib\site-packages\aws_cdk\__init__.py", line 1051, in <module>
    from ._jsii import *
  File "C:\<project>\.venv\lib\site-packages\aws_cdk\_jsii\__init__.py", line 13, in <module>
    __jsii_assembly__ = jsii.JSIIAssembly.load(
  File "C:\<project>\.venv\lib\site-packages\jsii\_runtime.py", line 43, in load
    _kernel.load(assembly.name, assembly.version, os.fspath(assembly_path))
  File "C:\<project>\.venv\lib\site-packages\jsii\_kernel\__init__.py", line 300, in load
    self.provider.load(LoadRequest(name=name, version=version, tarball=tarball))
  File "C:\<project>\.venv\lib\site-packages\jsii\_kernel\providers\process.py", line 341, in load
    return self._process.send(request, LoadResponse)
  File "C:\<project>\.venv\lib\site-packages\jsii\_kernel\providers\process.py", line 329, in send
    raise JSIIError(resp.error) from JavaScriptError(resp.stack)
jsii.errors.JSIIError: EPERM: operation not permitted, rename 'C:\<usertemp>\jsii-runtime-untar-4ppdjT' -> 'C:\<usertemp>\jsii-kernel-fxOoDQ\node_modules\aws-cdk-lib'

Reproduction Steps

Run any cdk command, e.g. cdk diff in a cdk project on Windows with A/V software enabled.

Possible Solution

The bug might have been introduced by #3724. Here, the fix from #992 is no longer present and the files are copied again, instead of being extracted directly to the target.

Additional Information/Context

The issue is not present in 1.66.0

SDK version used

1.67.0

Environment details (OS name and version, etc.)

Windows 10

@andipabst andipabst added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Sep 13, 2022
@Richardmbs12
Copy link

Richardmbs12 commented Sep 15, 2022

My whole team who is on windows is getting this error all at once since we upgraded our CDK stack and CLI version to 2.40, so it seems to be quite consistently happen for everyone on windows. This is a huge blocker for the team the past week as everyone is experiencing this. Please urgently address as this is a big bug that should not have gotten released after testing

@RomainMuller
Copy link
Contributor

this is a big bug that should not have gotten released after testing

Our testing (including Windows) worked flawlessly... The issue here is that we are unable to reproduce the issue with the problematic A/V software configurations due to various reasons (licenses, policies, etc...).


I see even setting JSII_RUNTIME_PACKAGE_CACHE environment variable to enabled would result in a renameSync operation being done, so even enabling the experimental feature will not save you...

@RomainMuller RomainMuller added p1 module/kernel Issues affecting the `jsii-kernel` module os/windows Related specifically to Windows behavior and removed needs-triage This issue or PR still needs to be triaged. labels Sep 26, 2022
@RomainMuller RomainMuller self-assigned this Sep 26, 2022
RomainMuller added a commit that referenced this issue Sep 26, 2022
Windows does not allow renaming files that are currently open, or
directories that contain open files. When antivirus software is used,
files are open for analysis by the A/V software, making it impossible to
rename files too quickly after they've been created.

This was already reported in #992 and addressed, however the issue was
re-introduced in #3724, in which tarballs were extracted into temporary
directories that were then renamed.

Changed the code back to a form taht extracts files directly into their
final place instead of staging via a temporary space, and added comments
warning maintainers about the specific issue being solved here, so that
hopefully the problem does not get re-introduced again in the future.

Fixes #3751
@mergify mergify bot closed this as completed in #3769 Sep 26, 2022
mergify bot pushed a commit that referenced this issue Sep 26, 2022
Windows does not allow renaming files that are currently open, or directories that contain open files. When antivirus software is used, files are open for analysis by the A/V software, making it impossible to rename files too quickly after they've been created.

This was already reported in #992 and addressed, however the issue was re-introduced in #3724, in which tarballs were extracted into temporary directories that were then renamed.

Changed the code back to a form taht extracts files directly into their final place instead of staging via a temporary space, and added comments warning maintainers about the specific issue being solved here, so that hopefully the problem does not get re-introduced again in the future.

Fixes #3751



---

By submitting this pull request, I confirm that my contribution is made under the terms of the [Apache 2.0 license].

[Apache 2.0 license]: https://www.apache.org/licenses/LICENSE-2.0
@github-actions
Copy link
Contributor

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

@RomainMuller
Copy link
Contributor

The fix is in and I will try to release a fixed version of the jsii runtimes tomorrow morning.

If you need un-blocked faster or want to early-test to make sure the issue is fixed, you can download this release-package.zip (version 1.68.1-dev.20220926.0) that includes pre-release binaries for the runtimes.

You can verify your download by using the following signature (my public keys are registered on GitHub and also keybase.io):

-----BEGIN PGP SIGNATURE-----

iHUEABYKAB0WIQQJYIEHVrtT2OLLCOi7PiEyTxi8sQUCYzHOUAAKCRC7PiEyTxi8
sVRVAP0Rfqi5bIxrBiT7OtZmgG7FEEDdxan8KTe/1hXWqV0vYQEA1bfRY3ENR9yA
ZWQHv7d3m1t4wnAF7HrCwBWb6qe3MQ0=
=cY1H
-----END PGP SIGNATURE-----

@andipabst
Copy link
Author

Thank you for the fix @RomainMuller, I just tested it successfully.
With the release 1.68.0 installed from pip, I could reproduce the issue. But using the patched version you linked above (1.68.1-dev.20220926.0), the error didn't occur any longer 🎉

@RomainMuller
Copy link
Contributor

Thank you for confirming! I'll kick off the release for 1.96.0 right now...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue is a bug. module/kernel Issues affecting the `jsii-kernel` module os/windows Related specifically to Windows behavior p1
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants