Rename operation throwing exception #992

FabioGentile · 2019-11-15T15:21:02Z

🐛 Bug Report

Affected Languages

TypeScript or Javascript
Python
Java
.NET (C#, F#, ...)

General Information

JSII Version:
Platform: Windows 10

What is the problem?

Invocation of cdk cli fails due to a permission error while renaming a file.
As reported here https://stackoverflow.com/a/50297753 the cause likely to be the antivirus that blocks the operation. In my scenario I couldn't disable the A/V due to domain policy, so instead I've added a retry mechanism in the doRename function (cdk_venv/python3.7/site-packages/jsii/_embedded/jsii/jsii-runtime.js:10408):
(please keep in mind that I have NO experience at all with JS, i'm sure that there are much better way to achieve this by using async/await for example, this is just to provide a minimal example)

function doRename (src, dest, overwrite) {
  let i = 0;
  
  for(i=0; i<10; i++){
    try {
      if (overwrite) {
        removeSync(dest)
        return rename(src, dest, overwrite)
      }
      if (fs.existsSync(dest)) throw new Error('dest already exists.')
      return rename(src, dest, overwrite)

    } 
    catch (err) {
      console.error(`Failed rename, try ${i}/10`);
      if(i==9) throw err

      var now = new Date().getTime();
      var millisecondsToWait = 250; /* i.e. 1 second */
      while ( new Date().getTime() < now + millisecondsToWait ) {}
    }
  }
}

Verbose Log

jsii.errors.JavaScriptError:
  Error: EPERM: operation not permitted, rename 'C:\Users\F87D8~1.GEN\AppData\Local\Temp\jsii-kernel-install-staging-7hkALG\package' -> 'C:\Users\F87D8~1.GEN\AppData\Local\Temp\jsii-kernel-EHW0R3\node_modules\@aws-cdk\core'
      at Object.renameSync (fs.js:643:3)
      at rename (C:\Users\f.gentile\Desktop\test\cdk-t\.win_env\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:10408:8)
      at doRename (C:\Users\f.gentile\Desktop\test\cdk-t\.win_env\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:10403:10)
      at Object.moveSync (C:\Users\f.gentile\Desktop\test\cdk-t\.win_env\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:10394:10)
      at Kernel.load (C:\Users\f.gentile\Desktop\test\cdk-t\.win_env\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:7479:16)
      at KernelHost.processRequest (C:\Users\f.gentile\Desktop\test\cdk-t\.win_env\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:7285:28)
      at KernelHost.run (C:\Users\f.gentile\Desktop\test\cdk-t\.win_env\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:7225:14)
      at Immediate._onImmediate (C:\Users\f.gentile\Desktop\test\cdk-t\.win_env\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:7228:37)
      at processImmediate (internal/timers.js:439:21)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "app.py", line 3, in <module>
    from aws_cdk import core
  File "C:\Users\f.gentile\Desktop\test\cdk-t\.win_env\lib\site-packages\aws_cdk\core\__init__.py", line 413, in <module>
    __jsii_assembly__ = jsii.JSIIAssembly.load("@aws-cdk/core", "1.16.3", __name__, "core@1.16.3.jsii.tgz")
  File "C:\Users\f.gentile\Desktop\test\cdk-t\.win_env\lib\site-packages\jsii\_runtime.py", line 40, in load
    _kernel.load(assembly.name, assembly.version, os.fspath(assembly_path))
  File "C:\Users\f.gentile\Desktop\test\cdk-t\.win_env\lib\site-packages\jsii\_kernel\__init__.py", line 209, in load
    self.provider.load(LoadRequest(name=name, version=version, tarball=tarball))
  File "C:\Users\f.gentile\Desktop\test\cdk-t\.win_env\lib\site-packages\jsii\_kernel\providers\process.py", line 330, in load
    return self._process.send(request, LoadResponse)
  File "C:\Users\f.gentile\Desktop\test\cdk-t\.win_env\lib\site-packages\jsii\_kernel\providers\process.py", line 318, in send
    raise JSIIError(resp.error) from JavaScriptError(resp.stack)
jsii.errors.JSIIError: EPERM: operation not permitted, rename 'C:\Users\F87D8~1.GEN\AppData\Local\Temp\jsii-kernel-install-staging-7hkALG\package' -> 'C:\Users\F87D8~1.GEN\AppData\Local\Temp\jsii-kernel-EHW0R3\node_modules\@aws-cdk\core'
Subprocess exited with error 1
Error: Subprocess exited with error 1
    at ChildProcess.<anonymous> (C:\Users\f.gentile\AppData\Roaming\npm\node_modules\aws-cdk\lib\api\cxapp\exec.ts:115:23)
    at ChildProcess.emit (events.js:210:5)
    at ChildProcess.EventEmitter.emit (domain.js:476:20)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:272:12)

Full log https://gist.github.com/FabioGentile/29350501e3890305a94feff4b54ff1f8

Could it make sense to introduce something similar in the upstream version?

The text was updated successfully, but these errors were encountered:

joyofdata · 2020-02-13T15:55:43Z

CDK 1.23.0 on Windows 10 Pro

Have been trying already about 20 times to run cdk list. The following is only a few of the error messages.

C:\repos\data-lake-infrastructure-as-code\xy_ds_datalake\cdk (xy_ds_datalake/feature/SCANA-344-distance-service-for-oi-p2p-etl -> origin)
(.env-cdk) λ cdk list
jsii.errors.JavaScriptError:
  Error: EPERM: operation not permitted, rename 'C:\Users\RAFFAE~1.VOG\AppData\Local\Temp\jsii-kernel-install-staging-QwC2Kp\package' -> 'C:\Users\RAFFAE~1.VOG\AppData\Local\Temp\jsii-kernel-a9dMAI\node_modules\@aws-cdk\cx-api'
      at Object.renameSync (fs.js:643:3)
      at rename (C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:10436:8)
      at doRename (C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:10431:10)
      at Object.moveSync (C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:10422:10)
      at Kernel.load (C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:7493:16)
      at KernelHost.processRequest (C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:7296:28)
      at KernelHost.run (C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:7236:14)
      at Object.<anonymous> (C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:7204:6)
      at __webpack_require__ (C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:20:30)
      at C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:84:18

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "app.py", line 5, in <module>
    from aws_cdk import core
  File "C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\aws_cdk\core\__init__.py", line 411, in <module>
    import aws_cdk.cx_api
  File "C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\aws_cdk\cx_api\__init__.py", line 31, in <module>
    __jsii_assembly__ = jsii.JSIIAssembly.load("@aws-cdk/cx-api", "1.23.0", __name__, "cx-api@1.23.0.jsii.tgz")
  File "C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\jsii\_runtime.py", line 40, in load
    _kernel.load(assembly.name, assembly.version, os.fspath(assembly_path))
  File "C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\jsii\_kernel\__init__.py", line 215, in load
    self.provider.load(LoadRequest(name=name, version=version, tarball=tarball))
  File "C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\jsii\_kernel\providers\process.py", line 330, in load
    return self._process.send(request, LoadResponse)
  File "C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\jsii\_kernel\providers\process.py", line 318, in send
    raise JSIIError(resp.error) from JavaScriptError(resp.stack)
jsii.errors.JSIIError: EPERM: operation not permitted, rename 'C:\Users\RAFFAE~1.VOG\AppData\Local\Temp\jsii-kernel-install-staging-QwC2Kp\package' -> 'C:\Users\RAFFAE~1.VOG\AppData\Local\Temp\jsii-kernel-a9dMAI\node_modules\@aws-cdk\cx-api'
Subprocess exited with error 1

C:\repos\data-lake-infrastructure-as-code\xy_ds_datalake\cdk (xy_ds_datalake/feature/SCANA-344-distance-service-for-oi-p2p-etl -> origin)
(.env-cdk) λ cdk list
jsii.errors.JavaScriptError:
  Error: EPERM: operation not permitted, rename 'C:\Users\RAFFAE~1.VOG\AppData\Local\Temp\jsii-kernel-install-staging-DILQOQ\package' -> 'C:\Users\RAFFAE~1.VOG\AppData\Local\Temp\jsii-kernel-uvMm1v\node_modules\@aws-cdk\aws-apigateway'
      at Object.renameSync (fs.js:643:3)
      at rename (C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:10436:8)
      at doRename (C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:10431:10)
      at Object.moveSync (C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:10422:10)
      at Kernel.load (C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:7493:16)
      at KernelHost.processRequest (C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:7296:28)
      at KernelHost.run (C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:7236:14)
      at Immediate._onImmediate (C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:7239:37)
      at processImmediate (internal/timers.js:439:21)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "app.py", line 9, in <module>
    from xy_ds_datalake.stacks import BaseStack, DataLakeStack
  File "c:\repos\data-lake-infrastructure-as-code\xy_ds_datalake\cdk\src\xy_ds_datalake\stacks\__init__.py", line 1, in <module>
    from .base import BaseStack
  File "c:\repos\data-lake-infrastructure-as-code\xy_ds_datalake\cdk\src\xy_ds_datalake\stacks\base\__init__.py", line 1, in <module>
    from .stack import BaseStack
  File "c:\repos\data-lake-infrastructure-as-code\xy_ds_datalake\cdk\src\xy_ds_datalake\stacks\base\stack.py", line 8, in <module>
    from xy_aws_shared.etl_model.fargate_helper import (
  File "c:\repos\data-lake-infrastructure-as-code\xy_aws_shared\src\xy_aws_shared\etl_model\fargate_helper.py", line 1, in <module>
    from aws_cdk import (aws_ec2 as ec2, aws_ecs as ecs, aws_iam as iam, core)
  File "C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\aws_cdk\aws_ecs\__init__.py", line 637, in <module>
    import aws_cdk.aws_route53_targets
  File "C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\aws_cdk\aws_route53_targets\__init__.py", line 98, in <module>
    import aws_cdk.aws_apigateway
  File "C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\aws_cdk\aws_apigateway\__init__.py", line 626, in <module>
    __jsii_assembly__ = jsii.JSIIAssembly.load("@aws-cdk/aws-apigateway", "1.23.0", __name__, "aws-apigateway@1.23.0.jsii.tgz")
  File "C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\jsii\_runtime.py", line 40, in load
    _kernel.load(assembly.name, assembly.version, os.fspath(assembly_path))
  File "C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\jsii\_kernel\__init__.py", line 215, in load
    self.provider.load(LoadRequest(name=name, version=version, tarball=tarball))
  File "C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\jsii\_kernel\providers\process.py", line 330, in load
    return self._process.send(request, LoadResponse)
  File "C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\jsii\_kernel\providers\process.py", line 318, in send
    raise JSIIError(resp.error) from JavaScriptError(resp.stack)
jsii.errors.JSIIError: EPERM: operation not permitted, rename 'C:\Users\RAFFAE~1.VOG\AppData\Local\Temp\jsii-kernel-install-staging-DILQOQ\package' -> 'C:\Users\RAFFAE~1.VOG\AppData\Local\Temp\jsii-kernel-uvMm1v\node_modules\@aws-cdk\aws-apigateway'
Subprocess exited with error 1

C:\repos\data-lake-infrastructure-as-code\xy_ds_datalake\cdk (xy_ds_datalake/feature/SCANA-344-distance-service-for-oi-p2p-etl -> origin)
(.env-cdk) λ cdk list
XY Lambda Layer requirements up-to-date.
Traceback (most recent call last):
  File "app.py", line 34, in <module>
    datalake_dev = DataLakeStack(app, f"data-lake-{env_name}", env=cdk_env, tags=tags)
  File "C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\jsii\_runtime.py", line 66, in __call__
    inst = super().__call__(*args, **kwargs)
  File "c:\repos\data-lake-infrastructure-as-code\xy_ds_datalake\cdk\src\xy_ds_datalake\stacks\data_lake\data_lake.py", line 153, in __init__
    self._aws_lambda_layer = xy_lambda_layer._create_awsdatawrangler_layer(self)
  File "c:\repos\data-lake-infrastructure-as-code\xy_lambda_layer\src\xy_lambda_layer\__init__.py", line 52, in _create_awsdatawrangler_layer
    layer_file = _aws_data_wrangler._build_layer()
  File "c:\repos\data-lake-infrastructure-as-code\xy_lambda_layer\src\xy_lambda_layer\_aws_data_wrangler.py", line 68, in _build_layer
    latest_release_url = _get_latest_release_download_url()
  File "c:\repos\data-lake-infrastructure-as-code\xy_lambda_layer\src\xy_lambda_layer\_aws_data_wrangler.py", line 29, in _get_latest_release_download_url
    with urllib.request.urlopen(LATEST_RELEASE_URL) as response:
  File "C:\Users\raffael.vogler\AppData\Local\Programs\Python\Python36\lib\urllib\request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Users\raffael.vogler\AppData\Local\Programs\Python\Python36\lib\urllib\request.py", line 532, in open
    response = meth(req, response)
  File "C:\Users\raffael.vogler\AppData\Local\Programs\Python\Python36\lib\urllib\request.py", line 642, in http_response
    'http', request, response, code, msg, hdrs)
  File "C:\Users\raffael.vogler\AppData\Local\Programs\Python\Python36\lib\urllib\request.py", line 570, in error
    return self._call_chain(*args)
  File "C:\Users\raffael.vogler\AppData\Local\Programs\Python\Python36\lib\urllib\request.py", line 504, in _call_chain
    result = func(*args)
  File "C:\Users\raffael.vogler\AppData\Local\Programs\Python\Python36\lib\urllib\request.py", line 650, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
Subprocess exited with error 1

C:\repos\data-lake-infrastructure-as-code\xy_ds_datalake\cdk (xy_ds_datalake/feature/SCANA-344-distance-service-for-oi-p2p-etl -> origin)
(.env-cdk) λ cdk list
jsii.errors.JavaScriptError:
  Error: EPERM: operation not permitted, rename 'C:\Users\RAFFAE~1.VOG\AppData\Local\Temp\jsii-kernel-install-staging-oOT22i\package' -> 'C:\Users\RAFFAE~1.VOG\AppData\Local\Temp\jsii-kernel-AwWuNw\node_modules\@aws-cdk\aws-lambda'
      at Object.renameSync (fs.js:643:3)
      at rename (C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:10436:8)
      at doRename (C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:10431:10)
      at Object.moveSync (C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:10422:10)
      at Kernel.load (C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:7493:16)
      at KernelHost.processRequest (C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:7296:28)
      at KernelHost.run (C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:7236:14)
      at Immediate._onImmediate (C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\jsii\_embedded\jsii\jsii-runtime.js:7239:37)
      at processImmediate (internal/timers.js:439:21)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "app.py", line 9, in <module>
    from xy_ds_datalake.stacks import BaseStack, DataLakeStack
  File "c:\repos\data-lake-infrastructure-as-code\xy_ds_datalake\cdk\src\xy_ds_datalake\stacks\__init__.py", line 1, in <module>
    from .base import BaseStack
  File "c:\repos\data-lake-infrastructure-as-code\xy_ds_datalake\cdk\src\xy_ds_datalake\stacks\base\__init__.py", line 1, in <module>
    from .stack import BaseStack
  File "c:\repos\data-lake-infrastructure-as-code\xy_ds_datalake\cdk\src\xy_ds_datalake\stacks\base\stack.py", line 8, in <module>
    from xy_aws_shared.etl_model.fargate_helper import (
  File "c:\repos\data-lake-infrastructure-as-code\xy_aws_shared\src\xy_aws_shared\etl_model\fargate_helper.py", line 1, in <module>
    from aws_cdk import (aws_ec2 as ec2, aws_ecs as ecs, aws_iam as iam, core)
  File "C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\aws_cdk\aws_ecs\__init__.py", line 623, in <module>
    import aws_cdk.aws_autoscaling
  File "C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\aws_cdk\aws_autoscaling\__init__.py", line 264, in <module>
    import aws_cdk.aws_elasticloadbalancingv2
  File "C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\aws_cdk\aws_elasticloadbalancingv2\__init__.py", line 283, in <module>
    import aws_cdk.aws_certificatemanager
  File "C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\aws_cdk\aws_certificatemanager\__init__.py", line 103, in <module>
    import aws_cdk.aws_cloudformation
  File "C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\aws_cdk\aws_cloudformation\__init__.py", line 125, in <module>
    import aws_cdk.aws_lambda
  File "C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\aws_cdk\aws_lambda\__init__.py", line 262, in <module>
    __jsii_assembly__ = jsii.JSIIAssembly.load("@aws-cdk/aws-lambda", "1.23.0", __name__, "aws-lambda@1.23.0.jsii.tgz")
  File "C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\jsii\_runtime.py", line 40, in load
    _kernel.load(assembly.name, assembly.version, os.fspath(assembly_path))
  File "C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\jsii\_kernel\__init__.py", line 215, in load
    self.provider.load(LoadRequest(name=name, version=version, tarball=tarball))
  File "C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\jsii\_kernel\providers\process.py", line 330, in load
    return self._process.send(request, LoadResponse)
  File "C:\repos\data-lake-infrastructure-as-code\.env-cdk\lib\site-packages\jsii\_kernel\providers\process.py", line 318, in send
    raise JSIIError(resp.error) from JavaScriptError(resp.stack)
jsii.errors.JSIIError: EPERM: operation not permitted, rename 'C:\Users\RAFFAE~1.VOG\AppData\Local\Temp\jsii-kernel-install-staging-oOT22i\package' -> 'C:\Users\RAFFAE~1.VOG\AppData\Local\Temp\jsii-kernel-AwWuNw\node_modules\@aws-cdk\aws-lambda'
Subprocess exited with error 1

RomainMuller · 2020-02-25T15:20:22Z

Hello,

Assuming hopefully the temporary files aren't cleaned up, are you able to perform the rename in question manually?

RomainMuller · 2020-02-25T15:23:45Z

Additionally - have you tried to contact your antivirus vendor and ask about the symptoms you observe? If they are indeed causing this, they may be able to provide better indications as to how this should be fixed "properly".

joyofdata · 2020-02-25T19:51:45Z

Hello @RomainMuller ,
I can rename and delete manually. The work station is maintained by the IT department of the company I'm working at. I contacted them requesting an exclusion of AppData\Local\Temp\ from the anti virus software. But I'm not optimistic that this will be possible.

RomainMuller · 2020-02-26T18:49:36Z

But does the Antivirus vendor have documentation for how software should be made to inter-operate best with their process? EPERM is generally speaking not a returnable error....

monnecc · 2020-02-27T17:06:25Z

Same thing here with .NET

karkohli · 2020-03-02T20:40:03Z

@RomainMuller
One of my customer is currently using .NET. They having similar error [Permission errors while trying to rename (temporary) npm packages].

They using Windows Defender that scans that file [see attached image] and looks like the file is closed before CDK is attempting the rename action. Manual rename operation also is not going through since CDK cleans up on failure.

Environment info:

Node 12.13.1
NPM 6.14.1
aws-cdk NPM package 1.26.0
AWS NuGet-packages: 1.25.0

Image:

MrArnoldPalmer · 2020-03-03T03:00:35Z

@karkohli Looks like some race condition may be occuring. We are looking into this.

monnecc · 2020-03-12T13:50:22Z

Hi, is there any news around this issue?

MrArnoldPalmer · 2020-03-12T21:21:19Z

@monnecc we haven't been able to reproduce this yet. We will update here when we get some time to dig in further.

fknittel · 2020-04-08T07:11:57Z

Configuring the anti virus software (Windows Defender in our case), does not appear to make a difference. We excluded the temporary directory in which the rename operation takes place ("Add-MpPreference -ExclusionPath ..."), but the "EPERM: operation not permitted, rename ..." error persists.

So either there's an additional scanner running on our Windows boxes (this is a corporate environment not fully under our control) or the AV angle is a red herring.

(This is CDK with Java and most team members gave up on running the command on Windows due to the low chances of a successful run.)

hansterwal · 2020-04-09T06:58:49Z

Hi

I'm also running into this error on average it takes me three times to be able to deploy my stack via cdk deploy. I'm running on Windows 10 Enterprise 1909.

MrArnoldPalmer · 2020-04-09T16:59:15Z

This is a hard issue for us to repro since its environment/configuration specific. Any more investigation any of you can do would be super helpful to getting this solved. I'd consider getting IT involved if they own the configuration of user permissions and AV policy.

argenstijn · 2020-05-18T11:35:00Z

Is it easy possible to change the tmp/cache directory? that could solve the problem as well.

RomainMuller · 2020-05-18T13:14:00Z

@argenstijn supposedly setting the TMPDIR environment variable effects where the temporary directories get placed. Although on Windows it could be TEMP or TMP instead (it's not too clear to me).

fknittel · 2020-05-18T13:34:43Z

@argenstijn Setting the environment variable TEMP caused the temp directory used for the jsii kernel to change. Proof: After the change, the EPERM error message included the new path. 😞 But maybe you'll have more luck.

The only approach that worked for our team is to patch the code. Specifically, we globally install the jsii runtime (npm i -g @jsii/runtime), set the JSII_RUNTIME environemt variable to <NODE_PATH>\node_modules\@jsii\runtime\webpack\jsii-runtime.js and then patch that jsii-runtime.js-file according to the instructions in this ticket's initial post.

devang-s · 2020-05-21T13:40:50Z

I got the same error today when running cdk ls. It is my personal Win 10 laptop.
I just right-clicked the Temp folder, and unchecked the Read-Only attribute and re-ran. And it worked fine. But hopefully, a better solution should be there.

RomainMuller · 2020-05-25T14:04:58Z

After investigating this and trying out a couple of different workarounds (all of which felt gross, but one's gotta do what one's gotta do), I found out there is a way to change how this particular jsii kernel API works so that it no longer involves a rename operation. This would hopefully fix this issue in a nice and clean way!

When loading a new library into the jsii kernel, the provided `tarball` was extracted to a temporary directory, then moved into it's final location. On Windows, this operation could fail on an `EACCESS` or `EPERM` error (often due to malware scanners accessing the file for inspection on file systems which do not support renaming files that are being accessed). This changes how the `load` API works so taht the `tarball` is extracted directly in it's final install location, so that no rename operation is needed. Fixes #992 --- By submitting this pull request, I confirm that my contribution is made under the terms of the [Apache 2.0 license]. [Apache 2.0 license]: https://www.apache.org/licenses/LICENSE-2.0

OlegBoulanov · 2020-06-03T14:29:19Z

Is there any resolution on this?
Win10, dotnet, McAfee maintained by IT - how to break through the rename EPERM error wall?
CDK --version
1.42.1 (build a4797b4)

RomainMuller · 2020-06-05T14:37:20Z

@OlegBoulanov - I reckon the problem should go away with CDK 1.43.0 or newer.

OlegBoulanov · 2020-06-05T20:53:47Z

@RomainMuller - thank you!

OlegBoulanov · 2020-06-08T12:49:42Z

Doesn't look fixed:

PS C:\olegbs\olegb\source\repos\company\project> cdk --version
1.44.0 (build 1cd832b)

PS C:\olegbs\olegb\source\repos\company\project> cdk ls
Unhandled exception. Amazon.JSII.Runtime.JsiiException: EPERM: operation not permitted, rename 'C:\olegbs\olegb\AppData\Local\Temp\jsii-kernel-install-staging-QoQm30\package' -> 'C:\olegbs\olegb\AppData\Local\Temp\jsii-kernel-Ni5uS1\node_modules@aws-cdk\core'
at Amazon.JSII.Runtime.Services.Client.TryDeserialize[TResponse](String responseJson)
at Amazon.JSII.Runtime.Services.Client.ReceiveResponseTResponse
at Amazon.JSII.Runtime.Services.Client.Send[TRequest,TResponse](TRequest requestObject)

RomainMuller · 2020-06-08T14:03:51Z

@OlegBoulanov - are you positive you're using version 1.44.0 of the libraries, too?

OlegBoulanov · 2020-06-08T19:05:18Z

@RomainMuller - Yes, absolutely. These are two sequential commands

OlegBoulanov · 2020-06-08T19:05:48Z

Maybe I need to upgrade something else, too?

OlegBoulanov · 2020-06-08T19:34:47Z

Oh, sorry. There are no libraries but CDK itself

RomainMuller · 2020-06-09T08:06:05Z

So you are likely using 2 different CDK components: the CLI (which your message mentions you have at 1.44.0) and the libraries (since you're doing .NET, it'd be the Amazon.CDK, Amazon.CDK.AWS.* dependencies of your project).

OlegBoulanov · 2020-06-09T20:57:08Z

@RomainMuller - sorry it took so long to confirm, and yes, referencing 1.45.0 packages fixes the problem

Thank you!

argenstijn · 2020-06-15T16:43:59Z

It works Sent from Outlook Mobile<https://aka.ms/blhgte>

…

________________________________ From: OlegBoulanov <notifications@github.com> Sent: Tuesday, June 9, 2020 10:57:22 PM To: aws/jsii <jsii@noreply.github.com> Cc: argenstijn <argenstijn@hotmail.com>; Mention <mention@noreply.github.com> Subject: Re: [aws/jsii] Rename operation throwing exception (#992) @RomainMuller<https://github.com/RomainMuller> - sorry it took so long to confirm, and yes, referencing 1.45.0 packages fixes the problem Thank you! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#992 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAA75LGW54A3PB3LZNPBQJTRV2O3FANCNFSM4JN4HQIA>.

laimonassutkus · 2020-10-14T11:09:01Z

I seem to have the latest version of jsii and i get this "not permitted" error.

Naxod-prog · 2022-09-09T11:44:40Z

im on cdk version 2.27.0 (build 8e89048) but im still facing the same issues,

Caused by: software.amazon.jsii.JsiiException: EPERM: operation not permitted, rename 'C:\Users\a2414\AppData\Local\Temp\1\jsii-runtime-untar-dLGyec' -> 'C:\Users\a2414\AppData\Local\Temp\1\jsii-kernel-QuOmP0\node_modules\aws-cdk-lib'
Error: EPERM: operation not permitted, rename 'C:\Users\a241554\AppData\Local\Temp\1\jsii-runtime-untar-dLGyec' -> 'C:\Users\a241554\AppData\Local\Temp\1\jsii-kernel-QuOmP0\node_modules\aws-cdk-lib'
at renameSync (fs.js:794:3)
at exports.Kernel._load (C:\Users\a241554\AppData\Local\Temp\1\jsii-java-runtime17696518093862229919\lib\program.js:5356:37)
at C:\Users

Windows does not allow renaming files that are currently open, or directories that contain open files. When antivirus software is used, files are open for analysis by the A/V software, making it impossible to rename files too quickly after they've been created. This was already reported in #992 and addressed, however the issue was re-introduced in #3724, in which tarballs were extracted into temporary directories that were then renamed. Changed the code back to a form taht extracts files directly into their final place instead of staging via a temporary space, and added comments warning maintainers about the specific issue being solved here, so that hopefully the problem does not get re-introduced again in the future. Fixes #3751

Windows does not allow renaming files that are currently open, or directories that contain open files. When antivirus software is used, files are open for analysis by the A/V software, making it impossible to rename files too quickly after they've been created. This was already reported in #992 and addressed, however the issue was re-introduced in #3724, in which tarballs were extracted into temporary directories that were then renamed. Changed the code back to a form taht extracts files directly into their final place instead of staging via a temporary space, and added comments warning maintainers about the specific issue being solved here, so that hopefully the problem does not get re-introduced again in the future. Fixes #3751 --- By submitting this pull request, I confirm that my contribution is made under the terms of the [Apache 2.0 license]. [Apache 2.0 license]: https://www.apache.org/licenses/LICENSE-2.0

FabioGentile added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Nov 15, 2019

FabioGentile mentioned this issue Nov 15, 2019

Permission problem on windows aws/aws-cdk#5050

Closed

SomayaB assigned RomainMuller Nov 18, 2019

RomainMuller added the p2 label Nov 21, 2019

SomayaB added the language/python Related to Python bindings label Jan 13, 2020

MrArnoldPalmer mentioned this issue Feb 4, 2020

Amazon.JSII.Runtime.JsiiException: EPERM aws/aws-cdk#6105

Closed

SomayaB removed the needs-triage This issue or PR still needs to be triaged. label Mar 4, 2020

RomainMuller mentioned this issue May 25, 2020

fix(kernel): error raised during rename operation on win32 #1702

Merged

mergify bot closed this as completed in #1702 May 26, 2020

andipabst mentioned this issue Sep 13, 2022

Windows: Rename operation throwing exception #3751

Closed

RomainMuller mentioned this issue Sep 26, 2022

fix(kernel): EPERM on rename operation on Windows #3769

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rename operation throwing exception #992

Rename operation throwing exception #992

FabioGentile commented Nov 15, 2019 •

edited

Loading

joyofdata commented Feb 13, 2020

RomainMuller commented Feb 25, 2020

RomainMuller commented Feb 25, 2020

joyofdata commented Feb 25, 2020

RomainMuller commented Feb 26, 2020

monnecc commented Feb 27, 2020

karkohli commented Mar 2, 2020

MrArnoldPalmer commented Mar 3, 2020

monnecc commented Mar 12, 2020

MrArnoldPalmer commented Mar 12, 2020

fknittel commented Apr 8, 2020

hansterwal commented Apr 9, 2020

MrArnoldPalmer commented Apr 9, 2020

argenstijn commented May 18, 2020

RomainMuller commented May 18, 2020

fknittel commented May 18, 2020

devang-s commented May 21, 2020

RomainMuller commented May 25, 2020

OlegBoulanov commented Jun 3, 2020 •

edited

Loading

RomainMuller commented Jun 5, 2020

OlegBoulanov commented Jun 5, 2020

OlegBoulanov commented Jun 8, 2020

RomainMuller commented Jun 8, 2020

OlegBoulanov commented Jun 8, 2020

OlegBoulanov commented Jun 8, 2020

OlegBoulanov commented Jun 8, 2020

RomainMuller commented Jun 9, 2020

OlegBoulanov commented Jun 9, 2020

argenstijn commented Jun 15, 2020 via email

laimonassutkus commented Oct 14, 2020

Naxod-prog commented Sep 9, 2022

Rename operation throwing exception #992

Rename operation throwing exception #992

Comments

FabioGentile commented Nov 15, 2019 • edited Loading

🐛 Bug Report

Affected Languages

General Information

What is the problem?

Verbose Log

joyofdata commented Feb 13, 2020

RomainMuller commented Feb 25, 2020

RomainMuller commented Feb 25, 2020

joyofdata commented Feb 25, 2020

RomainMuller commented Feb 26, 2020

monnecc commented Feb 27, 2020

karkohli commented Mar 2, 2020

MrArnoldPalmer commented Mar 3, 2020

monnecc commented Mar 12, 2020

MrArnoldPalmer commented Mar 12, 2020

fknittel commented Apr 8, 2020

hansterwal commented Apr 9, 2020

MrArnoldPalmer commented Apr 9, 2020

argenstijn commented May 18, 2020

RomainMuller commented May 18, 2020

fknittel commented May 18, 2020

devang-s commented May 21, 2020

RomainMuller commented May 25, 2020

OlegBoulanov commented Jun 3, 2020 • edited Loading

RomainMuller commented Jun 5, 2020

OlegBoulanov commented Jun 5, 2020

OlegBoulanov commented Jun 8, 2020

RomainMuller commented Jun 8, 2020

OlegBoulanov commented Jun 8, 2020

OlegBoulanov commented Jun 8, 2020

OlegBoulanov commented Jun 8, 2020

RomainMuller commented Jun 9, 2020

OlegBoulanov commented Jun 9, 2020

argenstijn commented Jun 15, 2020 via email

laimonassutkus commented Oct 14, 2020

Naxod-prog commented Sep 9, 2022

FabioGentile commented Nov 15, 2019 •

edited

Loading

OlegBoulanov commented Jun 3, 2020 •

edited

Loading