CKAN complains about temp files when upgrading a mod and installing another one at the same time

[judemille]

When I download a large amount of mods, CKAN barfs, complaining that it can't access this ONE temp file.

Unhandled exception:
System.IO.IOException: The process cannot access the file 'C:\Users\FireFaced\AppData\Local\Temp\tmp8C0B.tmp' because it is being used by another process.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.File.InternalDelete(String path, Boolean checkHost)
at CKAN.NetAsyncDownloader.ModuleDownloadsComplete(Ne tFileCache cache, Uri[] urls, String[] filenames, Exception[] errors)
at CKAN.NetAsyncDownloader.<DownloadModules>c__AnonSt orey2.<>m__1(Uri[] _uris, String[] paths, Exception[] errors)
at System.Threading.ExecutionContext.RunInternal(Exec utionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionCon text executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.QueueUserWorkItemCallback.System. Threading.IThreadPoolWorkItem.ExecuteWorkItem()
at System.Threading.ThreadPoolWorkQueue.Dispatch()

[Postremus]

May be related to #801

[judemille]

I don't think so. @Postremus

[judemille]

Okay, I cleared my temp file, issue doesn't seem to come up. Closing this, and it seems that it is related to #801

[judemille]

All right, I'm sick of this. Now it's holding onto a different temp file EVEN THOUGH my temp folder is merely 1MB

[judemille]

facepalm Remind me NOT to click the close button

[judemille]

Ugh. @pjf can you please reopen this? I didn't mean to click "comment and close"

[judemille]

Thank you, I didn't mean to close it after I reopened it.

[squizzeak]

I am able to reliably reproduce temp file issues by trying to both upgrade a mod and install a new one at the same time. Not sure if it's related.

[judemille]

Yeah, usually when I install mods, I also just run through the upgrades. Strange

[judemille]

Anyone?

[Postremus]

Can you Point the issue down to a specific mod?

[judemille]

No, it's pretty broad. It usually comes up when I install a lot of mods AND upgrade a bunch of mods at the same time.

[squizzeak]

I've had it occur with as little as a single new mod install and a single mod update in the same change list.

[judemille]

Huh. Yeah, the issue is quite broad.

[Postremus]

@firefaced @squizzeak I am unable to reproduce this bug. Would you mind testing a dev version which may fix this issue?

[judemille]

Sure

[Postremus]

https://dl.dropboxusercontent.com/u/46071056/CKAN/ckan.exe
Please paste the new exception.

[judemille]

@plague006 I don't think that's relevant, TBH. It seems to not matter which mod is installed/upgraded, just that you upgrade and install mods at the same time.

[mheguy]

You're correct that it's not about which mods. I just provide information if it's available.

[judemille]

Okay, that's reasonable, since people ask for ALL THE DETAILS

[judemille]

Anybody? I REALLY just want to have CKAN manage mods properly.

[Postremus]

@firefaced @plague006 Does the .zip of the mod already exists in your CKAN/dowloads Folder before you install / update the mod? Does it still exists after the install / update ?

[judemille]

@Postremus, it wouldn't matter, the ZIP file has a different name in each version. Also, the mods I've been installing in the tests were never in my cache.

[mheguy]

In my case the files weren't cached @Postremus, and the .zips are present after the transaction.

[Postremus]

Ok, ckan is srupid.

----- Ursprüngliche Nachricht -----
Von: "Matthew" notifications@github.com
Gesendet: ‎19.‎08.‎2015 14:46
An: "KSP-CKAN/CKAN" CKAN@noreply.github.com
Cc: "Martin Panzer" postremus1996@googlemail.com
Betreff: Re: [CKAN] CKAN complains about temp files when upgrading a mod andinstalling another one at the same time (#1367)

In my case the files weren't cached @Postremus, and the .zips are present after the transaction.
—
Reply to this email directly or view it on GitHub.

[judemille]

@Postremus
Very srupid.

[Postremus]

That was my mobil phone, not me. :)

----- Ursprüngliche Nachricht -----
Von: "FireFaced, AKA TheCrazyCarts" notifications@github.com
Gesendet: ‎19.‎08.‎2015 16:45
An: "KSP-CKAN/CKAN" CKAN@noreply.github.com
Cc: "Martin Panzer" postremus1996@googlemail.com
Betreff: Re: [CKAN] CKAN complains about temp files when upgrading a mod andinstalling another one at the same time (#1367)

@Postremus
Very srupid.
—
Reply to this email directly or view it on GitHub.

[judemille]

I know @Postremus

[Postremus]

@firefaced another dev version:
https://dl.dropboxusercontent.com/u/46071056/CKAN/ckan.exe

[squizzeak]

Tried another install + upgrade with the dev version you just posted, @Postremus. Results are below.

Install: Water Sounds
Upgrade: A visor for Valentina

---

Downloading "https://kerbalstuff.com/mod/1094/A visor for Valentina/download/1.01"
About to install...

• WaterSounds 1.1

Downloading "https://kerbalstuff.com/mod/1094/A visor for Valentina/download/1.01"
Downloading "https://kerbalstuff.com/mod/823/Water Sounds/download/1.1"

*************\* Exception Text **************
CKAN.FileNotFoundKraken: Trying to install WaterSounds 1.1, but it's not downloaded or download is corrupted
   at CKAN.ModuleInstaller.Install(CkanModule module, String filename)
   at CKAN.ModuleInstaller.InstallList(ICollection`1 modules, RelationshipResolverOptions options, IDownloader downloader)
   at CKAN.ModuleInstaller.InstallList(List`1 modules, RelationshipResolverOptions options, IDownloader downloader)
   at CKAN.Main.<>c__DisplayClass230_0.<InstallList>b__0()
   at CKAN.Main.WasSuccessful(Action action)
   at CKAN.Main.InstallList(HashSet`1 toInstall, RelationshipResolverOptions options, IDownloader downloader)
   at CKAN.Main.InstallMods(Object sender, DoWorkEventArgs e)
   at System.ComponentModel.BackgroundWorker.WorkerThreadStart(Object argument)

[judemille]

Unhandled exception:
System.IO.IOException: The process cannot access the file 'C:\Users\FireFaced\AppData\Local\Temp\tmp6B83.tmp' because it is being used by another process.
   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.File.InternalDelete(String path, Boolean checkHost)
   at CKAN.NetAsyncDownloader.ModuleDownloadsComplete(NetFileCache cache, Uri[] urls, String[] filenames, Exception[] errors)
   at CKAN.NetAsyncDownloader.<DownloadModules>c__AnonStorey2.<>m__1(Uri[] _uris, String[] paths, Exception[] errors)
   at CKAN.NetAsyncDownloader.FileDownloadComplete(Int32 index, Exception error)
   at System.ComponentModel.AsyncCompletedEventHandler.Invoke(Object sender, AsyncCompletedEventArgs e)
   at System.Net.WebClient.OnDownloadFileCompleted(AsyncCompletedEventArgs e)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
   at System.Threading.ThreadPoolWorkQueue.Dispatch() 

Still getting this D:

[Kemp-J]

I'm also seeing this issue. Has anyone confirmed that it doesn't happen when only installing or only upgrading? I only see people confirming that is does happen when doing both.

My initial guess was that AV software is locking the file based on the fact that it's a file that is created and then very soon after deleted. I've seen similar issues before. AV software is supposed to be nice about it, but many aren't.

[judemille]

@Kemp-J The point of a temp file is to be used for a bit, then deleted. But AV could be doing that. Also, for others: Workaround for now is just not to install AND upgrade mods in one changeset.

[MicroDoser]

"Workaround for now is just not to install AND upgrade mods in one changeset."

Could that not be the internal solution CKAN uses?

What is to stop an update being done first then install mods once that has completely finished?

If the issue is installing and upgrading at the same time then just don't do that by coding the 'apply changes' section of code to run twice, one for upgrade, one for install.

Then it does not matter what causes the bug as the trigger for the cause is avoided.

[Sakata-MC]

Still occurs. If this is going to be thrown under the bus as "won't fix", then make CKAN not allow the user to both select a mod to install while upgrades are selected. That should be doable with a few appropriately placed conditionals.

At least until the issue is found and resolved -- noticed that this is being looked at despite the searched ones being listed 'closed'.

Unhandled exception:
System.IO.IOException: The process cannot access the file 'C:\Users\xxxx\AppData\Local\Temp\tmpD6DE.tmp' because it is being used by another process.
   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.File.InternalDelete(String path, Boolean checkHost)
   at CKAN.NetAsyncDownloader.ModuleDownloadsComplete(NetFileCache cache, Uri[] urls, String[] filenames, Exception[] errors)
   at CKAN.NetAsyncDownloader.<DownloadModules>c__AnonStorey2.<>m__1(Uri[] _uris, String[] paths, Exception[] errors)
   at CKAN.NetAsyncDownloader.FileDownloadComplete(Int32 index, Exception error)
   at System.ComponentModel.AsyncCompletedEventHandler.Invoke(Object sender, AsyncCompletedEventArgs e)
   at System.Net.WebClient.OnDownloadFileCompleted(AsyncCompletedEventArgs e)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
   at System.Threading.ThreadPoolWorkQueue.Dispatch() 

[politas]

What makes you think this has been given a "won't fix" label? Look at the labels on this issue - Bug, CKAN.DLL, and urgent. There's a workaround, and we are giving out that advice until its gets fixed, but this is definitely going to be fixed.

It's quite common in bug tracking systems to close off duplicates of accepted bug reports, retaining the one that has the most useful information.

[HebaruSan]

• This call stack was changed drastically in Cache downloads individually upon completion #2756; instead of ModuleDownloadsComplete storing files to the cache in a batch at the end, they're stored individually as they finish
• Disable tx timeouts, add tx debug logging, static DLL pattern, fix docs #3512 disabled the arbitrary 10-minute transaction timeout that may have been involved (exception thrown by File.InternalDelete related to "because it is being used by another process")
• The download path is not a temp file anymore as of Resume failed downloads #3666
• Another user reported the same issue for installing without upgrading or vice versa (see Crashes while downloading update/new mods #1712), so that detail was probably a red herring

Closing as probably fixed. If there's still an issue here, we'd need to start over collecting info about it anyway because almost none of the related code still works the same.