CKAN stuck at loading available modules #2711

ghost · 2019-04-01T16:02:40Z

Background

CKAN Version: 1.26.0

KSP Version: 1.4.3.2152

Operating System: Windows 10 x64

Have you made any manual changes to your GameData folder (i.e., not via CKAN)? -> Yes

Problem

What steps did you take in CKAN?
-> selecting 1.4 as new compatible KSP game version

What did you expect to happen?
-> Game change the settings to an other compatible mode

What happened instead?
-> CKAN (freez) stuck at loading available modules in Loading Modules

Screenshots:
->

ghost · 2019-04-01T16:04:07Z

Its loading like that (several times(>1)) over an hour

DasSkelett · 2019-04-01T19:34:40Z

Which version(s) did you mark as compatible?

ghost · 2019-04-02T12:26:31Z

marked compatible version: 1.4

ghost · 2019-04-02T13:34:05Z

a noteable thing to ad is that that occured right after letting it udate

DasSkelett · 2019-04-02T16:52:44Z

Does it work now?
Else try to delete the ckan.exe and replace it with a manually downloaded v1.26.

randomguyingithub · 2019-04-03T07:41:42Z

try to move ckan folder outside. It seems compatible data file doesn't work with new ckan(the ckan won't even appear for me)
then replace the registry with the old one and do things again
(not sure if this is correct but it fixed for me)

ghost · 2019-04-03T15:07:39Z

No, it doesn't work either way, i tried so far everything including new install.
But anyway (Ikd if this is interresting full), when it's at the step "loading available modules" it doesn't require any of my capacities and i cant controle it anymore (e.g.: fullscreen, closing), I have to close ist via the task manager.

i still dont know how to fix this, cause every attempt end in the same result, there each step seems the same.

DasSkelett · 2019-04-03T21:08:32Z

Could you you try to run ckan.exe update in the command line? See if that works.

Edit: Can you zip the registry.json and upload it here? I want to take a look at it.

randomguyingithub · 2019-04-04T05:33:59Z

Don't tell me it's the old pop-up behind ckan problem

ghost · 2019-04-04T12:28:29Z

This is the registry.json :
registry.zip

still with the "ckan.exe update" it wont work(it didnt changed anything).

btw Grüße aus Deutschland

DasSkelett · 2019-04-04T20:07:01Z

Grüße zurück!

The registry works perfectly fine for me, even if I don't refresh after replacing the existing one.
Neither can I spot something wrong.
Can you try the complete clean and reinstall process?

ghost · 2019-04-04T20:49:20Z

the question is how should i do it. Every time i'm runing CKAN it will automaticly go into this endless loading status.

DasSkelett · 2019-04-04T21:42:32Z

Oh, yeah...
Another command to try first: ckan.exe repair registry (set the --ksp flag if this KSP install is not you default).
After that (and if the first command doesn't work):

Go into the CKAN/history folder and search for a .ckan file which looks valid (a lot of mods listed under "depends"). As recent as possible. Copy that one into the directory where your ckan.exe is (and rename it to something easy if you want, like "mods.ckan").
Do only Phase 2 of the Clean and reinstall process.
Run ckan.exe install -c mods.ckan (or whatever you choose as a file name).
Perform Phase 4.

jupiterbjy · 2019-04-05T09:39:16Z

I'm having same issue, and looks even more awesome when performed clean install. This grey screen is how it looks like in first startup of CKAN after clean install. After force closing this not responding process, I now then have exact same problem with NicLars04.

I tried to fix my backed up CKAN's registry and result is:

It just crashes. After 1.26 update, few CKAN-installed mods like "DeployableEngiens" and "Wildblue Tools" are shown as AutoDetect and was removed from registry for unknown reason.

Performing clean install CKAN with v1.25.4 without update is also crashing with exception as following:

System.ArgumentOutOfRangeException: 인덱스 및 길이는 문자열 내의 위치를 참조해야 합니다.
매개 변수 이름: length (Rough translation: The index and length must refer to the position within the string. Parameter name: length)
   위치: System.String.Substring(Int32 startIndex, Int32 length)
   위치: System.Linq.Lookup`2.Create[TSource](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector, IEqualityComparer`1 comparer)
   위치: System.Linq.GroupedEnumerable`3.GetEnumerator()
   위치: System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector, IEqualityComparer`1 comparer)
   위치: System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector)
   위치: CKAN.NetFileCache.GetCachedFilename(Uri url, Nullable`1 remoteTimestamp)
   위치: CKAN.NetFileCache.IsMaybeCachedZip(Uri url)
   위치: CKAN.GUIMod.UpdateIsCached()
   위치: CKAN.Main.<>c__DisplayClass253_0.<_UpdateModsList>b__1(CkanModule m)
   위치: System.Linq.Enumerable.WhereSelectListIterator`2.MoveNext()
   위치: System.Collections.Generic.HashSet`1.UnionWith(IEnumerable`1 other)
   위치: CKAN.Main._UpdateModsList(Boolean repo_updated, IEnumerable`1 mc)
   위치: CKAN.Main.PostUpdateRepo(Object sender, RunWorkerCompletedEventArgs e)
   위치: System.ComponentModel.BackgroundWorker.OnRunWorkerCompleted(RunWorkerCompletedEventArgs e)

From these results, I think CKAN is accessing some files outside of KSP folder or otherwise there's no reason for old version of CKAN to break after update. If so, tell us which files' being accessed.

randomguyingithub · 2019-04-05T14:14:58Z

I ran repair registry cmd, I got gravityturncontinued conflict with gravityturn error

ghost · 2019-04-05T14:21:08Z

exactly how jupiterbjy discribed, i have the same issues and same results as he presented.

DasSkelett · 2019-04-05T15:27:05Z

@NicLars04 did you try the second part of my comment?

Go into the CKAN/history folder and search for a .ckan file which looks valid (a lot of mods listed under "depends"), as recent as possible. Copy that one into the directory where your ckan.exe is (and rename it to something easy if you want, like "mods.ckan").

Do only Phase 2 of the Clean and reinstall process.

Run ckan.exe install -c mods.ckan (or whatever you choose for file name).

Perform Phase 4.

ghost · 2019-04-05T16:20:08Z

ehm... i tried but i am not that good in this cmd and programming stuf, so i am a bit strugelling how to do it. Could anyone explain it to me in more details please, i would realy appreciate it. 👍

(ps.: i dont understand how the end of step 2 shuold work, then i have all exept for the downloads deleted)

ghost · 2019-04-05T16:38:07Z

CKAN Version: v.1.25.4

Ok, So i have just installed the older version and know i am getting new errors like from jupiterbjy. If all dont work, Is there any way to get it working in an older version?

jupiterbjy · 2019-04-06T00:33:11Z

FIXED IT:

Run regedit.exe, find CKAN Folder located under 'HKEY_CURRENT_USER\Software\CKAN' or search CKAN.

Remove that CKAN Folder itself, and launch CKAN regardless of it's version.

And BAM. It works. Tested on v1.26.0.

I'm quite surprised CKAN does use registry.. To developers, please make sure to deploy hotfix that remove/fix actual registery, not registry.json file.

Have a (un)safe flight everyone!

DasSkelett · 2019-04-06T02:21:19Z

@jupiterbjy thanks for the clue.
I suspect the KSPBuilds key to be the problematic one.

@NicLars04, scratch the reinstallation. Please try to delete the registry keys, one at a time.

Run regedit.exe (via Win+R combination, or search for it tín the start menu)
Navigate to HKEY_CURRENT_USER\Software\CKAN
Right click on KSPBuilds -> Delete
Run CKAN, see if it works.
If not, delete DownloadCacheDir and run CKAN again
Repeat with another entry in the registry until CKAN works again.
Please tell us which key got it to work for you again.

jupiterbjy · 2019-04-06T10:51:36Z

I still got v1.25.4 CKAN on Laptop with untouched registy, I'll give result in 1 - 2 hours.

FYI, it's safe to assume KSPBuild is cause because after CKAN regenerated all keys, all other keys are same, except install directory key which I moved last year and still not changed.

jupiterbjy · 2019-04-06T11:52:27Z

Data previous v1.26 (collapsed)

{ "builds": { "464": "0.23.5.464", "559": "0.24.2.559", "642": "0.25.0.642", "705": "0.90.0.705", "830": "1.0.0.830", "842": "1.0.2.842", "861": "1.0.4.861", "1024": "1.0.5.1024", "1028": "1.0.5.1028", "1172": "1.1.0.1172", "1174": "1.1.0.1174", "1180": "1.1.0.1180", "1183": "1.1.0.1183", "1196": "1.1.0.1196", "1203": "1.1.0.1203", "1209": "1.1.0.1209", "1215": "1.1.0.1215", "1224": "1.1.0.1224", "1228": "1.1.0.1228", "1230": "1.1.0.1230", "1250": "1.1.1.1250", "1260": "1.1.2.1260", "1289": "1.1.3.1289", "1473": "1.2.0.1473", "1479": "1.2.0.1479", "1485": "1.2.0.1485", "1486": "1.2.0.1486", "1489": "1.2.0.1489", "1494": "1.2.0.1494", "1499": "1.2.0.1499", "1500": "1.2.0.1500", "1509": "1.2.0.1509", "1517": "1.2.0.1517", "1520": "1.2.0.1520", "1523": "1.2.0.1523", "1532": "1.2.0.1532", "1539": "1.2.0.1539", "1540": "1.2.0.1540", "1546": "1.2.0.1546", "1548": "1.2.0.1548", "1553": "1.2.0.1553", "1563": "1.2.0.1563", "1564": "1.2.0.1564", "1569": "1.2.0.1569", "1574": "1.2.0.1574", "1576": "1.2.0.1576", "1583": "1.2.0.1583", "1584": "1.2.0.1584", "1586": "1.2.0.1586", "1604": "1.2.1.1604", "1622": "1.2.2.1622", "1727": "1.2.9.1727", "1730": "1.2.9.1730", "1737": "1.2.9.1737", "1738": "1.2.9.1738", "1743": "1.2.9.1743", "1750": "1.2.9.1750", "1758": "1.2.9.1758", "1764": "1.2.9.1764", "1773": "1.2.9.1773", "1781": "1.2.9.1781", "1790": "1.2.9.1790", "1796": "1.2.9.1796", "1800": "1.2.9.1800", "1804": "1.3.0.1804", "1836": "1.3.1.1836", "1847": "1.3.1.1847", "1855": "1.3.1.1855", "1863": "1.3.1.1863", "1891": "1.3.1.1891", "2077": "1.4.0.2077", "2089": "1.4.1.2089", "2110": "1.4.2.2110", "2152": "1.4.3.2152", "2215": "1.4.4.2215", "2243": "1.4.5.2243", "2256": "1.4.5.2256", "2332": "1.5.0.2332", "2335": "1.5.1.2335", "2395": "1.6.0.2395", "2401": "1.6.1.2401" } }

Data after Update & Registry purge (collapsed)

{ "builds": { "464": "0.23.5.464", "559": "0.24.2.559", "642": "0.25.0.642", "705": "0.90.0.705", "830": "1.0.0.830", "842": "1.0.2.842", "861": "1.0.4.861", "1024": "1.0.5.1024", "1028": "1.0.5.1028", "1172": "1.1.0.1172", "1174": "1.1.0.1174", "1180": "1.1.0.1180", "1183": "1.1.0.1183", "1196": "1.1.0.1196", "1203": "1.1.0.1203", "1209": "1.1.0.1209", "1215": "1.1.0.1215", "1224": "1.1.0.1224", "1228": "1.1.0.1228", "1230": "1.1.0.1230", "1250": "1.1.1.1250", "1260": "1.1.2.1260", "1289": "1.1.3.1289", "1473": "1.2.0.1473", "1479": "1.2.0.1479", "1485": "1.2.0.1485", "1486": "1.2.0.1486", "1489": "1.2.0.1489", "1494": "1.2.0.1494", "1499": "1.2.0.1499", "1500": "1.2.0.1500", "1509": "1.2.0.1509", "1517": "1.2.0.1517", "1520": "1.2.0.1520", "1523": "1.2.0.1523", "1532": "1.2.0.1532", "1539": "1.2.0.1539", "1540": "1.2.0.1540", "1546": "1.2.0.1546", "1548": "1.2.0.1548", "1553": "1.2.0.1553", "1563": "1.2.0.1563", "1564": "1.2.0.1564", "1569": "1.2.0.1569", "1574": "1.2.0.1574", "1576": "1.2.0.1576", "1583": "1.2.0.1583", "1584": "1.2.0.1584", "1586": "1.2.0.1586", "1604": "1.2.1.1604", "1622": "1.2.2.1622", "1727": "1.2.9.1727", "1730": "1.2.9.1730", "1737": "1.2.9.1737", "1738": "1.2.9.1738", "1743": "1.2.9.1743", "1750": "1.2.9.1750", "1758": "1.2.9.1758", "1764": "1.2.9.1764", "1773": "1.2.9.1773", "1781": "1.2.9.1781", "1790": "1.2.9.1790", "1796": "1.2.9.1796", "1800": "1.2.9.1800", "1804": "1.3.0.1804", "1836": "1.3.1.1836", "1847": "1.3.1.1847", "1855": "1.3.1.1855", "1863": "1.3.1.1863", "1891": "1.3.1.1891", "2077": "1.4.0.2077", "2089": "1.4.1.2089", "2110": "1.4.2.2110", "2152": "1.4.3.2152", "2215": "1.4.4.2215", "2243": "1.4.5.2243", "2256": "1.4.5.2256", "2332": "1.5.0.2332", "2335": "1.5.1.2335", "2395": "1.6.0.2395", "2401": "1.6.1.2401" } }

Edited by @DasSkelett

DasSkelett · 2019-04-06T12:51:46Z

Those two lists are completely the same, at least Atom tells me so.
Did you spot a difference?

jupiterbjy · 2019-04-06T13:30:41Z

Checked with Notepad++, same here. To note here - At this time CKAN failed to initialize after update, so registry might not updated at all, sorry couldn't be helped.

ghost · 2019-04-06T13:31:58Z

@jupiterbjy, @DasSkelett
thanks for all 👍 . after succesfully deleting (1st KSPBuilds = not yet worked; 2nd = working fine) all works fine as allways (in CKAN v.1.25.4).

ghost · 2019-04-06T13:37:35Z

@DasSkelett
I dont know if you you wanna talk more about this probleme here with jupiterbjy , so I'd let you decide when to close it. 👍

jupiterbjy · 2019-04-06T13:45:39Z

This bug is not solved, so it'll stay as bug until fix is out.

ghost · 2019-04-06T14:05:41Z

ok

DasSkelett · 2019-04-07T00:15:10Z

So a quick summarize where we are:
After updating to CKAN v1.26, it can happen that CKAN stops working when loading the registry (after changing version compatibilities or a simple refresh probably too) without throwing an error.

If you downgrade again. you get the System.ArgumentOutOfRangeException. It has something to do with the cache, but I don't see the exact cause yet.
Interesting: if I downgrade, I get

System.ArgumentNullException: Value cannot be null.
Parametername: key
   bei System.ThrowHelper.ThrowArgumentNullException(ExceptionArgument argument)
   bei System.Collections.Generic.Dictionary`2.FindEntry(TKey key)
   bei System.Collections.Generic.Dictionary`2.TryGetValue(TKey key, TValue& value)
   bei CKAN.Registry.LatestAvailableWithProvides(String module, KspVersionCriteria ksp_version, RelationshipDescriptor relationship_descriptor)
   bei CKAN.Registry.allDependenciesCompatible(CkanModule mod, KspVersionCriteria ksp_version)
   bei CKAN.Registry.Available(KspVersionCriteria ksp_version)
   bei CKAN.Main._UpdateModsList(Boolean repo_updated, IEnumerable`1 mc)
   bei CKAN.Main.CurrentInstanceUpdated()
   bei CKAN.Main.OnLoad(EventArgs e)
   bei System.Windows.Forms.Form.OnCreateControl()
   bei System.Windows.Forms.Control.CreateControl(Boolean fIgnoreVisible)
   bei System.Windows.Forms.Control.CreateControl()
   bei System.Windows.Forms.Control.WmShowWindow(Message& m)
   bei System.Windows.Forms.Control.WndProc(Message& m)
   bei System.Windows.Forms.Form.WmShowWindow(Message& m)
   bei System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

which is probably because the newer metadata in the registry can't be read by CKAN and all those mods are returned as NULL, throwing this error. It is not a fatal bug, continuing a refreshing once gets CKAN to work again. Maybe doesn't have anything to do with this issue.

If you delete the registry key DownloadCacheDir, CKAN works again.
Maybe even a simple cache purge would suffice?
This, together with the exception points to a cache issue.

I wasn't able to recreate the issue yet. Can both of you @NicLars04 and @jupiterbjy send me an exported .ckan of your mods, so I can see if specific mods (suspecting those with spec version 1.26) cause the issue (if they are cached or something)?

ghost · 2019-04-07T13:16:26Z

mods_from_ckan.zip
this should be what you requested.

jupiterbjy · 2019-04-07T23:17:55Z

installed-portable-2019-04-05_17-42-51.txt

Lastest file right before CKAN failure. Convert .txt to .ckan.
This won't be as helpful as @NicLars04's since I let CKAN download at ramdisk.

Outside of topic, is there no temporary file removal on CKAN? Regardless of download successful or failure, CKAN keeps mod-sized temp files on temp folders and never removes it.

DasSkelett · 2019-04-09T20:57:46Z

Do you mean the cache? You can limit it in the settings.

jupiterbjy · 2019-04-10T01:02:33Z

CKAN download mod as tmp file at TEMP folder, and CKAN never moves it to cache folder unless all target mods are downloaded. This is not problematic unless there is crash or TEMP folder is limited.

Either using ramdisk or just system disk is full - this crash CKAN as TEMP folder containing drive get full before completing download. Then already created tmp files need to be manually removed or it stays there forever until reboot.

It would be much helpful if CKAN moved mod to cache folder after finishing each every downloads, so user won't have to worry about their capacity on TEMP folder and install large mods one by one.

DasSkelett · 2019-04-10T10:41:40Z

I think I know what you mean now.
Please open a new issue for this request, where we can discuss that without spamming this one.

HebaruSan · 2019-04-27T18:51:12Z

System.ArgumentOutOfRangeException: 인덱스 및 길이는 문자열 내의 위치를 참조해야 합니다.
매개 변수 이름: length (Rough translation: The index and length must refer to the position within the string. Parameter name: length)
   위치: System.String.Substring(Int32 startIndex, Int32 length)
   위치: System.Linq.Lookup`2.Create[TSource](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector, IEqualityComparer`1 comparer)
   위치: System.Linq.GroupedEnumerable`3.GetEnumerator()
   위치: System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector, IEqualityComparer`1 comparer)
   위치: System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector)
   위치: CKAN.NetFileCache.GetCachedFilename(Uri url, Nullable`1 remoteTimestamp)

That exception means that this block is throwing:

CKAN/Core/Net/NetFileCache.cs

Lines 175 to 180 in 44d44ea

    
           cachedFiles = files = allFiles() 
        
               .GroupBy(fi => fi.Name.Substring(0, 8)) 
        
               .ToDictionary( 
        
                   grp => grp.Key, 
        
                   grp => grp.First().FullName 
        
               );

Since it's complaining about the length, the cause may be a filename shorter than 8 characters in the cache folder. I can reproduce a freeze on load if I create such a file:

Two questions for those affected by this:

What path are you using for your CKAN download cache? (If you don't know, it'd be the default)
Are there any short filenames in that folder? If so, what are they and can you figure out how they got there?

EDIT: We have one answer to question 1 already:

@jupiterbjy, what else is in that folder?

jupiterbjy · 2019-04-28T06:03:52Z

@jupiterbjy, what else is in that folder?

Well, that E:\Tempy folder is not real cause of issue. Original folder was on ramdisk, with several temporary system files. So there's high chance to have file with short filename in there, but for now CKAN's temp file is shortest. (tmp38D5.tmp)

I think I know what you mean now.
Please open a new issue for this request, where we can discuss that without spamming this one.

Okay, will create soon.

DasSkelett added Bug Something is not working as intended GUI Issues affecting the interactive GUI labels Apr 1, 2019

HebaruSan mentioned this issue Apr 27, 2019

Fix freeze on non CKAN files in cache folder #2743

Merged

Olympic1 assigned HebaruSan Apr 27, 2019

HebaruSan pinned this issue Apr 27, 2019

HebaruSan closed this as completed in #2743 Apr 28, 2019

Olympic1 removed the Bug Something is not working as intended label Apr 28, 2019

HebaruSan unpinned this issue May 7, 2019

CKAN stuck at loading available modules #2711

CKAN stuck at loading available modules #2711

Comments

ghost commented Apr 1, 2019 • edited by HebaruSan Loading

Background

Problem

ghost commented Apr 1, 2019

DasSkelett commented Apr 1, 2019

ghost commented Apr 2, 2019

ghost commented Apr 2, 2019

DasSkelett commented Apr 2, 2019

randomguyingithub commented Apr 3, 2019 • edited Loading

ghost commented Apr 3, 2019

DasSkelett commented Apr 3, 2019 • edited Loading

randomguyingithub commented Apr 4, 2019

ghost commented Apr 4, 2019

DasSkelett commented Apr 4, 2019 • edited Loading

ghost commented Apr 4, 2019

DasSkelett commented Apr 4, 2019 • edited Loading

jupiterbjy commented Apr 5, 2019 • edited by DasSkelett Loading

randomguyingithub commented Apr 5, 2019

ghost commented Apr 5, 2019

DasSkelett commented Apr 5, 2019

ghost commented Apr 5, 2019

ghost commented Apr 5, 2019

CKAN Version: v.1.25.4

jupiterbjy commented Apr 6, 2019

DasSkelett commented Apr 6, 2019 • edited Loading

jupiterbjy commented Apr 6, 2019 • edited Loading

jupiterbjy commented Apr 6, 2019 • edited by DasSkelett Loading

DasSkelett commented Apr 6, 2019

jupiterbjy commented Apr 6, 2019

ghost commented Apr 6, 2019 • edited by ghost Loading

ghost commented Apr 6, 2019

jupiterbjy commented Apr 6, 2019

ghost commented Apr 6, 2019

DasSkelett commented Apr 7, 2019 • edited Loading

ghost commented Apr 7, 2019

jupiterbjy commented Apr 7, 2019 • edited Loading

DasSkelett commented Apr 9, 2019

jupiterbjy commented Apr 10, 2019 • edited Loading

DasSkelett commented Apr 10, 2019

HebaruSan commented Apr 27, 2019 • edited Loading

jupiterbjy commented Apr 28, 2019 • edited Loading

ghost commented Apr 1, 2019 •

edited by HebaruSan

Loading

randomguyingithub commented Apr 3, 2019 •

edited

Loading

DasSkelett commented Apr 3, 2019 •

edited

Loading

DasSkelett commented Apr 4, 2019 •

edited

Loading

DasSkelett commented Apr 4, 2019 •

edited

Loading

jupiterbjy commented Apr 5, 2019 •

edited by DasSkelett

Loading

DasSkelett commented Apr 6, 2019 •

edited

Loading

jupiterbjy commented Apr 6, 2019 •

edited

Loading

jupiterbjy commented Apr 6, 2019 •

edited by DasSkelett

Loading

ghost commented Apr 6, 2019 •

edited by ghost

Loading

DasSkelett commented Apr 7, 2019 •

edited

Loading

jupiterbjy commented Apr 7, 2019 •

edited

Loading

jupiterbjy commented Apr 10, 2019 •

edited

Loading

HebaruSan commented Apr 27, 2019 •

edited

Loading

jupiterbjy commented Apr 28, 2019 •

edited

Loading