How complex an install do we wish to handle?

[pjf]

Right now we have a very simple "take X, and place it in Y". That's great for the vast majority of mods, but some are a little more tricky. I'm going to use @BobPalmer's amazing Karbonite mod as an example. (Hi RoverDude! You rock!)

Karbonite zipfile structure

The current Karbonite zip looks like this, starting with GameData. I've
indicated extensive directory structure with an ellipsis (...):

GameData/CommunityResourcePack/...
GameData/Firespitter/Plugins/Firespitter.dll
GameData/ModuleManager.2.3.3.dll
GameData/TextureReplacer/CRP_ORS.cfg
GameData/UmbraSpaceIndustries
GameData/UmbraSpaceIndustries/Karbonite/...
GameData/UmbraSpaceIndustries/Karbonite.dll
GameData/UmbraSpaceIndustries/Karbonite.version
GameData/UmbraSpaceIndustries/Karbonite_PartPack/...
GameData/UmbraSpaceIndustries/OpenResourceSystem_1_2_0.dll
GameData/UmbraSpaceIndustries/ORSExtensions.dll
GameData/UmbraSpaceIndustries/USI.version
GameData/UmbraSpaceIndustries/USITools.dll
GameData/UmbraSpaceIndustries/USI_Converter.dll

While a simple CKAN install can just copy the entirety of GameData
across, that doesn't accurately reflect what's been bundled up
here. We have:

Karbonite Core:

• GameData/UmbraSpaceIndustries/Karbonite
• GameData/UmbraSpaceIndustries/Karbonite.dll
• GameData/UmbraSpaceIndustries/Karbonite.version
• GameData/UmbraSpaceIndustries/Karbonite_ParkPack

Open Resource System (bundled, required):

• GameData/UmbraSpaceIndustries/OpenResourceSystem_1_2_0.dll
• GameData/UmbraSpaceIndustries/ORSExtensions.dll

USI tools (bundled, required)

• GameData/UmbraSpaceIndustries/USI.version
• GameData/UmbraSpaceIndustries/USITools.dll
• GameData/UmbraSpaceIndustries/USI_Converter.dll

ModuleManager (bundled, required)

• GameData/ModuleManager.2.3.3.dll

Community Resource Pack (bundled, required)

• GameData/CommunityResourcePack

Firespitter (bundled, dll only, required)

• GameData/Firespitter/Plugins/Firespitter.dll

TextureReplacer tweak, for both CRP and ORS (oh god how do we classify this):

• GameData/TextureReplacer/CRP_ORS.cfg

Recommendations

I suggest that:

• We modify anywhere that allows a file to allow a list. We may even
  deprecate the key file with filelist. For example:

    "install" : [
        {
            "filelist" : [
                "GameData/UmbraSpaceIndustries/Karbonite",
                "GameData/UmbraSpaceIndustries/Karbonite.dll",
                "GameData/UmbraSpaceIndustries/Karbonite.version",
                "GameData/UmbraSpaceIndustries/Karbonite_ParkPack",
            ],
            "install_to" : "GameData"
        }
    ]

• We change the spec and implementation so that only paths up to and
  including GameData get stripped. (In the current implementation,
  all leading paths are stripped, which would break the above example.)

Open Questions

Do we need a more general "copy X to Y", such as:

"install" : [
    {
        "file"          : "YourMod/CopyTheseIfYouHaveScanSat/SCANsat/My_Scanners.cfg",
        "install_to"    : "GameData/SCANSat",
        "strip_leading" : true,
        "requires"      : "SCANsat"
    }
]

My opinion is we should not have this. It's complex, it's hard to understand,
and I'm very sure people will get it wrong. Most projects I've seen which do
this can solve their problems with :NEEDS in ModuleManager. It's okay if some mods can't be installed via the CKAN if that means we have additional surety that CKAN-compatible mods will install correctly.

Feedback and comments very welcome. Tagging @rafl, @NathanKell and @erendrake on this as folks who have good advice on CKAN in the past, along with @parisba because twitter.

[NathanKell]

I don't think we need a more generic copy system, but we may well need support for arbitrary actions (within a defined set). For example, consider RO. Due to how RT2 works (unless you folks have fixed this) the settings file is not exposed to MM. To change settings, RO includes a file to overwrite it. Ditto RO and TACLS. Now, those install steps can be handled by generic pathed copy. But it is easy to imagine a situation where RO might need to delete some file in a dependency.

[pjf]

@NathanKell : How does RO currently handle deleting a file? Is it mentioned in the forum thread and users miss it and then come and complain about how something doesn't work? Or is that RO may need to delete files in the future, but doesn't do so at all at the moment?

I'm hesitant to provide a rich action set, because that means tracking where each file has come from becomes very hard. To take Debian as an example, the systems know which files are owned by which packages, and so when updating or deleting a package, you can take well-defined actions. Debian makes it tremendously hard to overwrite someone else's file, because then it's not clear what should happen if their package or your package is updated or uninstalled. I presume that with the CKAN if something overwrites a file from another package, it should stay overwritten, even if that package is upgraded, but that's very non-trivial.

[BobPalmer]

Maybe leverage ksp-avc version files? Even my sub-dependencies (like the shared usi dlls) use those?

[pjf]

@BobPalmer : I do have plans to integrate with the AVC files at some point, but mainly in the form of an avc2ckan and ckan2avc generators, and sanity tools to make sure said files agree. Unfortunately, AVC doesn't encapsulate enough information to offer complete metadata about a mod, how it should be installed, and what it needs to run.

Out of curiosity, how are you updating your AVC files? Is it by hand, or is there a neat tool-chain that updates them based upon git version tags or similar?

[BobPalmer]

By hand, which is not a big deal even with the number of mods I have

[NathanKell]

RO doesn't need it now. I think it may have at one point, but thanks to new MM functionality it's fine without it at the moment. Indeed, only those few mods with unexposed configuration require special operations at all.

[pjf]

This ticket is now fabulously stale. We're already supporting the install of mods as complex as Karbonite, and a number of spec upgrades for 1.2 and 1.4 are enabling the installation of other mods with more complex requires. As such, I'm closing this out. ;)