Allow symlinks creation during model install rather than copying the model

[brandonrising]

What type of PR is this? (check all applicable)

• Refactor
• Feature
• Bug Fix
• Optimization
• Documentation Update
• Community Node Submission

Have you discussed this change with the InvokeAI team?

• Yes
• No, because:

Have you updated all relevant documentation?

• Yes
• No

Description

This PR allows users to elect into using symlinks for model installs rather than doubling the disk space usage of the models they're installing

[brandonrising]

Is there a reason not to do it this way?

[hipsterusername]

@brandonrising - Have you tested? If this works, it'd make sense to me.

[brandonrising]

@brandonrising - Have you tested? If this works, it'd make sense to me.

Only tested some on a mac so far. Planning on testing on windows/linux today, but it works really well on a mac for standard generations.

[lstein]

Symlinks require elevated permissions on windows. There needs to be a check for this

[lstein]

A couple of comments.

1. As noted above, symlinks won’t work on Windows unless the user is running as an Admin or has enabled Developer mode. So you should catch an exception and revert to physical copying if the call fails.

2. As an alternative to doing the symlinking, there is an unused option named inplace that can be passed to the install methods. If this is set, then local model file is not moved or copied; instead its current path is simply registered with the model configuration record. It looks like I simply neglected to expose this feature to the API, but I could do this. What do you think?

[brandonrising]

A couple of comments.

1. As noted above, symlinks won’t work on Windows unless the user is running as an Admin or has enabled Developer mode. So you should catch an exception and revert to physical copying if the call fails.
2. As an alternative to doing the symlinking, there is an unused option named inplace that can be passed to the install methods. If this is set, then local model file is not moved or copied; instead its current path is simply registered with the model configuration record. It looks like I simply neglected to expose this feature to the API, but I could do this. What do you think?

1. Do you think it's potentially fine if it throws errors without reverting to copy since it's an opt in feature in the config? Maybe we can just update the description of the config to say it requires admin rights on windows? As a user, if I set this I think I'd be annoyed if I found out it was copying when it failed to do symlinks even if there was a log in the terminal for it.

2. I think inplace would solve all the issues I made this PR for, but it is also kind of nice to have the folder on the file system telling you every model that's installed on invoke and where they're located since the alternative to that now is to open the sqlite db and query a table.

[psychedelicious]

Not a fan of platform-specific behaviours and special handling.

You shouldn't need to access the db to see model paths, you can just open the model manager tab right? We can easily expose the paths there if we haven't already.

[brandonrising]

Not a fan of platform-specific behaviours and special handling.

You shouldn't need to access the db to see model paths, you can just open the model manager tab right? We can easily expose the paths there if we haven't already.

Yeah that's fair. I didn't run into any issues running it on windows, but maybe I've just always been running it in admin mode. Honestly, in-place installs like @lstein suggested would resolve the issues I'm having anyway. I could switch this PR to do that instead.

[psychedelicious]

I'd much prefer an in-place install option. It probably shouldn't be the default, because it shifts the responsibility of managing the model files to the user. We can add a checkbox to the MM UI, with a blurb explaining the feature.

How does in-place interact with the diffusers conversion cache? What if my in-place model is on a different drive from my root models dir - say, a mounted network drive or external drive - is that going to tank performance of the JIT model conversion?

[psychedelicious]

That said, I don't think we need this for v4. We have a lot of changes to model installation happening, maybe we should work towards this for a minor release after we are confident in the model installation process.

[brandonrising]

Closing in favor of in place installs