Copy applications

[keith]

Based on a conversation on HackerNews a while back I was looking into the codebase to see about implementing the ability to copy applications rather than symlinking them. As I argued then, since externally installed applications often have their own upgrade mechanisms I believe it would make sense for cask to provide users with the ability to download, unpack and copy applications to the target directory. This would mean that cask would have no knowledge of the application after it was installed and it would not show up in the list. For all intensive purposes to cask it would be uninstalled. Yet this way the application would live on in the installed location updating itself internally as needed. While this may seemingly not entirely utilize cask's feature set I still think the download and unpacking process would be extremely useful regardless of what happens to the application after that.

I would love to hear thoughts about adding this feature. As for how that would be done, while briefly looking through the codebase I think I traced the installation process to the artifact installers depending on the source's type. It seems to me that this would present a problem. In the example of an application, since the app artifact inherits directly from the symlink installer, it would take some other changes to make copying work cleanly. Although another, much less clean option, would be to provide an option that changes the symlink command since simply changing it to a cp with the correct arguments nearly accomplishes what I'm suggesting (although it obviously wouldn't remove the original application probably in /opt/...).

[vitorgalvao]

Some clarifications on your points on HackerNews (some might’ve been true then, since it was over 3 months ago, and I’m not sure of the timeline without checking), that are (to some extent) inaccurate with how it currently works.

I don't think this synchronizes installations since the forumla are overwritten with each update. If someone updates one of them the specific version you downloaded elsewhere will not be downloaded again.

True to some degree. If the formula (called a cask, here) points to a url that always downloads the latest version, then this is the case. However, if the url is versioned, you can still make a tap via homebrew with the specific versions you want.

it's not like it symlinks your preference files so you're not actually cleaning up any junk if you uninstall the application through cask, it's the equivalent of just dragged it to the trash

True that it not interacts with preference files (although there as been some talk of it), but uninstalling is not necessarily the same as moving to the trash. It is in the case of .apps, but in the case of .pkg files, it will actually perform the correct (as best as we can figure out) uninstall, which is particularly useful for installers that do not deliver an uninstall command.

Also the applications will still attempt to update so it's not managing versions that way either.

True, but also being being worked on.

All that being said, even after becoming a collaborator I have for some time employed a script with a technique similar to yours (getting the symlinks to move the applications), but eventually started using this as it was designed. I’d welcome the possibility to do it natively, though, and as it was stated in HackerNews, it is a possibility to consider, if it garners enough interest.

[keith]

However, if the url is versioned, you can still make a tap via homebrew with the specific versions you want.

Definitely. Nice call. I only mentioned that because of someone arguing for cask
being an easy way to recreate an environment on multiple machines. That, in my
opinion, doesn't really affect this issue.

It is in the case of .apps, but in the case of .pkg files, it will actually perform the correct (as best as we can figure out) uninstall, which is particularly useful for installers that do not deliver an uninstall command.

Also good point. Everything I said was in terms of .app bundles extracted from
something, I wasn't even thinking about .pkgs or anything else.

True, but also being being worked on.

Cool idea. I still think it would be nice to use cask just as an installer.

...if it garners enough interest.

I have no problem doing the legwork here with a little guidance on how to
architect it, I just want to make sure if I spend the time on it it will be
considered.

[vitorgalvao]

I have no problem doing the legwork here with a little guidance on how to architect it, I just want to make sure if I spend the time on it it will be considered.

Let’s ping homebrew-cask’s creator, and a few collaborators and contributors on this one.

@phinze @rolandwalker @nanoxd @fanquake @alebcay What do you say? Here’s someone with a feature request that is also interested in doing the work to make it a reality. Could we have a brief discussion to see if this is something desirable for the future of homebrew-cask? It’s not the first time this has been talked about, so there is a some demand for it. We’re talking in the context of an additional way of working (as opposed to changing the way the project works, as that would go against its philosophy). It seems to me that under these conditions we can pretty much close the case on this question (in a general sense), giving a definite yes or no.

[alebcay]

Is there a function in Cask right now for specifying whether to copy or to symlink, maybe an environment variable or something, e.g. COPYAPPS=1, or something?

[rolandwalker]

+1, though it would have to be done with ditto instead of cp.

@phinze expressed approval of this idea as an optional behavior in #2312 (comment), and we noted this as a todo item in HACKING.md.

@Keithbsmiley I'd be glad to assist you with anything. One reason I did not tackle this one yet myself is that I figure it would work best as something users could set in a ~/.caskconfig file -- and we don't yet have such a thing.

(Environment variables as we use now are the wrong approach for configuration, according to the following logic: the configuration should be equally as persistent as the target.)

[rolandwalker]

Also pinging in @voanhduy1512 just in case he is interested.

[keith]

The idea of a configuration file sounds great to me especially for this. I also agree that the current environment variables aren't ideal for this.

[rolandwalker]

For config, my idea was to find a license-compatible parser for git-style config files, and to drop it in lib as we did with lib/plist/parser.rb. Something simple & robust.

I wasn't too happy with the alternatives that came up, and lost momentum.

[alebcay]

Since you found the nice little plist parser, have you considered making the config file with a plist instead of a git-style config file?

[voanhduy1512]

👍 I find myself need this feature a lot of times.

[booch]

I've been running into the problems with symlinks, and I think copying might be a good solution. At least for some packages --- we might want to include command-line switches in addition to config settings, to allow choosing linking or copying.

I'd like to suggest that we keep track of what we've installed though. It shouldn't be too much trouble to keep something in Caskroom with details about what was installed, so we can upgrade or uninstall it. As far as apps that update themselves, most apps have a way of determining the version, and we could check that before attempting an upgrade.

[keith]

To determine the version from the command line you would have to parse it out of the plist in the app bundle I believe.

[rolandwalker]

@booch, yes. See #3066 .
@Keithbsmiley, also yes. We should move towards using the version-number from the bundle plist as a standard.

[alebcay]

@rolandwalker Perhaps a devscript to read the version number from plist file inside apps? (It might be overkill though; just open the app bundle contents and find out)

[rolandwalker]

@alebcay as we are people of the command-line, a script is a good thing.

However, preferring the app-bundle version (as policy) should need to be discussed more among the maintainers.

[vitorgalvao]

A script to read the version could provide us with something else. With #1021 in mind, think of the idea of tweaking @alebcay’s cask-tasting and my cask-repair script to auto-update casks that currently have version 'latest'. They’d still need some kind of differentiation (so the script could know who they are, and to avoid unnecessary manual updates from contributors), but it could give us the best of both worlds, and settle the “checksummed versioned or no_checksum latest” issue.

[alebcay]

@vitorgalvao the script could certainly be retrofitted to have that functionality, but I would probably place it in another script for organizational purposes and have it called by the checkLinks.sh script instead. I imagine the task would involve:

1. Get list of Casks using version 'latest'.
2. For each such Cask, download the file specified at url. Record the SHA256 sum.
3. Keep running such checks until the SHA256 sum changes (automation via cron or other mechanism).
4. When some change occurs, do something. I'm not sure what we should do with this information of which Casks have had changes applied.

Regardless, this idea is off-topic for this issue - perhaps this discussion could continue in one of the issues dedicated to a brew upgrade mechanism of some sort.

[vitorgalvao]

Regarding point 4, with some simple changes, cask-repair (I’ve edited the previous comment to include it after posting, so you might’ve missed it while typing the reply) could already get your CaskNoSum.txt file and perform the updates (it’d be a matter of running it automatically as well). As stated, these casks would need some new meta-information (changing point 1), as versions would no longer be latest.

I agree this is off-topic here, I’ve already submitted the idea to the aforementioned #1021.

[vitorgalvao]

Just remembered something that has to be taken into consideration for copying applications. Some apps, like gridwars, actually require the directory they come in to be present, to function properly. Currently we only link the .app itself, as it’s sufficient for the implemented use case, but would’t be when copying. Since not every app that comes inside a directory needs it, cask files themselves would need some modification to accommodate this, as simply replacing linking with copying wouldn't suffice.

[alebcay]

In the finicky usage case mentioned by @vitorgalvao, it may then make sense to have both copy and link commands in the DSL, with copy being preferred and link being used in fallback situations such as with gridwars.

[ngan]

Is it possible to copy and still have cask manage your apps by writing a symlink in the opposite direction?

[vitorgalvao]

That doesn’t sound like a very robust solution, but sure, if we store some metadata on install, why not keep management options?

[username724]

@vitorgalvao Why reproduce metatdata that the system already stores for you? The system_profiler command should return any .app it can find, especially if the actual application is in /Applications.

To retrieve data needed for managing a given app, I use:
system_profiler SPApplicationsDataType | grep -B 7 'Sublime Text 2.app'
This will produce a nice little stream of data that can be stored as xml, or in any format you can parse.

[nanoxd]

@username724 The command is missing an s in SPApplicationsDataType

[szhu]

@username724 system_profiler will generate a report of all apps on the computer, and then we're using grep to filter out almost everything from the report. This is highly inefficient.

There's a much easier way to get the path to an app, using mdfind or osascript. mdfind is a command-line front end to Mac OS X's metadata (Spotlight) index. I assume system_profiler uses the same index, so it should return the same results. osascript is used to run AppleScript.

mdfind 'kind:application' -name 'Sublime Text 2'
# or
osascript -e 'POSIX path of (path to app "Sublime Text 2")'

(See function pathtoapp here. I named the function after equivalent AppleScript command.)

By the way, we should be identifying apps by, well, their identifier and not their name, since people might rename apps. (For example, I might want to rename "Google Chrome.app" to just "Chrome.app", which would break both commands above.) Also don't forget that there are some completely different apps that have the same name. Here's how to find the path to apps using their id:

mdfind -literal kMDItemCFBundleIdentifier= 'com.sublimetext.2'
# or
osascript -e 'POSIX path of (path to app id "com.sublimetext.2")'

AppleScript would probably be less preferred because it tends to open apps it's inspecting.

How did I know to use kMDItemCFBundleIdentifier? You can list all the metadata of a file using the aptly-named mdls command: mdls '/path/to/Sublime Text 2.app'.

btw, I would encourage you to upgrade to ST3 :)

[username724]

@nanoxd : Fixed. Thanks.

@szhu : I am reluctant to rely on mdfind as it relies on spotlight. I've ran in to several situations where I have had to turn spotlight off for whole semesters, because it conflicted with software required by my end users. From what I understand, system_profiler does not rely on spotlight.

To be clear, I was proposing using data already stored on the system for the purpose of management options. There are many ways to retrieve data already on the system, and all of them should be on the table, as long as they conform to the guidelines.

What I really want, is the ability to install directly to /Applications. It would be extremely useful in an enterprise environment.

Keep up the great work homebrew-cask team!!

[wamatt]

So for clarity, is the current thinking around copying or moving apps to /Applications dir?

[rolandwalker]

@wamatt copying would be more straightforward. I want to make it configurable, so that the user can choose copying, symlinking, or hardlinking against the staged copy under /opt/homebrew-caks/Caskroom.

[wamatt]

Cool. hardlinking is actually interesting since it saves space compared with copying.
Any known gotcha's with hardlinking? Other than maybe it's a bit more magical for the end user?

[rolandwalker]

It depends. Fonts have used hardlinks for a long time now (#2258). It works perfectly, unless you have set up /opt on a different volume.

However, for app bundles we would use an exotic creature: the directory hardlink (see #2312). Since directory-hardlinks are unusual, there might be some issues in practice (like certain utilities being unable to understand them). Thus #2312 is unmerged, because it needs to start out as an option chosen by the user. However, it might solve oddities that crop up with certain apps due to symlinking.

[wamatt]

Ok interesting. TIL there is a difference between dir hardlink and a file hardlink. :)

Hope you will seriously consider having a user option for "move" instead of copy. Actually though, in that case, it may as well just install to appdir=/Applications, without moving or copying.

That to me is the cleanest from an end user perspective.

[alebcay]

Hope you will seriously consider having a user option for "move" instead of copy. Actually though, in that case, it may as well just install to appdir=/Applications, without moving or copying.

That to me is the cleanest from an end user perspective.

Many Casks contain materials inside the downloaded package that should not belong in /Applications or your appdir of choice. Cask stages the operations in /opt/... to keep the appdir tidy.

[wamatt]

Many Casks contain materials inside the downloaded package that should not belong in /Applications or your appdir of choice. Cask stages the operations in /opt/... to keep the appdir tidy.

Ok. Maybe I'm missing something, but why not keep the other stuff you've mentioned in opt, then leave a single copy of the actual application in /Applications?

Maybe I'm misunderstanding copying. But if a user installs a 500mb app, it's going to take 1gb of their HD space, using the copy method. That would be an unexpected consideration for most users.

[budda]

👍 Would love to use Cask for simply copying a downloaded app in to /Applications -- set by an ENV variable for now would make me happy with Ansible :)

[keith]

I've created a homebrew formula with a patch that just switches ln -> cp as a temporary fix.

[szhu]

@keith can you give an example of how to install and use this formula? Thanks!

[keith]

brew tap caskroom/cask; brew install keith/formulae/brew-cask 

Use at your own risk!

[szhu]

Thanks! btw it seems to not work with casks that install command-line tools. Is it because the source is being deleted too early?

==> Downloading https://central.github.com/mac/latest
######################################################################## 100.0%
==> Copying App 'GitHub.app' to '/Users/Administrator/Applications/GitHub.app'
==> Running uninstall process for github; your password may be necessary
==> Removing launchctl service com.github.GitHub.Conduit
==> Removing launchctl service com.github.GitHub.GHInstallCLI
Error: It seems the copy source is not there: '/opt/homebrew-cask/Caskroom/github/latest/GitHub.app/Contents/MacOS/github_cli'

[keith]

Ah interesting. I didn't test with GitHub for mac. Looks like some things are suppose to get copied after I removed the source. In theory I don't have to remove the source, it's just unnecessary cruft. An immediate fix to this would be to remove the one you just installed and install my previous revision.

brew rm brew-cask
brew install https://raw.githubusercontent.com/keith/homebrew-formulae/ace116da352b6cc88d19b574a000af011017a2ed/Formula/brew-cask.rb

This one should solve that problem but the source will sit around in /opt/homebrew-cask/... forever.

[szhu]

Thanks for being on top of this! I think I'll stick with standard brew-cask for now. Would it be possible to change the patch so that it removes everything only after everything else is applied?

[keith]

Possibly.

[danielbayley]

+1 for this, except I think move rather than copy since we don't want 2 of everything taking up unnecessary space.

At the moment I have a Hazel rule watching /Applications which swaps the link with the App like this;

app=$(readlink "$1")
rm "$1" && mv "$app" /App*s && ln -s /App*s/"${app##*/}" "${app%/*}"

[vitorgalvao]

Closing in favour of these news: #13201.

[danielbayley]

In the mean time, if any other Hazel users (@BrianRoach) want to use my rule to do this, I've uploaded it here. Just import the rules and apply them to
/usr/local/opt/Caskroom (or wherever your Caskroom folder is…)

The main chunk of [bash] code is just;

version() { mdls -name kMDItemVersion "$1"| egrep -o [0-9.]+ ;}
App=(/App*s/"${1##*/}")

[[ ! -a $App || -L $App || $(version $1) > $(version $App) ]] &&
{   mv "$App" ~/.Trash
    mv "$1" "${App%/*}"
    ln -s "$App" "${1%/*}"/
}

[itsbrex]

Thanks @danielbayley for posting this. For whatever reason using /usr/local/bin/fish as the shell wouldn't work. Changed to /bin/bash and it worked.