chezmoi adds too many files

[itsaphel]

What exactly are you trying to do?

I want to slowly start using chezmoi, eg initially using it for ~/.config/nvim/ only

What have you tried so far?

I used the idea at the oh-my-zsh example at: https://www.chezmoi.io/reference/commands/add/#-i-include-types:

chezmoi add ~/.config/nvim/ --exact --recursive

When I do that, chezmoi starts trying to manage my entire ~/.config it seems:

> chezmoi managed
.config
.config/.DS_Store
.config/chezmoi
.config/fish
...
.config/zed

Consequently, when I do chezmoi diff, it wants to replace all of my files in other directories (aside from ~/.config/nvim) with null.

Additional context

I suspect I'm doing something obviously wrong above, as it seems like a simple use case.

The issue doesn't happen if I don't use --exact, but presumably I do want that flag.

May I suggest more examples in the quick start, and the add command, of adding entire directories to be managed with chezmoi? I'd think this is as common, if not more common, than adding individual files like ~/.zshrc. But https://www.chezmoi.io/quick-start has no examples of whole directories, and the add command reference only has one (~/.oh-my-zsh)

[halostatue]

There may be a bug in how --exact interacts with --recursive (I’ll leave that to @twpayne), but I'm not sure why you'd want to use --exact. What I would check is that you now have exact_dot_config/exact_nvim etc rather than dot_config/exact_nvim. However, the exact_ attribute tells chezmoi to remove anything not managed by chezmoi on chezmoi apply.

[twpayne]

What I think is happening here is:

The initial state is that there are no files currently managed by chezmoi.

When you run

chezmoi add ~/.config/nvim/ --exact --recursive

you're asking to chezmoi to manage ~/.config/nvim, but to manage ~/.config/nvim, chezmoi also has to manage the ~/.config directory.

Since you specified the --exact flag, chezmoi also applies this to the ~/.config directory, so it has to create entries for every existing subdirectory of ~/.config, which is why you see so much in the output of chezmoi managed.

[itsaphel]

I think that makes sense, but:

but to manage ~/.config/nvim, chezmoi also has to manage the ~/.config directory.

Why does it need to manage ~/.config to manage ~/.config/nvim? (is this true even when not using --exact?)

The reason why I thought --exact would be useful is if I remove some files in ~/.config/nvim, eg some ~/.config/nvim/lua/test.lua, I'd probably want this to be reflected in chezmoi and thus across machines?

[halostatue]

I think that makes sense, but:

but to manage ~/.config/nvim, chezmoi also has to manage the ~/.config directory.
Why does it need to manage ~/.config to manage ~/.config/nvim? (is this true even when not using --exact?)

Because you specified exact_. exact_ means that chezmoi is going to manage everything in that directory.

The reason why I thought --exact would be useful is if I remove some files in ~/.config/nvim, eg some ~/.config/nvim/lua/test.lua, I'd probably want this to be reflected in chezmoi and thus across machines?

That isn't the way that chezmoi works. If you remove ~/.config/nvim/lua/test.lua, chezmoi will regenerate ~/.config/nvim/lua/test.lua on the next apply. However, if you were to add ~/.config/nvim/lua/test2.lua — chezmoi will delete it on the next apply if you have specified exact.

chezmoi considers its source tree as the source of truth for what gets written. It's generally cautious (so it will prompt if something has been changed since the last apply), but to get new changes added in, you need to chezmoi merge-all (which I use all the time) or chezmoi readd (which I never use but some people use it regularly and successfully).

---

@twpayne Given what we've seen here, would it be fair to suggest that --exact and --recursive should probably not be used together? I don't want to prevent it, but I think that we should at least document it and maybe prompt / warn / require --force if you do want that behaviour?

[itsaphel]

Apologies, I should've been more clear.

What I mean to say is: if I deleted ~/.config/nvim/lua/test.lua in chezmoi's 'source' directory (through chezmoi cd), then with --exact, on the next apply, I'd expect ~/.config/nvim/lua/test.lua to be deleted.

If I don't use --exact, then on next apply, my file is still there.

Here's a quick example I did locally:

cd ~
mkdir test
mkdir test/test2
touch test/test.file
touch test/test2/test2.file

chezmoi add --recursive --exact test

chezmoi cd
rm exact_test/empty_test.file

chezmoi apply

test/test.file is now gone

conversely:

cd ~
mkdir test
mkdir test/test2
touch test/test.file
touch test/test2/test2.file

chezmoi add --recursive test

chezmoi cd
rm test/test.file

chezmoi apply

it isn't gone

Unless I'm missing something here, I think the --exact behaviour makes sense much of the time? For example: if I decide to refactor my configuration files on a machine - that is, say I have ~/.config/nvim/lua/base1.lua and ~/.config/nvim/lua/base2.lua, and I decide "oh, I want to merge these two files together" so that only a ~/.config/nvim/lua/base.lua exists. On the next apply on my other machines, I'd expect base1.lua and base2.lua to be deleted as stale files. Without --exact, they seem to linger around.

Perhaps there is a more proper way to achieve the above behaviour without --exact? Otherwise, the bit I don't quite get is why adding ~/.config/nvim/ to be tracked as --exact also means all of ~/.config is tracked as --exact - the directory I'm asking to be tracked here is ~/.config/nvim/, no?

[halostatue]

Apologies, I should've been more clear.

What I mean to say is: if I deleted ~/.config/nvim/lua/test.lua in chezmoi's 'source' directory (through chezmoi cd), then with --exact, on the next apply, I'd expect ~/.config/nvim/lua/test.lua to be deleted.

The way that I would have said that would be dot_config/nvim/lua/test.lua to be clear that you're referring to a source file. Thank you for the explanation.

…

Unless I'm missing something here, I think the --exact behaviour makes sense much of the time? For example: if I decide to refactor my configuration files on a machine - that is, say I have ~/.config/nvim/lua/base1.lua and ~/.config/nvim/lua/base2.lua, and I decide "oh, I want to merge these two files together" so that only a ~/.config/nvim/lua/base.lua exists. On the next apply on my other machines, I'd expect base1.lua and base2.lua to be deleted as stale files. Without --exact, they seem to linger around.

Perhaps there is a more proper way to achieve the above behaviour without --exact?

It depends on what you want and on the programs you are using. Neovim is a little more respectful of the XDG "standard" than some programs, but there are a lot of programs (chezmoi included¹) that break that with runtime files or have other files that you don't care to manage. With exact_, you have committed to having those in chezmoi and never having anything but what is in your chezmoi source for that, or explicitly adding files that might be machine specific-but-not-worth-managing be in .chezmoiignore.

With respect to exact_, it's not default behaviour. Your use case sounds like a correct use of it, but there are other ways to do this for multiple machines:

1. Manually checking chezmoi unmanaged periodically
2. Adding the files to be removed to .chezmoiremove when you remove them from one machine.

Otherwise, the bit I don't quite get is why adding ~/.config/nvim/ to be tracked as --exact also means all of ~/.config is tracked as --exact - the directory I'm asking to be tracked here is ~/.config/nvim/, no?

Tom explained this above. Try this:

mkdir -p ~/x/{y,z}/{a,b}
touch ~/x/z/{c,{a,b}/e}
chezmoi add --recursive --exact ~/x/z
chezmoi cd
find . -name '*x'

When you do the last, you will see something like ./exact_x or ./home/exact_x. If you then do (my chezmoi root is home):

$ find home/exact_x
home/exact_x
home/exact_x/exact_z
home/exact_x/exact_z/exact_a
home/exact_x/exact_z/exact_a/empty_e
home/exact_x/exact_z/exact_b
home/exact_x/exact_z/exact_b/empty_e
home/exact_x/exact_z/empty_c

You think that you’re just adding x/exact_z with chezmoi add --exact --recursive ~/x/z, but you’re actually adding exact_x/exact_z. On the other hand, if we do:

$ rm -rf home/exact_x ~/x
$ mkdir -p ~/x/{y,z}/{a,b}
$ touch ~/x/z/{c,{a,b}/e} ~/x/.keep
$ chezmoi add ~/x/.keep
$ chezmoi add --recursive --exact ~/x/z
$ chezmoi cd
$ find home/x
home/x
home/x/exact_z
home/x/exact_z/exact_a
home/x/exact_z/exact_a/empty_e
home/x/exact_z/exact_b
home/x/exact_z/exact_b/empty_e
home/x/exact_z/empty_c

Which is what you expect.

Therefore, if you want this, do the following:

touch ~/.config/.keep
chezmoi add ~/.config/.keep
chezmoi add --recursive --exact ~/.config/nvim

Footnotes

1. the state database is tracked next to the config file. There are very good reasons for this, but according to the XDG "standard", the state database should be in ~/.local/share/run or possibly in ~/.local/cache. Let's be clear: chezmoi made the right choice here, because the config and the state are tightly coupled and can't really run without each other. From a chezmoi perspective, it doesn't matter because these files can't be managed by chezmoi. ↩

[halostatue]

Since you specified the --exact flag, chezmoi also applies this to the ~/.config directory, so it has to create entries for every existing subdirectory of ~/.config, which is why you see so much in the output of chezmoi managed.

Having done a bit of experimentation above, I think that --exact and --recursive should be strongly discouraged, especially if there are paths before the last which have not been added to state. (Alternatively, --exact could apply only to the last item specified if it is a directory, and everything else gets "standard" attributes.)

The output of chezmoi managed is large because chezmoi apply will delete all of those files because all intervening directories have been added as exact_ as well, so chezmoi ~/.config/nvim without any dot_config or private_dot_config in source will result in exact_dot_config/exact_nvim, which is precisely what would not be wanted.