Support for submodules

[XDIJK]

Good afternoon,

I'd like to propose the ability to support sub-modules within the obsidian-git plugin.
This would effectively allow you to store multiple directories of notes in multiple repositories.
I think this could be beneficial to people who require strict separation between for example work notes and personal notes, without having to deal with multiple vaults thus instances of obsidian.

[Vinzent03]

I don't know submodules well. What needs to be changed?

[XDIJK]

Apologies for the delay in getting back to you.

I haven't spend much time thinking about the implementation nor am I familiar with the simple-git dependency, or the exact mechanics of how the obsidian-git plugin works by itself.

So the following will be rather pseudo like.
Since the plugin already assumes that there is a initialized git repository there should be no need to initialize the sub modules.
A list of submodules could be retrieved by running:

git config --file .gitmodules --get-regexp path or git submodule status
If these return non empty we could actually start doing operations. since submodules exist
These commands do not produce clean output to parse.

e.g

• pulling updates from all submodules in the repository
  git submodule update --recursive --remote
  If we want to consider the potential URL of the submodule changing we could run:
  git submodule sync --recursive before pulling the changes

• commit changes in submodules
  If we wanted to be precise we could parse the the output of the git config command to get the path and create a commit for that directory. or create a commit for every submodule under the parent by leveraging git submodule foreach e.g
  git submodule foreach 'git add -A'
git submodule foreach 'git commit'

• Pushing changes from all submodules from parent to submodules and parent
  git push --recurse-submodules=on-demand this may fail if the parent and submodule are on different branches
  git submodule foreach 'git push' should work.

After doing the submodule operations
The standard operations on the main branch can be continued and the submodule ref will also be commited on the main branch.

I am not particularly skilled at git so might have gotten some terms wrong.

If you do believe this is a worthwhile addition but with low priority or interest I can take a crack at adding it however won't have anytime to do so for the upcoming few weeks.

[Vinzent03]

Thanks a lot for the explanation! I will see what simple-git supports and what I can do. Would you prefer extra commands in Obsidian for submodules, or even for every submodule? Or just update submodules with the existing commands?

[kzhovn]

I've just been updating my submodule manually, and would love support to do it automatically - my preferred implementation would be a setting that when toggled enabled performing all operations on submodules by default. Don't know if this is what @XDIJK was envisioning, of course.

[XDIJK]

A toggle for hey i've got submodules please run operations on them would also be perfectly fine for my case. Just as @kzhovn suggested!

[Vinzent03]

Would somebody of you volunteer to test the submodules support? Just replace your manifest.json and main.js with the one in the attached zip in <your-vault>/.obsidian/plugins/obsidian-git. @XDIJK Thanks again for the detailed explanation.
The upstream branch should be set in the submodules so that git push works without origin and branch parameter.
obsidian-git.zip

[aparente]

I just installed the plugin for the first time and was wondering if this functionality exists!

I collaborate a lot with non-work colleagues and want to be able to share groups of notes (eg. notes on research papers) but don't want to share the full vault. If I'm reading this correctly it sounds like this has been implemented. Is there a detailed overview of how to run this?

[Vinzent03]

So far it is not released. I just posted a test version in my comment before. So you need to install that version. In the settings
is a toggle to update submodules. If you turn it on, backup and pull operates on all submodules too.

[XDIJK]

@Vinzent03 I gave it a quick test and it seems to be working fine.
Only things I ran into are in regards to git configuration.

I haven't tested anything in-depth as in conflicts or more in-depth rebases I also think that might be out of scope for the plugin.

Few issues I ran into in regards to git configuration were the following

• Setting up the submodule to track git submodule set-branch --branch <default-branch> -- <repo-name>
• Setting up the submodule update so that it doesn't checkout in a detached state but rebases git config -f .gitmodules submodule.<repo-name>.update rebase
  this can also be set to merge

For me personally the my use-case is covered, anything specific you'd like me test?

[XDIJK]

Just saw this error being thrown if the working tree is clean on the submodule but the parent repository working tree isn't:

Uncaught (in promise) Error: Entering 'submodule-name'
On branch main
Your branch is up to date with 'origin/main'.


nothing to commit, working tree clean
fatal: run_command returned non-zero status for submodule-name
.

    at Object.action (eval at <anonymous> (app.js:1), <anonymous>:2708:33)
    at PluginStore.exec (eval at <anonymous> (app.js:1), <anonymous>:2740:33)
    at eval (eval at <anonymous> (app.js:1), <anonymous>:3143:45)
    at new Promise (<anonymous>)
    at GitExecutorChain.handleTaskData (eval at <anonymous> (app.js:1), <anonymous>:3141:16)
    at GitExecutorChain.eval (eval at <anonymous> (app.js:1), <anonymous>:3125:46)
    at Generator.next (<anonymous>)
    at fulfilled (eval at <anonymous> (app.js:1), <anonymous>:3053:58)
    at processTicksAndRejections (internal/process/task_queues.js:93)

[Vinzent03]

Sorry for the delayed answer. @XDIJK Thanks a lot for testing!

Setting up the submodule update so that it doesn't checkout in a detached state but rebases git config -f .gitmodules submodule..update rebase
this can also be set to merge

Hmm that's weird, because I run git submodule update --recursive --remote --merge. I thought this would make the setting unnecessary.

Regarding your error I think it doesn't like to commit no changes. I though't that it is possible, because in non submodule that works. (I guess it still returns a non zero status, but that's otherwise no problem. The git submodule foreach though doesn't like that.)

[XDIJK]

@Vinzent03

Hmm that's weird, because I run git submodule update --recursive --remote --merge. I thought this would make the setting unnecessary.

Agreed I would have to test it on a completely fresh setup then it may have been an issue on my end.

Regarding your error I think it doesn't like to commit no changes. I though't that it is possible, because in non submodule that works. (I guess it still returns a non zero status, but that's otherwise no problem. The git submodule foreach though doesn't like that.)

I've been doing some googling and cannot find a way for foreach to return a non-zero exit code.
This happens for commit , push not for add,

I think the only way to achieve this is by parsing the list of sub modules using git config --file .gitmodules --get-regexp path or another command which you can get the path from read the results into an array then treat each submodule as a normal repository at least for commiting and pushing.

[Vinzent03]

Agreed I would have to test it on a completely fresh setup then it may have been an issue on my end.

Maybe your submodule was already detached? If you run git submodule update --init, it only checks the newest commit out and not the branch.

I didn't want to use the config file, because that wouldn't be recursive. I check for uncommited changes now in the foreach command itself. Nevertheless, thanks for the tip.

[XDIJK]

Thanks a bunch for implementing this feature!
Also clever solution to use further scripting within the foreach I hadn't considered that at all.

[eric-schleicher]

yes this is super awesome! thanks for @XDIJK for proposing and @Vinzent03 for implementing it. this makes obsidian so much more useful as I can have an integrated workflow over all obsidian content, but finely break out sections by audience. I love it. thanks!

[Yanksi]

Hi, would like to know if this feature is available on the mobile version of this plug-in? Seems that I am not able to see the Update submodules switch in the Obsidian Git Mobile plug-in.

[Vinzent03]

@Yanksi It's not supported by the underlying library isomorphic-git. Will add this to the README.

[rea1shane]

I check for uncommited changes now in the foreach command itself.

Hi, would like to know more details about your solution. Did you use git status in foreach? Or handling non-zero exceptions? Is these logic in commitAll({ message }) method? I can't read js code, so look forward to your explanation.

Thank you.

[rea1shane]

Hi @Vinzent03

I find a way to solve commit on no change submodule: Continue looping over submodules with the "git submodule foreach" command after a non-zero exit.

Did you done like that? If not hopefully it will give you some new inspiration.

[Vinzent03]

@rea1shane What exactly is your issue?

[rea1shane]

@rea1shane What exactly is your issue?

I'm sorry I didn't express clearly. I am new to submodules and would like to know how you handle non-zero status in git submodule foreach 'git commit', just that.

But just had a new issue, I can't stage submodule changes. Already enable update submodules.

I update README file in hahahaha submodule, and I can't stage it. Am I using the wrong way?

Obsidian: Version 1.0.0 (Installer 1.0.0)
obsidian-git: 2.8.0

[ArneNostitz]

Hi, I was wondering if this is only working folder based - or if this could also work on a tag-base or even a dataview query?

I have some work files I want to share - but they are scattered throughout my vault (folders: people, project, meeting-notes) but all of them have a "Project:: Project01" inline key ... so its not a subfolder but more a sub-set of files that i would like to push to a second repository for coworkers to get and update. and to have them ultimately sync back into my vault on request.
is that possible? thanks

[SmartSceptic]

I'm sorry I didn't express clearly. I am new to submodules and would like to know how you handle non-zero status in git submodule foreach 'git commit', just that.

I also use 2 repositories - submodules, and I have the same problem: at some point the repository stopped updating automatically.
Manual check through plugin commands, and side menu, showed the same problem as in the quoted message. Changes cannot be saved, as long as there are sub-module links among them.

Through the command line it still works fine...