Keeping corrected typos with CrowdAnki

[spechtjonas]

Hey guys,
first of: thanks for your work, it's amazing.

I don't know, if this is the right place, but i got a question:
When two people (A and B) collaborate on a deck and simultaneously correct typos on different notes, is there a way to sync both changes to the repository without losing any progress?
As I understand, the normal workflow would be to first import the deck to sync with any other changes people made. But by doing that, you would lose the changes you made to the content of notes, so all corrected typos would be reverted.
Looking at ultimate geographys github, i found that in order to preserve typo corrections, one is supposed to create a pull request.
As I only use git hub for crowdanki and don't know how to create a pull request for changes regarding certain notes (if that's how it works) I really would appreciate someone to enlighten my, if there are options to resolve this issue and if so, how you would use them.
Any help is much appreciated. Have a nice day!

[aplaice]

In brief: currently it's unfortunately not possible without manually using git, GitHub or a git/GitHub frontend. (We'd like to change that, but it'll take a while.)

---

Thanks for the question!

I don't know, if this is the right place

It's the best available place! :)

Looking at ultimate geographys github, i found that in order to preserve typo corrections, one is supposed to create a pull request.

Ultimate Geography uses brain-brew on top of CrowdAnki, so its approach is slightly different than what you could do, and I wouldn't base your workflow on the one it uses. (BrainBrew moves some of the complexity from the structure of the output — it's more human-readable than CrowdAnki's deck.json — to having yet another tool that you have to install and use.) Also, Ultimate Geography focuses on editing the deck in its "git"/"exported" form, while you want to be able to push changes edited within Anki. (The latter is also possible with BrainBrew but it's even more complicated.)

---

Suggestions

The following are obviously very, very far from convenient. The descriptions are based on CrowdAnki's Readme but differ from it, in some places. I'm focusing on the git CLI approach, as I've never used GitHub desktop.

I have no idea if it's not too convoluted for everyday use for people not familiar with git. I'm describing it in detail, since it might also help us (the CrowdAnki developers) as a specification for what we might want to automate, to make it easier for future users. :) (Implementing the improvements will take a while, though.)

Example workflow

I'll try to describe example workflows, for two people correcting typos. In both cases, I'm assuming that you already have a git repository (on GitHub, say at https://github.com/creator/deck — obviously replace this with a correct path), containing the deck.json.

From "scratch"

(From "scratch" means that we're assuming that the users haven't imported their decks, yet.)

Person A clones https://github.com/creator/deck/ in some directory /path/to/dir/:

cd /path/to/dir/
git clone https://github.com/creator/deck/

In Anki, they import the deck File > CrowdAnki: Import from disk and make their typo fixes.

In parallel, person B also clones https://github.com/creator/deck/, imports the deck and makes their typo fixes.

Person A then exports their deck (with CrowdAnki) back to the directory containing the git clone. In the git directory (in /path/to/dir/deck/), they add and commit their changes:

cd /path/to/dir/deck/
# The next line is not strictly necessary but will help avoid some
# issues.  It creates a branch, named `person_a_typo_fixes_1` — it
# could be any other name.
git checkout -b person_a_typo_fixes_1
git add '*'
git commit -m "Add person A's typo-fixes"

In order to push their changes they need a GitHub account, a fork of the deck on GitHub (just click Fork in the top right corner) and a way of authenticating on GitHub. They can then add their fork and push to it:

git remote add personA https://github.com/personA/deck/
git push personA --set-upstream person_a_typo_fixes_1

On GitHub they can then open a PR (Pull requests > New pull request > compare across forks > set "head repository" to personA/deck and compare to person_a_typo_fixes_1).

Person B can do the same thing (add, commit and push their changes):

cd /path/to/dir/deck/
# The next line is not strictly necessary but will help avoid some
# issues.  It creates a branch, named `person_b_typo_fixes_1` — it
# could be any other name.
git checkout -b person_b_typo_fixes_1
git add '*'
git commit -m "Add person B's typo-fixes" # This can be any message! :)

git remote add personB https://github.com/personB/deck/
git push personB --set-upstream person_b_typo_fixes_1

The manager/creator of the deck can then merge the two pull requests. If there are no merge conflicts (none of the same notes were modified by different people), this should be straightforward; if not, it might be messy. :(

(If the manager/creator of the deck is one of persons A or B, they can push directly to the main branch without opening PRs.)

Once the changes are merged, A and B can pull all the changes back from GitHub:

git checkout main # or master, the name of the main branch depends on your git version...
git pull origin

and import into Anki.

(If their changes haven't yet been merged upstream, but they want to get other people's changes that have been made since their previous import, they can do something like:

git pull origin # without first changing the branch

and import into Anki.)

Starting to use git mid-way

If git/GitHub wasn't used from the beginning of the sharing of the deck, then things are still salvageable.

The users A and B still git clone the deck, but they don't import into Anki from there since that would overwrite their changes. Instead, they just export their changes back into the git directory and push to GitHub.

Things get tricky (when merging) if the version of the deck that is in git is different from the version that A or B first imported. e.g. B imported the deck; the creator made some changes and published to GitHub; B then wants to contribute their own changes back. In that case, B's PR will contain both: 1. B's changes, 2. an undoing of the creator's changes. 2 will have to be manually edited out. :/

---

I hope that this is at all helpful!

[spechtjonas]

I could not have dreamed of a more profound answer, thank you so much for the write-up.

I think I understood most of it, but only testing will show, if I fully comprehend the workflow and underlying mechanics of git. As I am in the middle of studying, I hope to get around to testing this on the weekend.

I'd hit you up with any follow-up questions and feedback once I tried it out, if that's okay.

Thanks again, this was a huge help already!

[aplaice]

Thanks for the question — it motivated me to rethink the collaboration workflow with CrowdAnki from the Anki side and what are its pain-points (as mentioned, collaboration from the "git"/"repo" side, as in Ultimate Geography, is well developed, but it's far less convenient for end-users).

I'd hit you up with any follow-up questions and feedback once I tried it out, if that's okay.

Yes, of course!

As I am in the middle of studying

Good luck studying and in your exams!

[spechtjonas]

A quick update on my situation and the proposed workflow: It works really well, so far!

If done some testing, messing around with a 3 card test deck, using two seperate accounts, replicating two users collaborating on that deck. Merging the changes worked flawlessly (unless I messed up) and it's actually not that inconveniet imo.

Using GitHub Destkop, for a person collaborating on the deck I now broke it down to:

1. Create a fork (as you have to later anyways)
2. Clone the fork into GitHub Desktop (cloning to some directory)
3. Import the directory to Anki
4. Make changes
5. Export the directory
6. Create a new branch in GitHub Desktop
7. Commit changes to that branch
8. Create Pull Request
9. (Wait for changes to be accepted)
10. Update your fork
11. Import updated repo to Anki.

Honestly, once you get the hang of it, I think it's not to bad. For furthter updates, the person just has to repeat steps 3-10.

Probably more questions and issues will arise in the future, using a much more complicated deck, where more things can go wrong.
Thanks again for you help and explanations! If you have any further suggestions how to improve that workflow, please let me know. Maybe this little update is interesting regarding your ambitions to improve the anki-end user complatibility. :)

[aplaice]

Thanks for the update!

If you have any further suggestions how to improve that workflow, please let me know.

I don't think that with the current tooling things can be streamlined.

(The idea for improvements would be to roll 2-3 and 5-7 (and ideally also 1 and 8) into CrowdAnki. In principle 2-3 is already possible (CrowdAnki: Import git repository), but without allowing re-exporting to git* or giving easy access to the resulting git repository, it's only useful for "consumption", but not for "contributing back", i.e. it's not sufficient for collaboration purposes.)

* Technically snapshotting already allows exporting to git, but it's not geared for collaboration.

Merging the changes worked flawlessly (unless I messed up) and it's actually not that inconveniet imo.

Merging issues might start cropping up when people use different versions of Anki and when the differences between decks start becoming larger. (They shouldn't and hopefully won't, but I can't promise anything.)

[spechtjonas]

Update

Just to let you know: Merging worked fine for a 30.000+ card deck (at least once)!

Follow-up question

From my understanding, using a fork and a pull request, a contributer publishes only those changes, that are different between his fork and the new export from anki he made. Therefore, any changes made to the main repository are irrelevant, as only the difference on his side is checked and uploaded. In order for this to work porperly, whenever the fork ist updated to the current version of the main repository, the contributing person must import those changes to anki. Otherwise there will be issues on the next export as differences betwenn the now updated fork and the old version in anki, which were created by changes others commited in the mean time, will be interpreted as changes on the contributers side. On merge, those would revert these changes.

My question regards contributing as the creator: I, as the creator, import the repository and make some changes. In the mean time, others contribute changes, which I accept and merge with the main. Afterwards, I want upload my changes to the repository aswell, so I export my anki deck as respository. Now: As the main was changed inbetween my import to anki and the export now, wouldn't all changes made by others commited in the mean time be seen as differences to my exportet repository and thereby overwritten, when I push my changes to main?
I think I understood the concept for contributers very well, but I cant seem to wrap my head around the creator's side.

There is probably a very easy explanation, but with my very limited github knowledge I cant figure it out. Any help would be much appreciated! :)

[aplaice]

From my understanding, using a fork and a pull request, a contributer publishes only those changes, that are different between his fork and the new export from anki he made. Therefore, any changes made to the main repository are irrelevant, as only the difference on his side is checked and uploaded. In order for this to work porperly, whenever the fork ist updated to the current version of the main repository, the contributing person must import those changes to anki. Otherwise there will be issues on the next export as differences betwenn the now updated fork and the old version in anki, which were created by changes others commited in the mean time, will be interpreted as changes on the contributers side. On merge, those would revert these changes.

Yes, exactly!

My question regards contributing as the creator: I, as the creator, import the repository and make some changes. In the mean time, others contribute changes, which I accept and merge with the main. Afterwards, I want upload my changes to the repository aswell, so I export my anki deck as respository. Now: As the main was changed inbetween my import to anki and the export now, wouldn't all changes made by others commited in the mean time be seen as differences to my exportet repository and thereby overwritten, when I push my changes to main?

Yes, you're right — this is tricky, and my previous suggestion (of just pushing to main) was confusing and overall pretty terrible, as it could lead to various issues. Oops! Sorry!

(What I had been thinking was that you, as the creator, would keep your local main unchanged and just merge PRs in GitHub. Once you wanted to export your own Anki changes, again, you'd:

1. Export from Anki to your "local" main.
2. Try pushing from your "local" main to your GitHub main.
3. If that failed (due to there being changes in GitHub from contributors) you'd pull and merge the changes from GitHub into your "local" main, and then push to GitHub.

However, this is rather confusing and extremely error-prone — if you updated your "local" main from GitHub (as would generally make sense...) before exporting from Anki, then you'd have exactly the problem that you described, so on second thoughts, I don't recommend this workflow.)

Instead, I'd suggest that you keep a branch (say my_anki) that always tries to correspond to Anki, while merging all PRs to main. For instance, at the start, you have:

"local" main = GitHub main = "local" my_anki = Anki = v0

You (as creator of the deck and manager of the repository) merge some PRs from contributors (hence main changes, and let's assume you update both "local" main and "GitHub" main). In parallel, you make some changes in Anki.

"local" main = GitHub main = v_main_1
Anki = v_anki_1
"local" my_anki = v0
(v_main_1 ≠ v_anki_1 ≠ v0).

You switch your current local branch to my_anki, export from Anki, and "add" and commit the changes:

"local" main = GitHub main = v_main_1
"local" my_anki = Anki = v_anki_1

Here, you have two options. Either you can locally (in GitHub desktop) merge my_anki and main (merge one into the other, and back) and push main to GitHub or push from my_anki to GitHub, open a PR, merge the PR into main and update both local my_anki and main from GitHub main. In both cases you end up with:

"v_merged_1 = v_anki_1 + v_main_1"
"local" main = GitHub main = "local" my_anki = v_merged_1
Anki = v_anki_1

Finally, you import everything back into Anki:

"local" main = GitHub main = "local" my_anki = Anki = v_merged_1

I hope that this makes sense! (Also, apologies if I've made the explanations too long-winded.)

Sorry again for the confusion!

[spechtjonas]

Great solution, as always!

So basically, I would use a branch the way a contributer can use his fork, as both will not correspond directly to main, right? Put that way, it's actually not that complicated. I will let you now, if I run into any issues. Thanks!

[aplaice]

I would use a branch the way a contributer can use his fork, as both will not correspond directly to main, right?

Yes, that's the idea! :)