Feature Request - Ignore Field on Import/Export

[ohare93]

I love the collaboration that this add-on allows, but there is one thing that is stopping me from being able to share my decks with others. I want to begin sharing some of my personal language learning decks with others, however I use images in my cards. Using images is wonderful for memory, there's nothing better. But when my deck is filled to the brim with copyrighted images simply taken from Google Images for personal use, this makes my decks completely unshareable. Of course it would be better if I had copyright free images (which I do for some), but finding that for thousands of notes, especially vague concepts, is incredibly challenging.

Potential Solution: The ability to flag any field(s) so that their value will not be exported/imported. This would allow each user to set their own picture, while still being able to import the deck when it is upgraded without their manually set "Picture" field being overwritten. Exporting without certain fields would be nice, but I will be using other deck managers like AnkiDM, so Importing really is the main issue.

I imagine this could be implemented for importing in many ways.

1. State that the field should not import in the Note Model. The field should still be taken into account so you can match it to the correct Note Type, but it's value will not be put into the Note itself (thus leaving anything the user has set changed).

  "note_models": [
    {
      "__type__": "NoteModel",
      "crowdanki_uuid": "43e2586a-9a65-11e8-a777-a0481cc15658",
      "name": "Ultimate Geography",
      "flds": [
        {
        ...
        {
          "name": "Map",
          "ord": 7,
          "font": "Arial",
          "media": [],
          "rtl": false,
          "size": 20,
          "sticky": false
          "doNotImport": true        <-----------------------------
        }
      ],

2. State that the specific card does not overwrite specific fields, with some type of generic flag which CrowdAnki reads on Import.

  "notes": [
    {
      "__type__": "Note",
      "data": "",
      "fields": [
        "England",
        "Constituent country of the United Kingdom.",
        "London",
        "",
        "Constituent country",
        "<img src=\"ug-flag-england.svg\" />",
        "",
        false                 <----------------------------------
      ],
      "flags": 0,
      "guid": "e+/O]%*qfk",
      "note_model_uuid": "43e2586a-9a65-11e8-a777-a0481cc15658",
      "tags": [
        "UG::Europe"
      ]
    },

This feature would be useful in a number of different ways, not just stopping the sharing of copyrighted images. In the Ultimate Geography deck, for instance, there could be a field "Personal Notes" with the users own memory or personal helpful tip on that country. Everyone surely has these personal thoughts which could aid them in recall, but in a shared deck like this it is impossible to add in your own, without the risk of losing them when you next upgrade. With this feature, no longer will you be stuck deciding one or the other!

Thanks again for your great work with this add-on, it's a real boon to the community! 👍

[ohare93]

I feel like taking a crack at this issue myself, due to this being a major blocker for some of my other goals. As such I have forked this repo, and intended to have a look into this.

However I am a bit confused by the repos intended workflow. Where do your sync the folder to locally? I see the Anki-Addon files are in the crowd-anki folder, but do you store this in the addons folder og anki? I thought the top level folder must have an init.py to be recognised. If you don't store it there, how do you sync changes you make? Also, must I uninstall the production version of CrowdAnki first?

A contribution document would be helpful. Have I overlooked something like this? Thanks 👍

[Stvad]

Hey @ohare93! Sounds good! Looking forward to your contribution! Please add here what approach you're planning to take so we'd be on one page for it.

Re contribution document - it sounds like a good idea to have one :). I'll look into creating one over the weekend.

To get you started though:

1. Checkout repo
2. run package_plugin.sh
3. Remove the production version of the plugin from Anki
4. Add a symlink to the Anki plugins folder pointing to crowd_anki directory

At this point if you start Anki - it'd be using your development version of CrowdAnki. To test new changes you've made - you need to restart Anki.

If you want to add new library dependency - add them to the Pipfile and re-run package_plugin.sh

Unit tests:

• Install required dependencies: pipenv install --dev
• Run pipenv run mamba ./
• Add more to https://github.com/Stvad/CrowdAnki/tree/master/test 😉

[ohare93]

Ah, Linux! My mortal enemy 😅 well I have been meaning to get round to making a Linux VM, perhaps this will be the thing that kicks me into gear. Thank you though, this was very useful 👍

[Stvad]

You don't actually need Linux for this. I presume you use Windows? In that case you only need to install Pipenv and Bash for Windows (the latter is optional, but makes things easier)

[Stvad]

Created https://github.com/Stvad/CrowdAnki/blob/master/CONTRIBUTING.md let me know if you I think I should add something else there/explain something in more details

[ohare93]

Hey mate, sorry for the delay, been busy.

You should mention in your CONTRIBUTING.md that you need to make a symlink to a folder in the add-ons folder which is not named "crowd_anki" 😓 this simple error on my part (as I just named the symlink the same as the original folder) caused countless wasted hours, and the add-on would do nothing (but it loaded and was in the add-on list), as the first line in __init__.py checking if tests are running:

if __name__ != "crowd_anki":

So anyways since I got past that hiccup, I am now getting some requirement errors. Specifically for a module named multiprocessing not existing.

I have ran pipenv install --dev and fetch_dependencies.sh many times with no errors. Any help?

[Stvad]

You should mention in your CONTRIBUTING.md that you need to make a symlink to a folder in the add-ons folder which is not named "crowd_anki"

Ouch. I wonder how I never hit that one. Sorry about that, updated the CONTRIBUTING

multiprocessing is a part of standard library so it should be present 🤔. I remember hitting a problem when some other standard modules were not included in the Anki's distribution of Python (very annoying). I suspect it can be the case here with Anki packaging a custom distribution of Python for Win.

I thought things improved on this front (didn't hit any of those since migrating to 2.1). It's an avenue for investigation, I'll look more into it on the weekends.

[Stvad]

https://apps.ankiweb.net/docs/addons.html#standard-modules this 👿

[Stvad]

Wait, the prod version of CrowdAnki works for you, right?

[ohare93]

Oh, I should have mentioned: due to this project and a few others requiring a lot of requirements that were a big hassle to setup on my Windows machine I have created a Linux dual boot (specifically Kubuntu 18.04). So all of this work and future work will be done in there.

Wait, the prod version of CrowdAnki works for you, right?

Yes of co-... No... It does not! 🤔 strange, I never even considered that, as I've not installed any prod versions of add-ons on this development machine. So there is something missing on my machine? 🤔 I have Python installed (2.7 and 3.6) and running import multiprocessing in the command line works fine for each of them 😖

Will keep trying to fix, but any help would be greatly appreciated!

[Stvad]

I see. I haven't heard Linux people complaining about this. So I wonder what's happening here 🤔

Did you install your anki instance using the system's package manager (i.e. something like apt-get install anki ) or by downloading it from anki website? If you went with the second option - can you try the first one?

[ohare93]

Hey, that fixed it! Installing with sudo apt-get install anki then a quick sudo apt-get install python3-distutils fixed my issues. Thank you 👍 Now I can actually program 😅

[Stvad]

🎉

[Stvad]

If you have those images in github repo won't that still be considered personal use? I.e. you're not charging money for access or anything like that

[ohare93]

Good question, I am not an expert in this, however I believe it is still an issue. I would still sharing those images to others, especially if I wanted to upload the deck to AnkiWeb 😨 When sharing on AnkiWeb you have to tick a box which states:

Copyright
I declare under penalty of perjury that the material I am sharing is entirely my own work, or I have obtained a license from the intellectual property holder(s) to share it here.

🤔

[Stvad]

Scary 😛

[ohare93]

Alright, I'll pick this back up in a few weeks time 👍 saying it here to commit myself 😅

[ukanuk]

This feature would be useful in a number of different ways, not just stopping the sharing of copyrighted images. In the Ultimate Geography deck, for instance, there could be a field "Personal Notes" with the users own memory or personal helpful tip on that country. Everyone surely has these personal thoughts which could aid them in recall, but in a shared deck like this it is impossible to add in your own, without the risk of losing them when you next upgrade. With this feature, no longer will you be stuck deciding one or the other!

+1000! I have a kanji learning deck that I would to convert to shared GitHub deck, but it also has several fields for personal thoughts/notes that shouldn't be imported or overwritten!

A third potential implementation method (could be in addition to methods 1 & 2 above):
3. An import dialogue where users manually select which fields should overwrite their personal collection (the note model and/or card skip flags could still be used, where note model would set the default for which fields are/aren't imported, but users have checkboxes to overwrite this if they want to change what's imported one time).

Having the import easily configurable like this would be valuable to me for my kanji deck, because I have a "kanji meaning" field which should be initialized to default values when users first install the deck, but should be able to be changed by users afterwards without fear of getting overwritten. Using a one-word meaning to learn kanji is widely accepted even though kanji all have multiple meanings, but not everyone agrees on what that one word ought to be. So it's useful to be able to overwrite some set of default values, but also to be able to skip overwriting any fields which users may otherwise want to customize.

If an import dialogue were created, it would additionally be awesome to have a checkbox for whether card templates are overwritten, and for whether the options/configuration group is overwritten (as would be desirable per #61 and #31)

[ohare93]

@ukanuk

A third potential implementation method (could be in addition to methods 1 & 2 above):

3. An import dialogue where users manually select which fields should overwrite their personal collection

Giving users this override option is definitely the plan 👍 But I do believe that the setting of personal fields by deck creators is a necessity for this feature to function properly, otherwise users must correctly select the personal fields each time. Do it wrong once and don't notice right away and you lose all your personal information! 😱

So it's useful to be able to overwrite some set of default values, but also to be able to skip overwriting any fields which users may otherwise want to customize.

An already working feature in my current shitty test implementation is for a field to be marked personal, but also to have a value. What that would do is that it takes the value for new cards and does not take the value (i.e. keeps it personal) for existing cards. That would achieve what you want, but I agree there are other scenarios not covered by this that the personal user option may cover.

If an import dialogue were created, it would additionally be awesome to have a checkbox for whether card templates are overwritten, and for whether the options/configuration group is overwritten (as would be desirable per #61 and #31)

Cannot agree more 👏

I'll see how far I can push this functionality into an actual feature this weekend. I'll update my PR regardless, so others can play with if they like 👍

[ohare93]

Fyi for anyone interested, I have been linked to the Special Fields add-on, which implements the same general feature I am trying to accomplish with this PR in CrowdAnki. You mark fields so that their value will not be overwritten.

However it is rather limited 😓 As you can see in this video (12 mins in) there is no gui, and one must edit the protected fields in the code of the add-on each time a different set of fields need be protected 😞

It is heartening to see that others are trying to accomplish the same goals, and that people are using the Special Fields add-on with enthusiasm 😁 as my current implementation in #72 is much more user friendly with a proper gui, I am certain that people will use this feature 👍