You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
e26b088 from @atsukoba + help from @muthissar: REMI+ is now implemented! 🎉 This multitrack tokenization can be seen as an extension of REMI.
2962211Chord tokens can now represent the root note within tokens (versus only chord quality previously). Chord parameters have to be specified in additional_tokens argument, with the keys chord_maps, chord_tokens_with_root_note and chord_unknown. You can use the default value as an example.
e402b0d_in_as_seq decorator now automatically checks if the input ids are encoded with BPE
2064ee9 fix with BPE containing spaces in merges, could not load tokenizers after training
Compatibility
due to 2064ee9, bytes and merges are shifted from v2.0.0. BPE tokenizers will be incompatible and would have to be retrained, or the bytes from their vocabularies and merges would have to be shifted. This only applies for BPE.