Skip to content

V2.0.1 REMI+ and new Chord params

Compare
Choose a tag to compare
@Natooz Natooz released this 18 Mar 09:48
· 237 commits to main since this release

Changes

  • e26b088 from @atsukoba + help from @muthissar: REMI+ is now implemented! 🎉 This multitrack tokenization can be seen as an extension of REMI.
  • 2962211 Chord tokens can now represent the root note within tokens (versus only chord quality previously). Chord parameters have to be specified in additional_tokens argument, with the keys chord_maps, chord_tokens_with_root_note and chord_unknown. You can use the default value as an example.
  • e402b0d _in_as_seq decorator now automatically checks if the input ids are encoded with BPE
  • 2064ee9 fix with BPE containing spaces in merges, could not load tokenizers after training

Compatibility

  • due to 2064ee9, bytes and merges are shifted from v2.0.0. BPE tokenizers will be incompatible and would have to be retrained, or the bytes from their vocabularies and merges would have to be shifted. This only applies for BPE.