Lift the requirement of human fingering with RP1M

[zhaoyi11]

Lift the requirement of human fingering with RP1M

Motivation

To play piano with dexterous robot hands, Robopianist requires human-annotated fingering. However, the fingering information is only available for a limited set of songs. This poses challenges for playing new pieces. This PR implements the Optimal Transport (OT)-based fingering method from RP1M (https://arxiv.org/abs/2408.11048), allowing the agent to play MIDI files without relying on human provided fingering.

Dependencies

The OT-based fingering implementation requires linear_sum_assignment from scipy, which is already included in the project. No additional packages are needed.

Modifications

Changes were made to robopianist/suite/tasks/piano_with_shadow_hands.py. The OT-based fingering is used when human fingering is unavailable.

Performance

The implementation was tested on Twinkle Twinkle Litter Star with DroQ from https://github.com/kevinzakka/robopianist-rl/tree/main with default hyperparameters. Results are summarized in the table below:

	RP1M	RoboPianist w/ fingering	RoboPianist w/o fingering
F1 @3m	0.976	0.994	0.29
Recall @3m	0.963	0.990	0.247
Precision @3m	1	1	1
FPS	216	223	244

TODO

[ ] Add documentation on how to incorporate new songs.

[kevinzakka]

Thanks so much @zhaoyi11, this looks good to me!