Integrate with MTEB?

[Muennighoff]

Really cool work! In case you're interested I think it'd be great for the community & visibility of your project to have InstructIR available in MTEB and maybe also add a leaderboard tab for it 😊 We recently also integrated the amazing FollowIR work under Retrieval w/ Instructions in the LB: https://huggingface.co/spaces/mteb/leaderboard & in the code via this PR: embeddings-benchmark/mteb#408
cc @hanseokOh

[hanseokOh]

Thanks for having an interest on our work! Yes we would love to add our InstructIR to MTEB leaderboard.
How can we proceed it? @Muennighoff

[Muennighoff]

Great! The first step is to allow evaluation of InstructIR via MTEB. You would need to open a PR like one of these: LongEmbed ; Law ; FollowIR ; French ; Scandinavian. 🙌 We're happy to help along the way once the PR is open 😊

[hanseokOh]

Okay thanks for the description! I'll proceed it when we're ready :) Thanks for the suggestion 👍 I'll ping this thread when the PR is done.