Code

Leaderboard#1235

Code

Merged

x-tabdeveloping merged 40 commits intomainembeddings-benchmark/mteb:mainfrom leaderboard_devembeddings-benchmark/mteb:leaderboard_devCopy head branch name to clipboardOct 21, 2024

+954-267

Commits

Commits on Sep 14, 2024

• Add leaderboard dev

  KennethEnevoldsen

  committed
  a363594

  Copy full SHA for a363594

Commits on Sep 23, 2024

• Renamed MTEBResults to TaskResult

  x-tabdeveloping

  committed
  aa50296

  Copy full SHA for aa50296

• Moved model and model meta loading utilities into overview.py

  x-tabdeveloping

  committed
  17b17f7

  Copy full SHA for 17b17f7

• Added get_model_metas to retrieve filtered metadata for models

  x-tabdeveloping

  committed
  2ae9392

  Copy full SHA for 2ae9392

Commits on Sep 24, 2024

• Restructured results object and made it into a class instead of a dict

  x-tabdeveloping

  committed
  5eb66f1

  Copy full SHA for 5eb66f1

• Added utilities for filtering models on BenchmarkResults objects

  x-tabdeveloping

  committed
  9f75bf5

  Copy full SHA for 9f75bf5

• Added to_table utility function to BenchmarkResults

  x-tabdeveloping

  committed
  f3103c1

  Copy full SHA for f3103c1

• Merge branch 'main' into leaderboard_dev

  x-tabdeveloping

  authored
  942b1a7

  Copy full SHA for 942b1a7

• Added serialization utilities to BenchmarkResults

  x-tabdeveloping

  committed
  bb1e364

  Copy full SHA for bb1e364

• Attempted fixing tests

  x-tabdeveloping

  committed
  24e0e3e

  Copy full SHA for 24e0e3e

• Added get_model_metas to __init__

  x-tabdeveloping

  committed
  bc1941e

  Copy full SHA for bc1941e

• Added get_benchmarks to __init__ and made it return all benchmarks by default

  x-tabdeveloping

  committed
  37e0e25

  Copy full SHA for 37e0e25

• Added get_benchmarks to __init__

  x-tabdeveloping

  committed
  f0fb326

  Copy full SHA for f0fb326

• Made tasks hashable

  x-tabdeveloping

  committed
  691380c

  Copy full SHA for 691380c

• Added task filtering based on task objects on BenchmarkResults

  x-tabdeveloping

  committed
  111cfd5

  Copy full SHA for 111cfd5

• Added BenchmarkResults to __init__

  x-tabdeveloping

  committed
  a84764c

  Copy full SHA for a84764c

Commits on Oct 2, 2024

• Added additional arguments to get_scores on two classes

  x-tabdeveloping

  committed
  50062e3

  Copy full SHA for 50062e3

Commits on Oct 14, 2024

• Made get_scores smarter on BenchmarkResult

  x-tabdeveloping

  committed
  5a8fa73

  Copy full SHA for 5a8fa73

• Added basic multilingual benchmark

  x-tabdeveloping

  committed
  31ac648

  Copy full SHA for 31ac648

• Modified benchmark to be able to easily access results

  x-tabdeveloping

  committed
  4332612

  Copy full SHA for 4332612

• Added useful properties and filtering functions to BenchmarkResults

  x-tabdeveloping

  committed
  3e17e4c

  Copy full SHA for 3e17e4c

• Added minimal functioning example

  x-tabdeveloping

  committed
  e7ca3f8

  Copy full SHA for e7ca3f8

Commits on Oct 16, 2024

• Added smarter table, task-list updating and tried fixing dropdown scrolling

  x-tabdeveloping

  committed
  0d1d450

  Copy full SHA for 0d1d450

• Merge branch 'main' into leaderboard_dev

  x-tabdeveloping

  authored
  e8cca9d

  Copy full SHA for e8cca9d

Commits on Oct 17, 2024

• Made restrict_results into a private function

  Show description for 266394c
  

  x-tabdeveloping

  and

  KennethEnevoldsen

  authored
  266394c

  Copy full SHA for 266394c

• Removed old leaderboard scripts

  x-tabdeveloping

  committed
  327c8d6

  Copy full SHA for 327c8d6

• Hardcoded max and min model size

  x-tabdeveloping

  committed
  9ec49fb

  Copy full SHA for 9ec49fb

• Removed redundant utils file

  x-tabdeveloping

  committed
  ce2569d

  Copy full SHA for ce2569d

• Ran linting

  x-tabdeveloping

  committed
  228e7d3

  Copy full SHA for 228e7d3

• added leaderboard dependencies as optional

  x-tabdeveloping

  committed
  bee9e41

  Copy full SHA for bee9e41

• Fixed union type error on Python 3.9

  x-tabdeveloping

  committed
  5e6a42e

  Copy full SHA for 5e6a42e

• Removed references to Dict in task aggregation

  x-tabdeveloping

  committed
  781ee95

  Copy full SHA for 781ee95

Commits on Oct 18, 2024

• Fixed name errors in _restrict_task_results

  x-tabdeveloping

  committed
  ae5afb7

  Copy full SHA for ae5afb7

• Fixed _restrict_task_results

  x-tabdeveloping

  committed
  ca5014c

  Copy full SHA for ca5014c

• Made hf_subsets={'default'} when the task is monolingual in _restric_task_results

  x-tabdeveloping

  committed
  cb11921

  Copy full SHA for cb11921

• Task dropdown now gets filtered based on the other criteria

  x-tabdeveloping

  committed
  9fac012

  Copy full SHA for 9fac012

Commits on Oct 21, 2024

• Ran linting again

  x-tabdeveloping

  committed
  006b845

  Copy full SHA for 006b845

• Introduced hotfix for reranking test

  x-tabdeveloping

  committed
  dcca04d

  Copy full SHA for dcca04d

• Added BenchmarkResults to __all__ in __init__

  x-tabdeveloping

  committed
  0bf3746

  Copy full SHA for 0bf3746

• Fixed validate_and_filter_scores method, and replaced _restric_task_results with it

  x-tabdeveloping

  committed
  607c998

  Copy full SHA for 607c998