Code

Add sparse objects benchmark#536

Code

Merged

mschrimpf merged 97 commits intobrain-score:masterbrain-score/vision:masterfrom benlonnqvist:sparse_objectsbenlonnqvist/brain-score:sparse_objectsCopy head branch name to clipboardJun 12, 2024

+1,386

Commits

Commits on Jan 10, 2024

• integrate �brain-score/model-tools#75 to brain-score/vision

  benlonnqvist

  committed
  4690626

• Merge branch 'brain-score:master' into vision_microsaccades

  benlonnqvist

  authored
  26ed57a

Commits on Jan 15, 2024

• retrigger

  benlonnqvist

  committed
  1fee011

Commits on Jan 16, 2024

• retrigger

  benlonnqvist

  committed
  26313aa

Commits on Jan 17, 2024

• add exception for when temp file write fails

  benlonnqvist

  committed
  71ba207

• fix error with tf temp file management

  benlonnqvist

  committed
  3ea9a9c

Commits on Jan 18, 2024

• add bandaid to a DataAssembly index problem

  benlonnqvist

  committed
  3203e7b

Commits on Jan 24, 2024

• retrigger

  benlonnqvist

  committed
  a2e0a15

Commits on Feb 9, 2024

• initial commit

  benlonnqvist

  committed
  c6974a2

• Merge branch 'brain-score:master' into sparse_objects

  benlonnqvist

  authored
  a636742

• add data packaging

  benlonnqvist

  committed
  c4eb6b3

• revert inadvertent changes from another branch

  benlonnqvist

  committed
  d375873

• revert an inadvertent change from another branch

  benlonnqvist

  committed
  9ae0d9c

• revert inadvertent change from another branch

  benlonnqvist

  committed
  850c073

Commits on Feb 19, 2024

• Merge branch 'brain-score:master' into sparse_objects

  benlonnqvist

  authored
  9c425c0

Commits on Feb 26, 2024

• repackage data in a more readable format

  benlonnqvist

  committed
  df1a386

• Merge branch 'sparse_objects' of https://github.com/benlonnqvist/brain-score into sparse_objects

  benlonnqvist

  committed
  d42aa68

• Merge branch 'brain-score:master' into sparse_objects

  benlonnqvist

  authored
  b2d0d78

• add check for perfect accuracy in observed_consistency, that existed in the original implementation

  benlonnqvist

  committed
  40793d4

• Merge branch 'sparse_objects' of https://github.com/benlonnqvist/brain-score into sparse_objects

  benlonnqvist

  committed
  3fdf742

Commits on Feb 28, 2024

• add accuracy distance metric

  benlonnqvist

  committed
  3b82aeb

Commits on Feb 29, 2024

• add accuracy_distance ceilings

  benlonnqvist

  committed
  187f504

• rename benchmark

  benlonnqvist

  committed
  97a60e2

Commits on Mar 4, 2024

• add composite error consistency benchmark and engineering benchmark

  benlonnqvist

  committed
  3139434

• fix naming errors

  benlonnqvist

  committed
  d71f3ae

Commits on Mar 5, 2024

• add some benchmark tests

  benlonnqvist

  committed
  1089cf5

• add benchmark tests and test values

  benlonnqvist

  committed
  0241dbe

Commits on Mar 6, 2024

• add metric tests

  benlonnqvist

  committed
  bb81ae0

• correct accuracy distance computation, and the corresponding ceilings and expected raw scores

  benlonnqvist

  committed
  32b820f

Commits on Mar 12, 2024

• add benchmarks to s3

  benlonnqvist

  committed
  e69bf05

Commits on Mar 18, 2024

• Merge branch 'brain-score:master' into sparse_objects

  benlonnqvist

  authored
  fc65bed

• add some hashes

  benlonnqvist

  committed
  19ec821

Commits on Mar 20, 2024

• add assembly and stimulus hashes, versions, etc.

  benlonnqvist

  committed
  fa0fd1f

• Merge branch 'sparse_objects' of https://github.com/benlonnqvist/brain-score into sparse_objects

  benlonnqvist

  committed
  b1f374b

• Merge branch 'brain-score:master' into sparse_objects

  benlonnqvist

  authored
  3476320

• Merge branch 'master' into sparse_objects

  benlonnqvist

  authored
  806b861

• Merge branch 'master' into sparse_objects

  benlonnqvist

  authored
  b1bc727

Commits on Mar 22, 2024

• Merge branch 'master' into sparse_objects

  benlonnqvist

  authored
  59e5def

Commits on Mar 26, 2024

• Merge branch 'master' into sparse_objects

  benlonnqvist

  authored
  c47c75d

• Merge branch 'master' into sparse_objects

  benlonnqvist

  authored
  eb0d61a

• add correct bucket path for .nc files

  benlonnqvist

  committed
  5cc691c

• Merge branch 'master' into sparse_objects

  benlonnqvist

  authored
  8350c11

Commits on Mar 27, 2024

• fix metric tests

  benlonnqvist

  committed
  679c00a

• Merge branch 'sparse_objects' of https://github.com/benlonnqvist/brain-score into sparse_objects

  benlonnqvist

  committed
  92c96db

• fix bug with error consistency naming

  benlonnqvist

  committed
  cc6913a

Commits on Mar 30, 2024

• Merge branch 'master' into sparse_objects

  benlonnqvist

  authored
  6177803

Commits on Apr 2, 2024

• Merge branch 'master' into sparse_objects

  benlonnqvist

  authored
  6caefd1

Commits on Apr 3, 2024

• Merge branch 'master' into sparse_objects

  benlonnqvist

  authored
  12c8787

• correct benchmark identifiers

  benlonnqvist

  committed
  a9e0dae

Commits on Apr 5, 2024

• Merge branch 'master' into sparse_objects

  benlonnqvist

  authored
  1109093

Commits on Apr 8, 2024

• remove DATASETS that no longer exists

  benlonnqvist

  committed
  0c60660

• Merge branch 'sparse_objects' of https://github.com/benlonnqvist/brain-score into sparse_objects

  benlonnqvist

  committed
  f711cde

Commits on Apr 9, 2024

• Merge branch 'master' into sparse_objects

  benlonnqvist

  authored
  5509082

Commits on May 1, 2024

• Merge branch 'master' into sparse_objects

  benlonnqvist

  authored
  55cf8e2

Commits on May 8, 2024

• Merge branch 'master' into sparse_objects

  benlonnqvist

  authored
  6926b1c

• fix test bug

  benlonnqvist

  committed
  02fa379

• remove diffs to main

  benlonnqvist

  committed
  a2e78cf

• Merge branch 'sparse_objects' of https://github.com/benlonnqvist/brain-score into sparse_objects

  benlonnqvist

  committed
  e2f3237

• resolve conflict

  benlonnqvist

  committed
  186c8fe

• resolve conflict

  benlonnqvist

  committed
  4c8179c

• Merge branch 'master' into sparse_objects

  benlonnqvist

  authored
  446a1f0

Commits on May 30, 2024

• Merge branch 'master' into sparse_objects

  benlonnqvist

  authored
  1cf05dc

• schroedinger's benchmark registry: only by observing can you know whether it recognizes the benchmarks or not

  benlonnqvist

  committed
  52f448f

• remove benchmark initialization from registry

  benlonnqvist

  committed
  41ba2ec

• maybe fix issue with dataset loading

  benlonnqvist

  committed
  cf6cf19

Commits on May 31, 2024

• Merge branch 'master' into sparse_objects

  benlonnqvist

  authored
  8c075ce

Commits on Jun 4, 2024

• Merge branch 'master' into sparse_objects

  benlonnqvist

  authored
  5bf73de

Commits on Jun 5, 2024

• add stimulus prefix to stimulus sets

  benlonnqvist

  committed
  1fdc923

• change stimulus sets and data assemblies to have type(stimulus_id) == int, since saving and loading stimulus sets does not respect data type

  benlonnqvist

  committed
  b85445a

• change the .nc bucket

  benlonnqvist

  committed
  8bb0a41

• image_id => stimulus_id

  benlonnqvist

  committed
  2802674

• change composite to all

  benlonnqvist

  committed
  727f2e5

Commits on Jun 6, 2024

• Merge branch 'master' into sparse_objects

  benlonnqvist

  authored
  bede2ad

• data types

  benlonnqvist

  committed
  fbaf4db

• Merge branch 'sparse_objects' of https://github.com/benlonnqvist/brain-score into sparse_objects

  benlonnqvist

  committed
  31e5913

Commits on Jun 7, 2024

• change s3 .nc directory

  benlonnqvist

  committed
  42693fc

• .nc

  benlonnqvist

  committed
  9b773b1

• another .nc

  benlonnqvist

  committed
  ec63b78

• .nc now on bucket brainscore-vision

  benlonnqvist

  committed
  c1962bf

• remove / from remote_filepath?

  benlonnqvist

  committed
  a89daa4

• assembly -> target

  benlonnqvist

  authored
  b50c78f

• subject_assembly -> target

  benlonnqvist

  authored
  5a47c71

• assembly -> target

  benlonnqvist

  authored
  8a72940

Commits on Jun 8, 2024

• fix error with accuracydistance benchmark

  benlonnqvist

  committed
  84941a3

• fix raw score error with accuracydistance

  benlonnqvist

  committed
  d7b9083

• ensure score <= 1.0

  benlonnqvist

  committed
  ac0642d

• fix lazyload bug with stimulusset

  benlonnqvist

  committed
  2705fe5

• Merge branch 'master' into sparse_objects

  benlonnqvist

  authored
  2f4b29f

• add raw score to engineering benchmark

  benlonnqvist

  committed
  1cdaec2

• fix recursion in score calculation and fix expected scores

  benlonnqvist

  committed
  b7f3c08

• fix .nc name error in test

  benlonnqvist

  committed
  0c339a7

• nc filename retrieval

  benlonnqvist

  authored
  e9a2572

Commits on Jun 12, 2024

• @pytest.mark.private_access

  benlonnqvist

  authored
  c44fb0b

• ugly code -> pretty code

  benlonnqvist

  authored
  a5665f9

• remove unnecessary newline addition

  benlonnqvist

  authored
  3d28874

• add newline back

  benlonnqvist

  authored
  e93643f

• Merge branch 'master' into sparse_objects

  benlonnqvist

  authored
  0515591