Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

email fix for score_plugins workflow #581

Merged
merged 3 commits into from
Feb 29, 2024

Conversation

samwinebrake
Copy link
Collaborator

Related to closed PR #579

Broke that PR up into two parts:

  1. (THIS PR) a fix for the current score_plugins workflow that finds brain-score emails based on uid and funnels these emails to jenkins for correct email reporting of submission success/failure
  2. (FOR A FUTURE PR) failure emails for failing unit testing jenkins jobs

Everything is tested and working, as long as secrets.AWS_ACCESS_KEY_ID and secrets.AWS_SECRET_ACCESS_KEY are set within vision.

(See previous comments from here for input on choices made in this PR.)

Copy link
Contributor

@kvfairchild kvfairchild left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some changes requested but (as long as this is fully tested), looks good to me.

As we're adding logic for notifying the submitter on failure across multiple actions, you may want to look into reusable workflows to centralize the logic and reduce duplication.

.github/workflows/score_new_plugins.yml Outdated Show resolved Hide resolved
.github/workflows/score_new_plugins.yml Show resolved Hide resolved
.github/workflows/score_new_plugins.yml Outdated Show resolved Hide resolved
samwinebrake and others added 2 commits February 29, 2024 10:33
@samwinebrake
Copy link
Collaborator Author

samwinebrake commented Feb 29, 2024

Some changes requested but (as long as this is fully tested), looks good to me.

As we're adding logic for notifying the submitter on failure across multiple actions, you may want to look into reusable workflows to centralize the logic and reduce duplication.

Good point @kvfairchild ! I'll merge this for now so we have emails going out to users and prioritize creating a reusable user-notifier action that can be used for this and other CI results.

@samwinebrake samwinebrake merged commit 3614ff7 into master Feb 29, 2024
9 checks passed
@samwinebrake samwinebrake self-assigned this Mar 5, 2024
samwinebrake added a commit that referenced this pull request Mar 8, 2024
* email fix for score_plugins workflow (#581)

* Emails passed to jenkins

* Update .github/workflows/score_new_plugins.yml

Co-authored-by: Katherine Fairchild <kvg0@mit.edu>

* fix deprecated set output

---------

Co-authored-by: Katherine Fairchild <kvg0@mit.edu>

* add eBarlow_Vanilla to models (#587)

Co-authored-by: AutoJenkins <brainscore@info.org>

* add eBarlow_lmda_01 to models (#588)

Co-authored-by: AutoJenkins <brainscore@info.org>

* add cornet_s_0_0_0 to models (#589)

Co-authored-by: AutoJenkins <brainscore@info.org>

* add unet_ethan to models (#590)

Co-authored-by: AutoJenkins <brainscore@info.org>

* add unet_ethan to models (#591)

Co-authored-by: AutoJenkins <brainscore@info.org>

* Revert "add unet_ethan to models (#591)" (#594)

This reverts commit c89059e.

* Revert "add unet_ethan to models (#590)" (#595)

This reverts commit 9a8774a.

* Transfer plugin information by artifact for scoring runs (#592)

* upload and download json artifact between jobs

* fix testing changes

* brain-score.org (user:514) | Add new plugin(s): models: [cornet_s_0_0_0 cornet_s_0_1_0]  | (public:False) (#603)

* add cornet_s_0_0_0 to models

* add cornet_s_0_1_0 to models

---------

Co-authored-by: AutoJenkins <brainscore@info.org>

* Add microsaccades: interface and default model helper (#492)

* integrate brain-score/model-tools#75 to brain-score/vision

* add exception for when temp file write fails

* fix error with tf temp file management

* add bandaid to a DataAssembly index problem

* move microsaccades to the model side

* update comments

* remove indexing bug

* fix bug with activations.shape

* Apply suggestions from code review

Co-authored-by: Martin Schrimpf <mschrimpf@users.noreply.github.com>

* Delete brainscore_vision/data/scialom2024/__init__.py

* address review changes

* remove needless import

* fix bug with temporary file handling test

* assume number_of_trials=1 and require_variance=False when getting stored activations

* fix bug with access to ActivationsExtractorHelper.set_visual_degrees

* move extractor calls to ModelCommitment generic

* add check for whether activations_model exists

* fix bug with TestVisualDegrees

* add link to BrainModel issue

* remove shifts from stimulus set packaging

* change link signatures

* change microsaccade call signature

* microsaccades are now computed on both a pixel and degree basis

* Apply suggestions from code review

Co-authored-by: Martin Schrimpf <mschrimpf@users.noreply.github.com>

* fix outdated comments, type hints, etc.

* change function call to reduce repetition

* add kwargs to microsaccade helpers

* refactor microsaccade usage to their own class to improve readability

* refactor microsaccade coords into MicrosaccadeHelper

* refactor microsaccade building

* change the way MultiIndex is set

* fix tf/pytorch/keras bug with image shape calculation

* cv2 reshaping in translate

* add test for exact microsaccades

* fix microsaccade indexing

* rename test to be in line with current naming

* add require_variance to _from_paths_stored

* reduce unnecessarily long test times by reducing the number of trials tests run for, while keeping the test the same

---------

Co-authored-by: Martin Schrimpf <mschrimpf@users.noreply.github.com>

* Use MajajHong2015public.IT instead of mock data (#585)

* Changed _MockBenchmark to new format

* revert ceiler back to martin's suggestion

* implemented MockBenchmark to use MJ2015 Public

---------

Co-authored-by: Katherine Fairchild <kvg0@mit.edu>
Co-authored-by: AutoJenkins <brainscore@info.org>
Co-authored-by: Ben Lonnqvist <ben.lonnqvist@epfl.ch>
Co-authored-by: Martin Schrimpf <mschrimpf@users.noreply.github.com>
Co-authored-by: Michael Ferguson <mferg@mit.edu>
@deirdre-k deirdre-k deleted the sw/score_plugins_emails_fix branch June 12, 2024 18:57
@deirdre-k deirdre-k restored the sw/score_plugins_emails_fix branch June 12, 2024 18:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants