Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Experiment server #2132

Closed
wants to merge 9 commits into from
Closed

Experiment server #2132

wants to merge 9 commits into from

Conversation

sondreso
Copy link
Collaborator

@sondreso sondreso commented Oct 4, 2021

Issue
Resolves #1250

Approach

  1. Refactor out the tracking of an evaluation.
  2. Allow the evaluator to track multiple evaluations
  3. Add the concept of an experiment with its own status, containing multiple evaluations.
  4. Make the run models setup, start and finish the experiment at the correct times.
  5. Make experiment endpoint for clients (decide on "out-of-band" communication for experiments and evaluations vs all status in one channel)
  6. Make evaluator communication session
  7. Cleanup

@sondreso sondreso self-assigned this Oct 4, 2021
@sondreso sondreso marked this pull request as draft October 4, 2021 07:54
@sondreso
Copy link
Collaborator Author

sondreso commented Jan 3, 2022

Will reopen a new pull request in the future

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Authority on information about evaluations/experiments
1 participant