Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add benchmarks endpoints to Orion Web API #996

Merged

Commits on Nov 30, 2022

  1. Add benchmarks/ to WebAPI

    bouthilx authored and notoraptor committed Nov 30, 2022
    Configuration menu
    Copy the full SHA
    39acaa9 View commit details
    Browse the repository at this point in the history
  2. Lazy build of Benchmark study and analysis filters

    Why:
    
    Building the benchmark object (setup_experiments in particular) can be
    quite expensive (>60s). When we only want to execute one particular
    analysis, the build does a lot of useless fetch on the storage. The
    experiments should be built only when they are needed.
    Relatedly, the analysis method should allow selecting specific study.
    
    How:
    
    Only call setup_experiments on study when we try to access
    `study.get_experiments()`. The task_ids which seemed useless are removed
    from `experiments_info` as well.
    bouthilx authored and notoraptor committed Nov 30, 2022
    Configuration menu
    Copy the full SHA
    b37c3c4 View commit details
    Browse the repository at this point in the history
  3. Add benchmarks/name to webapi

    bouthilx authored and notoraptor committed Nov 30, 2022
    Configuration menu
    Copy the full SHA
    87dda11 View commit details
    Browse the repository at this point in the history
  4. Remove Benchmark {algorithm: algo, deterministic:}

    Why:
    
    The interface is bloated and difficult to memorise. One of the reason we
    decided to go this way is that we previously needed to instantiate an
    algo to access its attributes. We can now use algo_factory.get_class()
    to have the class object only and access the class attributes. Thanks to
    this we can now simply set deterministic as a class attribute for the
    algorithms and it simplifies the interface of the benchmark.
    
    Algorithms have deterministic set to False by default since they
    generally have sources of variations.
    bouthilx authored and notoraptor committed Nov 30, 2022
    Configuration menu
    Copy the full SHA
    2233d96 View commit details
    Browse the repository at this point in the history
  5. Adapt to ParallelAssessment

    bouthilx authored and notoraptor committed Nov 30, 2022
    Configuration menu
    Copy the full SHA
    7f1f75a View commit details
    Browse the repository at this point in the history
  6. Add missing import

    notoraptor committed Nov 30, 2022
    Configuration menu
    Copy the full SHA
    c9ade74 View commit details
    Browse the repository at this point in the history
  7. in orion serve:

    - Add gunicorn settings from YAML to Gunicon app
    - increase gunicorn timeout to 300 seconds
    notoraptor committed Nov 30, 2022
    Configuration menu
    Copy the full SHA
    6b0314b View commit details
    Browse the repository at this point in the history
  8. Fix rebase

    notoraptor committed Nov 30, 2022
    Configuration menu
    Copy the full SHA
    b1c0f4b View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    28006df View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    028ff6a View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    aa30077 View commit details
    Browse the repository at this point in the history
  12. Reformat code

    notoraptor committed Nov 30, 2022
    Configuration menu
    Copy the full SHA
    a5f16f0 View commit details
    Browse the repository at this point in the history
  13. Configuration menu
    Copy the full SHA
    b539a2f View commit details
    Browse the repository at this point in the history
  14. Add debug logging

    notoraptor committed Nov 30, 2022
    Configuration menu
    Copy the full SHA
    85ef34f View commit details
    Browse the repository at this point in the history
  15. Run pre-commit

    notoraptor committed Nov 30, 2022
    Configuration menu
    Copy the full SHA
    3c44c63 View commit details
    Browse the repository at this point in the history
  16. more debug

    notoraptor committed Nov 30, 2022
    Configuration menu
    Copy the full SHA
    1bfa803 View commit details
    Browse the repository at this point in the history
  17. Configuration menu
    Copy the full SHA
    a13df40 View commit details
    Browse the repository at this point in the history
  18. Pass timeout to lock constructor and use lock instead of lock.acquire…

    … in with-statement, to make sure lock.acquire and lock.release are called
    notoraptor committed Nov 30, 2022
    Configuration menu
    Copy the full SHA
    9202581 View commit details
    Browse the repository at this point in the history
  19. Fix pylint

    notoraptor committed Nov 30, 2022
    Configuration menu
    Copy the full SHA
    0d806e9 View commit details
    Browse the repository at this point in the history
  20. debug experiments

    notoraptor committed Nov 30, 2022
    Configuration menu
    Copy the full SHA
    07f7c8e View commit details
    Browse the repository at this point in the history
  21. More debug pickledb lock

    notoraptor committed Nov 30, 2022
    Configuration menu
    Copy the full SHA
    9e313b4 View commit details
    Browse the repository at this point in the history
  22. debug get database

    notoraptor committed Nov 30, 2022
    Configuration menu
    Copy the full SHA
    18c132d View commit details
    Browse the repository at this point in the history
  23. more debug db info

    notoraptor committed Nov 30, 2022
    Configuration menu
    Copy the full SHA
    c01db7f View commit details
    Browse the repository at this point in the history
  24. debug with dill

    notoraptor committed Nov 30, 2022
    Configuration menu
    Copy the full SHA
    b77d622 View commit details
    Browse the repository at this point in the history
  25. Configuration menu
    Copy the full SHA
    e2a24fe View commit details
    Browse the repository at this point in the history
  26. revert dill

    notoraptor committed Nov 30, 2022
    Configuration menu
    Copy the full SHA
    0bd5b37 View commit details
    Browse the repository at this point in the history
  27. Configuration menu
    Copy the full SHA
    680dc33 View commit details
    Browse the repository at this point in the history
  28. Configuration menu
    Copy the full SHA
    124168b View commit details
    Browse the repository at this point in the history
  29. Configuration menu
    Copy the full SHA
    4664602 View commit details
    Browse the repository at this point in the history
  30. Configuration menu
    Copy the full SHA
    71dae0a View commit details
    Browse the repository at this point in the history
  31. Fix tests

    notoraptor committed Nov 30, 2022
    Configuration menu
    Copy the full SHA
    4e37d0a View commit details
    Browse the repository at this point in the history

Commits on Dec 1, 2022

  1. Configuration menu
    Copy the full SHA
    a0590a0 View commit details
    Browse the repository at this point in the history