Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up tests #977

Merged
merged 15 commits into from
Oct 29, 2020
Merged

Speed up tests #977

merged 15 commits into from
Oct 29, 2020

Commits on Oct 27, 2020

  1. Cache _list_all we don't need the latest list

    The test does not require the list of flows to be updated, to a single
    cached version will do fine (this call otherwise would take ~40
    seconds).
    PGijsbers committed Oct 27, 2020
    Configuration menu
    Copy the full SHA
    9e3fb88 View commit details
    Browse the repository at this point in the history
  2. Reduce the amount of verified runs

    Downloading a run takes a non-significant amount of time (est. 300ms on
    my current setup). It is unnecessary to compare against all >=100 runs,
    while a handful should do fine (perhaps even just one should do).
    PGijsbers committed Oct 27, 2020
    Configuration menu
    Copy the full SHA
    bfb5304 View commit details
    Browse the repository at this point in the history
  3. Increase the batch size to avoid more than 2 pages

    The batch size required in some pages over 40 pages to be loaded, which
    increased the workload unnecessarily. These changing preserve pagination
    tests while lowering the amount of round trips required.
    PGijsbers committed Oct 27, 2020
    Configuration menu
    Copy the full SHA
    deddc04 View commit details
    Browse the repository at this point in the history
  4. Mark as test_get_run_trace as skip

    Since it is already covered by test_run_and_upload_randomsearch.
    PGijsbers committed Oct 27, 2020
    Configuration menu
    Copy the full SHA
    480cee0 View commit details
    Browse the repository at this point in the history
  5. Filter on dataset id serverside

    Speeds up ~25x, and reduces network traffic.
    PGijsbers committed Oct 27, 2020
    Configuration menu
    Copy the full SHA
    402a2ef View commit details
    Browse the repository at this point in the history
  6. Reduce the amount of pages loaded

    Loading a page takes ~600ms. I don't think testing with 3 pages is any
    worse than 10. I also think this is an ideal candidate of test that
    could be split up into (1) testing the url is generated correctly, (2)
    testing a pre-cached result is parsed correctly and (3) testing the url
    gives the expected response (the actual integration test).
    PGijsbers committed Oct 27, 2020
    Configuration menu
    Copy the full SHA
    fd02a9d View commit details
    Browse the repository at this point in the history
  7. Simplify model tested in swapped parameter test

    If the test is that swapped parameters work, we don't need a complicated pipeline or dataset.
    PGijsbers committed Oct 27, 2020
    Configuration menu
    Copy the full SHA
    a9fd67b View commit details
    Browse the repository at this point in the history
  8. Add a cli flag to toggle short/long scenarios

    Some tests support both, by checking e.g. only a few runs vs all runs.
    PGijsbers committed Oct 27, 2020
    Configuration menu
    Copy the full SHA
    7904ef6 View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    db26dc9 View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    bbf76f4 View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    6186121 View commit details
    Browse the repository at this point in the history

Commits on Oct 28, 2020

  1. Check the trace can be retrieved individually

    To cover for the skipping of test_get_run_trace
    PGijsbers committed Oct 28, 2020
    Configuration menu
    Copy the full SHA
    63e103b View commit details
    Browse the repository at this point in the history
  2. Remove old test

    PGijsbers committed Oct 28, 2020
    Configuration menu
    Copy the full SHA
    a7e6b73 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    e7f6b3d View commit details
    Browse the repository at this point in the history

Commits on Oct 29, 2020

  1. Fix decorator call

    PGijsbers committed Oct 29, 2020
    Configuration menu
    Copy the full SHA
    d85b77a View commit details
    Browse the repository at this point in the history