Automate Reconfiguration Performance Test

[bjee19]

Proposed changes

Automate the reconfiguration performance test.

Problem: We want to automate our reconfiguration performance test so that we don't need to run it manually.

Solution: Automated the reconfiguration performance test.

Testing: Checked that results are reasonable.

Closes #1367

Checklist

Before creating a PR, run through this checklist and mark each as complete.

• I have read the CONTRIBUTING doc
• I have added tests that prove my fix is effective or that my feature works
• I have checked that all unit tests pass after adding my changes
• I have updated necessary documentation
• I have rebased my branch onto main
• I will ensure my PR is targeting the main branch and pulling from my branch from my own fork

Release notes

If this PR introduces a change that affects users and needs to be mentioned in the release notes,
please add a brief note that summarizes the change.

NONE

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 88.85%. Comparing base (31628c7) to head (683cd36).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #2313   +/-   ##
=======================================
  Coverage   88.85%   88.85%           
=======================================
  Files         100      100           
  Lines        7513     7513           
  Branches       50       50           
=======================================
  Hits         6676     6676           
  Misses        781      781           
  Partials       56       56           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[sjberman]

LGTM, just be sure to delete the results file that was added.

Also be sure to do a full test run to make sure everything works. Nice job!

[sjberman]

Actually on second thought, we can include the results file, just be sure we have one for both plus and oss. Or don't include either, and we'll get them both when we run the scheduled pipeline.