Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

In-memory checkpoint simulation #1323

Merged
merged 43 commits into from
Jun 20, 2024

Conversation

adrienmaillard
Copy link
Contributor

@adrienmaillard adrienmaillard commented Feb 6, 2024

Description

The first part of this branch has been done by @mattdailis, it

  • enables to duplicate simulation engines
  • then designs a specific simulate procedure that is able to use the best cached engine to simulate a given plan

Then, I took this work and integrated it into the scheduling agent. Here are some elements of this work:

  • Separation between simulation and the computation of simulation results. E.g. rootfinding does not need simulation results (used only in scheduling)
  • Partial computation of simulation results for only a subset of the resources (used only in scheduling)
  • SimulationFacade has been completely overhauled to account for checkpoint simulation. It now takes care of updating the plan to simulate with child activities and activity durations.
  • The scheduler worker now maintain a cache of engines with a configurable size.

Other stuff:

  • A little bit of cleaning in the scheduler (DurationExpression mainly)
  • In the priority solver, goals and activity expressions now explicitely compute resource dependencies and we use the partial sim computation above + we mock the simulation results when no resource is need, that is, we compute fake simulation results from grounding the schedule into a SimulationResults object.
  • The interval of the last profile segment of constraint simulation results become closed-closed to be consistent with what the final state of the simulation engine means. When the engine is finishing at time t, everything at this time has been performed, meaning that the resource value at this time has been computed.

This is not visible in the tests because the metric for simulation performance in scheduling has changed but all of this is greatly decreasing the required simulation time (for the scheduling tests at least...).

Verification

Few tests were added.

Documentation

Needs some documentation, at least for the configuration of cache size.

Future work

  • Smarter checkpoint policy
  • Incremental simulation

@adrienmaillard adrienmaillard added performance A code change that improves performance feature A new feature or feature request scheduling Anything related to the scheduling domain simulation Anything related to the simulation domain labels Feb 6, 2024
@adrienmaillard adrienmaillard self-assigned this Feb 6, 2024
@adrienmaillard adrienmaillard force-pushed the prototype/duplicatable-sim-engine branch from 5350091 to b6b522f Compare February 6, 2024 20:22
@adrienmaillard adrienmaillard force-pushed the prototype/duplicatable-sim-engine branch from b6b522f to 28e9789 Compare February 6, 2024 22:00
@adrienmaillard adrienmaillard force-pushed the prototype/duplicatable-sim-engine branch from 28e9789 to db504bd Compare February 6, 2024 22:08
@adrienmaillard adrienmaillard force-pushed the prototype/duplicatable-sim-engine branch from db504bd to 9b7d6be Compare February 7, 2024 01:56
@adrienmaillard adrienmaillard force-pushed the prototype/duplicatable-sim-engine branch from 9b7d6be to 69c335b Compare February 7, 2024 19:21
@adrienmaillard
Copy link
Contributor Author

Matt, I have not rewritten your part of the history. I think there is some stuff to squash, let me know what you want to do.

@adrienmaillard adrienmaillard force-pushed the prototype/duplicatable-sim-engine branch from 69c335b to 746c3c0 Compare February 9, 2024 00:38
@adrienmaillard adrienmaillard force-pushed the prototype/duplicatable-sim-engine branch from 746c3c0 to df833e1 Compare February 9, 2024 00:40
@adrienmaillard adrienmaillard force-pushed the prototype/duplicatable-sim-engine branch from df833e1 to 16957a5 Compare February 9, 2024 00:43
@adrienmaillard adrienmaillard marked this pull request as ready for review February 9, 2024 00:57
@adrienmaillard adrienmaillard requested a review from a team as a code owner February 9, 2024 00:57
@adrienmaillard adrienmaillard requested review from skovati, Mythicaeda and mattdailis and removed request for skovati February 9, 2024 00:57
adrienmaillard and others added 24 commits June 18, 2024 13:44
When we exit the simulation at time t, the engine has finished all work at time t and this has updated the value of resources. Consequently, the last interval should not be closed/open but closed/closed to be consistent with the state of the simulation engine.
@dandelany dandelany force-pushed the prototype/duplicatable-sim-engine branch from 4405891 to 50ec917 Compare June 18, 2024 20:45
@adrienmaillard adrienmaillard merged commit 8ac4188 into develop Jun 20, 2024
6 checks passed
@adrienmaillard adrienmaillard deleted the prototype/duplicatable-sim-engine branch June 20, 2024 15:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking change A change that will require updating downstream code feature A new feature or feature request performance A code change that improves performance scheduling Anything related to the scheduling domain simulation Anything related to the simulation domain
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Integrating checkpoint simulation into scheduler Validate checkpoint simulation
4 participants