Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Heuristics to determine how much time to allocate the resource #38

Open
sparshev opened this issue Nov 16, 2022 · 0 comments
Open

Heuristics to determine how much time to allocate the resource #38

sparshev opened this issue Nov 16, 2022 · 0 comments
Labels
enhancement New feature or request

Comments

@sparshev
Copy link
Collaborator

Right now node just tells if it can or can not provide the label, but for cluster proper working it will be great to have more data about how much it will take to run the resource and properly elect the right executor.

The metrics:

  • Lifetime of the resource (Add resource timeout in label definition #28) - will tell how much more time the node driver/resources will be busy.
  • Time to get the images - quick HEAD request for the URLs will give some idea on the archives size and previous downloads will provide average download speed to calculate the required time.
  • Storage required for the images could be calculated out of images metadata (which is stored in the head of the tar.xz archive) which contains the size of unpacked files. Maybe it will be not a good idea to cleanup the old images just to fulfill the resource request if the other nodes in the cluster will be faster...

So something like that, quite sure there is alot more and the logic to calculate all those input data will be quite complex, but will optimize the cluster utilization and will help with nodes specialization.
@sparshev sparshev added the enhancement New feature or request label Nov 16, 2022
@sparshev sparshev mentioned this issue Nov 16, 2022
Merged
10 tasks
sparshev added a commit that referenced this issue Nov 16, 2022
@sparshev
Added lifetime for the Resources in Label definition (#39)
be38ee2 
With lifetime timeout we can be sure the application resource will not be executed forever if something bad happened and based on this timeout fish node will be able to do some heuristics on how much time left till provisioning the next resource (#38).

The duration is set in standard golang format as string "1h2m3s", if it's empty or 0 - the default from fish config will be used. And if it's negative (like "-1s") then the resource should exist until user say so by deallocating it.

This change really helps with clouds where we certainly don't need to left the resources for long time.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@sparshev