Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a Wait task (or Sleep/Delay task) #4879

Closed
anna-geller opened this issue Sep 10, 2024 · 29 comments
Closed

Add a Wait task (or Sleep/Delay task) #4879

anna-geller opened this issue Sep 10, 2024 · 29 comments
Assignees
Labels
area/backend Needs backend code changes enhancement New feature or request good first issue Great issue for new contributors kind/quick-win Seems to be quick to do

Comments

@anna-geller
Copy link
Member

anna-geller commented Sep 10, 2024

Feature description

Often, even for testing/demo, we need to make execution run for at least X number of seconds/minutes. Atm, we tend to run it in a container or local process, which can be costly + too many lines of code:

id: sleep
namespace: company.team
tasks:
  - id: wait
    type: io.kestra.plugin.scripts.shell.Commands
    taskRunner:
      type: io.kestra.plugin.core.runner.Process
    commands:
      - sleep 10

It would be useful to add a dedicated tasks for it:

id: sleep
namespace: company.team
tasks:
  - id: wait
    type: io.kestra.plugin.core.flow.Sleep
    duration: PT30S

Seems useful for unit testing too.

@anna-geller anna-geller added area/backend Needs backend code changes enhancement New feature or request kind/quick-win Seems to be quick to do labels Sep 10, 2024
@loicmathieu
Copy link
Member

Pause is already there for that, moreover pause can be resumed, and free the worker thread for other tasks.

@anna-geller
Copy link
Member Author

Yes correct, I'm aware. Pause has to be Resumed (unless you dive deeper to understand how timeout + delay works) -- we need a simple Wait task that will keep the Execution in a RUNNING state -- often needed for process orchestration too

@loicmathieu
Copy link
Member

No, pause can be resume, by default a Pause with a delay do exactly the same as what you propose here except that it frees the worker thread (and allow resuming if for some reason you don't want to wait for so long).

@shabith16
Copy link

Hi, is there anyone working on this issue.

@anna-geller
Copy link
Member Author

This one would be perfect for you @shabith16 if you want to take it 👍 thanks so much!

@shabith16
Copy link

yes I will work on this

@shabith16
Copy link

Hi ma'am, I am really interested in contributing to this project I know Java fundamentals (good with DSA) but not have really done any development what should i be learning in order to contribute to this project can you point me in the right direction.

@anna-geller
Copy link
Member Author

Maybe try setting up the local env and asking AI like e.g. ChatGPT to help with the first prototype of the plugin?

@shabith16
Copy link

yes ma'am.

@tchiotludo tchiotludo added good first issue Great issue for new contributors and removed kind/good-first-issue labels Oct 1, 2024
@shabith16
Copy link

shabith16 commented Oct 2, 2024

Hi ma'am/sir,
This post is to update you on my progress.
I have completed the Udemy course on Kestra and gained a understanding of the workflow and use case of Kestra. I have set up Kestra locally, ran a few flows (including the one mentioned earlier), and now understand what this issue demands.

The proposed idea is to create a plugin named "Sleep."
I consulted ChatGPT and generated a few sample Sleep classes. While they are not working properly, they gave me a general idea of what is required.

Currently, I am going through the Plugin Developer's Guide.

@loicmathieu
Copy link
Member

Hi @shabith16 we have a Sleep task for testing, you can move out of test directory and update it with the missing parts (mainly documentation).
It is here: https://github.com/kestra-io/kestra/blob/develop/core/src/test/java/io/kestra/core/tasks/test/Sleep.java
And should be moved here: https://github.com/kestra-io/kestra/tree/develop/core/src/main/java/io/kestra/plugin/core/debug

@shabith16
Copy link

yes sir

@shabith16
Copy link

sir,
i am well versed n Java DSA and oops and have developed many oops based projects using Java.
now what are the areas i should focus or learn inorder to understand the codebase clearly and contribute to it. ( i am very much interested in learning the backend of kestra)
currently I started to learn docker , kubernetes.
it will be greatly helpful and accelerating if you could point me in the right direction.
Thank you

@loicmathieu
Copy link
Member

The backend itself is a bit complex.
It uses Java 21, Micronaut, Lombok and JooQ.
It's easier to start with a plugin, and this is what this issue is about, please read the plugin developer guide for any help on this subject: https://kestra.io/docs/plugin-developer-guide

@shabith16
Copy link

yes sir thank you very much.

@shabith16
Copy link

Dear Team,
Due to my full-time job requiring me to relocate to the office, I will not be able to remain active and focus on my contributions for the time being. I kindly request a few days off to manage the transition.
Thank you.

@anna-geller
Copy link
Member Author

Time-off approved! 😄 thanks for being part of our community!

@yoyounik
Copy link
Contributor

Hey @anna-geller i know this work has been assigned, but as a beginner can i also try to work on it ?

@MilosPaunovic
Copy link
Member

Yup, go for it @yoyounik! 🚀

@yoyounik
Copy link
Contributor

image:

Hey @MilosPaunovic @anna-geller i successfully implemented a wait task on my local instance(localhost:8080).
{46300737-702D-4A13-9F72-DF925A372B0E}

Would this be acceptable for now, or should I investigate further to reduce the delay?

@MilosPaunovic
Copy link
Member

We can't say much like this, without you opening a PR, so will wait for that.

@yoyounik
Copy link
Contributor

Got it! I'll go ahead and prepare a PR with the implementation of the wait task. Once it's ready, I'll update here for your review. Thanks for the feedback!

@yoyounik
Copy link
Contributor

Hey @MilosPaunovic i have raised a PR titled "Updated time duration to 30 seconds". Please take a look when you get a chance. Thanks.

@Skraye
Copy link
Member

Skraye commented Oct 22, 2024

Hey @yoyounik I think you misunderstood the issue, the need is to create a new task, not modify the value of a testing flow

@yoyounik
Copy link
Contributor

Hey @Skraye
Thanks for the feedback!
I see now that the requirement was to create a new task rather than modify the value in the test flow. I misunderstood the issue earlier.
I'll work on creating a new task, specifically for the sleep functionality, and update the PR accordingly. Thanks for pointing that out!

@yoyounik
Copy link
Contributor

hello @Skraye
I've made some changes to this PR:

Added a new Java class: io.kestra.plugin.core.debug
Updated the YAML file: core/src/test/resources/flows/valids/wait-task-flow.yaml
The new Java class and YAML file are necessary for implementing the wait task functionality.

Note: I've temporarily ignored writing test cases to expedite the PR review process. I'll add comprehensive test cases in a follow-up commit once the core functionality is approved.

Please review these changes and let me know if you have any questions or require further clarification.

@Skraye Skraye closed this as completed in b99d9d4 Oct 23, 2024
@yoyounik
Copy link
Contributor

Thank you all, @Skraye @anna-geller @MilosPaunovic @loicmathieu @tchiotludo so much for the guidance and trust in allowing me to contribute to this project! I'm grateful for your feedback and the chance to work alongside such talented individuals. Your reviews and support helped me improve, and I look forward to contributing again soon! 🙏😊

@shabith16
Copy link

Hi, @yoyounik congratulations.
could you drop your X(twitter) we shall connect.

@yoyounik
Copy link
Contributor

sure @shabith16
twitter: @nikhil_sin71743

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/backend Needs backend code changes enhancement New feature or request good first issue Great issue for new contributors kind/quick-win Seems to be quick to do
Projects
Status: Done
Development

No branches or pull requests

7 participants