RFC: Infinite Tasks

[FrozenPandaz]

RFC: Infinite Tasks

Some tasks run through Nx are tasks which run infinitely until they are terminated. In this RFC, these tasks will be referred to as infinite tasks though the name is subject to change. Examples of infinite tasks are tasks which start servers, build and test tasks which have watch modes, and many more. Currently, Nx expects all tasks to end and does not really have much special handling for infinite tasks. In many cases, tasks have a relationship with these infinite tasks where they depend on them being running. If this is specified as a dependsOn today, Nx will never continue as it will wait for the depended upon infinite tasks to finish; which it will not.

Concept

Nx should have handling for infinite tasks. Infinite tasks are a subset of tasks which never end on their own accord; they must be terminated by something else.

Infinite tasks are much like other tasks. Infinite tasks can be run like any other tasks, either on their own, as part of a set of tasks, or as a dependency of another task. Infinite tasks can depend on other tasks; infinite and finite. And other tasks, infinite or not, can depend on infinite tasks.

Infinite tasks are different from finite tasks in some ways though. Infinite tasks do not end on their own and need to be terminated by something else. Infinite tasks also do not yield an output; this means that they cannot be cached. However, infinite tasks do yield a side effect which is likely the purpose why the task is kept alive.

Example Use Cases

Starting a web-server to e2e test

Currently in Nx, the e2e task is responsible for starting the web-server which it is testing. With dependencies on infinite tasks, Nx can takeover the responsibility of starting the web server while the e2e task is responsible for waiting for the server to be ready. Cypress, Playwright and other e2e tasks runners already do this.

Starting backends alongside a web server

{
  name: 'frontend',
  targets: {
    serve: {
      command: 'vite dev',
      dependsOn: ['content-backend:serve', 'auth-backend:serve']
    }
  }
}

This would start the required backends while running a frontend application.

Starting a db alongside a backend server

{
  name: 'api',
  targets: {
    serve: {
      command: 'node ./server.js',
      dependsOn: ['database:start']
    }
  }
}

Publishing to a local-registry before running e2e tests

{
  name: 'angular-e2e',
  targets: {
    serve: {
      command: 'jest e2e/angular',
      dependsOn: ['@nx/source:prepare-local-registry']
    }
  }
}

Feedback Requested

While you read this RFC and dive into the details, the Nx core team would like some feedback on this design before we implement it. Please let us know your thoughts after you have read it.

Is this a feature you would like to see implemented?

Firstly, is this a feature that excites you and does it solve the problems you face? And would this feature improve or enable the kinds of workflows that you need?

Naming/ Terminology

Secondly, naming things is hard. "infinite tasks" is terminology which is subject to change. We would like to hear feedback on the terminology we should use. We'd like the term to be easily understood and intuitively capture this type of task and the differences with the existing concept of tasks.

Infinite refers to the infinite duration of the tasks but other terms could be more specific. Is it obvious what infinite vs finite pertains to?

Perpetual could make sense because the tasks remain running as does a perpetual motion machine. Unfortunately, there is no real antonym as there is for infinite vs finite. Talking about not perpetual tasks is not as nice

We have also considered some other terms.

Tasks can be classified as neverending or endless as they will not end on their own accord. They would end though, when they are terminated.

These tasks could be considered long running tasks. But some tasks such as e2e tests can also run for a long time while not being infinitely running. And if these tasks don't last for that long... is it fair to call them long running?

Immortal vs Mortal is another way of describing these tasks. Immortallity is specific to life though but tasks aren't living. It also has religious connotations which may cause uncomfort.

Tasks should eventually complete. If something does not complete, is it even still a task? Should we introduce a new term altogether? Rather than referring to them as tasks, should we refer to them as processes? This would be easily confused for OS processes.

Timeline

We hope to begin implementation starting January 2025 so please provide as much feedback as you can before then.

Lifecycle

The lifecycle for an infinite task would have the following lifecycle:

1. Waits for dependent mortal tasks to complete and dependent infinite tasks to start
2. Task starts
3. Task is running
4. Task is terminated

Now we'll contrast this to the lifecycle for a finite task:

1. Waits for dependent finite tasks to complete and dependent infinite tasks to start
2. Task starts
3. Task is running
4. Task completes either successfully or not

There are 2 main differences from what is currently in Nx.

Tasks (both finite and infinite) depending on infinite tasks, Nx will not wait for the task to complete. It will instead, wait for the task to start. The task may not be fully ready to produce side effects yet. This will be discussed later on.

Infinite tasks need to be terminated. Nx would handle this but when this occurs depends on why Nx is running that infinite task.

Use Cases

Below, we'll dive into some different cases Nx will have to handle. Each of them will detail some current workarounds and their flaws. @vsavkin also produced a video with diagrams to go over this topic here: https://www.youtube.com/watch?v=-gezeX9zxuM

Isolated Infinite Task

Running an infinite task on its own with no dependencies and without other tasks depending on it is already fairly well supported by Nx. If you're using Nx to run app:serve, app:build --watch, or app:test --watch then these are already infinite tasks. From the user's point of view, this should be the exact same as if the user were to run that task outside of Nx.

When an infinite task starts running, it is expected to continue running until Nx ends it. Thus, Nx would throw an error if an infinite task were to exit before Nx terminates it. This will be true for the following cases as well.

Nx starts the infinite task solely because the user instructed it to. There is not much other known information as there will be in other use cases. Nx will thus wait for the user to interrupt Nx before it terminates the infinite task.

Infinite Tasks Depending on Finite Tasks

Infinite tasks can depend on finite tasks.

For example, app:serve could depend on itself and it's dependencies to be built before starting up.

Running an infinite task on its own with dependencies on other tasks but no dependencies on itself will be much like running a infinite task in isolation. The only difference is that Nx will wait for the finite tasks to complete before the infinite task is started. It is expected to continue running until Nx is interrupted (Ctrl + C).

Multiple Unrelated Infinite Tasks

Running multiple infinite tasks may be something users already do.

An example of this is serving two applications via nx run-many -t serve --projects frontend,backend such as a frontend and a backend.

As it already does currently, Nx will wait dependent tasks of each infinite task before starting them respectively. As with the prior cases, Nx would also terminate these tasks when the user interrupts Nx.

The main change here will be how Nx handle the terminal outputs. Currently, the user has a few different options for how to show the output. Only 2 of the 3 really make sense.

• Static
  • This mode would display outputs of dependent tasks when they are done but then would not display the outputs of infinite tasks since they do not end. This option doesn't make much sense.
• Stream
  • This mode interlaces the terminal outputs while prefixing them. This is alright but output is still harder to understand as it can be broken up.
• Dynamic (default) and "stream-without-prefixes" which is similar
  • This mode interlaces the terminal outputs without prefixing them.

Nx would be updated to handle multiple streams of outputs differently. What would be ideal in this case is if the user could switch focus between different infinite task outputs. None of the infinite tasks are auxilliary and all of them are main tasks. Terminal output will be covered further in more detail below.

Single Finite Task Depending on Infinite Task(s)

A finite task can depend on infinite task(s).

For example, e2e tests can depend on the application server. Nx would start the application server first, then start the e2e tests. Until the e2e tests complete, the application server will stay running. When the e2e tests finish, Nx will terminate the application server.

Currently, Nx does not handle this case. Because tasks wait for all dependent tasks to end and infinite tasks do not end, Nx will just stop at running the infinite task.

Nx plugins can do their best to handle this case by configuring tools to wait for another Nx task to start and produce side effects. Doing this does handle the same issue but is inefficient as it will yield multiple task graphs as opposed to being combined in a single task graph. This would mean that tasks which exist in multiple graphs, are run once for each graph being executed. Caching can be utilized to reuse as much existing work as possible. Not all work can be cached though so this could result in a lot of wasted work.

When a finite task depends on any infinite task(s), Nx will start the infinite task(s) before running the finite task. Unlike the previous cases, the user starting the infinite task directly. The infinite task is started to accompany the finite task. The finite task will eventually complete. When this happens, the infinite task(s) become orphaned and Nx will terminate them since they are no longer necessary.

As for the actual nature of the depenency, the finite task depending on the infinite task(s) likely needs some sort of side effect to be produced before the finite task can truly begin. If Nx were to start the finite task immediately as soon as the infinite task is started, the finite task may encounter issues if it does not handle the side effect not being present. For the initial implementation, Nx will classify that as an issue with the finite task. When the tasks are run outside of Nx, they still likely need some handling for when the side effect does not exist. Application servers need to handle when their accompanying services are unavailable. And test runners are already currently configured to wait for an application server to come up. However, Nx can still prioritize starting the infinite task(s) being depended upon as soon as it can. This would give more time for the side-effect to become available while Nx runs other dependent tasks. But ultimately, Nx will not further delay the execution of tasks depending on infinite task(s) after the infinite task has started.

The terminal output of the finite task is more important than the output of the infinite task(s) output in this case. The user initiated directly. Nx would have the finite task's terminal output shown most prominently. The infinite task(s) output could either be hidden entirely or optionally shown. Ideally, Nx would allow the user to see two terminal outputs at once. Again, terminal output will be covered in more detail further below.

Multiple Finite Tasks depending on a particular Infinite Task(s)

An extension of the previous case is when multiple finite tasks depend on a single infinite task or set of infinite tasks.

An example of this is multiple e2e test suites (app1-e2e:e2e and app2-e2e:e2e both depending app:serve) or atomized test suites (app1-e2e:e2e-ci--feature1.spec.ts and app1-e2e:e2e-ci--feature2.spec.ts both depending on app:serve).

As with the previous case, Nx Plugins do their best to handle this case by configuring the testing tool to start the infinite task. This has the same issues as the previous case but it is amplified because each and every task depending on the infinite task would waste some work. A lot of compute is wasted here.

When multiple finite tasks depend on a particular infinite task or a set of infinite tasks, Nx can reuse the infinite task(s). As with the previous case, Nx will start the infinite task(s) being depended upon as soon as possible. However, Nx will not terminate the infinite task(s) until all tasks which depend upon the infinite task completes.

This case is more complicated to distribute across different machines. Multiple finite tasks would be in the execution queue. An agent does not know which if any of those finite tasks it will receive. Agents could prematurely run any infinite tasks any task in the execution queue depends on but this may waste some work if some agents end up not being assigned any tasks depending on those infinite tasks. Agents could only run the infinite task necessary for tasks it is assigned when it is assigned but this would likely leave the tasks depending on the infinite tasks waiting for a bit until the side effects are available. Ideally, the orchestrator would accurately make the decision upfront about which subset of agents will receive the dependent tasks and run them efficiently utilizing only those agents. There's a lot of optimization which can be done but to start, it should be sufficient for the orchestrator to send infinite task(s) to an agent before it wants to send it a task which depends on it. We'll start there and explore more optimizations later on.

As for the terminal output, now multiple finite tasks would need to have their outputs shown while also showing the output of infinite tasks at the same time. Nx already handles showing multiple finite tasks' output, but not the output of dependent infinite task's. The mortal task output should still be more prominent and the dependent tasks could be optionally shown. Again, terminal output will be covered in more detail further below.

Infinite Task(s) Depending on Infinite Task(s)

An extension of all the previous cases, is when infinite task(s) depend on a particular infinite task or a set of infinite tasks.

Examples of this would be if:

• frontend:serve depends on backend:serve
• frontend:serve depends on backend1:serve and backend2:serve
• app1:serve and app2:serve both depend on backend:serve
• app2:serve and app2:serve both depend on both backend1:serve and backend2:serve
• backend:serve depends on database:serve
• app1:serve and app2:serve both depend on backend1:serve and backend2:serve which depend on database1:serve and database2:serve respectively

Currently, Nx does not handle this because the infinite tasks do not end and Nx will not continue.

Nx plugins again can do their best to handle this case using an executor to optionally start the dependent servers only of they are not available. This has a lot of intricacies. When multiple tasks are started simultaneously, there is still a chance multiple tasks will start the dependent servers as they are not available. There's a lot of wasted work as it's not included in the task graph. In theory, this can be handled perfectly.. but honestly, this case is not handled well for the most part at the moment.

Nx could handle this case with the same basic principles discussed in the previous sections. Dependent Infinite tasks are started first before any infinite tasks depending on them are started. The multiple infinite tasks started directly by the user remain running until the user interrupts Nx. And when this happens, the dependent infinite tasks are also terminated.

Solutions

Configuration

A target will declare that the task that it creates is an infinite task.

{
  name: 'app',
  targets: {
    serve: {
      command: 'vite',
      infinite: true
    }
  }
}

Other targets can specify that its task depend on the infinite task

{
  name: 'app-e2e',
  targets: {
    e2e: {
      command: 'playwright test',
      dependsOn: ['app:serve']
    }
  }
}

With this configuration, Nx will know that app:serve is infinite.

Readiness

Earlier we touched on the fact that tasks can take some time to produce side effects which parent tasks are expecting. For instance, starting servers might not be instantaneous. Infinite tasks may need a way to indicate that the side effects have been produced. Parent tasks would then wait for their dependent tasks to be ready rather than started. This concept of "readiness" does not currently exist.

Option 1: Do not handle it

Tasks are likely already built in a way where they can be started without their dependent infinite tasks being truly ready and wait until the side effects are available. It is definitely an option to not introduce the notion of "ready" to Nx and let the task itself handle this. Given this, Nx would start the parent task possibly immediately after the dependent infinite tasks have started.

Applications would need this sort of logic when running in production (not via Nx), so most applications should already be built to handle this kind of startup. For example, frontends would need to handle if their backends either haven't started or went down. Not delaying parent tasks allows multiple infinite tasks begin any pre work at the same time.

For tests, the plugins that Nx provides for Cypress and Playwright already handle waiting for the web server to startup. This waiting would continue to exist without Cypress and Playwright explicitly responsible for starting and terminating the dependent infinite tasks.

If the parent task doesn't handle waiting for the side effect, the parent task would fail and Nx would leave it up to developers to change the behavior of the parent task.

Option 2: Add configuration

Tasks can report to Nx that the sidecar task produces a side effect. Nx can handle waiting for these side effects. Doing so would delay parent task's from executing. Readiness might not be a boolean state though. Readiness may be more gradual as more side effects are produced. It is hard to configure more gradual startups where different features become ready at different times. The most generic way of specifying readiness is to allow a task to determine a stream or iterable of readiness states. And this stream would need to originate within the actual application itself and communicate with Nx. Even if readiness were simplified to be a boolean when all side effects are available, it would still have to be built into the application itself and cause delays in task execution.

Decision

For the initial implementation, option 1 of not introducing readiness state should suffice. Nx would still be able to add readiness handling in the future while remaining backwards compatible. Nx would default tasks to immediately become ready unless it has some other indicator for Nx to use. Readiness definitely needs to be handled but this will continue to stay within the tasks. As discussed before, both application servers and test tools already likely have some handling for this.

Terminal Output

As discussed before in the different use cases, terminal output is a large part of how Nx would need to handle infinite tasks.

In general, terminal outputs of tasks can either be main or auxilliary. Main tasks are those directly requested by users and should be given prominent treatment. Tasks which main tasks depend on are auxilliary and can be optionally shown possibly at the same time. Nx does not currently have the infrastructure in place to support this.

However, utilizing Rust has already allowed Nx to be more transparent terminal output from individual tasks. In the coming year, we will be rewriting the orchestration of multiple task outputs via Rust. There are powerful terminal libraries in Rust that Nx could utilize to create a new terminal UI which can handle the above use cases.

This new UI would have 2 new capabilities that the current UI does not have:

Separate areas of terminal output

Nx should be able to output terminal output to separate areas of the terminal. This is necessary to view both main and auxilliary output at the same time. The auxilliary output should be handled differently depending on the width of the terminal. Terminals which are not wide enough will be limited to switching between main and auxilliary output.

Switch between outputs of different tasks

Multiple tasks can be main or auxilliary so Nx should allow for developers to switch between different tasks. This prevents the outputs from being interlaced which could make the output hard to understand.

Current Terminal Output

The above terminal output will take some time to create so in the mean time, there still remains the existing display solutions for infinite tasks for the time being. When the new Terminal UI is ready, the developer experience using infinite tasks will improve. The following options would be switched by the developer with the current --outputStyle flag.

Option 1: Do not show terminal output for dependent tasks

In some situations, dependent tasks can very well be ignored. If all dependent tasks are working as they are expected to, then terminal outputs of dependent tasks could be hidden altogether. Nx already handles this today.

Option 2: Show terminal output interlaced with parent tasks

Interlacing terminal outputs is messy. Nx would prefix the outputs but either task's outputs could still be broken up in a way where it becomes hard to understand. In situations where a dependent task is not behaving as expected, this option can be used to debug the issue. This is not a great experience but Nx already currently has support for this.

Prior Art

https://kubernetes.io/docs/concepts/workloads/pods/sidecar-containers/

Kubernetes allows containers to specify other containers to run alongside it much like Nx's tasks expect infinite tasks to run alongside them.

[Previous Sidecars RFC]

This previous RFC designs out sidecars which are a subset of infinite tasks. In this design, infinite tasks depended upon by other tasks are synonymous to the sidecars mentioned in the previous design. This design encompasses more than just sidecar tasks.

[dwelch2344]

Yes please. Setting up things like SSH tunnels / etc would be so nice to do in a NX ecosystem