Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Doc updates #60

Merged
merged 2 commits into from
May 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions doc/child_workflows.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Child Workflows orchestrate invocations of Activities just like Workflows do.

Child Workflows should not be used for code organization, however they can be used to partition a Workflow execution's event history into smaller chunks which helps avoid the roughly *~50MB* Workflow event history limit, amongst other use cases.

You should visit the [workflows](https://cljdoc.org/d/io.github.manetu/temporal-sdk/CURRENT/workflows) page to learn more about Workflows, their constraints, and their executions in general.
You should visit the [workflows](./workflows.md) page to learn more about Workflows, their constraints, and their executions in general.

For more information about Child Workflows in general visit [Temporal Child Workflows](https://docs.temporal.io/encyclopedia/child-workflows)

Expand Down Expand Up @@ -61,4 +61,3 @@ See [Temporal Parent Close Policy](https://docs.temporal.io/encyclopedia/child-w
:workflow-execution-timeout 3600
:workflow-run-timeout 3600}))
```

45 changes: 22 additions & 23 deletions doc/workflows.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,42 +30,42 @@ A Workflow implementation consists of defining a (defworkflow) function. The pl

### Workflow Implementation Constraints

Temporal uses the [Event Sourcing pattern](https://docs.microsoft.com/en-us/azure/architecture/patterns/event-sourcing) to recover the state of a Workflow object, including its threads and local variable values. In essence, the Workflow code is re-executed from the beginning whenever a Workflow state requires restoration. During replay, successfully executed Activities are not re-executed but return the result previously recorded in the Workflow event history.
Temporal uses the [Event Sourcing pattern](https://docs.microsoft.com/en-us/azure/architecture/patterns/event-sourcing) to recover the state of a Workflow object, including its threads and local variable values. The Workflow code is re-executed from the beginning whenever a Workflow state requires restoration. During replay, successfully executed Activities are not re-executed but return the result previously recorded in the Workflow event history.

Even though Temporal has the replay capability, which brings resilience to your Workflows, you should never think about this capability when writing your Workflows. Instead, you should focus on implementing your business logic/requirements and write your Workflows as they would execute only once.

There are some things, however, to think about when writing your Workflows, namely determinism and isolation. We summarize these constraints here:

- Do not use any mutable global variables such as atoms in your Workflow implementations. This will ensure that multiple Workflow instances are fully isolated.
- Do not call any non-deterministic functions like non-seeded random or uuid-generators directly from the Workflow code.
- Do not use mutable global variables such as atoms in your Workflow implementations. Avoiding globals will ensure that multiple Workflow instances are fully isolated.
- Do not call non-deterministic functions like non-seeded random or uuid generators directly from the Workflow code. Instead, use Side Effects.
- Perform all IO operations and calls to third-party services on Activities and not Workflows, as they are usually non-deterministic.
- Do not use any programming language constructs that rely on system time. (Coming soon: API methods for time)
- Do not use threading primitives such as clojure.core.async/go or clojure.core.async/thread. (Coming soon: API methods for async function execution)
- Do not use any programming language constructs that rely on system time. All notions of time must come from Side Effects or Activities so that the results become part of the Event History.
- Do not use threading primitives such as clojure.core.async/go or clojure.core.async/thread.
- Do not perform any operations that may block the underlying thread, such as clojure.core.async/<!!.
- There is no general need for explicit synchronization because multi-threaded code inside a Workflow is executed one thread at a time and under a global lock.
- This Clojure SDK provides integration with [promesa](https://github.com/funcool/promesa) with a few limitations (See [Promises](#promises)) for asynchronous integration with safe blocking operations, such as waiting on an Activity.
- (Coming soon) Use versioning-support when making changes to the Workflow code. Without this, any deployment of updated Workflow code might break already running Workflows.
- Don’t access configuration APIs directly from a Workflow because changes in the configuration might affect a Workflow execution path. Pass it as an argument to a Workflow function or use an Activity to load it.
- Use [versioning-support](#versioning) when making changes to the Workflow code. Without this, any deployment of updated Workflow code might break already running Workflows.
- Don’t access configuration APIs directly from a Workflow because changes in the configuration might affect the Workflow's execution path. Pass the configuration as an argument to a Workflow function or use an Activity to load it.

## Registering Workflows

By default, Workflows are automatically registered simply by declaring a (defworkflow). You may optionally manually specify Workflows to register when creating Workers (see [temporal.client.worker/worker-options](https://cljdoc.org/d/io.github.manetu/temporal-sdk/CURRENT/api/temporal.client.worker#worker-options)).

*It should be noted that the name of the Workflow, the arguments and signals that the Workflow accepts, and the data that the workflow returns are all part of a contract that you need to maintain across potentially long-lived instances. Therefore, refactoring code involving Workflow logic should be treated with care to avoid inadvertently breaking your contract.*
*It should be noted that the name of the Workflow, the arguments and signals that the Workflow accepts, and the data that the workflow returns are all part of a contract you need to maintain across potentially long-lived instances. Therefore, you should take care when refactoring code involving Workflow logic to avoid inadvertently breaking your contract.*

## Starting Workflow Executions

In this Clojure SDK, developers manage Workflows with the following flow:

1. Invoke [temporal.client.core/create-workflow](https://cljdoc.org/d/io.github.manetu/temporal-sdk/CURRENT/api/temporal.client.core#create-workflow)
1. Invoke [temporal.client.core/create-workflow](https://cljdoc.org/d/io.github.manetu/temporal-sdk/CURRENT/api/temporal.client.core#create-workflow)
2. Invoke [temporal.client.core/start](https://cljdoc.org/d/io.github.manetu/temporal-sdk/CURRENT/api/temporal.client.core#start) or [temporal.client.core/signal-with-start](https://cljdoc.org/d/io.github.manetu/temporal-sdk/CURRENT/api/temporal.client.core#signal-with-start). The `params` passed to these functions will be forwarded to the Workflow and available as `args` in the request map of the Workflow.
3. Gather the asynchronous results with [temporal.client.core/get-result](https://cljdoc.org/d/io.github.manetu/temporal-sdk/CURRENT/api/temporal.client.core#get-result), which returns a promise and requires dereferencing before the result value is realized.
3. Gather the asynchronous results with [temporal.client.core/get-result](https://cljdoc.org/d/io.github.manetu/temporal-sdk/CURRENT/api/temporal.client.core#get-result), which returns a promise and requires dereferencing to realize the result.

### Example

```clojure
(defworkflow my-workflow
[{:keys [foo]}]
[{:keys [foo]}]
...)

(let [w (create-workflow client my-workflow {:task-queue "MyTaskQueue"})]
Expand All @@ -75,7 +75,7 @@ In this Clojure SDK, developers manage Workflows with the following flow:

## Safe blocking within Workflows

A Temporal Workflow instance behaves like a [Lightweight Process](https://en.wikipedia.org/wiki/Light-weight_process) or [Fiber](https://en.wikipedia.org/wiki/Fiber_(computer_science)). These types of designs support a high ratio of Workflow instances to CPUs, often in the range of 1000:1 or greater. Achieving this feat requires controlling the IO in and out of the instance to maximize resource sharing. Therefore, any LWP/Fiber implementation will generally provide its own IO constructs (e.g., mailboxes, channels, promises, etc.), and Temporal is no exception.
A Temporal Workflow instance behaves like a [Lightweight Process](https://en.wikipedia.org/wiki/Light-weight_process) or [Fiber](https://en.wikipedia.org/wiki/Fiber_(computer_science)). These designs support a high ratio of Workflow instances to CPUs, often in the range of 1000:1 or greater. Achieving this feat requires controlling the IO in and out of the instance to maximize resource sharing. Therefore, any LWP/Fiber implementation will generally provide its own IO constructs (e.g., mailboxes, channels, promises, etc.), and Temporal is no exception.

In this Clojure SDK, this support comes in a few different flavors:

Expand All @@ -92,7 +92,7 @@ Specific methods naturally return Workflow-safe Promises, such as invoking an Ac
- "Originating" primitives, such as [create](https://funcool.github.io/promesa/latest/promesa.core.html#var-create), [resolved](https://funcool.github.io/promesa/latest/promesa.core.html#var-resolved), and [let](https://funcool.github.io/promesa/latest/promesa.core.html#var-let)
- Aggregating primitives, such as [all](https://funcool.github.io/promesa/latest/promesa.core.html#var-all) and [race](https://funcool.github.io/promesa/latest/promesa.core.html#var-race)

Instead, you must ensure that all promises originate with an SDK provided function, such as [temporal.activity/invoke](https://cljdoc.org/d/io.github.manetu/temporal-sdk/CURRENT/api/temporal.activity#invoke) or [temporal.promise/rejected](https://cljdoc.org/d/io.github.manetu/temporal-sdk/CURRENT/api/temporal.promise#rejected). For aggregating operations, see Temporal Safe options for [temporal.promise/all](https://cljdoc.org/d/io.github.manetu/temporal-sdk/CURRENT/api/temporal.promise#all) and [temporal.promise/race](https://cljdoc.org/d/io.github.manetu/temporal-sdk/CURRENT/api/temporal.promise#race).
Instead, you must ensure that all promises originate with an SDK-provided function, such as [temporal.activity/invoke](https://cljdoc.org/d/io.github.manetu/temporal-sdk/CURRENT/api/temporal.activity#invoke) or [temporal.promise/rejected](https://cljdoc.org/d/io.github.manetu/temporal-sdk/CURRENT/api/temporal.promise#rejected). For aggregating operations, see Temporal Safe options for [temporal.promise/all](https://cljdoc.org/d/io.github.manetu/temporal-sdk/CURRENT/api/temporal.promise#all) and [temporal.promise/race](https://cljdoc.org/d/io.github.manetu/temporal-sdk/CURRENT/api/temporal.promise#race).

What this means in practice is that any promise chain should generally start with some Temporal-native promise.

Expand Down Expand Up @@ -143,7 +143,7 @@ For situations where `some-condition` is `false` because promesa will cast the s
(p/then (fn [x] ...)))
```

Doing so ensures that the origination rules are met regardless of the outcome of the conditional.
Doing so ensures that you meet the origination rules regardless of the conditional outcome.

### Await

Expand All @@ -157,7 +157,7 @@ Your Workflow may send or receive [signals](https://cljdoc.org/d/io.github.manet

##### Channel Abstraction using 'signal-chan'

This SDK provides a [core.async](https://github.com/clojure/core.async) inspired abstraction on [Temporal Signals](https://docs.temporal.io/workflows#signal) called signal-channels. To use signal channels, your Workflow may either block waiting with signals with [temporal.signals/<!](https://cljdoc.org/d/io.github.manetu/temporal-sdk/CURRENT/api/temporal.signals#%3C!) or use the non-blocking [temporal.signals/poll](https://cljdoc.org/d/io.github.manetu/temporal-sdk/CURRENT/api/temporal.signals#poll). Either way, your Workflow needs to first obtain the `signal-chan` context obtained by [temporal.signals/create-signal-chan](https://cljdoc.org/d/io.github.manetu/temporal-sdk/CURRENT/api/temporal.signals#create-signal-chan).
This SDK provides a [core.async](https://github.com/clojure/core.async) inspired abstraction on [Temporal Signals](https://docs.temporal.io/workflows#signal) called signal-channels. To use signal channels, your Workflow may either block waiting with signals with [temporal.signals/<!](https://cljdoc.org/d/io.github.manetu/temporal-sdk/CURRENT/api/temporal.signals#%3C!) or use the non-blocking [temporal.signals/poll](https://cljdoc.org/d/io.github.manetu/temporal-sdk/CURRENT/api/temporal.signals#poll). Either way, your Workflow needs first to obtain the `signal-chan` context obtained by [temporal.signals/create-signal-chan](https://cljdoc.org/d/io.github.manetu/temporal-sdk/CURRENT/api/temporal.signals#create-signal-chan).

###### Example

Expand Down Expand Up @@ -191,14 +191,14 @@ Alternatively, you may opt to handle signals directly with [temporal.signals/reg

Your Workflow may respond to [queries](https://cljdoc.org/d/io.github.manetu/temporal-sdk/CURRENT/api/temporal.client.core#query).

A temporal query is similar to a temporal signal, both are messages sent to a running Workflow.
The difference is that a signal intends to change the behaviour of the Workflow, whereas a query intends to inspect the current state of the Workflow.
Querying the state of a Workflow implies that the Workflow must maintain state while running, typically in a clojure [atom](https://clojuredocs.org/clojure.core/atom) or [ref](https://clojure.org/reference/refs).
A temporal query is similar to a temporal signal; both are messages sent to a running Workflow.
The difference is that a signal intends to change the behaviour of the Workflow, whereas a query intends to inspect its current state.
Querying the state of a Workflow implies that the Workflow must maintain its state while running, typically in a Clojure [atom](https://clojuredocs.org/clojure.core/atom) or [ref](https://clojure.org/reference/refs).

#### Registering a Query handler

To enable querying a Workflow, you may use [temporal.workflow/register-query-handler!](https://cljdoc.org/d/io.github.manetu/temporal-sdk/CURRENT/api/temporal.workflow#register-query-handler!).
The query handler is a function that has a reference to the Workflow state, usually by closing over it. It interprets the query and returns a response.
The query handler is a function that has a reference to the Workflow state, usually by closing over it. It interprets the query and returns a response.

```clojure
(defworkflow stateful-workflow
Expand All @@ -222,10 +222,10 @@ A query consists of a `query-type` (keyword) and possibly some `args` (any seria

## Exceptions

This SDK integrates with the [slingshot](https://github.com/scgilardi/slingshot) library. Stones cast with slingshot's throw+ are serialized and re-thrown across activity and workflow boundaries in a transparent manner that is compatible with slingshot idiomatic try+ based catch blocks.
This SDK integrates with the [slingshot](https://github.com/scgilardi/slingshot) library. Stones cast with Slingshot's throw+ are serialized and re-thrown across activity and workflow boundaries in a transparent manner that is compatible with Slingshot's `try+` based catch blocks.

### Managing Retries
By default, stones cast that are not caught locally by an activity or workflow trigger ApplicationFailure semantics and are thus subject to the overall Retry Policies in place. However, the developer may force a given stone to be non-retriable by setting the flag '::non-retriable?' within the object.
By default, stones cast that are not caught locally by an Activity or Workflow trigger ApplicationFailure semantics and are thus subject to the overall Retry Policies in place. However, the developer may force a given stone to be non-retriable by setting the flag '::non-retriable?' within the object.

Example:

Expand All @@ -244,7 +244,7 @@ The Temporal Platform requires that Workflow code be deterministic. Because of

Example:

Assume we have a workflow that invokes an activity using temporal.activity/invoke that we wish to convert to temporal.activity/local-invoke. Changing this for future workflows is not a problem. However, any existing workflows need to be careful as this change could introduce non-determinism.
Assume we have a workflow that invokes an activity using temporal.activity/invoke that we wish to convert to temporal.activity/local-invoke. Such a change is acceptable for any future instances of your Workflow. However, any existing instances must be careful as this logic change could introduce non-determinism during replay.

We can safely handle both the original and the new desired scenario by branching based on the results from calling temporal.workflow/get-version:

Expand All @@ -257,4 +257,3 @@ We can safely handle both the original and the new desired scenario by branching
(= version w/default-version) @(a/invoke versioned-activity :v1)
(= version 1) @(a/local-invoke versioned-activity :v2)))
```

2 changes: 1 addition & 1 deletion src/temporal/internal/child_workflow.clj
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
(ns temporal.internal.child-workflow
(ns ^:no-doc temporal.internal.child-workflow
(:require [temporal.common :as common]
[temporal.internal.utils :as u]
[temporal.internal.workflow :as w])
Expand Down