Skip to content

Commit 4074ede

Browse files
authored
docs(turbopack): Better document the Vc type, with references to ResolvedVc and VcOperation (#72524)
![Screenshot 2024-11-25 at 16-42-42 Vc in turbo_tasks - Rust.png](https://graphite-user-uploaded-assets-prod.s3.amazonaws.com/HAZVitxRNnZz8QMiPn4a/759f1e16-90b0-45e1-bfbf-19cbfd238748.png) ![Screenshot 2024-11-25 at 16-43-09 OperationVc in turbo_tasks - Rust.png](https://graphite-user-uploaded-assets-prod.s3.amazonaws.com/HAZVitxRNnZz8QMiPn4a/6b36e3aa-6fa2-4a9d-baf9-7f24987acf9f.png) ![Screenshot 2024-11-25 at 16-43-25 ResolvedVc in turbo_tasks - Rust.png](https://graphite-user-uploaded-assets-prod.s3.amazonaws.com/HAZVitxRNnZz8QMiPn4a/ed5794bb-0f5b-491e-ae6e-9ccf9878e609.png) ![Screenshot 2024-11-25 at 16-43-41 State in turbo_tasks - Rust.png](https://graphite-user-uploaded-assets-prod.s3.amazonaws.com/HAZVitxRNnZz8QMiPn4a/4a79d97f-f33b-44ca-b26e-930c2352089f.png)
1 parent eecc5f1 commit 4074ede

File tree

7 files changed

+247
-20
lines changed

7 files changed

+247
-20
lines changed

turbopack/crates/turbo-tasks/src/lib.rs

Lines changed: 17 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -2,25 +2,29 @@
22
//! execution.
33
//!
44
//! It defines 4 primitives:
5-
//! - functions: Unit of execution, invalidation and reexecution.
6-
//! - values: Data created, stored and returned by functions.
7-
//! - traits: Traits that define a set of functions on values.
8-
//! - collectibles: Values emitted in functions that bubble up the call graph and can be collected
9-
//! in parent functions.
5+
//! - **[Functions][macro@crate::function]:** Units of execution, invalidation, and reexecution.
6+
//! - **[Values][macro@crate::value]:** Data created, stored, and returned by functions.
7+
//! - **[Traits][macro@crate::value_trait]:** Traits that define a set of functions on values.
8+
//! - **[Collectibles][crate::TurboTasks::emit_collectible]:** Values emitted in functions that
9+
//! bubble up the call graph and can be collected in parent functions.
1010
//!
1111
//! It also defines some derived elements from that:
12-
//! - cells: The locations in functions where values are stored. The content of a cell can change
13-
//! after the reexecution of a function.
14-
//! - Vcs: A reference to a cell in a function or a return value of a function.
15-
//! - task: An instance of a function together with its arguments.
12+
//! - **[Tasks][book-tasks]:** An instance of a function together with its arguments.
13+
//! - **[Cells][book-cells]:** The locations associated with tasks where values are stored. The
14+
//! contents of a cell can change after the reexecution of a function.
15+
//! - **[`Vc`s ("Value Cells")][Vc]:** A reference to a cell or a return value of a function.
1616
//!
17-
//! A Vc can be read to get a read-only reference to the stored data.
17+
//! A [`Vc`] can be read to get [a read-only reference][ReadRef] to the stored data, representing a
18+
//! snapshot of that cell at that point in time.
1819
//!
19-
//! On execution of functions, turbo-tasks will track which Vcs are read. Once
20-
//! any of these change, turbo-tasks will invalidate the task created from the
21-
//! function's execution and it will eventually be scheduled and reexecuted.
20+
//! On execution of functions, `turbo-tasks` will track which [`Vc`]s are read. Once any of these
21+
//! change, `turbo-tasks` will invalidate the task created from the function's execution and it will
22+
//! eventually be scheduled and reexecuted.
2223
//!
2324
//! Collectibles go through a similar process.
25+
//!
26+
//! [book-cells]: https://turbopack-rust-docs.vercel.sh/turbo-engine/cells.html
27+
//! [book-tasks]: https://turbopack-rust-docs.vercel.sh/turbo-engine/tasks.html
2428
2529
#![feature(trivial_bounds)]
2630
#![feature(min_specialization)]

turbopack/crates/turbo-tasks/src/macro_helpers.rs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,7 @@ macro_rules! stringify_path {
6666
/// implement [`ShrinkToFit`].
6767
///
6868
/// This is used by the derive macro for [`ShrinkToFit`], which is called by the
69-
/// [turbo_tasks::value][crate::value] macro.
69+
/// [turbo_tasks::value][macro@crate::value] macro.
7070
///
7171
/// [autoderef]: http://lukaskalbertodt.github.io/2019/12/05/generalized-autoref-based-specialization.html
7272
pub struct ShrinkToFitDerefSpecialization<'a, T> {

turbopack/crates/turbo-tasks/src/raw_vc.rs

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,15 @@ impl Display for CellId {
5454
}
5555
}
5656

57+
/// A type-erased representation of [`Vc`].
58+
///
59+
/// Type erasure reduces the [monomorphization] (and therefore binary size and compilation time)
60+
/// required to support `Vc`.
61+
///
62+
/// This type is heavily used within the [`Backend`][crate::backend::Backend] trait, but should
63+
/// otherwise be treated as an internal implementation detail of `turbo-tasks`.
64+
///
65+
/// [monomorphization]: https://doc.rust-lang.org/book/ch10-01-syntax.html#performance-of-code-using-generics
5766
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
5867
pub enum RawVc {
5968
TaskOutput(TaskId),

turbopack/crates/turbo-tasks/src/state.rs

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -96,6 +96,25 @@ impl<T> Drop for StateRef<'_, T> {
9696
}
9797
}
9898

99+
/// An [internally-mutable] type, similar to [`RefCell`][std::cell::RefCell] or [`Mutex`] that can
100+
/// be stored inside a [`VcValueType`].
101+
///
102+
/// **[`State`] should only be used with [`OperationVc`] and types that implement
103+
/// [`OperationValue`]**.
104+
///
105+
/// Setting values inside a [`State`] bypasses the normal argument and return value tracking
106+
/// that's tracks child function calls and re-runs tasks until their values settled. That system is
107+
/// needed for [strong consistency]. [`OperationVc`] ensures that function calls are reconnected
108+
/// with the parent/child call graph.
109+
///
110+
/// When reading a `State` with [`State::get`], the state itself (though not any values inside of
111+
/// it) is marked as a dependency of the current task.
112+
///
113+
/// [internally-mutable]: https://doc.rust-lang.org/book/ch15-05-interior-mutability.html
114+
/// [`VcValueType`]: crate::VcValueType
115+
/// [strong consistency]: crate::Vc::strongly_consistent
116+
/// [`OperationVc`]: crate::OperationVc
117+
/// [`OperationValue`]: crate::OperationValue
99118
#[derive(Serialize, Deserialize)]
100119
pub struct State<T> {
101120
serialization_invalidator: SerializationInvalidator,

turbopack/crates/turbo-tasks/src/vc/mod.rs

Lines changed: 114 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -38,18 +38,127 @@ use crate::{
3838
CellId, CollectiblesSource, RawVc, ResolveTypeError, SharedReference, ShrinkToFit,
3939
};
4040

41-
/// A Value Cell (`Vc` for short) is a reference to a memoized computation
42-
/// result stored on the heap or in persistent cache, depending on the
43-
/// Turbo Engine backend implementation.
41+
/// A "Value Cell" (`Vc` for short) is a reference to a memoized computation result stored on the
42+
/// heap or in persistent cache, depending on the Turbo Engine backend implementation.
4443
///
45-
/// In order to get a reference to the pointed value, you need to `.await` the
46-
/// [`Vc<T>`] to get a [`ReadRef<T>`][crate::ReadRef]:
44+
/// In order to get a reference to the pointed value, you need to `.await` the [`Vc<T>`] to get a
45+
/// [`ReadRef<T>`][`ReadRef`]:
4746
///
4847
/// ```
4948
/// let some_vc: Vc<T>;
5049
/// let some_ref: ReadRef<T> = some_vc.await?;
5150
/// some_ref.some_method_on_t();
5251
/// ```
52+
///
53+
/// `Vc`s are similar to a [`Future`] or a Promise with a few key differences:
54+
///
55+
/// - The value pointed to by a `Vc` can be invalidated by changing dependencies or cache evicted,
56+
/// meaning that `await`ing a `Vc` multiple times can give different results. A [`ReadRef`] is
57+
/// snapshot of the underlying cell at a point in time.
58+
///
59+
/// - Reading (`await`ing) `Vc`s causes the current task to be tracked a dependent of the `Vc`'s
60+
/// task or task cell. When the read task or task cell changes, the current task may be
61+
/// re-executed.
62+
///
63+
/// - `Vc` types are always [`Copy`]. Most [`Future`]s are not. This works because `Vc`s are
64+
/// represented as a few ids or indicies into data structures managed by the `turbo-tasks`
65+
/// framework. `Vc` types are not reference counted, but do support [tracing] for a hypothetical
66+
/// (unimplemented) garbage collector.
67+
///
68+
/// - Unlike futures (but like promises), the work that a `Vc` represents [begins execution even if
69+
/// the `Vc` is not `await`ed](#execution-model).
70+
///
71+
/// For a more in-depth explanation of the concepts behind value cells, [refer to the Turbopack
72+
/// book][book-cells].
73+
///
74+
///
75+
/// ## Subtypes
76+
///
77+
/// There are a couple of "subtypes" of `Vc`. These can both be cheaply converted back into a `Vc`.
78+
///
79+
/// - **[`ResolvedVc`]:** A reference to a cell constructed within a task, as part of a [`Vc::cell`]
80+
/// or `value_type.cell()` constructor. As the cell has been constructed at least once, the
81+
/// concrete type of the cell is known (allowing [downcasting][ResolvedVc::try_downcast]). This is
82+
/// stored as a combination of a task id, a type id, and a cell id.
83+
///
84+
/// - **[`OperationVc`]:** The synchronous return value of a [`turbo_tasks::function`]. Internally,
85+
/// this is stored using a task id. Exact type information of trait types (i.e. `Vc<Box<dyn
86+
/// Trait>>`) is not known because the function may not have finished execution yet. Operations
87+
/// must first be [`connected`][OperationVc::connect]ed before being read.
88+
///
89+
/// [`ResolvedVc`] is almost always preferred over the more awkward [`OperationVc`] API, but
90+
/// [`OperationVc`] can be useful inside of [`State`] or when dealing with [collectibles].
91+
///
92+
/// In addition to these potentially-explicit representations of a `Vc`, there's another internal
93+
/// representation of a `Vc`, known as a "Local `Vc`".
94+
///
95+
/// - **Local Operation or Cell:** Same as [`ResolvedVc`] or [`OperationVc`], but these values are
96+
/// stored in task-local state that is freed after their parent non-local task exits. These values
97+
/// are sometimes created when calling a [`turbo_tasks::function`] as an optimization. [Converting
98+
/// a local `Vc` to a `ResolvedVc`][Vc::to_resolved] will construct a new
99+
/// [non-local][NonLocalValue] cell.
100+
///
101+
/// These many representations are stored internally using a type-erased [`RawVc`]. Type erasure
102+
/// reduces the [monomorphization] (and therefore binary size and compilation time) required to
103+
/// support `Vc` and its subtypes.
104+
///
105+
/// <div class="warning">
106+
/// <p>
107+
/// Local <code>Vc</code>s are not valid outside of their parent task, so they must be implicitly
108+
/// (e.g. as an argument or return type) or explicitly (e.g. via <a
109+
/// href="#method.to_resolved"><code>Vc::to_resolved</code></a>) be converted to a non-local <a
110+
/// href="struct.ResolvedVc.html"><code>ResolvedVc</code></a> or <a
111+
/// href="struct.VcOperation.html"><code>VcOperation</code></a> before crossing task boundaries.
112+
/// </p>
113+
/// <p>
114+
/// For this reason, <code>Vc</code> types (which are potentially local) will be disallowed as
115+
/// fields in <a href="attr.value.html"><code>turbo_tasks::value</code></a>s in the future.
116+
/// </p>
117+
/// </div>
118+
///
119+
/// | | Representation? | [Non-Local?] | Equality? | Can be Downcast? |
120+
/// |-----------------|-----------------------------|--------------|-------------------------|----------------------------|
121+
/// | [`Vc`] | One of many | ⚠️ Maybe | ❌ Not recommended | ⚠️ After resolution |
122+
/// | [`ResolvedVc`] | Task Id + Type Id + Cell Id | ✅ Yes | ✅ Yes, [see docs][rvc] | ✅ [Yes, cheaply][resolve] |
123+
/// | [`OperationVc`] | Task Id | ✅ Yes | ✅ Yes, [see docs][ovc] | ⚠️ After resolution |
124+
///
125+
/// [Non-Local]: NonLocalValue
126+
/// [rvc]: ResolvedVc
127+
/// [ovc]: ResolvedVc
128+
/// [resolve]: ResolvedVc::try_downcast
129+
///
130+
/// See the documentation for [`ResolvedVc`] and [`OperationVc`] for more details about these
131+
/// subtypes.
132+
///
133+
///
134+
/// ## Execution Model
135+
///
136+
/// While task functions are expected to be side-effect free, their execution behavior is still
137+
/// important for performance reasons, or to code using [collectibles] to represent issues or
138+
/// side-effects.
139+
///
140+
/// Function calls are neither "eager", nor "lazy". Even if not awaited, they are guaranteed to
141+
/// execute (potentially emitting collectibles) before the root task finishes or before the
142+
/// completion of any strongly consistent read containing their call. However, the exact point when
143+
/// that execution begins is an implementation detail. Functions may execute more than once due to
144+
/// dirty task invalidation.
145+
///
146+
///
147+
/// ## Equality & Hashing
148+
///
149+
/// Because `Vc`s can be equivalent but have different representation, it's not recommended to
150+
/// compare `Vc`s by equality. Instead, you should convert a `Vc` to an explicit subtype first
151+
/// (likely [`ResolvedVc`]). Future versions of `Vc` may not implement [`Eq`], [`PartialEq`], or
152+
/// [`Hash`].
153+
///
154+
///
155+
/// [tracing]: crate::trace::TraceRawVcs
156+
/// [`ReadRef`]: crate::ReadRef
157+
/// [`turbo_tasks::function`]: crate::function
158+
/// [monomorphization]: https://doc.rust-lang.org/book/ch10-01-syntax.html#performance-of-code-using-generics
159+
/// [`State`]: crate::State
160+
/// [book-cells]: https://turbopack-rust-docs.vercel.sh/turbo-engine/cells.html
161+
/// [collectibles]: CollectiblesSource
53162
#[must_use]
54163
#[derive(Serialize, Deserialize)]
55164
#[serde(transparent, bound = "")]

turbopack/crates/turbo-tasks/src/vc/operation.rs

Lines changed: 41 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,43 @@ use crate::{
88
Upcast, Vc, VcValueTrait,
99
};
1010

11+
/// A "subtype" (can be converted via [`.connect()`]) of [`Vc`] that
12+
/// represents a specific call (with arguments) to [a task][macro@crate::function].
13+
///
14+
/// Unlike [`Vc`], `OperationVc`:
15+
///
16+
/// - Does not potentially refer to task-local information, meaning that it implements
17+
/// [`NonLocalValue`], and can be used in any [`#[turbo_tasks::value]`][macro@crate::value].
18+
///
19+
/// - Has only one potential internal representation, meaning that it has a saner equality
20+
/// definition.
21+
///
22+
/// - Can be [reconnected][OperationVc::connect] to the strongly-consistent compilation graph after
23+
/// being placed inside of a [`State`].
24+
///
25+
/// - Makes sense with [collectibles][`CollectiblesSource`], as it represents a function call, and
26+
/// only function calls can have issues or side-effects.
27+
///
28+
///
29+
/// ## Equality & Hashing
30+
///
31+
/// Equality between two `OperationVc`s means that both have an identical in-memory representation
32+
/// and point to the same task function call. The implementation of [`Hash`] has similar behavior.
33+
///
34+
/// If [connected] and then `.await`ed at the same time, both would likely resolve to the same
35+
/// [`ReadRef`], though it is possible that they may not if the task or cell is invalidated between
36+
/// `.await`s.
37+
///
38+
/// Because equality is a synchronous operation that cannot read the cell contents, even if the
39+
/// `OperationVc`s are not equal, it is possible that if `.await`ed, both `OperationVc`s could point
40+
/// to the same or equal values.
41+
///
42+
/// [`.connect()`]: OperationVc::connect
43+
/// [reconnected]: OperationVc::connect
44+
/// [connected]: OperationVc::connect
45+
/// [`NonLocalValue`]: crate::NonLocalValue
46+
/// [`State`]: crate::State
47+
/// [`ReadRef`]: crate::ReadRef
1148
#[must_use]
1249
pub struct OperationVc<T>
1350
where
@@ -21,6 +58,9 @@ impl<T: ?Sized> OperationVc<T> {
2158
///
2259
/// The caller must ensure that the `Vc` is not a local task and it points to a a single
2360
/// operation.
61+
///
62+
/// **This API is a placeholder and will likely be removed soon** in favor of a future API that
63+
/// uses macros and static (compile-time) assertions in place of runtime assertions.
2464
pub fn new(node: Vc<T>) -> Self {
2565
// TODO to avoid this runtime check, we should mark functions with `(operation)` and return
2666
// a OperationVc directly
@@ -32,7 +72,7 @@ impl<T: ?Sized> OperationVc<T> {
3272
}
3373

3474
/// Marks this operation's underlying function call as a child of the current task, and returns
35-
/// a dereferenced [`Vc`] that can be [resolved][Vc::to_resolved] or read with `.await?`.
75+
/// a [`Vc`] that can be [resolved][Vc::to_resolved] or read with `.await?`.
3676
///
3777
/// By marking this function call as a child of the current task, turbo-tasks will re-run tasks
3878
/// as-needed to achieve strong consistency at the root of the function call tree. This explicit

turbopack/crates/turbo-tasks/src/vc/resolved.rs

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,52 @@ use crate::{
1616
ResolveTypeError, Upcast, VcRead, VcTransparentRead, VcValueTrait, VcValueType,
1717
};
1818

19+
/// A "subtype" (via [`Deref`]) of [`Vc`] that represents a specific [`Vc::cell`]/`.cell()` or
20+
/// [`ResolvedVc::cell`]/`.resolved_cell()` constructor call within [a task][macro@crate::function].
21+
///
22+
/// Unlike [`Vc`], `ResolvedVc`:
23+
///
24+
/// - Does not potentially refer to task-local information, meaning that it implements
25+
/// [`NonLocalValue`], and can be used in any [`#[turbo_tasks::value]`][macro@crate::value].
26+
///
27+
/// - Has only one potential internal representation, meaning that it has a saner equality
28+
/// definition.
29+
///
30+
/// - Points to a concrete value with a type, and is therefore [cheap to
31+
/// downcast][ResolvedVc::try_downcast].
32+
///
33+
///
34+
/// ## Construction
35+
///
36+
/// There are a few ways to construct a `ResolvedVc`, in order of preference:
37+
///
38+
/// 1. Given a [value][VcValueType], construct a `ResolvedVc` using [`ResolvedVc::cell`] (for
39+
/// "transparent" values) or by calling the generated `.resolved_cell()` constructor on the value
40+
/// type.
41+
///
42+
/// 2. Given an argument to a function using the [`#[turbo_tasks::function]`][macro@crate::function]
43+
/// macro, change the argument's type to a `ResolvedVc`. The [rewritten external signature] will
44+
/// still use [`Vc`], but when the function is called, the [`Vc`] will be resolved.
45+
///
46+
/// 3. Given a [`Vc`], use [`.to_resolved().await?`][Vc::to_resolved].
47+
///
48+
///
49+
/// ## Equality & Hashing
50+
///
51+
/// Equality between two `ResolvedVc`s means that both have an identical in-memory representation
52+
/// and point to the same cell. The implementation of [`Hash`] has similar behavior.
53+
///
54+
/// If `.await`ed at the same time, both would likely resolve to the same [`ReadRef`], though it is
55+
/// possible that they may not if the cell is invalidated between `.await`s.
56+
///
57+
/// Because equality is a synchronous operation that cannot read the cell contents, even if the
58+
/// `ResolvedVc`s are not equal, it is possible that if `.await`ed, both `ResolvedVc`s could point
59+
/// to the same or equal values.
60+
///
61+
///
62+
/// [`NonLocalValue`]: crate::NonLocalValue
63+
/// [rewritten external signature]: https://turbopack-rust-docs.vercel.sh/turbo-engine/tasks.html#external-signature-rewriting
64+
/// [`ReadRef`]: crate::ReadRef
1965
#[derive(Serialize, Deserialize)]
2066
#[serde(transparent, bound = "")]
2167
pub struct ResolvedVc<T>

0 commit comments

Comments
 (0)