Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add docs around allow_refs feature #862

Merged
merged 3 commits into from
Oct 3, 2023
Merged

Add docs around allow_refs feature #862

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
620b0c5
Add docs around allow_refs feature
philippjfr Oct 2, 2023
a5a7850
Address comments
philippjfr Oct 2, 2023
9da7108
Incorporate comments
philippjfr Oct 3, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
119 changes: 119 additions & 0 deletions doc/user_guide/How_Param_Works.ipynb
View file
Open in desktop
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • The rest of this page is pretty mechanistic until this section. e.g. what is the mechanism of the syncing? what is happening to unsync when b2.p is set?
  • What happens if you try to do b2 = B(p=b1.param.p) if allow_refs was False? Why?
  • Is there a limit on how many references can be made to the same object? impact on performance?
  • Any guards against circular referencing?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion for new intro text:

Beyond the custom attribute access mechanisms of a single Parameter, Param can link together multiple Parameters. With the allow_refs option, a Parameter can act as a dynamic reference to another Parameter. This enables the values of two or more Parameters to stay in sync, allowing for reactive development. Any change to the referenced Parameter's value is automatically reflected in all Parameters that reference it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any guards against circular referencing?

@philippjfr what is your take on bi-directionally linking Parameters in Param? I'd guess there's a way for this to be implemented, if so let's open an issue to discuss the API? I'd love Param 2.1 to have it to be able to deprecate Panel's .link.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, would like to see the same.

Original file line number Diff line number Diff line change
Expand Up @@ -237,6 +237,125 @@
"You can re-execute the above code changing to `per_instance=True` and/or `instantiate=True` on Parameter `p` and see how the behavior differs. With `per_instance=True` (which would normally be the default), `a1` and `a2` would each have independent copies of the `Parameter` object, and with `instantiate=True`, each instance would get its own copy of the class's default value, making it immune to later changes at the class level."
]
},
{
"cell_type": "markdown",
"id": "b86f55e3-5ae5-4879-9b4c-273e859b6fec",
"metadata": {},
"source": [
"## References\n",
"\n",
"The custom attribute access allows accessing the concrete value of a `Parameter` but via the `.param` namespace we can also access the underlying Parameter value. In many scenarios the `Parameter` object can be treated as a reference that represents the current value of that parameter. Specifically we can tell a `Parameter` that it should `allow_refs`, i.e. it should be able to accept a reference to another `Parameter` and resolve it dynamically, ensuring that the two `Parameter` values remain synced:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "fba9fbdc-0776-4cd3-8250-e00487593d57",
"metadata": {},
"outputs": [],
"source": [
"class B(Parameterized):\n",
" \n",
" p = Parameter(default=1, allow_refs=True)"
]
},
{
"cell_type": "markdown",
"id": "27beb80b-415f-41cb-b909-adb6822e7e9a",
"metadata": {},
"source": [
"Having declared a `Parameter` that allows references we can now pass the parameter `p` of `b1` to parameter `p` of `b2`:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "22a51c29-c286-4e0f-b1a6-1e02bc4149da",
"metadata": {},
"outputs": [],
"source": [
"b1 = B(p=14)\n",
"b2 = B(p=b1.param.p)"
]
},
{
"cell_type": "markdown",
"id": "5eab5e13-66df-4001-8d70-2cab70449641",
"metadata": {},
"source": [
"Inspecting `b2.p` we will see that `p` of `b2` now reflects the value of `b1.p`:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "fdc58565-f327-405f-958d-b49caa70adb3",
"metadata": {},
"outputs": [],
"source": [
"b2.p"
]
},
{
"cell_type": "markdown",
"id": "7322e310-9105-48c9-a0b8-41cb80bcf4e6",
"metadata": {},
"source": [
"Even when we update the value of `b1.p` the value of `b2.p` will reflect the change:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "961aebc0-45b5-458d-919c-1ba6a3c286e7",
"metadata": {},
"outputs": [],
"source": [
"b1.p = 7\n",
"\n",
"b2.p"
]
},
{
"cell_type": "markdown",
"id": "75c5eb72-3132-458c-9b30-b5cd1e64b63d",
"metadata": {},
"source": [
"If we explicitly set `b2.p` however the two values will become unsynced:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2e02cb21-a4ab-48fb-a516-da17feaf0da0",
"metadata": {},
"outputs": [],
"source": [
"b2.p = 3\n",
"\n",
"print(b1.p, b2.p)"
]
},
{
"cell_type": "markdown",
"id": "701e26c3-6eb0-4c76-b453-cc37be06ac41",
"metadata": {},
"source": [
"and any subsequent changes to `b1.p` will not be reflected by `b2`:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "50aedbea-7902-4f1a-8e98-6866a965eda8",
"metadata": {},
"outputs": [],
"source": [
"b1.p = 27\n",
"\n",
"b2.p"
]
},
{
"cell_type": "markdown",
"id": "c3e3d3aa",
Expand Down
150 changes: 150 additions & 0 deletions doc/user_guide/Parameters.ipynb
View file
Open in desktop
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's my suggestion for new intro text, including a more descriptive header:

Dynamically linking parameters

Building on the previous discussion about the dual roles of a Parameter - as both a value holder and a metadata container - let's explore how parameters can go a step further by acting as dynamic references to other parameters. When configured with allow_refs=True, a Parameter can serve as a live link to another Parameter, mirroring its current value. This capability enables more intricate relationships between parameters, allowing for automatic value synchronization and forming the basis for reactive programming.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand how the example used to demonstrate nested_refs is an example of nesting; u1 and u2 don't contain any references...?

I would have expected something more like this:

class U(param.Parameterized):
    a = param.Number()

class V(param.Parameterized):
    b = param.Number(allow_refs=True)

class W(param.Parameterized):
    c = param.Number(allow_refs=True, nested_refs=True)

u = U(a=5)
v = V(b=u.param.a)
w = W(c=v.param.b)

Copy link
Member Author

@philippjfr philippjfr Oct 3, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is what demonstrates nesting, i.e. the references are nested inside some other container:

w = W(c=[u1.param.a, u2.param.a])

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Chaining references like you have done doesn't require any special support.

Original file line number Diff line number Diff line change
Expand Up @@ -784,6 +784,156 @@
"This approach can provide significant speedup and memory savings in certain cases, but should only be used for good reasons, since it can cause confusion for any code expecting instances to be independent as they have been declared."
]
},
{
"cell_type": "markdown",
"id": "2579c782-8dfd-4ded-995f-dc8498f4d275",
"metadata": {},
"source": [
"## Parameter references\n",
"\n",
"Having looked at the difference between the `Parameter` value and the `Parameter` object let us consider the idea that a `Parameter` is a reference or proxy for its underlying value. Parameters may be declared to `allow_refs` which effectively means that they will accept other Parameter **objects** (and other valid references) and resolve and reflect their current values."
philippjfr marked this conversation as resolved.
Show resolved Hide resolved
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "111e4911-4b73-40c3-ab57-70cd42eb7fb8",
"metadata": {},
"outputs": [],
"source": [
"class U(param.Parameterized):\n",
" \n",
" a = param.Number()\n",
" \n",
"class V(param.Parameterized):\n",
" \n",
" b = param.Number(allow_refs=True)\n",
" \n",
"u = U(a=3.14)\n",
"v = V(b=u.param.a)\n",
"\n",
"v.b"
]
},
{
"cell_type": "markdown",
"id": "324475cf-65a3-4902-8508-57af19697b5f",
"metadata": {},
"source": [
"By declaring that `V.b` allows references we have made it possible to pass the Parameter `U.b`, which means `v.b` will reflect the value of `u.a`:"
philippjfr marked this conversation as resolved.
Show resolved Hide resolved
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "62016252-5a7d-4bb7-be7a-029ed1a47025",
"metadata": {},
"outputs": [],
"source": [
"u.a = 1.57\n",
"\n",
"v.b"
]
},
{
"cell_type": "markdown",
"id": "c6f62936-89bb-49e3-b197-cef43891b2f7",
"metadata": {},
"source": [
"This unidirectional link will be in effect until something else tries to set the value:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "28094868-4f23-4399-be7b-da0c73854be7",
"metadata": {},
"outputs": [],
"source": [
"v.b = 14.1\n",
"u.a = 13.2\n",
"\n",
"v.b"
]
},
{
"cell_type": "markdown",
"id": "48878238-e062-4fd4-bb99-0e9020d9b26f",
"metadata": {},
"source": [
"In other words, if the value is overridden from the outside the link will be automatically removed.\n",
"\n",
"Simple references are resolved when `allow_refs=True` but to allow nested references we separately have to set `nested_refs=True`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d63d949d-79a2-471f-b0bb-232086e486bf",
"metadata": {},
"outputs": [],
"source": [
"class W(V):\n",
" \n",
" c = param.List(allow_refs=True, nested_refs=True)\n",
" \n",
"u1 = U(a=3)\n",
"u2 = U(a=13)\n",
"\n",
"w = W(c=[u1.param.a, u2.param.a]) \n",
"\n",
"w.c"
]
},
{
"cell_type": "markdown",
"id": "e7b3489f-4a75-4767-a7d2-b7066a976ce4",
"metadata": {},
"source": [
"When we modify either `u1.a` or `u2.a`, `w.c` will update:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c2d9fc30-d5cb-49ac-aaf7-d4d6f1d74cb0",
"metadata": {},
"outputs": [],
"source": [
"u1.a = 7\n",
"\n",
"w.c"
]
},
{
"cell_type": "markdown",
"id": "ae2d21b3-c78b-4648-8622-282c237a1d64",
"metadata": {},
"source": [
"Note that `Parameter` types are not the only types of valid references. The full list of valid references include:\n",
"\n",
"- Class and instance `Parameter` objects\n",
"- Functions or methods annotated with `param.depends`\n",
"- Functions wrapped with `param.bind`\n",
"- Reactive expressions declared using `param.rx`\n",
"- Asynchronous generators\n",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Allowing bare asynchronous generators isn't going to cause problems with param.Callable? Is it the kind of case were users will have to set allow_refs=False and won't have the option between passing a normal async generator or a "reference async generator"?
Super edge-case I know, it's just to understand the consequences.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is going to cause problems if you enable allow_refs=True but for a callable there's rarely ever a good reason to do so.

Copy link
Member Author

@philippjfr philippjfr Oct 2, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The deeper point though is that currently we don't have a way to allow some reference types and not others, which I guess is a problem.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The trick I originally used in Panel was that I would first check if a value would pass _validate before I attempted to resolve the reference, the issue there is that Dynamic parameters (including Number) allow functions/generators and you want to allow references even on param.Parameter types.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is going to cause problems if you enable allow_refs=True but for a callable there's rarely ever a good reason to do so.

Is allow_refs set to True for pn.viewable.Viewer objects?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not currently, but Jim strongly suggested it should be.

"- Custom objects transformed into a valid reference with a hook registered with `param.parameterized.register_reference_transform`.\n",
maximlt marked this conversation as resolved.
Show resolved Hide resolved
"\n",
"There are two utility functions which allow resolving the value of a reference and all parameters the reference depends on:"
philippjfr marked this conversation as resolved.
Show resolved Hide resolved
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8cb30e85-5cfe-49d6-a641-f98c84a7acb2",
"metadata": {},
"outputs": [],
"source": [
"from param.parameterized import resolve_ref, resolve_value\n",
philippjfr marked this conversation as resolved.
Show resolved Hide resolved
"\n",
"resolve_ref(u1.param.a), resolve_value(u1.param.a)"
]
},
{
"cell_type": "markdown",
"id": "678b7a0e",
Expand Down