Non referentially stable refs cause multiple updates

[yuchi]

When a React element receives a different ref on re-renderer the previous ref is updated to null and the new one is given the current value — regardless of changes in value.

I made a small CodeSandbox to verify this. In it you can also find a “solution”, which is to transform the API into an hook. This let us introduce useCallback to give the user a referentially stable ref.

This doesn’t solve the problem completely: if one of the refs changes all other refs receive a double (and useless) update too.

[gregberge]

I implemented a useMergeRefs that works in all cases, what do you think?

https://codesandbox.io/s/merge-refs-experiment-h5lgc

[yuchi]

Thanks for following up!

In your solution you use a Effect while refs are updated even before the execution of LayoutEffect, i suggest to move to useLayoutEffect.

But even with this correction in place you still have an issue, that refs are updated too late.
I made another CodeSandbox based on yours that shows the problem:

1. There are 2 new refs at the start of the component
2. standardRef is attached directly to a native element (ref={standardRef})
3. manualRef is attached by setting its .current during funcRef3’s execution
4. There’s a useLayoutEffect that expects to find both refs populated (after clicking once on the button)… but manualRef is attached after this effect runs.

[gregberge]

@yuchi OK, so we need to use useUniversalEffect to be compatible with SSR.

[yuchi]

I have no idea about SSR. Currently there’s no way to attach an effect or a callback of some sort before other layout effects are run (only the actual order of execution matters).

I’m seriously thinking this library should strike a balance between correctness and limitations, and an RFC should be made to cover this on the library itself.

[gregberge]

@yuchi we should take a decision to make it right. Could you submit a PR to fix problems?

[agriffis]

I don't think this needs effects at all. Doesn't this work to preserve the resulting callback ref?

/* eslint-disable react-hooks/exhaustive-deps */
import React from 'react'

export const useMergedRefs = refs =>
  React.useCallback(current => {
    refs.forEach(ref => {
      if (typeof ref === 'function') {
        ref(current)
      } else if (ref && !Object.isFrozen(ref)) {
        ref.current = current
      }
    })
  }, refs)

The reason for disabling react-hooks/exhaustive-deps is that we're passing the raw refs array to useCallback, which means that eslint can't statically check it. But it works fine for the actual checking which preserves and updates the callback.

Also, the Object.isFrozen test shows up for me in tests. I'm not sure under what conditions the ref becomes a frozen, empty object, but we just need to avoid assigning in that case.

[theKashey]

There are a lot of different versions of this pattern. Almost every project came with something similar:

• https://react-hooks.org/docs/use-fork-ref
• https://github.com/mitchellhamilton/apply-ref
• https://github.com/theKashey/use-callback-ref
• https://github.com/mui-org/material-ui/blob/9bd4277ecd660ebe2fd4cb08960f58e98ddb6d43/packages/material-ui/src/utils/useForkRef.js

And (strangely!) many are suffering from this issue.

Imho the current API should be just deprecated and hook(useMergedRefs) based version released instead of it.

[oliviertassinari]

Regarding the version in Material-UI. I believe we handle it correctly: https://codesandbox.io/s/merge-refs-experiment-forked-zcokt?file=/src/App.js.

import { unstable_useForkRef as useForkRef } from "@material-ui/utils";

We have faced this issue when rewriting all the components to be hook based. @yuchi would you confirm?

[theKashey]

@oliviertassinari - I can confirm that your realization is stable.

But we can get another problem in the future as the Strict and Concurrent modes are not very any ref tricks friendly.

[yuchi]

MUI's version is stable, but only in a single specific sense, and formally fragile. Let me explain.

A perfect ref merger should:

1. Be stable against re-renders, if no input ref has changed the resulting ref must be the same as the previous one, so that it doesn't uselessly trigger an update.
2. Be transparent, from the outside, from the standpoint of a single ref creator, there should be no difference between using the ref as-is and merged with other refs.

MUI's current implementation has the following problems:

1. It uses useMemo which as we all know as historically been defined as an optimization tool, and should not be used to have "singleton" semantics. In this case avoiding recreating a ref is some kind of an optimization trick, but since ref updating is in the imperative world (a function ref can have side effects ^{[citation needed]}), we need more guarantees.
2. When one of the ref changes a new merged ref is created and all refs gets updated, again ruining the expectation about when and how function refs are called.

[yuchi]

Again, I feel like this cannot be solved in user space.

[agriffis]

You could add a WeakMap to avoid calling any refs with the same current repeatedly:

/* eslint-disable react-hooks/exhaustive-deps */
import {useCallback, useRef} from 'react'

export const useMergedRefs = refs => {
  const weak = useRef(new WeakMap()).current
  return useCallback(current => {
    refs.forEach(ref => {
      if (!weak.has(ref) || weak.get(ref) !== current) {
        if (typeof ref === 'function') {
          ref(current)
        } else if (ref && !Object.isFrozen(ref)) {
          ref.current = current
        }
        weak.set(ref, current)
      }
    })
  }, refs)
}

[yuchi]

@agriffis I was thinking about it too, but I’m still trying to come up with a case where a ref is called again with the same value. If we find that case than this approach cannot work. Also, your implementation has a bug, when refs change the merged ref changes, so the previous merged ref is called with null by React, therefore setting al refs to null in the WeakMap…

[agriffis]

@yuchi Very interesting. Well here is another attempt...

1. Returns a stable callback (it will never change)
2. Call all sub-refs with the same effective lifecycle as React does (no extra calls)
3. Handles adding and removing sub-refs gracefully
4. Should work in concurrent mode, too, by moving side effects (ref assignments) out of the render phase, as @diegohaz shows at https://twitter.com/diegohaz/status/1324168516061245441
5. Cool new name

Demo sandbox with console logging at https://codesandbox.io/s/use-reflector-52n36

import {useCallback, useEffect, useLayoutEffect, useRef} from 'react'

const useIsomorphicLayoutEffect =
  typeof window === 'undefined' ? useEffect : useLayoutEffect

const UNSET = {} // unique object

export default function useReflector(refs) {
  const value = useRef(UNSET)
  const captured = useRef(refs)
  const assigned = useRef(new WeakMap())

  // This effect runs on every commit, to capture refs that were passed during
  // render. This effect also emulates the behavior of React of nulling/setting
  // refs when they are added or removed. This is necessary because the callback
  // returned by useReflector is stable, so it will not get called when the
  // refs passed into the hook change.
  useIsomorphicLayoutEffect(() => {
    // Starve (set to null) any refs that have fallen out.
    starve(refs, captured, assigned)

    // Capture passed refs, so we can feed them.
    captured.current = refs

    // Feed (set to current value) any new refs that have been added, unless the
    // callback has not yet been called.
    if (value.current !== UNSET) {
      feed(value, captured, assigned)
    }
  })

  // Return a stable ref callback that will feed the captured refs.
  return useCallback(v => {
    value.current = v
    feed(value, captured, assigned)
  }, [])
}

/**
 * Feed the value to each of the captured refs, if it was not previously
 * assigned to the same value.
 */
function feed(value, captured, assigned) {
  const v = value.current
  const a = assigned.current
  captured.current.forEach(ref => {
    if (v === undefined ? !a.has(ref) : a.get(ref) !== v) {
      assign(ref, v)
      a.set(ref, v)
    }
  })
}

/**
 * Assign null to each of the captured refs which is no longer in the set.
 */
function starve(refs, captured, assigned) {
  const a = assigned.current
  captured.current.forEach(ref => {
    if (a.has(ref) && !refs.includes(ref)) {
      assign(ref, null)
      a.delete(ref)
    }
  })
}

/**
 * Assign to a ref, which might be a useRef or a callback ref.
 */
function assign(ref, value) {
  if (typeof ref === 'function') {
    ref(value)
  } else if (ref && !Object.isFrozen(ref)) {
    ref.current = value
  }
}

[yuchi]

@agriffis This is a very elegant solution with a major drawback… Please don’t hate me.

function MyComponent() {
  const [rerender, setRerender] = React.useState(false);
  const ref = React.useRef(null);

  React.useLayoutEffect(() => {
    console.log(ref.current); // <---- problem!
    setRerender(true);
  });

  const merged = useReflector(rerender ? [ref] : []);

  return rerender ?  <span ref={merged} /> : <div ref={merged} />
}

Since you are updating refs at Effect/LayoutEffect time all Effect/LayoutEffect run before useReflector is called don’t get the values update for new refs. Previously present refs get correctly update at the right time in the returned function ref.

[yuchi]

What scares me is the terribly subtlety of the semantic differences. So, somehow, I prefer a more raw approach of simply updating the ref every time a ref changes, and explicitly adivse the user against these problems when using function refs.

[agriffis]

@yuchi No worries, I concur. This was as close to a proper solution as I think we can get with existing React helpers.

• If we return a new callback, it will get called by React every time with the null-then-value sequence. We are powerless in the callback to know the value will be restored in a following call, so we can't protect sub-refs from the noise.

• If we return a stable callback, it won't get called by React, even when the list of sub-refs has changed. We can work around that with an effect, but then we can't guarantee refs will be updated before the effects that depend on them.

Concurrent mode doesn't change anything. Both classic and concurrent rendering modes are subject to these constraints.

So for now, the original solution I posted on this issue might be the best after all: #5 (comment)

It would be really nice if React would accept an array of refs and manage this internally.

[yuchi]

If we had to implement the perfect solution in user space the only required hook would be useRefUpdate which is called before all LayoutEffects, but at this point it would be easier for the React team to implement the functionality in core.

[ZachHaber]

@yuchi, I figured I'd throw my hat in the ring for this. Even though it's been a long time since the last discussion in this space, I still think it's the most in-depth discussion of merging refs I can find.

This variant makes it so that you can just use this one ref regardless of whether or not a ref was actually passed into the forwardRef component, which is one of the frustrating things that ends up requiring merging the refs to begin with.

The only thing I'm not sure of is whether properties on the ref object have to go on current it doesn't seem that way, and I could fairly easily adjust it to work without that by adding another useRef to store the current refs input value, or sticking an object on the ref and making the proxy handle get/set current to work with the value itself without the refs.

Usage:

    const Input = React.forwardRef<HTMLInputElement, {}>((props, fRef) => {
      const hookRef = useHookThatReturnsARefToDoThings();
      const ref = useRefs<HTMLInputElement>(null, [fRef, hookRef]);
      React.useEffect(() => {
        if (ref.current) {
          // Focus this input on mounting for some reason
          ref.current.focus();
        }
      }, []);

      return <input {...props} ref={ref} />;
    });

useRefs

import type * as React from "react";
import { useRef, useState } from "react";

const refsSymbol = Symbol("refs");
type AcceptedRef<T> = React.MutableRefObject<T> | React.LegacyRef<T>;

/**
 * `useRefs` returns a mutable ref object whose .current property is initialized to the passed argument (initialValue).
 * The returned object will persist for the full lifetime of the component.
 *
 * This is generally equivalent to `useRef` with the added benefit to keep other refs in sync with this one
 *
 * Note that `useRefs()` is useful for more than the ref attribute. It’s handy for keeping any mutable value around similar to how you’d use instance fields in classes.
 * @param initialValue The initial value for the ref. If it's `null` or `undefined`, the initially provided refs won't be updated with it
 * @param refs Optional refs array to keep updated with this ref
 * @returns Mutable Ref object to allow both reading and manipulating the ref from this hook.
 */
export function useRefs<T>(
  initialValue: T,
  refs?: Array<AcceptedRef<T>>
): React.MutableRefObject<T>;
/**
 * `useRefs` returns a mutable ref object whose .current property is initialized to the passed argument (initialValue).
 * The returned object will persist for the full lifetime of the component.
 *
 * This is generally equivalent to `useRef` with the added benefit to keep other refs in sync with this one
 *
 * Note that `useRefs()` is useful for more than the ref attribute. It’s handy for keeping any mutable value around similar to how you’d use instance fields in classes.
 * @param initialValue The initial value for the ref. If it's `null` or `undefined`, the initially provided refs won't be updated with it
 * @param refs Optional refs array to keep updated with this ref
 * @returns Mutable Ref object to allow both reading and manipulating the ref from this hook.
 */
export function useRefs<T>(
  initialValue: T | null,
  refs?: Array<AcceptedRef<T | null>>
): React.RefObject<T>;
/**
 * `useRefs` returns a mutable ref object whose .current property is initialized to the passed argument (initialValue).
 * The returned object will persist for the full lifetime of the component.
 *
 * This is generally equivalent to `useRef` with the added benefit to keep other refs in sync with this one
 *
 * Note that `useRefs()` is useful for more than the ref attribute. It’s handy for keeping any mutable value around similar to how you’d use instance fields in classes.
 * @param initialValue The initial value for the ref. If it's `null` or `undefined`, the initially provided refs won't be updated with it
 * @param refs Optional refs array to keep updated with this ref
 * @returns Mutable Ref object to allow both reading and manipulating the ref from this hook.
 */
export function useRefs<T = undefined>(
  initialValue?: undefined,
  refs?: Array<AcceptedRef<T | undefined>>
): React.RefObject<T | undefined>;
/**
 * `useRefs` returns a mutable ref object whose .current property is initialized to the passed argument (initialValue).
 * The returned object will persist for the full lifetime of the component.
 *
 * This is generally equivalent to `useRef` with the added benefit to keep other refs in sync with this one
 *
 * Note that `useRefs()` is useful for more than the ref attribute. It’s handy for keeping any mutable value around similar to how you’d use instance fields in classes.
 * @param initialValue The initial value for the ref. If it's `null` or `undefined`, the initially provided refs won't be updated with it
 * @param refs Optional refs array to keep updated with this ref
 * @returns Mutable Ref object to allow both reading and manipulating the ref from this hook.
 */
export function useRefs<T>(
  initialValue: T,
  refs?: Array<AcceptedRef<T>>
): React.MutableRefObject<T> {
  const refToProxy = useRef<T>(
    initialValue as any
  ) as React.MutableRefObject<T> & {
    [refsSymbol]: Array<AcceptedRef<T>>;
  };
  // Create the proxy inside useState to ensure it's only ever created once
  const [proxiedRef] = useState(() => {
    function applyRefValue(ref: AcceptedRef<T>, value: T | null) {
      if (typeof ref === "function") {
        ref(value);
      } else if (ref && !Object.isFrozen(ref)) {
        (ref as React.MutableRefObject<T | null>).current = value;
      }
    }
    const proxy = new Proxy(refToProxy, {
      set(target, p, value, receiver) {
        if (p === "current") {
          target[refsSymbol].forEach((ref) => {
            applyRefValue(ref, value);
          });
        } else if (p === refsSymbol && Array.isArray(value)) {
          const { current } = target;
          if (current != null) {
            // Check which refs have changed.
            // There will still be some duplication if the refs passed in change
            // *and* the ref value changes in the same render
            const prevSet = new Set(target[refsSymbol]);
            const newSet = new Set(value as AcceptedRef<T>[]);
            prevSet.forEach((ref) => {
              // Clear the value from removed refs
              if (!newSet.has(ref)) {
                applyRefValue(ref, null);
              }
            });
            newSet.forEach((ref) => {
              // Add the value to new refs
              if (!prevSet.has(ref)) {
                applyRefValue(ref, current);
              }
            });
          }
        }
        return Reflect.set(target, p, value, receiver);
      },
    });
    return proxy;
  });
  // ensure the refs is always an array
  // Update the current refs on each render
  // Refs are mutable and thus a bit
  proxiedRef[refsSymbol] = refs || [];
  return proxiedRef;
}

[theKashey]

@ZachHaber - big kudos for splitting ref declarations. You actually don't need to specify JSDocs for every single one as they will share the one with each other.

From another note - my own solution for this works quite similar to your one. Just without Proxies - on vanilla getters and setters - https://github.com/theKashey/use-callback-ref/blob/master/src/useRef.ts#L29

[agriffis]

@ZachHaber I set out this morning to create a test repo to demonstrate the problems with your approach:

https://github.com/agriffis/merge-refs-playground

The test repo specifically checks for the things that @yuchi and I were talking about. As we'd discussed, my useMergedRefs hook fails one nit-picky test, and my useReflector fails another nit-picky test. Both of these are nit-picky in the sense that they probably don't affect real-world use, but the mere fact that the merged ref diverges slightly is... annoying.

Your approach fails... um... let's see... count the ways... that can't be right... try again... clean glasses... hmmm...

Your approach works great and passes the nit-picky tests. Very nice! I never considered using a merged ref that wasn't a callback.

@theKashey I'm out of gas, but interested in your modification. Do you want to make me a quick PR with your hook?

[agriffis]

@ZachHaber I guess I'm still not sure about being safe for concurrent mode. The reason my useReflector takes the approach it does, is to ensure that we're not changing stateful data during render, instead doing it in useLayoutEffect. The problem is that it can be out of step with other useLayoutEffect invocations.

Is there any chance that your refs array will be incorrect in concurrent mode? I'm not sure how to demonstrate with a test.

(Same is true for @theKashey's hook, which also updates stateful data during render.)

[theKashey]

🙇 there are definitely a few moments to fix

---

UPD: Actually, after reviewing your tests, especially the last one - I am not sure they are correct.
Like yes - useRef is a portal to mutable world, but those tests are a little too mutable

• all built-in hooks are not always enforcing stable usage, but also throws if you pass unstable number of dependencies
• tests failed above do rely on "unstable" dependencies which is
  • against the rule of hooks "sense"
  • can be done by external logic, does not have to be backed-in merge refs

[agriffis]

@theKashey One of the goals we were trying to achieve is for the hook to handle changed refs transparently. That was implied by @yuchi's comment about MUI's implementation: "When one of the ref changes a new merged ref is created and all refs gets updated, again ruining the expectation about when and how function refs are called."

It would be a reasonable implementation to state: "refs cannot change once passed" or "the number of refs can't change from the first call" but I'm more interested in a hook that doesn't have these restrictions.

Therefore the tests check for these scenarios. Just because a hook fails some tests doesn't mean it isn't useful, it just means that there may be restrictions on how it can or should be called.

[ZachHaber]

@ZachHaber - big kudos for splitting ref declarations. You actually don't need to specify JSDocs for every single one as they will share the one with each other.

Using VSCode, I've generally found that they don't share, making it so you quickly end up with having to flip through the listings to find the one with the documentation.

@ZachHaber I guess I'm still not sure about being safe for concurrent mode. The reason my useReflector takes the approach it does, is to ensure that we're not changing stateful data during render, instead doing it in useLayoutEffect. The problem is that it can be out of step with other useLayoutEffect invocations.

Is there any chance that your refs array will be incorrect in concurrent mode? I'm not sure how to demonstrate with a test.

Honestly, I'm worried about that too. As long as you are careful in using it, I believe it should be fine. I.e. memoizing any callback ref inputs and not changing the number of refs being merged. I legitimately can't think of a user-land way to make the implementation truly perfect.

Perhaps the best we can do currently is have a set of multiple hooks with the various trade-offs explicitly clear?

[agriffis]

Perhaps the best we can do currently is have a set of multiple hooks with the various trade-offs explicitly clear?

I think that's what we have!

	concurrent safe	avoids layout effect	doesn't toggle refs	adds/removes refs
agriffis/useMergedRefs	✓	✓		✓
agriffis/useReflector	✓		✓	✓
ZachHaber/useRefs		✓	✓	✓
theKashey/use-callback-ref		✓	✓

Definitions:

• concurrent safe: definitely works in concurrent mode (i.e. avoids state updates during render).
• avoids layout effect: doesn't need to be sequenced by caller to ensure that ref is set before layout effects run.
• doesn't toggle refs: when list of passed refs changes, protects sub-refs from null-to-value flip-flop.

Personally, I would rather avoid any additional complication arising from the hook itself, since this hook is used in places that are already inherently complicated. That means that, ideally, I would like to use a hook that is assuredly concurrent-safe and doesn't have special requirements for sequencing with layout effects.

So the last question is, what's the big deal with toggling refs? Quick explanation: When useMergedRefs receives a modified list of refs, it returns a new callback. When React receives a new callback ref, it first calls the old one with null and then calls the new one with the proper value. Therefore any refs that are in both old/new sets are temporarily set to null before being reset to the proper value.

The interesting part is that these two steps happen directly in sequence. There's no render pass while the ref is nulled, so the application code never sees the null. The only way it can know that something happened is if it is passing a callback ref, but even then, it likely only means some kind of teardown and then buildup. No big deal.

Based on all that, I'll probably stick with useMergedRefs. Also, I like that the code is quite small: https://github.com/agriffis/merge-refs-playground/blob/main/src/hooks/agriffis/useMergedRefs.js 😄

[ZachHaber]

@agriffis, don't forget about the tradeoffs of stable reference vs unstable i.e.
would it ever trigger an effect via the dependency array.

In this case, an unstable reference could make it so that passing a mergedRef into another mergedRef would cause extra flip-flops on every render due to the change in one merged ref causing a change in the next merged ref. Which might actually pose a problem if someone passed in a setter from useState into the first merged ref!

Also, don't undervalue the ability to both merge the ref and actually make it controllable like a normal ref object without another ref to merge in to be able to see the current value! ☺️

[agriffis]

@ZachHaber Great points!

I added three more tests:

• returns a stable ref over sub-ref updates
• returns a gettable ref
• returns a settable ref

I also added useMergedRefs2 which is a spin on the original to use a settable/gettable ref instead of a callback. However it still prioritizes concurrent mode compatibility over returning a stable ref when the list of sub-refs changes. This one is now very similar to yours except for that detail.

Current test result output is here

[ZachHaber]

@agriffis I made a slight modification to mine to make it concurrent safe (using useLayoutEffect) agriffis/merge-refs-playground#2

I thought I had figured out something that would work perfectly by trying to use useImperativeHandle to trigger the updates to the refs array, unfortunately, that ends up as a layoutEffect in React's internals, so I just went with a layout effect in the end.

I think the main difference is stable ref vs unstable ref. I don't think it's possible to make an unstable ref safely avoid the value flip-flops, just like how I don't think it's possible currently to make a stable ref work before layout effects without making it potentially unsafe for concurrent mode.

There's a few bugs mentioning this in react: facebook/react#7769 (comment) for one. I wonder if we should put up a separate issue to request that useImperativeHandle is applied at the same time as other refs in normal DOM nodes. Though I worry that will be considered a breaking change.

[agriffis]

I think the main difference is stable ref vs unstable ref. I don't think it's possible to make an unstable ref safely avoid the value flip-flops, just like how I don't think it's possible currently to make a stable ref work before layout effects without making it potentially unsafe for concurrent mode.

Yup.

I think it would be great to request or RFC or PR a change to React to fix this, either by supplying a proper mechanism for writing a ref merger, or by supplying a ref merger. The reason I haven't personally is that I'm pretty sure they've been aware of the issue for a long time and haven't done anything about it. But maybe I shouldn't be so pessimistic (and/or lazy).