Refresh does not request pages in infinite hits again

[Haroenv]

Describe the bug 🐛

When updating the refresh token, infinite hits will not react to the refetch.

To Reproduce 🔍

Steps to reproduce the behavior:

1. go to the example
2. trigger next page
3. refresh
4. see no changes in the infinite hits, even if the index would have changed

The example's index can be changed to an index you control where you make changes to make the effect clearer.

https://codesandbox.io/s/react-instantsearch-app-dq2ro

Expected behavior 💭

Infinite hits reacts to refresh, either by:

1. clearing the cache completely and restarting on the current page
2. clearing the cache and the state and restarting on page 0

Likely we should go for option 2, unless there's a showPrevious.

Additional context

Add any other context about the problem here.

When refresh happens, we need to make sure the infinite hits internal cache is also invalidated.

On React Native, where the list component itself is stateful, we can not rely on the "key" hack, because it rerenders with an empty state when we simply clear the cache. What could be an option is:

1. clear cache
2. redo current search
3. save in cache
4. rerender

The problem is that you can't do that as a user reasonably, since you don't have access to the helper state.

A possible solution is:

In the function refresh in InstantSearchManager, emit an event to all widgets. Then in InfiniteHits, listen to that event, and clear the internal cache as we expect.

Another potential solution is to return a promise from the search which happens in refresh. This should allow people to rerender the InfiniteHits component manually.

Relevant pieces of code:

https://github.com/algolia/react-instantsearch/blob/ec9e0fbd6106d1c3e47f1dbfa6eaac3e20af6bd5/packages/react-instantsearch-core/src/core/createInstantSearchManager.js#L69-L72

Relevant issues:

algolia/react-instantsearch#2464

[alexpchin]

Hi all,

Just to give you a demonstration of what happens here if you refresh using a key. Unfortunately, you momentarily receive hits of length 0.

I have created an example to use ClearCache and a searchClient object - which does not work. https://snack.expo.io/@alexpchin/aglolia-refresh-problem
Please run on iOS and look out for the red flash.

The full code is here: https://github.com/alexpchin/algolia-refresh-issue/

The docs say this method is (browserOnly)?
https://www.algolia.com/doc/api-client/advanced/cache-browser-only/javascript/?language=javascript

I think it’s reasonable that the index updates and the results go back to page 0.

Thanks,
Alex

[Haroenv]

browserOnly implies not for node, it works in React Native. We are aware of this issue, but aren't sure what the solution could be unfortunately.

Without having a key, it behaves correctly, except for InfiniteHits, which keeps its cache and doesn't update. The problem is that we need to change things to be able to clear the cache on InfiniteHits too.

There's multiple options here:

1. add an option to clear the internal cache of InfiniteHits (same problem though, when do you clear it?)
2. Clear the internal cache of InfiniteHits when refresh is called (how?)

[alexpchin]

Thank you for the clarification. I believe I was originally the person who contacted RE: the problem and was just adding some more information so that people could see clearly the effect that is causes with the hits being momentarily empty when using a key to refresh.

Our project relies heavily on Algolia to display infinite results.

The idea solution would be that when refresh is given to InstantSearch, an event is emitted to the InfiniteHits so that we do not have to re-mount the component with a key.

To the user, I believe the disruption is only seen by hits length becoming momentarily 0
When the component is re-mounted.

[alexpchin]

Just to be 100% clear, what is the benefit of using the connectInfiniteHits connector over building a search using the Algolia Search API and passing filters to it? Something not dissimilar to (with a bit more work for pagination etc)?

import algoliasearch from 'algoliasearch/lite';
import * as C from 'src/constants';

const client = algoliasearch(C.ALGOLIA_APP_ID, C.ALGOLIA_API_KEY);

export const algoliaSearch = async ({
  indexName,
  query = '',
  filters = '',
  hitsPerPage = 20,
  attributesToRetrieve = ['*'],
}) => {
  try {
    const index = client.initIndex(indexName);
    const response = await index.search(query, {
      filters,
      attributesToRetrieve,
      hitsPerPage,
    });
    return response;
  } catch (err) {
    console.log(err);
    console.log(err.debugData);
  }
};

[Haroenv]

If all you are doing is showing results, there's no real advantage to use InfiniteHits in your case. Since we haven't fully solved "refresh" for InfiniteHits, it's a fair solution to fall back to the JS client.

To refresh on it, call client.clearCache and call your search/render function again.

Sorry for not yet fixing this refresh issue in the mean time :)

[alexpchin]

Hi all,

I was just wondering whether there was any update to this? I'm still having a little trouble "rolling my own" infinite hits using the API. My setup requires the ability to search, show no duplicated results, have infinite scroll etc. Everything with the ConnectInfiniteHits works except the problem stated above.

Thanks in advance,
Alex

[Haroenv]

Sorry @alexpchin, this is very complicated from our end to get something like this out the door, and it has not yet been prioritised as an issue essential to fix right now.

[alexpchin]

No problem! Thanks for the swift response 👍

[alexpchin]

Hi @Haroenv any plans to implement this?

[Haroenv]

Not currently, since you're the only person who has noted this bug / wrong behaviour. I totally agree that it's a problem, but since the solution is unclear we don't have fixing this planned. Sorry!

[yilinjuang]

Hi @Haroenv I'm also having this issue. Tried to implement scroll to top and refresh the search when user taps the tab bar. Using infinitehits on react native. Any workarounds besides re-mounting the entire search component?

[Haroenv]

You can remount the <InfiniteHits> component alone, at the same time as using refresh, which will clear its cache

[DRoghanian]

Just dropping in to say I've encountered the same issue.

[hamedhemmati]

I have also encountered this issue. I use the pull to refresh of Flatlist with a callback which sets the refresh to true and remounts the <InfiniteHits>. The first time I pull it doesn't show an updated list but the second time I pull then it shows the updated list.

update: as a workaround, I remount the <InfiniteHits> after a second using setTimeout and it seems to be working. Not ideal but at least it works for now.

[JimTeva]

Refresh with InifiniteHits seems not to work

[JimTeva]

My only workaround was to clear cache of inifiniteHits manually using searchClient.clearCache()

[eunjae-lee]

You can control the internal cache of InfiniteHits now.

import isEqual from 'react-fast-compare';

function getStateWithoutPage(state) {
  const { page, ...rest } = state || {};
  return rest;
}

function getInMemoryCache() {
  let cachedHits = undefined;
  let cachedState = undefined;
  return {
    read({ state }) {
      return isEqual(cachedState, getStateWithoutPage(state))
        ? cachedHits
        : null;
    },
    write({ state, hits }) {
      cachedState = getStateWithoutPage(state);
      cachedHits = hits;
    },
    clear() {
      cachedHits = undefined;
      cachedState = undefined;
    }
  };
}

const cache = getInMemoryCache();

<InfiniteHits 
  ...
  cache={cache}
/>

And later, you can call cache.clear() right before you refresh.

The internal cache of InfiniteHits should be cleared when refreshing. But, until it's properly fixed, that can be another workaround.

[meck93]

@eunjae-lee can this also be done when using connectInfiniteHits?
I didn't see anything in the documentation. Can I just pass it as an additional prop (see point 3)?

// 0. Initialize the cache
const sessionStorageCache = createInfiniteHitsSessionStorageCache();

// 1. Create a React component
const InfiniteHits = () => {
  // return the DOM output
};

// 2. Connect the component using the connector
const CustomInfiniteHits = connectInfiniteHits(InfiniteHits);

// 3. Use your connected widget
<CustomInfiniteHits cache={cache} />

[Haroenv]

Yes, the connector will accept the cache prop as well @meck93. You can check that by using a custom cache which has some logging :)

[meck93]

Great, thanks a lot 👍

[andrewbeckman]

I am also encountering this issue. Spent hours at this point trying to figure out what's going on. @Haroenv if Algolia isn't going to fix it, it'd be really helpful to at least have a warning in the docs so that others don't find themselves wasting time like I did.

Also react-instantsearch-native does not appear to have the same ability to override the cache.

[Haroenv]

react native InstantSearch should work exactly the same, since the connector is just forwarded from the "core" version, so I expect something else is going on.

This is something we're planning to fix in the future, however I don't see how we can give a warning in this case unfortunately?

[alexpchin]

Hi @Haroenv I'm just circling back to this. I've managed to get this work on the web by using:

<CustomHits
  cache={createInfiniteHitsSessionStorageCache("ais.tattoos")}
/>

Combined with:

import _objectWithoutProperties from "@babel/runtime/helpers/esm/objectWithoutProperties"
import isEqual from "react-fast-compare"

function getStateWithoutPage(state) {
  var _ref = state || {},
    // page = _ref.page,
    rest = _objectWithoutProperties(_ref, ["page"])

  return rest
}

function hasSessionStorage() {
  return (
    typeof window !== "undefined" &&
    typeof window.sessionStorage !== "undefined"
  )
}

export default function createInfiniteHitsSessionStorageCache(
  KEY = "ais.infiniteHits"
) {
  return {
    read: function read(_ref2) {
      var state = _ref2.state

      if (!hasSessionStorage()) {
        return null
      }

      try {
        var cache = JSON.parse(window.sessionStorage.getItem(KEY))
        return cache && isEqual(cache.state, getStateWithoutPage(state))
          ? cache.hits
          : null
      } catch (error) {
        if (error instanceof SyntaxError) {
          try {
            window.sessionStorage.removeItem(KEY)
          } catch (err) {
            // do nothing
          }
        }

        return null
      }
    },
    write: function write(_ref3) {
      var state = _ref3.state,
        hits = _ref3.hits

      if (!hasSessionStorage()) {
        return
      }

      try {
        window.sessionStorage.setItem(
          KEY,
          JSON.stringify({
            state: getStateWithoutPage(state),
            hits: hits,
          })
        )
      } catch (error) {
        // do nothing
      }
    },
  }
}

This allows me to create a different store for each page.
However, I'm still a bit stuck on how best to implement this in react-native.

Do you have a guide for clearing the cache for multiple connectInfiniteHits and there no solution where we can clear the internal cache without having to create our own?

Thanks and Happy New Year!

[alexpchin]

Hi @Haroenv Thanks for the 👍 Do you have a suggestion for resolving this in react-native? I still cannot seem to get things working well. Is it related to: algolia/react-instantsearch#2995 ?

[Haroenv]

I was still looking what's going on, but I don't have a good react native sandbox and setup, so that took a while, and I had to work on other things. If you can recreate the bad behaviour in a GitHub example, that would help a lot!

[alexpchin]

Hi @Haroenv

I did make a github repository a while ago: #5263 However, it is a little outdated now so I made a new one here:

https://github.com/alexpchin/react-instant-search-refresh

You will need to change the APP_ID and Index values to one where you can delete an object from an index to see the behaviour.

[alexpchin]

Hi @Haroenv did you have any suggestion for this fix? Currently, I'm having to use the connectHits and loading a large number of results because I can't seem to get the cache to clear well on connectInfiniteHits in react-native.

[Haroenv]

can you contact support@algolia.com with the full reproduction? this way there will be a reminder to reply to you?

[eunjae-lee]

I don't think it needs to be persistent (I don't know how persistent it is, though). In my opinion, the memory cache will be sufficient for many cases unless you intentionally keep the cache for a long time.
And what I meant by the memory cache, that implementation is exactly what I had in mind.

[alexpchin]

Hi @eunjae-lee Do you by any chance have a working react-native example with a custom cache (as above) and connectInfiniteHits. I'm still struggling to get my code to work as expected.

I am using a Configure to pass a search filter, when using a custom cache, I don't seem to be getting the new hits adding into the hits, rather, they are replacing the existing ones.

[eunjae-lee]

@alexpchin Sorry but we don't have any working example yet. It seems like your state isn't consistent and InfiniteHits thinks it's a new state and overrode the cache. Can you add logs into your custom cache implementation and check how the state changes?

function getInMemoryCache() {
  let cachedHits = undefined;
  let cachedState = undefined;
  return {
    clear() {
      cachedHits = undefined;
      cachedState = undefined;
    },
    read({ state }) {
      return isEqual(cachedState, getStateWithoutPage(state))
        ? cachedHits
        : null;
    },
    write({ hits, state }) {
      cachedState = getStateWithoutPage(state);
      cachedHits = hits;
    },
  };
}

[alexpchin]

Hi, I'm just circling back to this as I was on other tasks. I have logged some things:

import isEqual from 'react-fast-compare';

function getStateWithoutPage(state) {
  // eslint-disable-next-line no-unused-vars
  const { page, ...rest } = state || {};
  return rest;
}

function getInMemoryCache() {
  let cachedHits = undefined;
  let cachedState = undefined;
  return {
    clear() {
      cachedHits = undefined;
      cachedState = undefined;
    },
    read({ state }) {
      console.log('cachedState', JSON.stringify(cachedState, null, 4));

      console.log(
        'getStateWithoutPage',
        JSON.stringify(getStateWithoutPage(state), null, 4),
      );

      return isEqual(cachedState, getStateWithoutPage(state))
        ? cachedHits
        : null;
    },
    write({ hits, state }) {
      cachedState = getStateWithoutPage(state);
      cachedHits = hits;

      console.log('cachedHits', JSON.stringify(cachedHits, null, 4));
    },
  };
}

const storage = {};

function buildCache(key) {
  if (storage[key]) {
    return storage[key];
  }
  storage[key] = getInMemoryCache();
  return storage[key];
}

export const cache = (key) => buildCache(key);

This is an example of what I am seeing for cachedState

{
    "indices": {
        "search-tattoos": {
            "configure": {
                "filters": "type:flash OR type:tattoo OR type:collection",
                "hitsPerPage": 21
            },
            "page": 2
        },
        "search-artists": {
            "configure": {
                "filters": "role:artist AND verified:\"true\" OR status:instagram",
                "hitsPerPage": 21
            },
            "page": 1
        },
        "search-studios": {
            "configure": {
                "filters": "role:studio AND verified:\"true\" OR status:instagram",
                "hitsPerPage": 21
            },
            "page": 1
        },
        "search-collectors": {
            "configure": {
                "filters": "role:customer",
                "hitsPerPage": 21
            },
            "page": 1
        }
    },
    "query": ""
}

So I updated the getStateWithoutPage to:

import omitDeep from 'omit-deep-lodash';

const getStateWithoutPage = (state = {}) => {
  return omitDeep(state, 'page');
};

This helped to remove the references to page in the nested object.

{
    "indices": {
        "search-artists": {
            "configure": {
                "filters": "role:artist AND verified:\"true\" OR status:instagram",
                "hitsPerPage": 21
            }
        },
        "search-studios": {
            "configure": {
                "filters": "role:studio AND verified:\"true\" OR status:instagram",
                "hitsPerPage": 21
            }
        },
        "search-collectors": {
            "configure": {
                "filters": "role:customer",
                "hitsPerPage": 21
            }
        },
        "photos-places": {},
        "search-tattoos": {
            "configure": {
                "filters": "type:flash OR type:tattoo OR type:collection",
                "hitsPerPage": 21
            }
        },
        "tags": {}
    },
    "query": ""
}

I am now just going through to test a little further...

[alexpchin]

@eunjae-lee

Hi all, I'm still struggling to get a good solution here... I have got most of it working with a custom cache, however, when updating the search query - my cache does not seem to display the correct results. They seem to be stale by one result. If I change to a controlled component:

<InstantSearch
  indexName={indexName}
  onSearchStateChange={onSearchStateChange}
  searchClient={searchClient}
  searchState={searchState}
>
.
.
</InstantSearch>

Then the query updates the search, but the connectInfiniteHits then hides the previous hits after calling refineNext.

Ideally, the refresh prop should just clear the internal cache. Or alternatively, clear by calling searchClient.clearCache()?
This way, we don't need to mess around with custom caches at all?

Update

I've just seen:

• Search old results by one state react-instantsearch#3018

[Haroenv]

Do you have a GitHub repo with this latest state @alexpchin?

[alexpchin]

Hi @Haroenv I finally think I've sorted the issues. One of the issues was fixed with incorporating omit-deep-lodash to remove the page from all of the indices in the state (as I was using multi-index). Another was fixed by creating a cache using a key:

const storage = {};

function buildCache(key) {
  if (storage[key]) {
    return storage[key];
  }
  storage[key] = getInMemoryCache();
  return storage[key];
}

Then finally, your fix for algolia/react-instantsearch#3018 seemed to fix the stale hits that I was seeing.

I am just releasing a new version of my project, after I do that, I will create an example to share.

[eunjae-lee]

Hello @alexpchin Sorry I was off. I'm glad it seems to have fixed your issue. Let us know how it goes!

[skam22]

@alexpchin

do you have a working react native example with connectInfiniteHits that loads newly created items on refresh?

[allanohorn]

Is there any complete working example? I read the entire thread but not 100% clear on what the recommended solution is

[skam22]

@allanohorn

I got refresh to work in my App by manually clearing the Algolia Client cache. I'm using a reactive variable titled searchVar with Apollo client, but any global state manager will work. My searchVar has this shape:

SearchVar.tsx

  import {SearchState} from 'react-instantsearch-core';

  export interface SearchVar {
  refresh: boolean;
  searchState: SearchState;
  }

SearchScreen.tsx

  const {refresh, searchState} = useReactiveVar(searchVar)

  useEffect(() => {
    if (refresh) {
      algoliaClient
        .clearCache()
        .then(() => {
         
          //  LOGIC TO REFRESH THE TOKEN 
          //  AND RESET THE SEARCH STATE TO AN EMPTY OBJECT

          searchVar({
            refresh: false,         
            searchState: {}
          });
        })
        .catch(err => console.log('[SearchScreen] refresh ERROR', err));
    }
  }, [refresh]);

Then for the InstantSearch component I utilize a controlled searchState, onSearchStateChange, along with a refresh prop (but I'm not confident that actually does anything).

My InstantSearch looks like this:

      <InstantSearch
        searchClient={algoliaClient}
        indexName="items"
        searchState={searchState}
        onSearchStateChange={(newState: SearchState) => {

          //  UPDATE searchState BUT LEAVE REFRESH TOKEN UNTOUCHED
          searchVar({...searchVar(), searchState: newState});

        }}
        refresh={refresh}>

        <Header showHeader={showHeader} />

        <Animated.View style={animatedStyles.content}>
          <ConnectedSearchBox />
          <ConnectedSearchResults />
          <RefinementList />
        </Animated.View>

        <Footer showFooter={showFooter} />

      </InstantSearch>

let me know if that helps or if you have any questions

[josuebustos]

Does this also apply to react-instantsearch-hooks for react native projects?

The refresh={true} has no effect and the lib doesn't fetch new objects from the Indices.

I'm using the starter code provided here: https://www.algolia.com/doc/guides/building-search-ui/going-further/native/react-hooks/

import algoliasearch from 'algoliasearch'; // algoliasearch/light doesn't exist the docs need to be updated.
import { InstantSearch } from 'react-instantsearch-hooks'; // not to be confused with react-instantsearch-native
......
const [refresh, setRefresh] = useState(false);

const updateList = () => {
   setRefresh(true)
   
   setTimeout(() => {
      setRefresh(false);      
    }, 1000);
}

<InstantSearch
        indexName={"my-index-name"}
        searchClient={searchClient}       
        refresh={refresh}
      >
.....

<Button 
   onPress={updateList}
> Refresh List </Button>

[Haroenv]

In React InstantSearch Hooks refresh is a function returned from useInstantSearch, but it indeed doesn't clear the infinite hits cache either. Sorry!

[josuebustos]

In React InstantSearch Hooks refresh is a function returned from useInstantSearch, but it indeed doesn't clear the infinite hits cache either. Sorry!

Is there a possible workaround?

[Haroenv]

It depends what you want the behaviour to be. If you're ok with the "already displayed pages" to be reset to 0, you can call setUiState(state => { state.myindex.page = 0; return state }); refresh() for example, but there's other tradeoffs possible depending on exactly which experience you want to create.

[josuebustos]

It depends what you want the behaviour to be. If you're ok with the "already displayed pages" to be reset to 0, you can call setUiState(state => { state.myindex.page = 0; return state }); refresh() for example, but there's other tradeoffs possible depending on exactly which experience you want to create.

How can this be implement on react native using <InstantSearch> props and or <InfiniteHits> props?

[sarahdayan]

@josuebustos For now there are no APIs to do this by passing props to <InstantSearch> or <InfiniteHits> directly, but @Haroenv's suggestion might work depending on your use case.

Can you tell us more what you're trying to achieve from a functional point of view?

[josuebustos]

@josuebustos For now there are no APIs to do this by passing props to <InstantSearch> or <InfiniteHits> directly, but @Haroenv's suggestion might work depending on your use case.

Can you tell us more what you're trying to achieve from a functional point of view?

When I update the algoia index with a new object, I want the react native app to fetch the latest data and display it. A simple refresh mechanism to pull down the latest information from the algolia indexes.

Let me know if you need more information.

[skam22]

@josuebustos For now there are no APIs to do this by passing props to <InstantSearch> or <InfiniteHits> directly, but @Haroenv's suggestion might work depending on your use case.
Can you tell us more what you're trying to achieve from a functional point of view?

When I update the algoia index with a new object, I want the react native app to fetch the latest data and display it. A simple refresh mechanism to pull down the latest information from the algolia indexes.

Let me know if you need more information.

The example code I posted above is a working production implementation of pull to refresh in a react native app. It works by using a controlled searchState.

[sbkl]

the useInfiniteHits hooks from "react-instantsearch-hooks-web" seems to have the right behaviour while "react-instantsearch-hooks" used in react native doesnt. Any solution on this?

[NeanderNerd]

I Need to comment on this issue since I have been dealing with something similar in nature.

I have implemented ais-infinite-hits on my webpage using vue-InstantSearch and I haven't been able to get the pages to work correctly since the implementation. The behavior I am observing is, when I navigate to the search, it starts on a random page, most of the time the previous page that I was on before I navigated away from the search. then when i refresh the page the page number increments every time. If i scroll down to the bottom of the hits, the trigger for refineNext() reveals the next group of items on the search but the page number jumps to the highest page number with results. I am using the cache with createInfiniteHitsSessionStorageCache().

I almost feel like the cache is what is causing the issue but I need it in the application so that if a user navigates away from the search and clicks the back button they go back to the search in the state that it was in before they left.

After removing the Cache, the pages are still being weird. They jump around to random values, but, I would say the infinite-hits component is working as it should. on the other hand, I have confirmed that taking out the cache did in fact make it impossible for the user to navigate away from the search and come back to the page where they left off.