-
Notifications
You must be signed in to change notification settings - Fork 36
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
JavaScript heap out of memory #13
Comments
@mkarkachov This could be related to #10. May I know what version of the extension you're running? |
typesense/firestore-typesense-search@0.2.5 |
Hmmm. Could you share a sample firestore document? I'm curious to know the size of each document particularly and the data types. |
I get the same error |
@matslb Could you share a sample firestore document? |
Here is a typical document in my firestore:
These are the fields being indexed by Typesense, set in the extension configuration in firebase:
The fields match the collection schema |
Reading the extension code I think heap issues are also to be expected in the backfill function, as it first loads the complete set of documents from the firestore collection before it starts batching documents and sending them to the typesense API. I work on a project with a users collection that is nearing 1M records, containing user profile info and application settings. You do not want to load all of that upfront to the heap before you start processing chunks. I wrote utility functions to deal with these kinds of scenarios, but I haven't gotten around to packaging and releasing them to the public. |
I have >40k records in firestore and I got issue when I try to index more than 3 fields. Please check screenshots
The text was updated successfully, but these errors were encountered: