Enhance Bulk Reindexing with custom_import_scope for Faster Partial Indexing #1707
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR enhances Searchkick's bulk reindexing by introducing custom_import_scope, allowing selective indexing of associated data. Previously, reindexing would load all related records via search_import, even if only a subset was needed. This resulted in slower performance and unnecessary memory usage.
With this enhancement, developers can explicitly define which associations to include(for partial reindexing), making indexing up to 70% faster in cases where only partial data (e.g., client details) is required.
Key Changes
✅ Added custom_import_scope to limit loaded associations during reindexing.
✅ Ensures backward compatibility with search_import.
✅ Performance boost by avoiding unnecessary data fetches.
Example Usage
1. Full Reindexing (Legacy Approach)
This would load all related data, making reindexing slower.
Internally, this includes all associations present in import scope. such as: ( Vehicle.includes(:make, :model, :variant, inventory: [:user, :city, :tasks]) )
2. Optimized Partial Reindexing (New Approach)
Internally, this only includes only make : such as: ( Vehicle.includes(:make) )
Why This Matters?
This update significantly improves performance while ensuring existing functionality remains intact. Let me know if any refinements are needed! 🚀