DRAFT: Implementing Cross-Library Search

[carddev81]

Description

This issue outlines the proposed approach for implementing search functionality across multiple libraries using keywords. Unlike the single-library search (similar to Kiwix’s existing functionality), this feature would allow users to search across all libraries they have access to. Results will be grouped by library and also allow users the ability to page through results if there are too many to display on a single screen. This issue was created based on research completed from #559 and #558.

Proposed Approaches

1. Option 1: User-Driven Search Index
   • Description:
     • Create a dynamic search index by saving link labels and URLs into a dedicated search table as users navigate through libraries.
   • Advantages:
     • Grows dynamically based on user activity.
     • Data is stored persistently, ensuring historical data is retained.
     • Allows searching across all libraries once indexed.
   • Limitations:
     • Initial search results will be limited for new libraries until users interact with them.
2. Option 2: Pre-Populated Search Index
   • Description:
     • Develop a process to extract metadata directly from ZIM files and store the link labels and URLs in a search table.
   • Advantages:
     • Provides a complete set of searchable content from the outset.
     • Containerized storage structure ensures data safety and minimizes frequent rebuilding.
     • Allows searching across all libraries once indexed.

Development Tasks

Backend Tasks

1. API Changes:
   • Develop a new API handler for handling cross-library search queries, paginating the results.
2. Search Table:
   • Implement data persistence for search tables.
   • If using the pre-populated table approach, extract metadata from ZIM files and populate the database in a one-time process.

Frontend Tasks

1. Search Interface:
   • Add a search text field and either a button or search icon next to the Library Viewer.
   • Include a drop-down box next to the text field with options for library selection (e.g., specific libraries or "ALL").
   • Ensure the interface aligns with UnlockEd's branding and design principles.
2. Search Results Rendering:
   • Display search results grouped by library, each with their corresponding library icon.
   • Paginate results to handle large datasets.
   • Ensure results format is consistent with the UnlockEd interface.

Identified Risks and Challenges

1. Performance Impact:
   • Large-scale searches across multiple libraries could increase server load.
2. Security Concerns:
   • Properly restrict search results to libraries the user is authorized to access.
3. Initial Data Availability:
   • Dynamic indexing may result in limited search results until users interact with libraries.

Rejected Option

1. Utilize Existing Kiwix Search:
   • Kiwix’s existing search functionality is limited to individual libraries.

Next Steps

• Finalize the approach (Option 1 or Option 2).
• Break down tasks into smaller issues for parallel development if possible.
• Begin with backend modifications and API updates, followed by frontend integration.
• Conduct testing to ensure search functionality integrates well with UnlockEd.