[HOLD for payment 2023-08-24] [$2000] [ExpensifyHelp] Implement a search functionality on https://help.expensify.com

[marcochavezf]

We want to enhance the user experience on our help site by integrating a search functionality. This will allow users to find specific information quickly and efficiently.

Our plan is to utilize Google's Programmable Search Engine for this task, as it is a reliable and easy-to-setup solution. Specifically, we'd like to use the XML configuration (https://developers.google.com/custom-search/docs/basics) rather than manual setup through the Google Control Panel UI. This will facilitate future changes by our engineering team through the usual pull request process.

In your proposal, please be sure to address the following key aspects:

• Modification Plan: Outline which files and what code of the ExpensifyHelp project will be modified to incorporate the search functionality.

• Right-hand Panel Implementation: Describe how the right-hand panel will be designed and implemented, and how it can be displayed and hidden. The right-hand panel should host the search bar and the results display.

• Results Display: Explain how the search results will be displayed in the right-hand panel.

• Programmable Search Engine Implementation: Describe how the Programmable Search Engine will be implemented to only show results from https://help.expensify.com/.

The proposal doesn't actually require extensive detail, but it's a bonus if you provide it. We aim to confirm your familiarity with the ExpensifyHelp code and ensure that you can carry out the implementation without issues.

Mockups:

View all open jobs on GitHub

Upwork Automation - Do Not Edit

• Upwork Job URL: https://www.upwork.com/jobs/~016883d3a09768972a
• Upwork Job ID: 1686243072505339904
• Last Price Increase: 2023-08-01

[melvin-bot]

Job added to Upwork: https://www.upwork.com/jobs/~016883d3a09768972a

[melvin-bot]

Triggered auto assignment to @sophiepintoraetz (External), see https://stackoverflow.com/c/expensify/questions/8582 for more details.

[melvin-bot]

Triggered auto assignment to Contributor-plus team member for initial proposal review - @eVoloshchak (External)

[marcochavezf]

Changing the base payment to $2000 given the complexity of the implementation.

[melvin-bot]

Upwork job price has been updated to $2000

[payal-lathidadiya]

This is Interesting. I'd like to work on this new feature implementation.
I'll analyse in details and prepare a proposal.

[cubuspl42]

@marcochavezf This may by a silly question, but where does the relevant source, configuration, etc. reside? I always believed that expensify.com is so-called "old dot" and external contributors don't have access to it.

[samh-nl]

Proposal

Please re-state the problem that we are trying to solve in this issue.

Implement a search functionality on https://help.expensify.com

What is the root cause of that problem?

N/A

What changes do you think we should make in order to solve the problem?

Modification Plan

• In default.html, add a search icon button to the lhn-header container. The search icon will be placed in the assets/images/ directory.
• In assets/js/main.js, we define a function toggleSearchMenu(), which should also toggle a backdrop. Clicking the backdrop should close the search menu.
• We create a new file _includes/search-menu.html, or we can make it an implementation of a generic RHN menu. The latter, however, may be an unnecessary complication.
• In default.html, we include this newly created file.
• We should not change the route as we would lose information about the visited page when refreshing. Instead, we can use the q query parameter appended by searching to identify that we should open the search menu on initial page load. Alternatively, we can change the route but revert it when the search menu is closed.
• Lastly, the layout should be mobile friendly, with the search menu becoming screen-wide, similar to the LHN.

Right-hand Panel Implementation

• We can rely on gcse-searchbox and gcse-searchresults, which limits the scope to styling. Alternatively, we can implement a custom search input and use a stylized standard search results container. The latter may give more freedom for styling the search input. Lastly, the JSON API gives the most control, however it's unlikely that this proves to be necessary.
• Additionally, a close button is added to the top-right corner of the menu.

Results Display

• A search result would be composed of an anchor element, containing a header and a description.
• In _sass/_main.scss, we define the appropriate styling for the search results. Alternatively, we can separate out the search styling in a new file. However, this would be inconsistent with the current format, where _main.scss contains the bulk of the styling and other files are merely used for importing variables.
• A message should optionally be shown if there are no matching results.

Programmable Search Engine Implementation

• To restrict the search coverage to https://help.expensify.com/ (or only to articles, i.e. https://help.expensify.com/articles/), we can declare the route in the XML Annotations:
  https://developers.google.com/custom-search/docs/annotations
• I don't believe we have to expand the search query through defining synonyms, as the help articles already steer away from using jargon.

What alternative solutions did you explore? (Optional)

N/A

[nzmsakib]

Greetings.
I am interested to work on this feature implementation.
I can implement it according to your provided mockup files.
I need to add a portion of code inside the body tag of the existing code of the ExpensifyHelp project and some CSS.
I have used the Control Panel to change the layout and restrict the search domain. Here is a screenshot I implemented with Bootstrap 5: https://drive.google.com/file/d/1h0gkGT9aY8vQqCXzHUzon1Qe4vzeHati/view?usp=drive_link
I can customize it with only HTML, CSS, and JavaScript.

Though I am new to this repository, I am highly interested to work if you want something similar to the screenshot I provided. Can you tell me more about using the Context files? Where will it be stored in your repository? From control panel, I see there is an option to upload/download the XML files. But it doesn't provide much information how it can be integrated into projects.

[melvin-bot]

📣 @nzmsakib! 📣
Hey, it seems we don’t have your contributor details yet! You'll only have to do this once, and this is how we'll hire you on Upwork.
Please follow these steps:

1. Get the email address used to login to your Expensify account. If you don't already have an Expensify account, create one here. If you have multiple accounts (e.g. one for testing), please use your main account email.
2. Get the link to your Upwork profile. It's necessary because we only pay via Upwork. You can access it by logging in, and then clicking on your name. It'll look like this. If you don't already have an account, sign up for one here.
3. Copy the format below and paste it in a comment on this issue. Replace the placeholder text with your actual details.
   
   Format:

Contributor details
Your Expensify account email: <REPLACE EMAIL HERE>
Upwork Profile Link: <REPLACE LINK HERE>

[nzmsakib]

Contributor details
Your Expensify account email: engrsakibcse@outlook.com
Upwork Profile Link: https://www.upwork.com/freelancers/~011df441739393295c

[melvin-bot]

✅ Contributor details stored successfully. Thank you for contributing to Expensify!

[aditygrg2]

This seems cool!
Interested to contribute in this project. Would submit a proposal soon.

[melvin-bot]

📣 @aditygrg2! 📣
Hey, it seems we don’t have your contributor details yet! You'll only have to do this once, and this is how we'll hire you on Upwork.
Please follow these steps:

1. Get the email address used to login to your Expensify account. If you don't already have an Expensify account, create one here. If you have multiple accounts (e.g. one for testing), please use your main account email.
2. Get the link to your Upwork profile. It's necessary because we only pay via Upwork. You can access it by logging in, and then clicking on your name. It'll look like this. If you don't already have an account, sign up for one here.
3. Copy the format below and paste it in a comment on this issue. Replace the placeholder text with your actual details.
   
   Format:

Contributor details
Your Expensify account email: <REPLACE EMAIL HERE>
Upwork Profile Link: <REPLACE LINK HERE>

[aditygrg2]

Contributor details
Your Expensify account email: adityagarg.1747@gmail.com
Upwork Profile Link: https://www.upwork.com/freelancers/~01889386d30b9647d6

[melvin-bot]

✅ Contributor details stored successfully. Thank you for contributing to Expensify!

[studentofcoding]

Proposal

Modification Plan

• Add flex container to contain Logo & search icon button inside lhn-header container In default.html.
• Edit meta to contain all related links (e.g. https://cse.google.com)
• Add toggleSidebarSearch() in assets/js/main.js, to handle showing the Sidebar.
• Add search-sidebar element into default.html & use <div class="gcse-searchbox"></div> <div class="gcse-searchresults"></div> into it
• Change the style of search-bar & result component to accommodate the template in _main.scss

Right-hand Panel Implementation

• Add search-sidebar element into default.html & use <div class="gcse-searchbox"></div> <div class="gcse-searchresults"></div> into it
• Change the style of search-bar & result component to accommodate the template

Results Display

• Right now it's automatically showing the result onPress Enter and the result will be shown on div class="gcse-searchresults"></div> element.

Programmable Search Engine Implementation


Strict the result to only covering https://help.expensify.com/* via annotations.xml

<Annotations>
    <Annotation about="Expensify help" score="1.0">
        <Label name="_include_"/>
        <AdditionalData attribute="original_url" value="www.help.expensify.com/*"/>
    </Annotation>
</Annotations>

POC

• The Functionality (enter or click search icon to load result) with toggle Sidebar is live here: https://xml-docs-studentofcoding.vercel.app/

https://www.loom.com/share/a35931897c1742f4a2de28eddf68ed69?sid=67186f0d-06d8-496e-bfda-69eba843c611

cc: @marcochavezf

[Nikhil-Vats]

Proposal

Please re-state the problem that we are trying to solve in this issue.

Implement a search functionality on https://help.expensify.com

What is the root cause of that problem?

This is a new feature to improve the user experience.

What changes do you think we should make in order to solve the problem?

Modification Plan

A high level overview of modifications is -

1. Inside lhn-header div in layouts/default.html, add a search button. We can use the same svg we are using in main app - magnifying-glass.svg.
2. Add the RHN component to display search bar and search results. For this, we have to create new templates in _includes and use it in default.html.
3. We also need to create a backdrop component which can be part of the RHP component or separate. Upon clicking the backdrop, RHP should close.
4. Add the styles for both of these in _main.scss.
5. Add functions openSearchPage, closeSearchPage, etc. in assets/js.
6. In _data/_routes.yml, add a route for search page so it has its own URL which enables search page to be directly opened via URL.

Right-hand Panel Implementation

Design for RHP -

1. Create a new component for search bar in _includes.
2. Create a new template for showing a single result in _includes.
3. Create a new component for RHP in _includes.
4. Use the new RHP component and the other components described above in a new component SearchPage which will be rendered in default.html.

Other implementation details -

1. Search page should open when user clicks the search icon. Optionally, we can also add a listener for CMD + K. For this, we can make a function openSearchPage.
2. Search page should close when user clicks on the "X" button or when user clicks outside the RHP. For this, we can make a function closeSearchPage.

Results Display

For displaying the search results, there will be a separate search result template described in the section above. We can create a custom snippet to display additional structured information like tags/category of a page, etc. or we can use the default structure which is a header followed by a link and description.

Programmable Search Engine Implementation

GPSE's annotations.xml allow us to specify the pages that should be searched.

<?xml version="1.0" encoding="UTF-8"?>
<Annotations>
    <Annotation about="help.expensify.com/*" timestamp="0x000601d65e668437" score="1.0">
        <Label name="_include_"></Label>
        <AdditionalData attribute="original_url" value="https://help.expensify.com/*"></AdditionalData>
    </Annotation>
</Annotations>

These can be configured in annotations.xml.

Other formats possible are -
Individual pages:https://help.expensify.com/articles/split-bills/paying-friends/Request-and-Split-Bills
Entire site: https://help.expensify.com/*
Parts of site: https://help.expensify.com/hubs/split-bills/* or https://help.expensify.com/hubs/request-money
Entire domain: *.expensify.com

We can use the synonyms XML file to improve the search result

Google already has a large corpus of synonyms and we only need to define the synonyms for terms specific to Expensify like IOU, policy, etc.

<Synonyms>
  <Synonym term="IOU">
    <Variant>Request money</Variant>
    <Variant>Send money</Variant>
    <Variant>Split bill</Variant>
  </Synonym>
  <Synonym term="Policy">
    <Variant>Workspace</Variant>
    <Variant>Organisation</Variant>
  </Synonym>
</Synonyms>

Filtering and sorting search results

We can add PageMap data to all articles to allow structured search and display custom snippets in results to show the important information about result like date, category, etc.

To do this, we have to -

1. Add important information which we would like to be in the PageMap in the top of articles markdown files like we have added title, description.
2. Use those attributes in the meta tag in default.html like we are accessing title in hub.html - {{ hub.title }}.

Use context.xml for changing other configurations to improve search

We can configure the attributes to enhance user experience. ex -

1. enableHistory
2. mobileLayout
3. enableAutoComplete
4. sort_by

These can be edited via context.xml configuration file.

What alternative solutions did you explore? (Optional)

NA

[melvin-bot]

✅ Contributor details stored successfully. Thank you for contributing to Expensify!

[aditygrg2]

Proposal

Please re-state the problem that we are trying to solve in this issue.

Implementing search functionality on https://help.expensify.com

What is the root cause of that problem?

This is a new improvement to provide better user experience.

What changes do you think we should make in order to solve the problem?

Modification Plan:

The below paths are to be taken in reference to the https://help.expensify.com/ code i.e, inside the docs folder

1. Inside lhn-header div inside default.html, a search icon needs to be added.
2. The search bar div can be created in a different file, which can be included outside lhn
3. The design part including the backdrop and transitions of this needs to implemented in _main.scss
4. The toggle part of the right-panel is to be implemented in main.js

Right-hand Panel Implementation:

1. In the default.html file, a new div element containing a search bar will be introduced besides the existing "ExpensifyHelp" title. This div will be placed outside the element with the ID "lhn."
2. In the same default.html file, another div will be included outside the element with ID "lhn" to hold both the search bar and the content. Initially, this div will be hidden using the CSS class "negative-right" with the CSS property "right" set to "-100%."
3. The functionality to show and hide this div will be implemented in the main.js file. Specifically, when the user clicks the search bar, the "negative-right" class will be added to the div, causing it to slide into view from the right. Clicking on the x icon or outside the search div will remove the "negative-right" class, causing the div to slide out of view, returning to its initial hidden state.
4. The main.js file will handle the logic to toggle the visibility of the hidden div based on user interaction with the search bar.

Results Display

1. The search results will contain a title and a short description which can be interpreted using GPSE
2. The CSS of it will already be managed on the RHP implementation as discussed above
3. If no results appear on this page for a user's query "Send a message button" can be shown as seen on the homepage of the Expensify help page.

Programmable Search Engine Implementation:

1. The present sitemap is not good for search queries, so we might need to create a new sitemap for the Expensify help page on Google Search Console. This can be easily done by using sitemap creators online and improving that manually.
2. To just search for specific links, Annotations can be used that allow specific pages to be searched which can be implemented in a new annotations.xml file
3. We can use LLM models, to get some important synonyms and use BOOST, FILTER, and ELIMINATE to score the search results properly. General synonyms can be added in the initial run but this can be a major improvement for future versions.
4. Attributes can be used to implement several other features like mobileLayout, enableHistory. A separate configuration file for this can be set up.

What alternative solutions did you explore?

We might save search queries in the local storage of the user, so revisiting the help page will show the top 2-3 queries searched in history.

The feature is a basic help search feature and should not be much complex. A basic on click search, with quick results is fine with not so much complexities and should be easy to use for a user.

cc: @marcochavezf

[rayane-djouah]

Proposal

Please re-state the problem that we are trying to solve in this issue.

To enhance user experience, a search functionality should be implemented on . This will allow users to find specific information in an efficient and effective manner.

What is the root cause of that problem?

Currently, the ExpensifyHelp site does not include a search feature. As a result, users may experience difficulty locating specific information.

What changes do you think we should make in order to solve the problem?

Modification Plan:

Add a search icon button in default.html:

We will introduce a search icon button inside the lhn-header container in the default.html file, which is located in the /_layouts directory. This icon will serve as the entry point to the search functionality.

Create toggleSearchPanel() in main.js:

We will define a new function named toggleSearchPanel() within the main.js file, located in the /assets/js directory. This function will be responsible for showing and hiding the search panel when the user interacts with the search icon button.

Create search-panel.html:

We will create a new file, search-panel.html, within the /_includes directory. This file will define the layout and structure of the search panel, including the search bar and the area for displaying search results.

Include search-panel.html in default.html:

We will modify the default.html file to include the newly created search-panel.html. This will embed the search panel into the site's layout.

Create search-panel.scss:

We will create a new CSS file, search-panel.scss, within the /_sass directory. This file will define all the styles for the search panel and its elements, such as the search bar and search results.

Implement search query handling in main.js:

Lastly, within the main.js file, we will implement functionality to capture the search query when the user submits the search form, and send it to the Programmable Search Engine.

Right-hand Panel Implementation:

Structure:

The right-hand panel will be a div container that will include the search bar and an area for displaying search results. This panel will be hidden by default and will slide into view when the user interacts with the search icon button.

Visibility:

The toggleSearchPanel() function in main.js will control the visibility of the right-hand panel. When invoked, it will add or remove a CSS class that moves the panel into or out of view.

Search Bar:

The search bar will be a simple form with a text input and a submit button. The text input will capture the user's search query, while the submit button will trigger the search when clicked.

Results Display:

The results display will be a div container inside the right-hand panel. This container will be populated with search results returned by the Programmable Search Engine.

Results Display:

Structure:

The search results will be displayed as a list. Each item in the list will represent a single search result and will be a clickable div containing the title and a snippet of the result.

Styling:

The styling for the search results will be defined in the search-panel.scss file. This will ensure that the search results are presented in a clear and readable manner.

Programmable Search Engine Implementation:

Configuration:

We will configure Google's Programmable Search Engine (PSE) to perform the search functionality. We will use PSE's XML configuration to limit the search results to only include content from https://help.expensify.com/.

Search Query Handlin

When the user submits a search query through the search bar, we will send the query to the PSE using its API. The API call will be made in the main.js file when the search form is submitted.

Results Processing

We will process the results returned by the PSE API and populate the results display in the right-hand panel with these results. This process will involve extracting the necessary data from the results and creating HTML elements to display this data.

What alternative solutions did you explore? (Optional)

[kmwamasali]

@marcochavezf @studentofcoding glad to know my solution will be used for your benefit

[marcochavezf]

@marcochavezf @studentofcoding glad to know my solution will be used for your benefit

@kmwamasali I think we can arrange a little compensation for you if it's included in the PR. I was unfamiliar with the custom search site API, but it sounds like a good addition. We can discuss more about it after the PR is up.

[kmwamasali]

@marcochavezf that sounds good, I will wait to see how it turns out

[melvin-bot]

Triggered auto assignment to @shawnborton (Design), see these Stack Overflow questions for more details.

[marcochavezf]

Hi @shawnborton, can we have your review for the UI changes posted in the videos here?

[melvin-bot]

Based on my calculations, the pull request did not get merged within 3 working days of assignment. Please, check out my computations here:

• when @studentofcoding got assigned: 2023-08-04 19:46:32 Z
• when the PR got merged: 2023-08-11 22:28:35 UTC
• days elapsed: 5

On to the next one 🚀

[melvin-bot]

Reviewing label has been removed, please complete the "BugZero Checklist".

[melvin-bot]

The solution for this issue has been 🚀 deployed to production 🚀 in version 1.3.54-13 and is now subject to a 7-day regression period 📆. Here is the list of pull requests that resolve this issue:

• Improvement: Search Feature for help expensify #24299

If no regressions arise, payment will be issued on 2023-08-24. 🎊

After the hold period is over and BZ checklist items are completed, please complete any of the applicable payments for this issue, and check them off once done.

• External issue reporter
• Contributor that fixed the issue
• Contributor+ that helped on the issue and/or PR

For reference, here are some details about the assignees on this issue:

• @studentofcoding requires payment

As a reminder, here are the bonuses/penalties that should be applied for any External issue:

• Merged PR within 3 business days of assignment - 50% bonus
• Merged PR more than 9 business days after assignment - 50% penalty

[melvin-bot]

Based on my calculations, the pull request did not get merged within 3 working days of assignment. Please, check out my computations here:

• when @studentofcoding got assigned: 2023-08-04 19:46:32 Z
• when the PR got merged: 2023-08-19 02:01:28 UTC
• days elapsed: 10

On to the next one 🚀

[sophiepintoraetz]

Payouts due:

Issue Reporter: $250 N/A (Marco created the issue)
Contributor: $2000 @studentofcoding
Contributor+: $2000 N/A (Marco)

Eligible for 50% #urgency bonus? N

@marcochavezf - I think the original timeframe was in this comment here (5 days), right? So there's no penalty applied?

[sophiepintoraetz]

Payment done - closing

[studentofcoding]

Payouts due:

Issue Reporter: $250 N/A (Marco created the issue)

Contributor: $2000 @studentofcoding

Contributor+: $2000 N/A (Marco)

Eligible for 50% #urgency bonus? N

@marcochavezf - I think the original timeframe was in this comment here (5 days), right? So there's no penalty applied?

Yes it's on 5 days @sophiepintoraetz

Thanks for approving