Google Apps Script: Fix data errors in spreadsheet used for wins page

[SAUMILDHANKAR]

Overview

The data used to generate our wins page should be error free. For this issue, we will analyze and fix the data discrepancies between the responses and review sheets used to build the wins page.

Action Items

• This issue was reopened because it failed the QA. Read through the comments, look at the PR, and then tackle this issue
• Obtain access to the Google Apps Script in the admin drive from the technical lead
• Become familiar with the workflow process in creating a wins entry
• In the admin drive, create a copy of the Wins-form (Responses) and Secret files in your own drive to recreate the workflow
• Identify the reason for data discrepancy between the responses and review sheets
• Fix the code logic so that the data is consistent amongst various sheets
• Add an error catching mechanism so that error like this is not repeated again
• From this comment, fix the following problems:
  • There is something wrong with the Wins spreadsheet, where Wins form entry made on 11/5/2021 wasn't added to the Review sheet and so the true/false in the "Display?" and "Homepage?" are off.
• Test the code so that on a new form submit, both the Review sheet and Responses sheet gets updated appropriately
  • See this comment
• Demo the code to the technical lead and team

For merge team:

• Once approved, update the current production code
• Release dependency on Google Apps Script: Add unique IDs as object keys for each wins entry #2385 and Enhance wins sheets to support selection for homepage randomizer #2505
  • Update action items in the dependency issue if needed

Resources/Instructions

Google Apps Script
_wins-data file
wins page JS
Response sheet

[github-actions]

Hi @SAUMILDHANKAR.

Good job adding the required labels for this issue. The merge team will review the issue and add a "Ready for Milestone" label once it is ready for prioritization.

Additional Resources:

• Wiki: How to create issues
• Wiki: How to read and interpret labels

[luke-karis]

ETA: March 23
Availability: Monday, Tuesday, Wednesday from 4 pm - 5 pm, Saturday and Sunday 10 am - 12 pm

[luke-karis]

Updated ETA and Availability:
ETA: March 28
Availability Friday, Saturday, Sunday 1-2 hours each day
Progress: Got access to Drive folder, I plan to really get going on this during the weekend
Blockers: None

[JessicaLucindaCheng]

@luke-karis Thanks for providing an updated ETA and availability but if you are providing a progress update please also include Progress and Blockers. Here is more information about providing a progress update: https://github.com/hackforla/website/blob/gh-pages/CONTRIBUTING.md#25-reporting-progress-on-your-issue

Once you have completed a progress report that includes progress, blocker, availability, and ETA, you can check off that item in your pre-work checklist and then check off the "Once all tasks are completed, close this issue." task and then close your pre-work issue. Thanks.

[luke-karis]

Updated ETA and Availability:
ETA: April 8
Availability: Monday, Tuesday, Wednesday, Thursday 4-5/5:30 pm
Progress: Started working on figuring out what the issue is
Blockers: none

[luke-karis]

ETA and Availability:
ETA: April 18
Availability: Tuesday 5:00-7:00, Thursday 4:00-5:00, Saturday and Sunday 10:00-12:00
Blockers: Kinda, but a little hard to articulate. I'm going to work more on this on Tuesday and hopefully come up with blockers to bring up a stand up.

[roslynwythe]

Availability: 1/15: 4-5:30 pm ; 1/16 10-noon, 3-5 pm; 1/17 1 -4 pm; 1/18 noon - 5 pm
ETA: 1/22/2022

[roslynwythe]

@jdingeman This issue was reopened because Bonnie had noticed that when she changed display to TRUE on a wins entry, it was not displaying. After testing, I could not find evidence of the bug originally described in this issue. I believe the behavior Bonnie observed is due to the fact that _wins-data.json is no longer being posted to GitHub, due to the out of date token, so no updates to wins data can be expected.

The last merge of data from the PR #3024 (comment)
is dated Nov 1, 2022. However, when I run git log -p _wins-data.json, it gives a date of Dec. 2, 2021 (commit 3b178e7). Several reasons were given for the pause in merging, but based on this comment: #3417 (comment) it sounds like the previous problems with the _wins-data.json feed were resolved and so the outstanding issue is that the GH wins token needs to be updated: #3748 (comment)

While debugging the app script, I examined the wins json feed prior to base64 encoding, and I saw no evidence of the previously reported problems such as empty JSON field or UniqueIDs.

If an additional error catching mechanism is desired for the bug in which Review and Response get out of sync, I suggest that in the function "createIssue" in code.gs, we check for an equal number of rows in Review and Response sheets, and if they are not equal, we a new GitHub issue to indicate the error. We should do the same in 'main', and if Review and Response do have an equal number of rows, we should not send a wins json feed to GitHub. That would be more a complex mechanism than was already created, because it will involve also changes to the ghrequest library code, but it would be a more effective mechanism.

[jdingeman]

Hi @roslynwythe, I think I'm finally understanding how the Google Apps Script is "integrated" into the repo, and it is quite strange to be honest. It seems that the changes are only committed to #3024 when the script is ran manually and successfully. I just ran it a few minutes ago and saw that the changes in Leon's issue #2675 were implemented, but they were not published to the site. If you have time to meet some time in the next week, I'd like to hear your explanation face to face so I can get a better understanding from your perspective.

[ExperimentsInHonesty]

@roslynwythe @jdingeman it looks like you both are working on this issue, so I am going to put it into the in progress column. Please move issues there when you self assign.

[jdingeman]

Met with dev to discuss how to move forward with issue. Determined that changes made in the Google Apps Script does not cause any bugs and is being pushed into #3024 which is not merged yet.

@ExperimentsInHonesty - the reason the changes haven't appeared on the website is because the PR that's holding all the changes going into the .json file hasn't been merged. Right now, the website only displays the data that's in the repo's current wins-data.json and that gets updated when the data coming from the Google Apps Script gets pushed and merged.

I let dev know that we will discuss this issue technically being satisfied since the bug was fixed on the script itself, but won't appear until #3024 is merged. I think when all issues related to Google Apps Script have been finished and satisfied on the script side, we can merge the PR so that changes are put on the published site.

Dev suggested optimizing the debugging methods and will write up a new issue for review.

[roslynwythe]

Please see Justin's comment #2901 (comment) above.
I am awaiting decision from PM and I've created draft issue #3881 to document my suggestions for improved error detection

[github-actions]

@roslynwythe

Please add update using the below template (even if you have a pull request). Afterwards, remove the 'To Update !' label and add the 'Status: Updated' label.

1. Progress: "What is the current status of your project? What have you completed and what is left to do?"
2. Blockers: "Difficulties or errors encountered."
3. Availability: "How much time will you have this week to work on this issue?"
4. ETA: "When do you expect this issue to be completed?"
5. Pictures (optional): "Add any pictures of the visual changes made to the site so far."

If you need help, be sure to either: 1) place your issue in the developer meeting discussion column and ask for help at your next meeting, 2) put a "Status: Help Wanted" label on your issue and pull request, or 3) put up a request for assistance on the #hfla-site channel.

_{You are receiving this comment because your last comment was before Monday, February 6, 2023 at 11:17 PM PST.}

[roslynwythe]

Closing, as the bug described in this issue was not observed. Most likely, the bug was fixed when the issue was originally closed by @luke-karis, although no detail was given. Currently, the _wins-data.json feed appears to be correct and if changes in the wins response sheet display column were not reflected in the website, it is because the feed was not being merged into the live site.

Please see #3024 for details on merging wins data.

See #3881 for proposed changes to provide detection of the bug, and notification in the form of a Github bug report.