Find or create a reliable Docker image for HFLA Website team

[ericvennemeyer]

Overview

We need to replace by finding or creating a reliable Docker image that is pinned to the dependency versions used by GH-Pages so the Website team can do development work on their local machines. The current one is no longer supported.

Action Items

• Confirm that no preexisting, externally-maintained Docker image exists that meets the above criteria.
• Explore the possibility of creating a custom HFLA image.
  • Build, test and confirm successful functioning of a custom image.
  • Determine feasibility and potential cost of creating a HFLA account on Docker Hub to host a custom image.
  • Determine procedure for tracking changes in GH-Pages dependencies and keeping custom image up-to-date.

Details

The image currently in use has switched to using versions of Ruby and Jekyll that throw errors when Website devs attempt to run the site locally in a Docker container.

As a new HFLA Website volunteer, I found that I and my fellow new volunteers were experiencing an error when trying to run the site locally with Docker. I determined this error was caused by an incompatibility between Ruby 3 and Jekyll 4. The Website team uses an image called jekyll:pages from the official Jekyll repo on Docker Hub in conjunction with the docker-compose up command when starting a container locally. In looking through the Jekyll Docker repo on GitHub, I found the error we experienced began roughly around the time the official Jekyll Docker images were updated from Ruby 2.7 to Ruby 3.1.

I emailed the maintainer of the Jekyll Docker repo directly to inquire about a solution, then followed up about a week later by opening an issue to request that the jekyll:pages image be updated to match the dependencies used by GitHub Pages. It has now been over a month without a response.

In the meantime, I decided to go ahead and fork the entire Jekyll Docker repo and modify their dockerfile to include the dependencies we need. The resulting image seemed to work--I was able to spin up a Docker container and run the H4LA website locally with it. This image is currently hosted at my personal Docker Hub account, and the code is currently in a repo at my GitHub account.

Resources/Instructions

List of GitHub Pages dependency versions

Potential preexisting GitHub Pages Docker images:
Madduci docker-github-pages repo
Madduci/docker-github-pages image on Docker Hub
Bretfisher/jekyll-serve repo
bretfisher/jekyll images in Docker Hub

Docker pricing info:
Docker subscription overview
Docker pricing and subscriptions
Docker image pull consumption tiers (scroll to bottom)

Docker docs:
Basic info on building Docker images
Guide to creating a new Docker Hub repo and pushing an image to it

GitHub Actions How-Tos:
Learn GitHub Actions
GH Action to build and push Docker images

Internal resources

• Docker hub account managed via HackforLA 1Password
• Repo for image: https://github.com/hackforla/ghpages-docker
• Scripts home here: https://script.google.com/home/all

[ericvennemeyer]

Update

From the Ops meeting on Sunday 6/5: It was decided that HFLA will begin maintaining/hosting our own GitHub Pages Docker image. To achieve this, I will carry out the newly-determined Action Items below.

Action Items

• Create a new HFLA account on Docker Hub and set up a repo for the GitHub Pages image.
• Push the Dockerfile and related code written for our new image from my personal repo to the newly-created hackforla/ghpages-docker repo.
  • Write a README.md in the above repo with usage instructions for the Dockerfile.
• Create a new bot ("Daniel Ridge") to periodically check the dependency versions for GitHub Pages and open a new issue in the Ops repo when something is updated.
• Write a new Wiki in the Ops repo to detail the process for building/maintaining the Docker image.

[ericvennemeyer]

i. Progress: Spent the majority of the week since the Ops meeting on 6/5 refining my Dockerfile code. It's now complete. I will be able to address the remaining action items above next week, as well as update the docker-compose.yml file in the website repo to use our new custom image, and open a pull request to close the original issue.

ii. Blockers: None.

iii. Availability: Tuesday 6/14 and Thursday 6/16, as needed to get the above completed.

iv. ETA: I hope to have all of the above done by end of next week, but it may be the following week. My first priority, however, will be to update the docker-compose.yml file to point to the new image in the new HFLA Docker Hub repo and make a pull request to close out this issue.

[JasonEb]

@ericvennemeyer we have setup a hackforla docker account, and you should be able to access it via the 1password Ops vault.

[ExperimentsInHonesty]

Q: would it be possible for me to get access to the Website vault on 1Password so I can have a look at the Elizabeth Honest script?

A: @ericvennemeyer I need you to add yourself to the roster so that I can give you access to the script.

Also, please put all your notes about the issue and any requests in the issue. You can always click on the three dots at the top of a comment to paste the notes into slack. But putting them in slack only will either slow down our response or keep you from getting one.

[ericvennemeyer]

@ExperimentsInHonesty I apologize, I will put all notes and requests in the issue itself moving forward. Thank you for addressing the questions I posted in Slack.

@JasonEb Thanks for looking into that over the weekend. I don't mean to drag this out, but unfortunately I'm still experiencing some issues with the Docker Hub account. To clarify: when you say you set up a hackforla docker account, do you mean that you created the account on hub.docker.com, or only that a new login for Docker Hub was created within 1Password?

I do have access to the 1Password Ops vault, and I can see the Docker Hub login item. But when I attempt to use those credentials to login at hub.docker.com, I get the following error: "Cannot log into an organization account." After some quick research, it appears that organization accounts (as opposed to regular user accounts) can not be logged into, but only managed by the users listed in the owners team. I don't completely understand the distinction. I only know that there was already a hackforla organization account on hub.docker.com that was last updated ca. four years ago, and so I was unable to use the hackforla name. Were you able to access that existing hackforla account, or did you start a new one?

Re: Elizabeth Honest, I misunderstood how the bot worked. I thought the script it ran on existed within GitHub, but I've since been informed by @JessicaLucindaCheng that it's actually a Google Apps script that sends messages to an associated GitHub account. Now that Bonnie has granted me Website vault access, I have been able to look at the Elizabeth Honest script via the google account. Thank you.

[ericvennemeyer]

Update:

I created a GitHub action in my personal repo, where the Dockerfile currently lives, and used it to successfully build and push the Docker image to my personal Docker Hub account. Next step is to migrate that code to the hackforla/ghpages-docker repo.

Request:

• In order to make the GitHub action work with the HFLA Docker Hub account, I'll need admin access to the hackforla/ghpages-docker repo so I can create secrets to store the login credentials.

[ericvennemeyer]

Question:

• After speaking with @JessicaLucindaCheng and Saumil about the functioning of the Elizabeth Honest bot, I'm not certain that we need the Daniel Ridge bot to work the same way. Elizabeth Honest uses a Google Apps script because her function is to integrate data submitted via Google Forms into the "wins" page of our website. So in that case, it makes sense to use a script that attaches to a Google Doc or Sheet. In the case of Daniel Ridge, there is no Google involvement; I think all functionality could be created via GitHub Actions. With that in mind, is there a reason to create Daniel Ridge as a Google Apps script?

[JasonEb]

@ExperimentsInHonesty I can't add @ericvennemeyer to https://github.com/orgs/hackforla/teams/ops-write/. The option doesn't seem available to the me but if we can add him to the that team, then he should be able to have access to this repo.

[ExperimentsInHonesty]

@ericvennemeyer I have added you to the read and write ops teams

[ericvennemeyer]

@ExperimentsInHonesty @JasonEb Thanks for adding me to the Ops and Ops-Write repos. I just successfully pushed my Dockerfile code to the hackforla/ghpages-docker repo. Unfortunately, I still don't have the permissions I need to add secrets to the repo so I can store the login credentials for our Docker Hub account.

Requests:

• Please make me an admin in the hackforla/ghpages-docker repo. If you would prefer not to, I will need someone with admin access to create secrets within the repo, entitled DOCKER_USERNAME and DOCKER_PASSWORD, using the respective username and password for our Docker Hub account as stored in 1Password.

• I'm still having trouble logging into the Docker Hub account using the aforementioned 1Password credentials. When I try to login as user hackforla I get the following error:

  "Cannot log into an organization account."

  When I try to login with the email ops@hackforla.org, I get this other error:

  "Incorrect authentication credentials."

  @JasonEb said he set up a hackforla Docker Hub account, but I suspect he may have only established the credentials in 1Password, and the problem is that I'm trying to log in to a pre-existing Docker Hub account called "hackforla" that is using other credentials. Please advise.

[ericvennemeyer]

Hey @JasonEb I received a notification that you had added me as an admin, though I can't seem to find the comment now.

Anyway, I can now see the settings tab in the menu, which I couldn't before. But I still don't have the option to create secrets. I'm not sure what additional permissions I would need.

[JasonEb]

@ericvennemeyer I have set DOCKER_USERNAME and DOCKER_PASSWORD to the credentials from the ops 1password vault.

[ericvennemeyer]

i. Progress: The new hackforlaops/ghpages image is now live, and the code to use it instead of the old jekyll/jekyll image has been merged into the website repo. Currently awaiting feedback as more new volunteers are onboarded and old ones update their local repos, but there should be no more Docker errors going forward.

Still left to do:

• Write a README.md for the hackforla/ghpages-docker repo.
• Write a Wiki to explain the technology and process for maintaining it.
• Create a bot to check version numbers and open an issue in the Ops repo when updating is required.

ii. Blockers: None.

iii. Availability: On vacation through July 10, will be able to return to this the week of the 11th.

iv. ETA: Will have the README and Wiki written by end of the week of July 11. Still experimenting with the bot, but hope to have it finished that same week.

[ericvennemeyer]

Apologies that the remaining steps of this issue have taken me longer than expected, but I'm still working on it.

i. Progress: I've written a first draft of a README.md in the ghpages-docker repo, and have written half of the wiki. For the wiki, I'd still like to write a section where I go through the Dockerfile and explain what each part does, so anyone needing to modify it in the future will have a thorough guide. After that, the final step will be to create the bot that creates an issue when a version number changes.

ii. Blockers: None.

iii. Availability: I will be out of town and offline through Sunday 7/24, but will get back to this issue next week.

iv. ETA: Wiki will be completed by end of next week, hopefully the bot as well.

[JasonEb]

Thanks for the update. No worries - let us know if there's anything we can help out with.

[JasonEb]

Any issues or blockers we can help you with? Just keeping a pulse, no worries.

[ericvennemeyer]

Hey @JasonEb thanks for checking in!

Yesterday I completed the rest of the wiki, so that and the readme are done. I'd like to go through them one last time, and if you or anyone else in ops would like to give them a look, I'd love to hear any feedback you might have.

I've left the trickiest part for last, i.e. the bot that opens issues when something needs updating. I plan to spend all of tomorrow on that, so hopefully it will get done. But that may be the source of some potential blockers.

One odd thing I did notice is that the GitHub action that builds and pushes the image to Docker Hub works fine when run manually, but seems to fail when pushing a commit with changes to the Dockerfile. I'm going to look at that tomorrow as well, but this may become a blocker too.

[ericvennemeyer]

Hey @JasonEb here's a brief update:

i. Progress:
I've got the bot script basically working. All that's left now is to tidy up some details, namely the following:

• Have the bot create a test issue in the Ops repo
• Make sure the cron scheduling is working
• Write a README for the bot explaining how it's built
• Make final edits to ghpages-docker README and Wiki
• Create a workflow in ghpages-docker that opens an issue in the Ops repo when the image build fails

Once all of the above is finished, this issue should be closed. Regarding the first step, maybe I can reach out to you next week when I'm about to run the test so you won't be surprised by the random issues showing up? Would also be good to get feedback on the specific text of the issue title and body.

ii. Blockers: None

iii. Availability: Tuesday 8/2 - 3 hours

iv. ETA: EOD Tuesday

[ericvennemeyer]

Hey @JasonEb I've completed all the tasks listed in this issue. There's one more favor I'd like to ask: could you add another secret to the ghpages-docker repo? It's a PAT that will allow the workflow within that repo to open an issue using the @danielridgebot account when the Docker image build fails. I'll slack you with details.

[JasonEb]

added daniel ridge pat secret

[ericvennemeyer]

Thanks @JasonEb that worked perfectly!

Note: I ultimately decided to implement the automated version checking functionality using workflows within an empty repo owned by @danielridgebot, rather than using Google Apps scripts as initially suggested.

From now on, the workflows within the check-ghpages-versions repo will run every day around 1200 UTC, or 5:00am PDT. (Technically, the workflow to check the GitHub-Pages Gem version number will run first, and the workflow to check the Ruby version will be triggered by the completion of the first one.)

Whenever GitHub Pages upgrades to a new version of either/both of those, @danielridgebot will open an issue within the Ops repo to notify you of the change. Once updates to the Dockerfile are made and the image rebuilt and pushed to Docker Hub, @danielridgebot will open an issue notifying of the success or failure of that build.

For more info, please consult the ghpages-docker wiki, ghpages-docker README, and check-ghpages-versions README.

Thanks for all your help in creating a solution to this issue, everyone!

[ericvennemeyer]

Hey @JasonEb Thanks for doing that. I can now see the settings tab in the menu, which I couldn't before. Unfortunately, I still don't have the option to create secrets.

…

On Wed, Jun 22, 2022 at 2:54 PM Jason E ***@***.***> wrote: @ericvennemeyer <https://github.com/ericvennemeyer> I have added you directly as a admin to the repo so you can add secrets to the Let us know if that doesn't enable you to set secrets. — Reply to this email directly, view it on GitHub <#28 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ATWZOVITG442M5KFKJHCXJLVQODRTANCNFSM5WWY5NOQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[ExperimentsInHonesty]

Here are the projects at hack for LA that are https://github.com/search?q=org%3Ahackforla+secrets.GITHUB_TOKEN&type=code