Keep local Docker images up to date

[ericvennemeyer]

Dependency

• Investigate use of Daniel Ridge bot its repo 'check-ghpages-versions'  #5897 when the bot is working again, review issue to see what we need to do.

Overview

As a member of the website development team, I want to make sure my local Docker environment always matches the GitHub Pages environment. This issue is to create a procedure for developers to periodically update their local Docker images.

Action Items

• Review the background notes below to understand the reason Docker images need to be updated locally
• Develop a procedure that will result in Docker images being updated regularly
  • Possible solution: add instructions in the How to Contribute wiki, potentially in section 2.7 Working on an issue, about how to delete the locally-stored image and pull the latest version.
  • Alternative: research the possibility of automating this process by modifying the docker-compose.yml file, or adding additional flags to the docker compose up command developers are instructed to use.

Resources/Instructions

• How to Contribute wiki
• Working on an issue wiki section
• docker compose pull documentation
• docker rmi documentation
• Docker desktop/dashboard documentation
• Details about a bot HfLA created re this project https://github.com/hackforla/ops/wiki/@danielridgebot-Ops-Wiki

Notes on local Docker images

Although the Docker image the website team uses (hackforlaops/ghpages) is updated every time GitHub Pages updates its software dependencies, the newly-updated image is not automatically downloaded to a developer's local computer every time they run docker compose up. Instead, Docker will look to see if the image already exists locally and, if it does, will continue to use that version. This may mean that whichever version of the image existed when a developer first ran Docker is the one they are still using, even if new versions have since been published.

The most straightforward way of addressing this is for developers to periodically remove their local copy of the image, either via command line or through the Docker Dashboard GUI, before running docker compose up. This will force Docker to automatically download the latest version from Docker Hub.

A possible solution would be to make removing the image a new step in another workflow that developers regularly use, such as creating a local feature branch to work on an issue, or updating their local gh-pages before pushing a finished issue to their forked repo.

[JessicaLucindaCheng]

To dev leads:
Are any of the devs up to speed and know what this issue is about? If no devs are, then is there a strong backend developer that can get up to speed on the issue and then meet up with Bonnie and Eric to get up to speed on the next steps, and get a hand off from Eric?

[blulady]

To dev leads: Are any of the devs up to speed and know what this issue is about? If no devs are, then is there a strong backend developer that can get up to speed on the issue and then meet up with Bonnie and Eric to get up to speed on the next steps, and get a hand off from Eric?

I am familiar with the issue. I think having the extra step in the work flows is good for familiarizing the developers with simple docker commands as well as being a simple solution to this problem. @JessicaLucindaCheng is there a particular reason a meeting is necessary?

[JessicaLucindaCheng]

To dev leads: Are any of the devs up to speed and know what this issue is about? If no devs are, then is there a strong backend developer that can get up to speed on the issue and then meet up with Bonnie and Eric to get up to speed on the next steps, and get a hand off from Eric?

I am familiar with the issue. I think having the extra step in the work flows is good for familiarizing the developers with simple docker commands as well as being a simple solution to this problem. @JessicaLucindaCheng is there a particular reason a meeting is necessary?

We discussed this at the 2022-11-14 Dev/PM meeting. Here are the notes copied from the meeting:

• Setup an issue for meeting with Eric
  • Sarah will link to issue
  • Bonnie will add her links
  • Whoever is going to lead the meeting read through these issues before the meeting:
    • Keep local Docker images up to date #3488
    • Find or create a reliable Docker image for HFLA Website team ops#28 pre work
    • https://github.com/hackforla/ops/issues?q=is%3Aopen+is%3Aissue+label%3A%22feature%3A+docker%22
    • Daniel Ridge Bot issue template for updates requested needs to be improved ops#52
    • https://github.com/hackforla/ops/wiki/@danielridgebot-Ops-Wiki
  • Get up to speed and figure out what needs to be done, such as having a meeting or updating the wiki

[blulady]

@JessicaLucindaCheng @ExperimentsInHonesty Hey guys, there was a link that you showed me at that meeting to a glitching Docker Image. Before arranging this meeting, can someone link the faulty Docker Image? Thanks!!!

[github-actions]

@jdingeman

Please add update using the below template (even if you have a pull request). Afterwards, remove the '2 weeks inactive' label and add the 'Status: Updated' label.

1. Progress: "What is the current status of your project? What have you completed and what is left to do?"
2. Blockers: "Difficulties or errors encountered."
3. Availability: "How much time will you have this week to work on this issue?"
4. ETA: "When do you expect this issue to be completed?"
5. Pictures (optional): "Add any pictures of the visual changes made to the site so far."

If you need help, be sure to either: 1) place your issue in the developer meeting discussion column and ask for help at your next meeting, 2) put a "Status: Help Wanted" label on your issue and pull request, or 3) put up a request for assistance on the #hfla-site channel. Please note that including your questions in the issue comments- along with screenshots, if applicable- will help us to help you. Here and here are examples of well-formed questions.

_{You are receiving this comment because your last comment was before Tuesday, May 23, 2023 at 12:17 AM PST.}

[github-actions]

@jdingeman

Please add update using the below template (even if you have a pull request). Afterwards, remove the '2 weeks inactive' label and add the 'Status: Updated' label.

1. Progress: "What is the current status of your project? What have you completed and what is left to do?"
2. Blockers: "Difficulties or errors encountered."
3. Availability: "How much time will you have this week to work on this issue?"
4. ETA: "When do you expect this issue to be completed?"
5. Pictures (optional): "Add any pictures of the visual changes made to the site so far."

If you need help, be sure to either: 1) place your issue in the developer meeting discussion column and ask for help at your next meeting, 2) put a "Status: Help Wanted" label on your issue and pull request, or 3) put up a request for assistance on the #hfla-site channel. Please note that including your questions in the issue comments- along with screenshots, if applicable- will help us to help you. Here and here are examples of well-formed questions.

_{You are receiving this comment because your last comment was before Tuesday, May 30, 2023 at 12:15 AM PST.}

[github-actions]

@jdingeman

Please add update using the below template (even if you have a pull request). Afterwards, remove the '2 weeks inactive' label and add the 'Status: Updated' label.

1. Progress: "What is the current status of your project? What have you completed and what is left to do?"
2. Blockers: "Difficulties or errors encountered."
3. Availability: "How much time will you have this week to work on this issue?"
4. ETA: "When do you expect this issue to be completed?"
5. Pictures (optional): "Add any pictures of the visual changes made to the site so far."

If you need help, be sure to either: 1) place your issue in the developer meeting discussion column and ask for help at your next meeting, 2) put a "Status: Help Wanted" label on your issue and pull request, or 3) put up a request for assistance on the #hfla-site channel. Please note that including your questions in the issue comments- along with screenshots, if applicable- will help us to help you. Here and here are examples of well-formed questions.

_{You are receiving this comment because your last comment was before Tuesday, June 13, 2023 at 12:16 AM PST.}

[ExperimentsInHonesty]

@roslynwythe I moved this into the new issue review column because the dependency is resolved. Please add a ready for prioritization label if its ready.

[HackforLABot]

Hi @roslynwythe, HfLA appreciates your interest in this issue, but please note that it is in the "New Issue Approval" column of the Project Board because it has not been finalized, approved, or prioritized, and so it is not ready for assignment. For this reason, you have been unassigned from this issue. Please remember to assign issues only from the "Prioritized Backlog" column.

The only exceptions to this rule are if you are writing an issue and the Draft label is applied, or if you are self-assigning to your "Pre-work Checklist" (the issue includes the Complexity: Prework label).

[roslynwythe]

I recommend that when an updated image hackforlaops/ghpages is available on Dockerhub, we notify developers via Slack and at meetings. I suggest these instructions:

• open Docker Desktop
• display Images (from left column menu)
• use the GUI to pull updates to the image.
• allow extra time for the next 'docker-compose, since a container will have to be recreated

[roslynwythe]

Pulling a new image and rebuilding the container is time consuming so I would rather avoid doing that needlessly.
There are some ways to automate this so that the docker client will check to see if a more recent version of an image is available.
In https://stackoverflow.com/questions/47326900/check-for-newer-docker-build-for-image?rq=3
Two methods are proposed:

1. a project called Watchtower (https://github.com/v2tec/watchtower), which watches the running container and if there is a new version with the same tag available, it will pull the new image and restart the container.
2. scripting within docker-compose.yml (that invokes a python script) to detect if the local docker image is out of date, but I'm not sure that is feasible as it requires python.

I might not have time for analysis of the above so please advise, should I modify the Action Items and unassign?

[ExperimentsInHonesty]

@roslynwythe will figure out what kind of check that people writing code for an issue, reviewing or submitting PRs could do, that would tell them if they have the most current docker image https://hub.docker.com/r/hackforlaops/ghpages

If there is an easy way to tell, then we write something for contributing on how to check and then how to update if needed.

[roslynwythe]

Draft: Proposed changes to CONTRIBUTING.md Section 2.7.b

• Change the title of Section 2.7.b from "Working on an issue (2): Create a new branch where you will work on your issue" to "Working on an issue (2): Creating and testing your issue branch"
• Replace the section starting with "i. What if you cannot see your changes locally within Docker" with:
  i. To test your issue branch locally, run the command "docker-compose up" from a terminal window. The website will then be accessible at http://localhost:4000 . A few notes regarding Docker:
  • the Docker Desktop application must be running, otherwise `docker-compose up` will not work
  • Docker pulls an image from Dockerhub.com which enables Docker to built the website locally with the same environment used to build the site at GitHub. As the environment and dependencies change, HfLA engineers update the Dockerhub image; however, the image updated at Dockerhub is not automatically downloaded, so it is necessary to periodically check to determine if the local image is outdated, and if so, to pull the updates from Dockerhub. Use this process:
  • In the Docker Desktop application, from the `Images` page, click the Action menu (3 dots) for the image `hackforlaops/ghpages`, select option `view details`.
    
    How to view details for a Docker image

    

  • Note the `CREATED` column
    
    How to determine when an image was last updated in Docker Desktop

    

  • VIew the details of the "latest" tag from [hackforlaops Dockerhub image tags](https://hub.docker.com/r/hackforlaops/ghpages/tags) and determine when an image was pushed to Docker hub
    
    How to determine when an image was pushed to Docker hub

    

    If the "latest" tagged image at Docker hub is more recent than the image in Docker Desktop, use the Docker Desktop Action menu (3 dots) to "pull" down the updated image
    
    Pull image from DockerHub

    

  • Note that the next `docker-compose up` will take extra time to recreate the Docker container based on the updated image.

[roslynwythe]

• See Update CONTRIBUTING.md Section 2.7.b with information about Docker #7576