From f1d4cf60085b20eb2e8a490969d7fdc472df561b Mon Sep 17 00:00:00 2001 From: IronManion <36028632+IronManion@users.noreply.github.com> Date: Fri, 16 Aug 2024 11:47:18 -0700 Subject: [PATCH] Initial commit --- .github/dependabot.yml | 6 ++ .github/steps/-step.txt | 1 + .github/steps/0-welcome.md | 1 + .github/steps/1-removing-sensitive-data.md | 40 ++++++++++ .github/steps/2-removing-with-bfg.md | 47 ++++++++++++ .github/steps/3-avoiding-future-commits.md | 36 +++++++++ .github/steps/X-finish.md | 25 +++++++ .github/workflows/0-welcome.yml | 72 ++++++++++++++++++ .../workflows/1-removing-sensitive-data.yml | 74 ++++++++++++++++++ .github/workflows/2-removing-with-bfg.yml | 72 ++++++++++++++++++ .../workflows/3-avoiding-future-commits.yml | 74 ++++++++++++++++++ .gitignore | 37 +++++++++ LICENSE | 7 ++ README.md | 75 +++++++++++++++++++ 14 files changed, 567 insertions(+) create mode 100644 .github/dependabot.yml create mode 100644 .github/steps/-step.txt create mode 100644 .github/steps/0-welcome.md create mode 100644 .github/steps/1-removing-sensitive-data.md create mode 100644 .github/steps/2-removing-with-bfg.md create mode 100644 .github/steps/3-avoiding-future-commits.md create mode 100644 .github/steps/X-finish.md create mode 100644 .github/workflows/0-welcome.yml create mode 100644 .github/workflows/1-removing-sensitive-data.yml create mode 100644 .github/workflows/2-removing-with-bfg.yml create mode 100644 .github/workflows/3-avoiding-future-commits.yml create mode 100644 .gitignore create mode 100644 LICENSE create mode 100644 README.md diff --git a/.github/dependabot.yml b/.github/dependabot.yml new file mode 100644 index 0000000..8ac6b8c --- /dev/null +++ b/.github/dependabot.yml @@ -0,0 +1,6 @@ +version: 2 +updates: + - package-ecosystem: "github-actions" + directory: "/" + schedule: + interval: "monthly" diff --git a/.github/steps/-step.txt b/.github/steps/-step.txt new file mode 100644 index 0000000..573541a --- /dev/null +++ b/.github/steps/-step.txt @@ -0,0 +1 @@ +0 diff --git a/.github/steps/0-welcome.md b/.github/steps/0-welcome.md new file mode 100644 index 0000000..9ff13a5 --- /dev/null +++ b/.github/steps/0-welcome.md @@ -0,0 +1 @@ + diff --git a/.github/steps/1-removing-sensitive-data.md b/.github/steps/1-removing-sensitive-data.md new file mode 100644 index 0000000..e585b2c --- /dev/null +++ b/.github/steps/1-removing-sensitive-data.md @@ -0,0 +1,40 @@ + + +## Step 1: Removing sensitive data + +_Welcome to "Change commit history"! :wave:_ + +We'll start by working with `.env` files. These files usually contain sensitive content. For this course, we'll work on removing that file and all traces in the Git history. The first step is to remove the file from repository. We'll alter the history later. + +We'll assume you're using the command line, but you can complete the course using your preferred tooling. + +**What is _sensitive content_?** Sensitive content is anything that is checked into your repository history that may put you or your organization at risk. This content usually comes in the form of credentials (i.e., passwords, access keys). The best practice for accidentally exposed sensitive content is to invalidate it (i.e., revoke a personal access token), completely remove it from all repository copies, and take measures to prevent future exposure. + +See [Deleting a file on GitHub Docs](https://docs.github.com/en/repositories/working-with-files/managing-files/deleting-files-in-a-repository#deleting-a-file) if you need additional help removing a file. + +### :keyboard: Activity: Remove `.env` in the project root directory + +1. Open your terminal of choice, clone this repository, and switch to your repository directory. + ```shell + git clone + cd + ``` +2. Delete `.env` from the root directory. + ```shell + git rm .env + ``` +3. Commit the removal of `.env`. + ```shell + git commit -m "remove .env file" + ``` +4. Push the removal to GitHub: + ```shell + git push + ``` +5. Wait about 20 seconds then refresh this page (the one you're following instructions from). [GitHub Actions](https://docs.github.com/en/actions) will automatically update to the next step. diff --git a/.github/steps/2-removing-with-bfg.md b/.github/steps/2-removing-with-bfg.md new file mode 100644 index 0000000..4606ac8 --- /dev/null +++ b/.github/steps/2-removing-with-bfg.md @@ -0,0 +1,47 @@ + + +## Step 2: Removing a file from Git history using BFG Repo-Cleaner + +_You removed `.env` from the repository's root directory! :tada:_ + +Now that we've deleted the file, people that browse the repository on GitHub.com or anyone looking at just the head commit won't see the file. However, due to Git's nature, the file is still present in the history. In this step, we'll work on removing the file from the repository history. + +**What is a _head commit_**? In Git, HEAD points to a branch or a commit. When we say [head commit](https://docs.github.com/en/get-started/quickstart/github-glossary#head), we usually mean the most recent commit in the repository's history. + +There are multiple tools available for removing Git history, we'll use BFG Repo-Cleaner in this step. You can find additional documentation on [Using the BFG in GitHub Docs](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/removing-sensitive-data-from-a-repository#using-the-bfg). + +**What is _BFG Repo-Cleaner_**? BFG Repo-Cleaner is software that can help you search through and alter repository history. Git can natively do this using [`git filter-repo`](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/removing-sensitive-data-from-a-repository#using-git-filter-repo), but it can be more complex. + +### :keyboard: Activity: Use BFG Repo-Cleaner to remove the `.env` file + +1. Update the local copy of your repository to ensure you have the most recent version of the course files. + ```shell + git pull + ``` +2. Install BFG Repo-Cleaner on your machine. You can follow the [instructions on the web site](https://rtyley.github.io/bfg-repo-cleaner/) to do so or you can use a package manager for your operating system. +3. Confirm the `.env` file is removed from the root directory. The command should return empty. + ```shell + find . -name ".env" + ``` +4. Search for .env in the repository's history. The command should return at least 2 commits: the addition of `.env` when you copied this template repository, and the removal of `.env`. + ```shell + git log --stat --all -- .env + ``` +5. Use BFG Repo-Cleaner to delete all references to `.env` that exist in the repository. + ```shell + bfg --delete-files .env + ``` +6. The tool will run and make some suggestions about some follow-up commands. Run those to get your local repository cleaned up. +7. Repeat the search for `.env` in the repository's history. This time, the command should return empty. + ```shell + git log --stat --all -- .env + ``` +8. Push your changes to GitHub. Note we're using the `--force` argument in this step since we're altering Git history. + ```shell + git push --force + ``` +9. Wait about 20 seconds then refresh this page (the one you're following instructions from). [GitHub Actions](https://docs.github.com/en/actions) will automatically update to the next step. diff --git a/.github/steps/3-avoiding-future-commits.md b/.github/steps/3-avoiding-future-commits.md new file mode 100644 index 0000000..9faa445 --- /dev/null +++ b/.github/steps/3-avoiding-future-commits.md @@ -0,0 +1,36 @@ + + +## Step 3: Avoiding future commits with `.env` + +_Nice work removing the file from entire history of the repository! :sparkles:_ + +The steps we've taken so far ensure that any _new_ clones of the repository don't contain the sensitive data. But what about collaborators that may already have a copy of the repository? You should ask anyone with a copy of the repository to delete it and clone the repository fresh. In a real-life scenario, you'd also take [additional steps](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/removing-sensitive-data-from-a-repository#fully-removing-the-data-from-github) to ensure no sensitive data is cached on GitHub.com. + +Now that we've mitigated the risk of exposing sensitive content, we'll be proactive and prevent its addition. + +We'll now configure Git so it ignores a future addition of sensitive content by adding `.env` to `.gitignore`. If someone should that file to the local copy of their repository, it will remain only on the contributor's machine and won't be pushed to GitHub. + +**What is `.gitignore`?** This special file allows us to tell Git naming patterns to ignore. You can read more about it in [Ignoring files on GitHub Docs](https://docs.github.com/en/get-started/getting-started-with-git/ignoring-files). + +### :keyboard: Activity: Add `.env` to `.gitignore` + +1. Update the local copy of your repository to ensure you have the most recent version of the course files. + ```shell + git pull + ``` +2. Locate the file we added to the repository titled `.gitignore`. +3. Add `.env` to the file. +4. Stage, commit the file: + ```shell + git add .gitignore + git commit -m "ignore .env files" + ``` +5. Push the file to GitHub.com + ```shell + git push + ``` +6. Wait about 20 seconds then refresh this page (the one you're following instructions from). [GitHub Actions](https://docs.github.com/en/actions) will automatically update to the next step. diff --git a/.github/steps/X-finish.md b/.github/steps/X-finish.md new file mode 100644 index 0000000..53f51ed --- /dev/null +++ b/.github/steps/X-finish.md @@ -0,0 +1,25 @@ + + +## Finish + +_Congratulations friend, you've completed this course!_ + +celebrate + +Here's a recap of all the tasks you've accomplished in your repository: + +- Removed a file from the head commit of the repository +- Used BFG Repo-Cleaner to removed a file from all Git history in the repository +- Avoided accidental commits by telling Git to ignore a specific file name + +### What's next? + +- The Docs on [Removing sensitive data](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/removing-sensitive-data-from-a-repository) are comprehensive and recommended reading after this course. +- Try your hand at some of the other features listed in the [BFG Repo-Cleaner documentation](https://rtyley.github.io/bfg-repo-cleaner/). +- [We'd love to hear what you thought of this course](https://github.com/orgs/skills/discussions/categories/change-commit-history). +- [Take another GitHub Skills course](https://github.com/skills). +- [Read the GitHub Getting Started docs](https://docs.github.com/en/get-started). +- To find projects to contribute to, check out [GitHub Explore](https://github.com/explore). diff --git a/.github/workflows/0-welcome.yml b/.github/workflows/0-welcome.yml new file mode 100644 index 0000000..d620864 --- /dev/null +++ b/.github/workflows/0-welcome.yml @@ -0,0 +1,72 @@ +name: Step 0, Welcome + +# This step triggers after the learner creates a new repository from the template. +# This workflow updates from step 0 to step 1. + +# This will run every time we create push a commit to `main`. +# Reference: https://docs.github.com/en/actions/learn-github-actions/events-that-trigger-workflows +on: + workflow_dispatch: + push: + branches: + - main + +# Reference: https://docs.github.com/en/actions/security-guides/automatic-token-authentication +permissions: + # Need `contents: read` to checkout the repository. + # Need `contents: write` to update the step metadata. + contents: write + +jobs: + # Get the current step to only run the main job when the learner is on the same step. + get_current_step: + name: Check current step number + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v4 + - id: get_step + run: | + echo "current_step=$(cat ./.github/steps/-step.txt)" >> $GITHUB_OUTPUT + outputs: + current_step: ${{ steps.get_step.outputs.current_step }} + + on_start: + name: On start + needs: get_current_step + + # We will only run this action when: + # 1. This repository isn't the template repository. + # 2. The step is currently 0. + # Reference: https://docs.github.com/en/actions/learn-github-actions/contexts + # Reference: https://docs.github.com/en/actions/learn-github-actions/expressions + if: >- + ${{ !github.event.repository.is_template + && needs.get_current_step.outputs.current_step == 0 }} + + # We'll run Ubuntu for performance instead of Mac or Windows. + runs-on: ubuntu-latest + + steps: + # We'll need to check out the repository so that we can edit the README. + - name: Checkout + uses: actions/checkout@v4 + with: + fetch-depth: 0 # Let's get all the branches. + + - name: Create a commit with a .env file + run: | + git config --global user.email "github-actions[bot]@users.noreply.github.com" + git config --global user.name "github-actions[bot]" + echo "SOME_SECURE_TOKEN=abc123" >> .env + git add .env + git commit -m "Add .env file" + git push + + # In README.md, switch step 0 for step 1. + - name: Update to step 1 + uses: skills/action-update-step@v2 + with: + token: ${{ secrets.GITHUB_TOKEN }} + from_step: 0 + to_step: 1 diff --git a/.github/workflows/1-removing-sensitive-data.yml b/.github/workflows/1-removing-sensitive-data.yml new file mode 100644 index 0000000..d4e81a4 --- /dev/null +++ b/.github/workflows/1-removing-sensitive-data.yml @@ -0,0 +1,74 @@ +name: Step 1, Removing sensitive data + +# This workflow updates from step 1 to step 2. + +# This will run every time we push a `.env` file. +# Reference: https://docs.github.com/en/actions/learn-github-actions/events-that-trigger-workflows +on: + workflow_dispatch: + # Add events that trigger this workflow. + push: + paths: + - .env + +# Reference: https://docs.github.com/en/actions/security-guides/automatic-token-authentication +permissions: + # Need `contents: read` to checkout the repository. + # Need `contents: write` to update the step metadata. + contents: write + +jobs: + # Get the current step to only run the main job when the learner is on the same step. + get_current_step: + name: Check current step number + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v4 + - id: get_step + run: | + echo "current_step=$(cat ./.github/steps/-step.txt)" >> $GITHUB_OUTPUT + outputs: + current_step: ${{ steps.get_step.outputs.current_step }} + + on_push-dotenv-file: + name: On push of .env file + needs: get_current_step + + # We will only run this action when: + # 1. This repository isn't the template repository. + # 2. The step is currently 1. + # Reference: https://docs.github.com/en/actions/learn-github-actions/contexts + # Reference: https://docs.github.com/en/actions/learn-github-actions/expressions + if: >- + ${{ !github.event.repository.is_template + && needs.get_current_step.outputs.current_step == 1 }} + + # We'll run Ubuntu for performance instead of Mac or Windows. + runs-on: ubuntu-latest + + steps: + # We'll need to check out the repository so that we can edit the README. + - name: Checkout + uses: actions/checkout@v4 + with: + fetch-depth: 0 # Let's get all the branches. + + # Logic for step 1 + - name: Ensure .env is gone from the root directory + run: | + if [ -f .env ]; then + echo "The .env file still exists in the root directory, please remove it." + exit 1 + else + echo "Great! The .env is no longer in the root directory." + exit 0 + fi + + # In README.md, switch step 1 for step 2. + - name: Update to step 2 + uses: skills/action-update-step@v2 + with: + token: ${{ secrets.GITHUB_TOKEN }} + from_step: 1 + to_step: 2 diff --git a/.github/workflows/2-removing-with-bfg.yml b/.github/workflows/2-removing-with-bfg.yml new file mode 100644 index 0000000..9335288 --- /dev/null +++ b/.github/workflows/2-removing-with-bfg.yml @@ -0,0 +1,72 @@ +name: Step 2, Removing a file with BFG Repo-Cleaner + +# This workflow updates from step 2 to step 3. + +# This will run every time we push to the repository. +# Reference: https://docs.github.com/en/actions/learn-github-actions/events-that-trigger-workflows +on: + workflow_dispatch: + # Add events that trigger this workflow. + push: + +# Reference: https://docs.github.com/en/actions/security-guides/automatic-token-authentication +permissions: + # Need `contents: read` to checkout the repository. + # Need `contents: write` to update the step metadata. + contents: write + +jobs: + # Get the current step to only run the main job when the learner is on the same step. + get_current_step: + name: Check current step number + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v4 + - id: get_step + run: | + echo "current_step=$(cat ./.github/steps/-step.txt)" >> $GITHUB_OUTPUT + outputs: + current_step: ${{ steps.get_step.outputs.current_step }} + + on_push: + name: On push + needs: get_current_step + + # We will only run this action when: + # 1. This repository isn't the template repository. + # 2. The step is currently 2. + # Reference: https://docs.github.com/en/actions/learn-github-actions/contexts + # Reference: https://docs.github.com/en/actions/learn-github-actions/expressions + if: >- + ${{ !github.event.repository.is_template + && needs.get_current_step.outputs.current_step == 2 }} + + # We'll run Ubuntu for performance instead of Mac or Windows. + runs-on: ubuntu-latest + + steps: + # We'll need to check out the repository so that we can edit the README. + - name: Checkout + uses: actions/checkout@v4 + with: + fetch-depth: 0 # Let's get all the branches. + + # Logic for step 2 + - name: Ensure .env is gone from history + run: | + if [ -n "$(git log --stat --all -- .env)" ]; then + echo "File .env is still in the repository's history, please remove it." + exit 1 + else + echo "File .env doesn't exist in the repository's history." + exit 0 + fi + + # In README.md, switch step 2 for step 3. + - name: Update to step 3 + uses: skills/action-update-step@v2 + with: + token: ${{ secrets.GITHUB_TOKEN }} + from_step: 2 + to_step: 3 diff --git a/.github/workflows/3-avoiding-future-commits.yml b/.github/workflows/3-avoiding-future-commits.yml new file mode 100644 index 0000000..ce103cb --- /dev/null +++ b/.github/workflows/3-avoiding-future-commits.yml @@ -0,0 +1,74 @@ +name: Step 3, Avoiding future commits with .env + +# This workflow updates from step 3 to step 4. + +# This will run every time we push `.gitignore`. +# Reference: https://docs.github.com/en/actions/learn-github-actions/events-that-trigger-workflows +on: + workflow_dispatch: + # Add events that trigger this workflow. + push: + paths: + - .gitignore + +# Reference: https://docs.github.com/en/actions/security-guides/automatic-token-authentication +permissions: + # Need `contents: read` to checkout the repository. + # Need `contents: write` to update the step metadata. + contents: write + +jobs: + # Get the current step to only run the main job when the learner is on the same step. + get_current_step: + name: Check current step number + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v4 + - id: get_step + run: | + echo "current_step=$(cat ./.github/steps/-step.txt)" >> $GITHUB_OUTPUT + outputs: + current_step: ${{ steps.get_step.outputs.current_step }} + + on_push-gitignore: + name: On push of .gitignore + needs: get_current_step + + # We will only run this action when: + # 1. This repository isn't the template repository. + # 2. The step is currently 3. + # Reference: https://docs.github.com/en/actions/learn-github-actions/contexts + # Reference: https://docs.github.com/en/actions/learn-github-actions/expressions + if: >- + ${{ !github.event.repository.is_template + && needs.get_current_step.outputs.current_step == 3 }} + + # We'll run Ubuntu for performance instead of Mac or Windows. + runs-on: ubuntu-latest + + steps: + # We'll need to check out the repository so that we can edit the README. + - name: Checkout + uses: actions/checkout@v4 + with: + fetch-depth: 0 # Let's get all the branches. + + # Logic for step 3 + - name: Check if .gitignore contains .env + run: | + if grep -qw ".env" .gitignore; then + echo ".env is in .gitignore" + exit 0 + else + echo ".env is not in .gitignore" + exit 1 + fi + + # In README.md, switch step 3 for step 4. + - name: Update to step 4 + uses: skills/action-update-step@v2 + with: + token: ${{ secrets.GITHUB_TOKEN }} + from_step: 3 + to_step: X diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..773bfd6 --- /dev/null +++ b/.gitignore @@ -0,0 +1,37 @@ +# Compiled source # +################### +*.com +*.class +*.dll +*.exe +*.o +*.so + +# Packages # +############ +# it's better to unpack these files and commit the raw source +# git has its own built in compression methods +*.7z +*.dmg +*.gz +*.iso +*.jar +*.rar +*.tar +*.zip + +# Logs and databases # +###################### +*.log +*.sql +*.sqlite + +# OS generated files # +###################### +.DS_Store +.DS_Store? +._* +.Spotlight-V100 +.Trashes +ehthumbs.db +Thumbs.db diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..6c5bc3d --- /dev/null +++ b/LICENSE @@ -0,0 +1,7 @@ +Copyright (c) GitHub, Inc. + +Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. diff --git a/README.md b/README.md new file mode 100644 index 0000000..442f5a6 --- /dev/null +++ b/README.md @@ -0,0 +1,75 @@ +
+ + + +# Remove commit history + +Accidental commits can be tricky to remove with Git. In this GitHub Skills course, you'll use BFG Repo-Cleaner to change the history of a Git repository. You can apply what you learn in this course to fully remove sensitive material from your own repository. + +
+ + + +## Welcome + +A trustworthy commit history is the backbone of version control with Git. As a result, altering commit history is difficult by design. Sometimes, the history needs to be altered to remove credentials or other sensitive data that is mistakenly checked in. In this course, we'll learn how to remove content from repository's complete history and apply best practices for preventing accidental commits in the future. + +- **Who is this for**: Intermediate users of Git, organizations +- **What you'll learn**: How to remove a file from Git's entire history +- **What you'll build**: You'll manipulate the history of a Git repository +- **Prerequisites**: We recommend you clone this repository to your machine and use the command line to follow along. You'll also need to install BFG Repo-Cleaner, but the steps will be covered in the course. +- **How long**: This course takes less than 1 hour to complete. + +In this course, you will: + +1. Remove content from the root directory of a repository +2. Use BFG Repo-Cleaner to remove content from repository history +3. Avoid future accidental commits by adding a pattern to `.gitignore` + +### How to start this course + + + +[![start-course](https://user-images.githubusercontent.com/1221423/235727646-4a590299-ffe5-480d-8cd5-8194ea184546.svg)](https://github.com/new?template_owner=skills&template_name=change-commit-history&owner=%40me&name=skills-change-commit-history&description=My+copy+of+the+skills+course+on+changing+commit+history&visibility=public) + +1. Right-click **Start course** and open the link in a new tab. +2. In the new tab, most of the prompts will automatically fill in for you. + - For owner, choose your personal account or an organization to host the repository. + - We recommend creating a public repository, as private repositories will [use Actions minutes](https://docs.github.com/en/billing/managing-billing-for-github-actions/about-billing-for-github-actions). + - Scroll down and click the **Create repository** button at the bottom of the form. +3. After your new repository is created, wait about 20 seconds, then refresh the page. Follow the step-by-step instructions in the new repository's README. + +