Clean up GitHub Workflows #3059

Pwuts · 2023-04-23T21:16:05Z

Background

The workflows are a bit of a mess currently

Changes

Big clean-up, see diff: https://github.com/Significant-Gravitas/Auto-GPT/pull/3059/files

Notable:

Improvements to Docker CI
- Build summaries (ballpark example: https://github.com/badgeteam/mch2022-firmware-esp32/actions/runs/4697249967)
- Layer caching
- Build -> Buildx
Deleted auto_format.yml as it doesn't do anything and we have linting workflows
Renamed dockerhub-imagepush.yml -> docker-release.yml
Renamed docker-image.yml -> docker-ci.yml

Test Plan

Thorough review, and then we'll see if it works after merge.

PR Quality Checklist

My pull request is atomic and focuses on a single change.
I have thoroughly tested my changes with multiple different prompts.
I have considered potential risks and mitigations for my changes.
I have documented my changes clearly and comprehensively.
I have not snuck in any "extra" small tweaks changes

codecov · 2023-04-23T21:19:26Z

Codecov Report

Patch coverage has no change and project coverage change: -0.02 ⚠️

Comparison is base (f8dfedf) 49.65% compared to head (b84a574) 49.63%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3059      +/-   ##
==========================================
- Coverage   49.65%   49.63%   -0.02%     
==========================================
  Files          64       64              
  Lines        3021     3022       +1     
  Branches      505      505              
==========================================
  Hits         1500     1500              
- Misses       1401     1402       +1     
  Partials      120      120

Impacted Files	Coverage Δ
autogpt/commands/web_selenium.py	`0.00% <0.00%> (ø)`

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

.github/workflows/benchmarks.yml

.github/workflows/docker-ci.yml

.github/workflows/docker-release.yml

ntindle · 2023-04-23T21:23:48Z

.github/workflows/pr-label.yml

@@ -34,7 +36,7 @@ jobs:
      pull-requests: write
    runs-on: ubuntu-latest
    steps:
-      - uses: codelytv/pr-size-labeler@v1.7.0
+      - uses: codelytv/pr-size-labeler@v1


Undo this. v1 version has a bug. add a note to keep specifically v1.7.0

According to these comments, it's a matter of setting the right permissions and not having the wrong API URL in the config:
CodelyTV/pr-size-labeler#27 (comment)
CodelyTV/pr-size-labeler#40 (comment)
CodelyTV/pr-size-labeler#40 (comment)

The workflow as it is in this PR should work.

.github/workflows/pr-label.yml

.github/workflows/sponsors_readme.yml

.github/workflows/docker-release.yml

Out of date

.github/workflows/ci.yml

* Clean up GitHub Workflows (Significant-Gravitas#3059) * initial cleanup of github workflows * only run pr-label workflow on push to master * move docker ci/release summaries to scripts * add XS label for PR's under 2 lines * draft test job for Docker CI * fix & activate Docker CI test job * add debug step to docker CI * fix Docker CI test container env * Docker CI build matrix * fixup build summaries * fix pipes in summary * optimize Dockerfile for layer caching * more markdown escaping * add gha cache scopes * add Docker CI cache clean workflow * More robust null byte checking * Adjust test_json_parser file (Significant-Gravitas#1935) Co-authored-by: Reinier van der Leer <github@pwuts.nl> * Add indirection layer between cli and application start * Agent and agent manager tests (Significant-Gravitas#3116) * Update Python version and benchmark file in benchmark.yml * Refactor main function and imports in cli.py * Update import statement in ai_config.py * Add set_temperature and set_memory_backend methods in config.py * Remove unused import in prompt.py * Add goal oriented tasks workflow * Added agent_utils to create agent * added pytest and vcrpy * added write file cassette * created goal oriented task write file with cassettes to not pay openai tokens * solve conflicts * add ability set azure because github workflow needs it off * solve conflicts in cli.py * black because linter fails * solve conflict * setup github action to v3 Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com> * fix conflicts Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com> * Plugins: debug line always printed in plugin load * add decorator to tests Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com> * move decorator higher up Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com> * init * more tests * passing tests * skip gitbranch decorator on ci * decorator skiponci * black * Update tests/utils.py decorator of skipping ci Co-authored-by: Nicholas Tindle <nicktindle@outlook.com> * black * I oopsed the name * black * finally * simple tests for agent and manager * ísort --------- Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com> Co-authored-by: Merwane Hamadi <merwanehamadi@gmail.com> Co-authored-by: Merwane Hamadi <merwane.hamadi@redica.com> Co-authored-by: Richard Beales <rich@richbeales.net> Co-authored-by: Nicholas Tindle <nick@ntindle.com> Co-authored-by: BillSchumacher <34168009+BillSchumacher@users.noreply.github.com> Co-authored-by: Nicholas Tindle <nicktindle@outlook.com> * anontation fix (Significant-Gravitas#3018) * anontation fix * fix param name and type --------- Co-authored-by: Richard Beales <rich@richbeales.net> * Hotfix/global agent manager workaround (Significant-Gravitas#3157) * Add indirection layer to entry point * Get around singleton pattern for AgentManager to fix tests * Remove dead permanent memory module (Significant-Gravitas#3145) * Remove dead permanent memory module * Delete sqlite db that snuck in * Refactor/move singleton out of config module (Significant-Gravitas#3161) * Add option to install plugin dependencies (Significant-Gravitas#3068) Co-authored-by: Nicholas Tindle <nick@ntindle.com> * Re-add install-plugin-deps to CLI (Significant-Gravitas#3170) * Update ci.yml (Significant-Gravitas#3179) * Update bulletin to warn about deprication (Significant-Gravitas#3181) * Ensure Fore.RED is followed by Fore.RESET (Significant-Gravitas#3182) This properly resets the terminal, ensuring that the red text is red and the normal text remains unaffected. Co-authored-by: Richard Beales <rich@richbeales.net> * Add a little more descriptive installation instruction (Significant-Gravitas#3180) * add Dev Container installation instruction to installation.md * add Dev Container installation instruction to installation.md * Update installation.md --------- Co-authored-by: Richard Beales <rich@richbeales.net> * Update installation.md (Significant-Gravitas#3166) * Refactor/decouple logger from global configuration (Significant-Gravitas#3171) * Decouple logging from the global configuration * Configure logging first * Clean up global voice engine creation * Remove class vars from logger * Remove duplicate implementation of --------- Co-authored-by: Richard Beales <rich@richbeales.net> * docs fix to image generation (Significant-Gravitas#3186) * Update README.md * Feature: Added Self Feedback (Significant-Gravitas#3013) * Feature: Added Self Feedback * minor fix: complied to flake8 * Add: Self Feedback To Usage.md * Add: role/goal allignment * Added: warning to usage.md * fix: Formatted with black --------- Co-authored-by: Richard Beales <rich@richbeales.net> * Update README.md * this changes it so the file from config is used, rather than a hardcoded name that might not exist (Significant-Gravitas#3189) * Test `Agent.create_agent_feedback` (Significant-Gravitas#3209) * remove plugin notice from CONTRIBUTING.md (Significant-Gravitas#3227) * Extract OpenAI API retry handler and unify ADA embeddings calls. (Significant-Gravitas#3191) * Extract retry logic, unify embedding functions * Add some docstrings * Remove embedding creation from API manager * Add test suite for retry handler * Make api manager fixture * Fix typing * Streamline tests * Use explicit API keys when querying openai rather than import time manipulation of the package attributes (Significant-Gravitas#3241) * Fix readme centering (Significant-Gravitas#3243) * Remove import time loading of config from llm_utils (Significant-Gravitas#3245) * adding probably erroneously removed return value from execut_shell, giving 'None' in return always otherise - not ideal (Significant-Gravitas#3212) Co-authored-by: James Collins <collijk@uw.edu> * Load .env in package init (Significant-Gravitas#3251) * Fix PR size autolabeler message (Significant-Gravitas#3194) * Fix `get_ada_embedding` return type (Significant-Gravitas#3263) --------- Signed-off-by: Merwane Hamadi <merwanehamadi@gmail.com> Co-authored-by: Reinier van der Leer <github@pwuts.nl> Co-authored-by: James Collins <collijk@uw.edu> Co-authored-by: YOUNESS ZEMZGUI <31807902+younessZMZ@users.noreply.github.com> Co-authored-by: Nicholas Tindle <nick@ntindle.com> Co-authored-by: Media <12145726+rihp@users.noreply.github.com> Co-authored-by: Merwane Hamadi <merwanehamadi@gmail.com> Co-authored-by: Merwane Hamadi <merwane.hamadi@redica.com> Co-authored-by: Richard Beales <rich@richbeales.net> Co-authored-by: BillSchumacher <34168009+BillSchumacher@users.noreply.github.com> Co-authored-by: Nicholas Tindle <nicktindle@outlook.com> Co-authored-by: Lei Zhang <isleizhang@outlook.com> Co-authored-by: Daniel Chen <daniel.young.chen@gmail.com> Co-authored-by: Deso <ethan.johnson1997@outlook.com> Co-authored-by: Lawrence Neal <lwneal@users.noreply.github.com> Co-authored-by: Duong HD <79622476+resduo@users.noreply.github.com> Co-authored-by: ✔️ITtechtor <104917057+MarKiSS2000@users.noreply.github.com> Co-authored-by: Toran Bruce Richards <toran.richards@gmail.com> Co-authored-by: AbTrax <45964236+AbTrax@users.noreply.github.com> Co-authored-by: Peter Petermann <ppetermann80@googlemail.com> Co-authored-by: Peter Svensson <psvensson@gmail.com> Co-authored-by: Robin Richtsfeld <robin.richtsfeld@gmail.com>

This reverts commit 1fd604d.

initial cleanup of github workflows

99a77c7

github-actions bot added the size/xl label Apr 23, 2023

Pwuts requested a review from ntindle April 23, 2023 21:18

only run pr-label workflow on push to master

58f7798

ntindle previously requested changes Apr 23, 2023

View reviewed changes

move docker ci/release summaries to scripts

435a346

Pwuts force-pushed the github-workflow-cleanup branch from 08bef65 to 435a346 Compare April 23, 2023 21:59

ntindle reviewed Apr 23, 2023

View reviewed changes

.github/workflows/sponsors_readme.yml Show resolved Hide resolved

Pwuts linked an issue Apr 23, 2023 that may be closed by this pull request

Speed op Docker CI #2693

Closed

ntindle reviewed Apr 23, 2023

View reviewed changes

.github/workflows/docker-release.yml Show resolved Hide resolved

add XS label for PR's under 2 lines

bbae196

draft test job for Docker CI

4d0e5ce

Pwuts force-pushed the github-workflow-cleanup branch from d3fc319 to 4d0e5ce Compare April 23, 2023 23:05

Pwuts mentioned this pull request Apr 23, 2023

Improve docker setup & config #1843

Merged

5 tasks

ntindle previously approved these changes Apr 23, 2023

View reviewed changes

Pwuts mentioned this pull request Apr 23, 2023

Fix Docker CI to also test running the image #2670

Closed

Pwuts linked an issue Apr 23, 2023 that may be closed by this pull request

Fix Docker CI to also test running the image #2670

Closed

Pwuts removed a link to an issue Apr 24, 2023

Fix Docker CI to also test running the image #2670

Closed

ntindle reviewed Apr 24, 2023

View reviewed changes

.github/workflows/ci.yml Show resolved Hide resolved

ntindle mentioned this pull request Apr 24, 2023

Added testing timout in gh actions #3087

Closed

richbeales previously approved these changes Apr 24, 2023

View reviewed changes

Merge branch 'master' into github-workflow-cleanup

6bdebad

Pwuts dismissed stale reviews from richbeales and ntindle via 6bdebad April 24, 2023 13:37

Pwuts mentioned this pull request Apr 24, 2023

Matrix all builds and tests across windows, macOS and Linux #3133

Closed

Pwuts force-pushed the github-workflow-cleanup branch 3 times, most recently from 4f8e80c to b7b7914 Compare April 24, 2023 14:55

fix & activate Docker CI test job

2685a22

Pwuts force-pushed the github-workflow-cleanup branch from b7b7914 to 2685a22 Compare April 24, 2023 14:55

Pwuts added 6 commits April 24, 2023 16:58

add debug step to docker CI

0e5e4d1

fix Docker CI test container env

a223c2d

Docker CI build matrix

ba9e506

fixup build summaries

30da9cf

fix pipes in summary

4a13283

optimize Dockerfile for layer caching

b287b30

Pwuts linked an issue Apr 24, 2023 that may be closed by this pull request

Fix Docker CI to also test running the image #2670

Closed

Pwuts added 3 commits April 24, 2023 18:15

more markdown escaping

af0b15d

add gha cache scopes

29b5217

add Docker CI cache clean workflow

b84a574

Pwuts added ci testing labels Apr 24, 2023

p-i- approved these changes Apr 24, 2023

View reviewed changes

Pwuts self-assigned this Apr 24, 2023

p-i- merged commit ffdc652 into master Apr 24, 2023

p-i- deleted the github-workflow-cleanup branch April 24, 2023 17:03

misterak8 added a commit to misterak8/Auto-GPT that referenced this pull request Apr 26, 2023

Revert "Clean up GitHub Workflows (Significant-Gravitas#3059) (#1)"

ffdc9b9

This reverts commit 1fd604d.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Clean up GitHub Workflows #3059

Clean up GitHub Workflows #3059

Pwuts commented Apr 23, 2023 •

edited

Loading

codecov bot commented Apr 23, 2023 •

edited

Loading

ntindle Apr 23, 2023

Pwuts Apr 23, 2023

Clean up GitHub Workflows #3059

Clean up GitHub Workflows #3059

Conversation

Pwuts commented Apr 23, 2023 • edited Loading

Background

Changes

Test Plan

PR Quality Checklist

codecov bot commented Apr 23, 2023 • edited Loading

Codecov Report

ntindle Apr 23, 2023

Choose a reason for hiding this comment

Pwuts Apr 23, 2023

Choose a reason for hiding this comment

Pwuts commented Apr 23, 2023 •

edited

Loading

codecov bot commented Apr 23, 2023 •

edited

Loading