Skip to content

Conversation

@mauwii
Copy link
Contributor

@mauwii mauwii commented Oct 16, 2022

in favor of #1118 🙈

mauwii and others added 2 commits October 16, 2022 19:19
* fix mkdocs deployment

* update path to python bin

* add trigger for current branch

* change path to python_bin for mac as well

* try to use setup-python@v4 instead of setting env

* remove setup conda action

* try to use $CONDA

* remove overseen action

* change branch from master to main

* sort out if then else for faster syntax

* remove more if functions

* add updates to create-caches as well

* eliminate the rest of if functions

* try to unpin pytorch and torchvision

* restore pinned versions

* try switching from set-output to use env

* update test-invoke-conda as well

* fix env var creation

* quote variable

* add second equal to compare

* try another way to use outputs

* fix outputs

* pip install for mac before creating conda env

* fix output variable

* fix python bin path

* remove pip install for before creating conda env

* unpin streamlit version in conda mac env

* try to make git-workflows better readable

* remove 4gotten trigger

* Update-gh-actions (#6)

* fix mkdocs deployment

* update path to python bin

* add trigger for current branch

* change path to python_bin for mac as well

* try to use setup-python@v4 instead of setting env

* remove setup conda action

* try to use $CONDA

* remove overseen action

* change branch from master to main

* sort out if then else for faster syntax

* remove more if functions

* add updates to create-caches as well

* eliminate the rest of if functions

* try to unpin pytorch and torchvision

* restore pinned versions

* try switching from set-output to use env

* update test-invoke-conda as well

* fix env var creation

* quote variable

* add second equal to compare

* try another way to use outputs

* fix outputs

* pip install for mac before creating conda env

* fix output variable

* fix python bin path

* remove pip install for before creating conda env

* unpin streamlit version in conda mac env

* try to make git-workflows better readable

* use macos-latest

* try to update conda before creating mac env

* better conda update trial

* re-pin streamlit version

* re-added trigger to run workflow in current branch

* try to find out if conda mac env could be updated

* install cmake, protobuf and rust b4 conda

* add yes to conda update

* lets try anaconda3-2022.05

* try environment.yml for mac as well

* reenable conda mac env, add pip install
also fix gitignore by changing from dream to invoke

* remove
- unecesary virtualenv creation
- conda update

change != macos back to == linux

* remove cmake from brew install since pre-installed

* disable opencv-python pip requirement

* fixed commands to find latest package versions

* update requirements for mac env

* back to the roots - only install conda env
depending on runner_os with or without extra env variables

* check out macOS in azure-pipelines
since becoming kind of tired of the GitHub Runner which is broken as ...

* let's try to setup python and update conda env

* initialize conda before using it

* add trigger in azure-pipelines.yml

* And another go for update first ....

* update azure-pipelines.yml
- add caching
- add checkpoint download
- add paths to trigger
and more

* unquote checkpoint-url

* fix chekpoint-url variable

* mkdir before downloading model

* set pr trigger to main, rename anaconda cache

* unique cacheHitVariables

* try to use macos-latest instead of macos-12

* update test-invoke-conda.yml:
- remove unecesarry echo step
- use s-weigand/setup-conda@v1
- remove conda update from install deps step since updated with action

* update test-invoke-conda.yml:
- rename conda env cache from ldm to invokeai
- reorder steps:
  1. checkout sources
  2. setup python
  3. setup conda
  4. keep order after set platform variables

* change macos back to 12 since also fails with 11

* update condition in run the tests
make difference between main or not main

* fix path to cache invokeai conda env

* fix invokeai conda env cache path

* update mkdocs-flow.yml

* change conda-channel priority

* update create-caches

* update conda env also when cache was used

* os dependend conda env cache path

* use existing CONDA env pointing to conda root

* create CONDA_ROOT output from $CONDA

* use output variable to define test prompts

* use setup-python v4, get rid of PYTHON_BIN env

* add runner.os to result artifacts name

* update test-invoke-conda.yml:
- reuse macos-latest
- disable setup python 3.9
- setup conda with default python version
- create or update conda environment depending on cache success
- remove name parameter from conda update since name is set in env yml

* improve mkdocs-flow.yml

* disable cache-hugginface-torch
since preload_models.py downloads to more than one location

* update mkdocs-flow.yml with new name

* rename mkdocs action to mkdocs-material

* try to ignore error when creating conda env
maybe it would still be usable, lets see ;P

* remove bloat

* update environment-mac.yml
to match dependencies of invoke-ai/InvokeAI's main branch

* disable conda update, tweak prompt condition

* try to set some env vars for macOS to fix conda

* stop ignoring error, use env instead of outputs

* tweak `[[` connditions

* update python and pip dependencie
makes a difference of 1 sec per itteration compared to 3.9!!!
also I see no reason why using a old pip version would be beneficial

* remove unecesarry env for macOS
everything was pre-tested on my MacBook Air 2020 with M1

* update conda env in setup step

* activate conda env after installation

* update test-invoke-conda.yml
- set conda env dependent on matrix.os
- set CONDA_ENV_NAME to prevent breaking action when renaming conda env
- fix conda env activation

* fix activate conda env

* set bash -l as default shell

* use action to activate conda env

* add conda env file to env activation

* try to replace s-weigeand with conda-incubator

* remove azure-pipelines.yml
funniest part is that the macos runner is the same as the one on github!

* include environment-file in matrix
- also disable auto-activate-base and auto-update-conda
- include macos-latest and macos-12 for debugging purpose
- set miniforge-version in matrix

* fix miniforge-variant, set fail-fast to false

* add step to setup miniconda
- make default shell a matrix variable
- remove bloat

* use a mac env yml without pinned versions

* unpin nomkl, pytorch and torchvision
also removed opencv-pyhton

* cache conda pkgs dir instead of conda env

* use python 3.10, exclude macos-12 from cache

* fix expression

* prepare for PR

* fix doubled id

* reuse pinned versions in mac conda env
- updated python pip version
- unpined pytorch and torchvision
- removed opencv-python
- updated versions to most recent (tested locally)

* fix classical copy/paste error

* remove unused env from shell-block comment

* fix hashFiles function to determine restore-keys

* reenable caching `~.cache`, update create-caches

* unpin all versions in mac conda env file
this was the only way I got it working in the action, also works locally
tested on MacBook Air 2020 M1
remove environment-mac-unpinned.yml

* prepare merge by removing this branch from trigger

* include pull_request trigger for main and dev

* remove pull_request trigger
@mauwii
Copy link
Contributor Author

mauwii commented Oct 16, 2022

@lstein @tildebyte @Any-Winter-4079
Here you go ;P

Copy link
Contributor

@tildebyte tildebyte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAICT, 'create-caches.yml' has a couple of issues:

  • "Use Cached Stable Diffusion v1.4 Model" is duplicated
  • "Use cached Conda Environment" and "Use cached Conda Packages" look to be almost exactly the same

Also, please unpin 'environment.yml'

mauwii and others added 2 commits October 16, 2022 13:56
* fix mkdocs deployment

* update path to python bin

* add trigger for current branch

* change path to python_bin for mac as well

* try to use setup-python@v4 instead of setting env

* remove setup conda action

* try to use $CONDA

* remove overseen action

* change branch from master to main

* sort out if then else for faster syntax

* remove more if functions

* add updates to create-caches as well

* eliminate the rest of if functions

* try to unpin pytorch and torchvision

* restore pinned versions

* try switching from set-output to use env

* update test-invoke-conda as well

* fix env var creation

* quote variable

* add second equal to compare

* try another way to use outputs

* fix outputs

* pip install for mac before creating conda env

* fix output variable

* fix python bin path

* remove pip install for before creating conda env

* unpin streamlit version in conda mac env

* try to make git-workflows better readable

* remove 4gotten trigger

* Update-gh-actions (#6)

* fix mkdocs deployment

* update path to python bin

* add trigger for current branch

* change path to python_bin for mac as well

* try to use setup-python@v4 instead of setting env

* remove setup conda action

* try to use $CONDA

* remove overseen action

* change branch from master to main

* sort out if then else for faster syntax

* remove more if functions

* add updates to create-caches as well

* eliminate the rest of if functions

* try to unpin pytorch and torchvision

* restore pinned versions

* try switching from set-output to use env

* update test-invoke-conda as well

* fix env var creation

* quote variable

* add second equal to compare

* try another way to use outputs

* fix outputs

* pip install for mac before creating conda env

* fix output variable

* fix python bin path

* remove pip install for before creating conda env

* unpin streamlit version in conda mac env

* try to make git-workflows better readable

* use macos-latest

* try to update conda before creating mac env

* better conda update trial

* re-pin streamlit version

* re-added trigger to run workflow in current branch

* try to find out if conda mac env could be updated

* install cmake, protobuf and rust b4 conda

* add yes to conda update

* lets try anaconda3-2022.05

* try environment.yml for mac as well

* reenable conda mac env, add pip install
also fix gitignore by changing from dream to invoke

* remove
- unecesary virtualenv creation
- conda update

change != macos back to == linux

* remove cmake from brew install since pre-installed

* disable opencv-python pip requirement

* fixed commands to find latest package versions

* update requirements for mac env

* back to the roots - only install conda env
depending on runner_os with or without extra env variables

* check out macOS in azure-pipelines
since becoming kind of tired of the GitHub Runner which is broken as ...

* let's try to setup python and update conda env

* initialize conda before using it

* add trigger in azure-pipelines.yml

* And another go for update first ....

* update azure-pipelines.yml
- add caching
- add checkpoint download
- add paths to trigger
and more

* unquote checkpoint-url

* fix chekpoint-url variable

* mkdir before downloading model

* set pr trigger to main, rename anaconda cache

* unique cacheHitVariables

* try to use macos-latest instead of macos-12

* update test-invoke-conda.yml:
- remove unecesarry echo step
- use s-weigand/setup-conda@v1
- remove conda update from install deps step since updated with action

* update test-invoke-conda.yml:
- rename conda env cache from ldm to invokeai
- reorder steps:
  1. checkout sources
  2. setup python
  3. setup conda
  4. keep order after set platform variables

* change macos back to 12 since also fails with 11

* update condition in run the tests
make difference between main or not main

* fix path to cache invokeai conda env

* fix invokeai conda env cache path

* update mkdocs-flow.yml

* change conda-channel priority

* update create-caches

* update conda env also when cache was used

* os dependend conda env cache path

* use existing CONDA env pointing to conda root

* create CONDA_ROOT output from $CONDA

* use output variable to define test prompts

* use setup-python v4, get rid of PYTHON_BIN env

* add runner.os to result artifacts name

* update test-invoke-conda.yml:
- reuse macos-latest
- disable setup python 3.9
- setup conda with default python version
- create or update conda environment depending on cache success
- remove name parameter from conda update since name is set in env yml

* improve mkdocs-flow.yml

* disable cache-hugginface-torch
since preload_models.py downloads to more than one location

* update mkdocs-flow.yml with new name

* rename mkdocs action to mkdocs-material

* try to ignore error when creating conda env
maybe it would still be usable, lets see ;P

* remove bloat

* update environment-mac.yml
to match dependencies of invoke-ai/InvokeAI's main branch

* disable conda update, tweak prompt condition

* try to set some env vars for macOS to fix conda

* stop ignoring error, use env instead of outputs

* tweak `[[` connditions

* update python and pip dependencie
makes a difference of 1 sec per itteration compared to 3.9!!!
also I see no reason why using a old pip version would be beneficial

* remove unecesarry env for macOS
everything was pre-tested on my MacBook Air 2020 with M1

* update conda env in setup step

* activate conda env after installation

* update test-invoke-conda.yml
- set conda env dependent on matrix.os
- set CONDA_ENV_NAME to prevent breaking action when renaming conda env
- fix conda env activation

* fix activate conda env

* set bash -l as default shell

* use action to activate conda env

* add conda env file to env activation

* try to replace s-weigeand with conda-incubator

* remove azure-pipelines.yml
funniest part is that the macos runner is the same as the one on github!

* include environment-file in matrix
- also disable auto-activate-base and auto-update-conda
- include macos-latest and macos-12 for debugging purpose
- set miniforge-version in matrix

* fix miniforge-variant, set fail-fast to false

* add step to setup miniconda
- make default shell a matrix variable
- remove bloat

* use a mac env yml without pinned versions

* unpin nomkl, pytorch and torchvision
also removed opencv-pyhton

* cache conda pkgs dir instead of conda env

* use python 3.10, exclude macos-12 from cache

* fix expression

* prepare for PR

* fix doubled id

* reuse pinned versions in mac conda env
- updated python pip version
- unpined pytorch and torchvision
- removed opencv-python
- updated versions to most recent (tested locally)

* fix classical copy/paste error

* remove unused env from shell-block comment

* fix hashFiles function to determine restore-keys

* reenable caching `~.cache`, update create-caches

* unpin all versions in mac conda env file
this was the only way I got it working in the action, also works locally
tested on MacBook Air 2020 M1
remove environment-mac-unpinned.yml

* prepare merge by removing this branch from trigger

* include pull_request trigger for main and dev

* remove pull_request trigger
@tildebyte tildebyte added the CI-CD Continuous integration / Continuous delivery label Oct 16, 2022
@mauwii
Copy link
Contributor Author

mauwii commented Oct 16, 2022

AFAICT, 'create-caches.yml' has a couple of issues:

  • "Use Cached Stable Diffusion v1.4 Model" is duplicated
  • "Use cached Conda Environment" and "Use cached Conda Packages" look to be almost exactly the same

Also, please unpin 'environment.yml'

May I ask what's the purpose behind "create-caches.yml" at all? maybe we could even delete it completeley, going to unpin versions in environment.yml then, but since it had no problems (even works on Linux and Windows) and version pinning is a good thing in the end (when it is working for everybody ofc), I would have left them pinned until further solution is worked out.

@mauwii
Copy link
Contributor Author

mauwii commented Oct 16, 2022

AFAICT, 'create-caches.yml' has a couple of issues:

  • "Use Cached Stable Diffusion v1.4 Model" is duplicated
  • "Use cached Conda Environment" and "Use cached Conda Packages" look to be almost exactly the same

Also, please unpin 'environment.yml'

Conda is actually right, it is caching Environment and Packages

  • Use cached Conda Environment
  • Use cached Conda Packages

Environment caching is not best practice, but since I thought it would run more than it will do now, it made totally sence (since conda pkgs/environment won't change often, but Code would).

@mauwii
Copy link
Contributor Author

mauwii commented Oct 16, 2022

@tildebyte to come back to the "bare minimal test", imho the test which is running is pretty huge:

all but refs/heads/main:

  • banana sushi -Ak_lms -S42
    this could be broken town to something with only 10 iterations when testing a PR

refs/heads/main:

  • banana sushi -Ak_lms -S42
  • banana sushi -Addim -S42
  • banana sushi -Ak_lms -W640 -H480 -S42
  • banana sushi -Ak_lms -S42 -G1 -U 2 0.5
  • banana sushi -Ak_lms -S42 -v0.2 -n3
  • banana sushi -Ak_lms -S42 -V1349749425:0.1,4145759947:0.1
  • snake -I outputs/preflight/000006.4145759947.png -S42
  • snake -I outputs/preflight/000006.4145759947.png -S42 -W640 -H640 --fit
  • strawberry sushi -I./image-and-mask.png -S42 -f0.9 -s100 -C15

this does a lot (but is fine to make sure it is still working ofc, but I also agree that for validating a PR this is pure overkill)

This can be changed by editing the two files in tests/*prompts.txt, also can be advanced ofc

@mauwii mauwii requested review from tildebyte and removed request for Any-Winter-4079 and lstein October 16, 2022 19:21
@mauwii
Copy link
Contributor Author

mauwii commented Oct 16, 2022

What - how can I remove a requested review xD Sorry, did only pres the "Re-require review" button to point out that the asked changes are implemented 🙆🏻‍♂️

@mauwii
Copy link
Contributor Author

mauwii commented Oct 16, 2022

this would be banana sushi -Ak_lms -S42 -s10:

image

imho this is far enough to do a quick PR validation while taking only 20% of the time 🤓

@mauwii
Copy link
Contributor Author

mauwii commented Oct 16, 2022

Also, please unpin 'environment.yml'

needed to revert this since unpinned environment yml failed:

Installing pip dependencies: ...working... ::warning::Pip subprocess error:%0AERROR: Could not detect requirement name for 'git+https://github.com/openai/CLIP.git@main#egg', please specify one with #egg=your_package_name%0A%0A
Pip subprocess error:
ERROR: Could not detect requirement name for 'git+https://github.com/openai/CLIP.git@main#egg', please specify one with #egg=your_package_name

Ran pip subprocess with arguments:
['/usr/share/miniconda3/envs/invokeai/bin/python', '-m', 'pip', 'install', '-U', '-r', '/home/runner/work/InvokeAI/InvokeAI/condaenv.6u8ex5u2.requirements.txt']
Pip subprocess output:

failed
Warning: 
CondaEnvException: Pip failed

also want to show how much faster it will run now with cached pkgs
@mauwii
Copy link
Contributor Author

mauwii commented Oct 16, 2022

So atm we have:

  • 35 minutes without cache (like f.E. a changed environment.yml)
  • 20 minutes with cache

which imho should be totally worth the time vs merging something which then breaks buildability, reverting this, ....

@tildebyte @lstein @Any-Winter-4079 What's your opinion? (not to mention that open source projects get the runtime 4 free 🤑)

And to have done something "useful" to get a rerun with caches I just included a step to list current conda packages (since a pip freeze > ...txt gives me something not very helpful (less pkg-versions than installed locations...)

@mauwii
Copy link
Contributor Author

mauwii commented Oct 17, 2022

should be ready to merge @tildebyte @lstein

mauwii and others added 23 commits October 17, 2022 22:27
* fix mkdocs deployment

* update path to python bin

* add trigger for current branch

* change path to python_bin for mac as well

* try to use setup-python@v4 instead of setting env

* remove setup conda action

* try to use $CONDA

* remove overseen action

* change branch from master to main

* sort out if then else for faster syntax

* remove more if functions

* add updates to create-caches as well

* eliminate the rest of if functions

* try to unpin pytorch and torchvision

* restore pinned versions

* try switching from set-output to use env

* update test-invoke-conda as well

* fix env var creation

* quote variable

* add second equal to compare

* try another way to use outputs

* fix outputs

* pip install for mac before creating conda env

* fix output variable

* fix python bin path

* remove pip install for before creating conda env

* unpin streamlit version in conda mac env

* try to make git-workflows better readable

* remove 4gotten trigger

* Update-gh-actions (#6)

* fix mkdocs deployment

* update path to python bin

* add trigger for current branch

* change path to python_bin for mac as well

* try to use setup-python@v4 instead of setting env

* remove setup conda action

* try to use $CONDA

* remove overseen action

* change branch from master to main

* sort out if then else for faster syntax

* remove more if functions

* add updates to create-caches as well

* eliminate the rest of if functions

* try to unpin pytorch and torchvision

* restore pinned versions

* try switching from set-output to use env

* update test-invoke-conda as well

* fix env var creation

* quote variable

* add second equal to compare

* try another way to use outputs

* fix outputs

* pip install for mac before creating conda env

* fix output variable

* fix python bin path

* remove pip install for before creating conda env

* unpin streamlit version in conda mac env

* try to make git-workflows better readable

* use macos-latest

* try to update conda before creating mac env

* better conda update trial

* re-pin streamlit version

* re-added trigger to run workflow in current branch

* try to find out if conda mac env could be updated

* install cmake, protobuf and rust b4 conda

* add yes to conda update

* lets try anaconda3-2022.05

* try environment.yml for mac as well

* reenable conda mac env, add pip install
also fix gitignore by changing from dream to invoke

* remove
- unecesary virtualenv creation
- conda update

change != macos back to == linux

* remove cmake from brew install since pre-installed

* disable opencv-python pip requirement

* fixed commands to find latest package versions

* update requirements for mac env

* back to the roots - only install conda env
depending on runner_os with or without extra env variables

* check out macOS in azure-pipelines
since becoming kind of tired of the GitHub Runner which is broken as ...

* let's try to setup python and update conda env

* initialize conda before using it

* add trigger in azure-pipelines.yml

* And another go for update first ....

* update azure-pipelines.yml
- add caching
- add checkpoint download
- add paths to trigger
and more

* unquote checkpoint-url

* fix chekpoint-url variable

* mkdir before downloading model

* set pr trigger to main, rename anaconda cache

* unique cacheHitVariables

* try to use macos-latest instead of macos-12

* update test-invoke-conda.yml:
- remove unecesarry echo step
- use s-weigand/setup-conda@v1
- remove conda update from install deps step since updated with action

* update test-invoke-conda.yml:
- rename conda env cache from ldm to invokeai
- reorder steps:
  1. checkout sources
  2. setup python
  3. setup conda
  4. keep order after set platform variables

* change macos back to 12 since also fails with 11

* update condition in run the tests
make difference between main or not main

* fix path to cache invokeai conda env

* fix invokeai conda env cache path

* update mkdocs-flow.yml

* change conda-channel priority

* update create-caches

* update conda env also when cache was used

* os dependend conda env cache path

* use existing CONDA env pointing to conda root

* create CONDA_ROOT output from $CONDA

* use output variable to define test prompts

* use setup-python v4, get rid of PYTHON_BIN env

* add runner.os to result artifacts name

* update test-invoke-conda.yml:
- reuse macos-latest
- disable setup python 3.9
- setup conda with default python version
- create or update conda environment depending on cache success
- remove name parameter from conda update since name is set in env yml

* improve mkdocs-flow.yml

* disable cache-hugginface-torch
since preload_models.py downloads to more than one location

* update mkdocs-flow.yml with new name

* rename mkdocs action to mkdocs-material

* try to ignore error when creating conda env
maybe it would still be usable, lets see ;P

* remove bloat

* update environment-mac.yml
to match dependencies of invoke-ai/InvokeAI's main branch

* disable conda update, tweak prompt condition

* try to set some env vars for macOS to fix conda

* stop ignoring error, use env instead of outputs

* tweak `[[` connditions

* update python and pip dependencie
makes a difference of 1 sec per itteration compared to 3.9!!!
also I see no reason why using a old pip version would be beneficial

* remove unecesarry env for macOS
everything was pre-tested on my MacBook Air 2020 with M1

* update conda env in setup step

* activate conda env after installation

* update test-invoke-conda.yml
- set conda env dependent on matrix.os
- set CONDA_ENV_NAME to prevent breaking action when renaming conda env
- fix conda env activation

* fix activate conda env

* set bash -l as default shell

* use action to activate conda env

* add conda env file to env activation

* try to replace s-weigeand with conda-incubator

* remove azure-pipelines.yml
funniest part is that the macos runner is the same as the one on github!

* include environment-file in matrix
- also disable auto-activate-base and auto-update-conda
- include macos-latest and macos-12 for debugging purpose
- set miniforge-version in matrix

* fix miniforge-variant, set fail-fast to false

* add step to setup miniconda
- make default shell a matrix variable
- remove bloat

* use a mac env yml without pinned versions

* unpin nomkl, pytorch and torchvision
also removed opencv-pyhton

* cache conda pkgs dir instead of conda env

* use python 3.10, exclude macos-12 from cache

* fix expression

* prepare for PR

* fix doubled id

* reuse pinned versions in mac conda env
- updated python pip version
- unpined pytorch and torchvision
- removed opencv-python
- updated versions to most recent (tested locally)

* fix classical copy/paste error

* remove unused env from shell-block comment

* fix hashFiles function to determine restore-keys

* reenable caching `~.cache`, update create-caches

* unpin all versions in mac conda env file
this was the only way I got it working in the action, also works locally
tested on MacBook Air 2020 M1
remove environment-mac-unpinned.yml

* prepare merge by removing this branch from trigger

* include pull_request trigger for main and dev

* remove pull_request trigger
@tildebyte this would not have been pointed out without PR-Validation
also want to show how much faster it will run now with cached pkgs
also switch to python between 3.9-3.10 and use conda-forge again
also change content of hashFile-function
remove restore-keys, make keys uniuqe
@mauwii
Copy link
Contributor Author

mauwii commented Oct 18, 2022

I hope I did the rebase correct now 🙈

@lstein lstein merged commit febf86d into invoke-ai:main Oct 18, 2022
@mauwii mauwii deleted the fix-gh-actions branch October 18, 2022 12:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI-CD Continuous integration / Continuous delivery

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants