Aws/trashai staging (#41)

* Coco POC #8 #9 (#19) (#26)

* Fixed some package.json issues, updated some entries in .gitignore and .in (for those who would use it)

* saving work

* saving work... having issues with async reading file then loading it in coco

* coco confirmed working

* Added JSON metadata

* Arg! this should work

* Removed Pro port mapping from localstack docker-compose

* Added multiple file upload and cleaned up presentation a bit

Looks like there is a problem with uploading multiple files at once, the
predictions don't always trigger

* Minor updates to MacOS details (#24)

Co-authored-by: Jim <jimewald@gmail.com>

Co-authored-by: Jim Ewald <jim.ewald@gmail.com>
Co-authored-by: Jim <jimewald@gmail.com>

Co-authored-by: Jim Ewald <jim.ewald@gmail.com>
Co-authored-by: Jim <jimewald@gmail.com>

* removed symlink

* Testing coco (#28)

* Fixed some package.json issues, updated some entries in .gitignore and .in (for those who would use it)

* saving work

* saving work... having issues with async reading file then loading it in coco

* coco confirmed working

* Added JSON metadata

* Arg! this should work

* Removed Pro port mapping from localstack docker-compose

* Added multiple file upload and cleaned up presentation a bit

Looks like there is a problem with uploading multiple files at once, the
predictions don't always trigger

* Minor updates to MacOS details (#24)

Co-authored-by: Jim <jimewald@gmail.com>

* removed symlink to web_model

* Coco POC #8 #9 (#19) (#26)

* Fixed some package.json issues, updated some entries in .gitignore and .in (for those who would use it)

* saving work

* saving work... having issues with async reading file then loading it in coco

* coco confirmed working

* Added JSON metadata

* Arg! this should work

* Removed Pro port mapping from localstack docker-compose

* Added multiple file upload and cleaned up presentation a bit

Looks like there is a problem with uploading multiple files at once, the
predictions don't always trigger

* Minor updates to MacOS details (#24)

Co-authored-by: Jim <jimewald@gmail.com>

Co-authored-by: Jim Ewald <jim.ewald@gmail.com>
Co-authored-by: Jim <jimewald@gmail.com>

Co-authored-by: Jim Ewald <jim.ewald@gmail.com>
Co-authored-by: Jim <jimewald@gmail.com>

* removed symlink

Co-authored-by: Jim Ewald <jim.ewald@gmail.com>
Co-authored-by: Jim <jimewald@gmail.com>

* Changed upload behaviour

* Refactored upload dialog / drop area

* added more model samples
* removed dropzone package

* Fixed issue with remove / key and array order

* Removed lab / test for deployed versions of environment

* Added about page

* Forgot to update the metadata text

* Yolov5-Taco web model #7 (#30)

* Testing tf_web model export

* removing pyright

* Working version of taco dataset + initial build instructions

* Created links to code4sac and the github project

Added jupyter notebook files for new training as well as using existing
training data

* Adjusted new training jupyter notebook

* Testing change to model loading in AWS

* Try 2 on loading model

* Try 3 on loading model

* Reverting changes (has to do with basic auth url)

* Removed old react frontend directory

* Removed old react frontend directory (#34)

* Moving domain from codefordev to trashai.org

* Fixes for #36, #35, #32 + download all feature, and backend refactor w/ dedup (#38)

* Replaced Backend with python, fixed infa deploy logic # 35

* backend now tracks images by their sha256 value to avoid duplication
* added new manual action to infra deploy logic # 35
* added log retention logic to backend deploy # 36
* added new fields to deploy_map
    "log_retention_days": 30,
    "dns_domain_map_root": true,
* made github_actions script aware of domain, and "is_root_domain" deploy_map setting

* Addresses # 32, added new features amongst them a download all button

* added new packages to UI
    * "dexie" local storage w/ indexeddb browser store, localstore
      couldn't cut it
    * "file-saver": allows us to do a saveas with the zip file
    * "jszip": zip libarary to address #
* indicators for backend upload success, and if the image has been seen
  before
* now looks good in mobile as well as desktop
* also using indexeddb introduced caching the file objects (for download later) and caching of the tf model

* Added support for python packages

* Adjusted permission to add/remove layers in the prefix namespace

* fixed permissions issue for layers / backend deploy

* Added hash and other metadata to the metadata display and hover over filename

* Fixed metadata download and info button

* bugfix on s3 naming with file extension (#40)

Co-authored-by: Steven Hollingsworth <shollingsworth@barracuda.com>
Co-authored-by: Jim Ewald <jim.ewald@gmail.com>
Co-authored-by: Jim <jimewald@gmail.com>

.github/workflows/infra.yaml

@@ -15,6 +15,17 @@ jobs:
             - name: Git clone the repository
               uses: actions/checkout@v1
 
+            - name: Setup python
+              uses: actions/setup-python@v3
+              with:
+                python-version: '3.8'
+                architecture: 'x64'
+
+            - name: setup boto3
+              run: |
+                pip3 install --upgrade pip
+                pip3 install boto3
+
             - name: register variables
               id: setregion
               run: |

.github/workflows/manual_run.yaml

@@ -19,6 +19,7 @@ on:
           - all_stack
           - backend_stack
           - frontend_stack
+          - deploy_role_stack
           - global_stack
           - region_stack
           - first_run_stack
@@ -36,6 +37,17 @@ jobs:
             - name: Git clone the repository
               uses: actions/checkout@v1
 
+            - name: Setup python
+              uses: actions/setup-python@v3
+              with:
+                python-version: '3.8'
+                architecture: 'x64'
+
+            - name: setup boto3
+              run: |
+                pip3 install --upgrade pip
+                pip3 install boto3
+
             - name: register variables
               id: setregion
               run: |

.gitignore

@@ -16,3 +16,4 @@ data
 **/*.zip
 pyrightconfig.json
 archive/
+tmp/

backend/.gitignore

@@ -1,2 +1,5 @@
 node_modules
 .serverless
+tmp/
+layers/requirements/
+.in

backend/Makefile

@@ -0,0 +1,56 @@
+.DEFAULT_GOAL := help
+help:
+	@echo "Available commands:"
+	@cat Makefile | grep '^\w' | cut -d ':' -f 1 | grep -v '^help$$'
+
+
+
+
+STAGE := $(shell git rev-parse --abbrev-ref HEAD | sed 's/^aws\///')
+REGION := us-west-1
+export STAGE
+export REGION
+
+offline:
+	yarn run local
+
+deploy:
+	serverless deploy \
+		-s "${STAGE}" \
+		-r "${REGION}"
+
+destroy:
+	serverless remove \
+		-s "${STAGE}" \
+		-r "${REGION}"
+
+print:
+	serverless print \
+		-s "${STAGE}" \
+		-r "${REGION}"
+
+list:
+	serverless deploy list functions \
+		-s "${STAGE}" \
+		-r "${REGION}"
+
+logs_web:
+	serverless logs \
+		--function webapp \
+		-t \
+		-s "${STAGE}" \
+		-r "${REGION}"
+#
+# logs_rate:
+#     serverless logs \
+#         --function ratehandler \
+#         -t \
+#         -s "${STAGE}" \
+#         -r "${REGION}"
+#
+# logs_cron:
+#     serverless logs \
+#         --function cronhandler \
+#         -t \
+#         -s "${STAGE}" \
+#         -r "${REGION}"