Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DL models as serverless functions #1767

Merged
merged 133 commits into from
Jul 29, 2020
Merged
Show file tree
Hide file tree
Changes from 131 commits
Commits
Show all changes
133 commits
Select commit Hold shift + click to select a range
583deca
Initial experiments with nuclio
Apr 21, 2020
f9c58e6
Update nuclio prototype
Apr 22, 2020
2920c5b
Improve nuclio prototype for dextr.
Apr 23, 2020
0fa76fb
Merge remote-tracking branch 'origin/develop' into nm/serverless
Apr 24, 2020
2a76d71
Dummy lambda manager
Apr 25, 2020
13978ec
OpenFaaS prototype (dextr.bin and dextr.xml are empty).
Apr 6, 2020
0cd4127
Moved openfaas prototype.
Apr 25, 2020
638ae73
Merge remote-tracking branch 'origin/develop' into nm/serverless
Apr 25, 2020
d089f49
Add comments
Apr 25, 2020
3481721
Add serializers and HLD for lambda_manager
Apr 26, 2020
096eb2d
Merge remote-tracking branch 'origin/develop' into nm/serverless
May 1, 2020
144f7be
Initial version of Mask RCNN (without debugging)
May 6, 2020
e85f4b7
Merge remote-tracking branch 'origin/develop' into nm/serverless
May 6, 2020
e0f3aea
Initial version for faster_rcnn_inception_v2_coco
May 9, 2020
af5dda4
Fix faster_rcnn_inception_v2_coco
May 9, 2020
6d061fc
Implemented mask_rcnn_inception_resnet_v2_atrous_coco
May 10, 2020
8dbfeb9
Implemented yolo detector as a lambda function
May 11, 2020
b9c7a86
Merge remote-tracking branch 'origin/develop' into nm/serverless
May 14, 2020
bb92b4a
Removed dextr app.
May 18, 2020
f9cfdf8
Merge remote-tracking branch 'origin/develop' into nm/serverless
May 18, 2020
fd06913
Added types for each function (detector and interactor)
May 18, 2020
985aa6d
Merge remote-tracking branch 'origin/develop' into nm/serverless
May 19, 2020
d79646a
Initial version of lambda_manager.
May 20, 2020
4ec9da9
Implement a couple of methods for lambda:
May 20, 2020
55aeb80
Merge remote-tracking branch 'origin/develop' into nm/serverless
May 28, 2020
5930d07
Merge remote-tracking branch 'origin/develop' into nm/serverless
May 29, 2020
666e71c
First working version of dextr serverless function
May 31, 2020
c160941
First version of dextr which works in UI.
May 31, 2020
86f9ab2
Modify omz.public.faster_rcnn_inception_v2_coco
Jun 1, 2020
b7c4768
Improve omz.public.mask_rcnn_inception_resnet_v2_atrous_coco
Jun 1, 2020
6b232d6
Improve omz.public.yolo-v3-tf function
Jun 1, 2020
737ccce
Implemented the initial version of requests for lambda manager.
Jun 1, 2020
be16027
Merge remote-tracking branch 'origin/develop' into nm/serverless
Jun 10, 2020
3b44879
First working version of POST /api/v1/lambda/requests
Jun 11, 2020
67c3337
Merge remote-tracking branch 'origin/develop' into nm/serverless
Jun 16, 2020
220d905
Updated specification of function.yaml (added labels and used annotat…
Jun 17, 2020
aea4665
Added health check for containers (nuclio dashboard feature)
Jun 17, 2020
5d1d775
Merge remote-tracking branch 'origin/develop' into nm/serverless
Jun 17, 2020
30d4579
Merge remote-tracking branch 'origin/develop' into nm/serverless
Jun 18, 2020
9490d1a
Read labels spec from function.yaml.
Jun 18, 2020
ed19ccf
Added settings for NUCLIO
Jun 18, 2020
393e631
Merge remote-tracking branch 'origin/develop' into nm/serverless
Jun 18, 2020
b103703
Fixed a couple of typos. Now it works in most cases.
Jun 18, 2020
fc3d6c6
Remove Plugin REST API
Jun 19, 2020
ff6d89e
Merge remote-tracking branch 'origin/develop' into nm/serverless
Jun 19, 2020
7a84e4c
Remove tf_annotation app (it will be replaced by serverless function)
Jun 19, 2020
093e4e7
Remove tf_annotation and cuda components
Jun 19, 2020
31858e9
Cleanup docs and Dockerfile from CUDA component.
Jun 19, 2020
88fed71
Just renamed directories inside serverless
Jun 19, 2020
38fd4e3
Remove redundant files and code
Jun 19, 2020
a396822
Remove redundant files.
Jun 19, 2020
b04407e
Remove outdated files
Jun 19, 2020
f518ba8
Remove outdated code
Jun 19, 2020
d0b5cfb
Delete reid app and add draft of serverless function for reid.
Jun 20, 2020
66711b8
Model list in UI.
Jun 21, 2020
92463af
Fixed the framework name (got it from lambda function).
Jun 21, 2020
54f64e9
Add maxRequestBodySize for functions, remove redundant code from UI f…
Jun 22, 2020
fa89f96
Merge remote-tracking branch 'origin/develop' into nm/serverless
Jun 22, 2020
e136827
Update view of models page.
Jun 22, 2020
61d20ca
Unblock mapping for "primary" models.
Jun 23, 2020
5317e62
Merge remote-tracking branch 'origin/develop' into nm/serverless
Jun 23, 2020
cfaf97f
Implement cleanup flag for lambda/requests and labeling mapping for f…
Jun 23, 2020
461a4de
Implement protection from running multiple jobs for the same task.
Jun 23, 2020
459aeaa
Fix invocation of functions in docker container.
Jun 24, 2020
fd6ba2f
Fix Dockerfile.ci
Jun 25, 2020
e14705e
Remove unused files from lambda_manager
Jun 25, 2020
32d4c59
Fix codacy warnings
Jun 25, 2020
c8b26f9
Fix codacy issues.
Jun 26, 2020
102a4e3
Fix codacy warnings
Jun 26, 2020
3652922
Implement progress and cancel (aka delete) operation.
Jun 26, 2020
3c43b9d
Send annotations in batch.
Jun 26, 2020
793b661
Fix UI. Now it can retrieve information about inference requests in p…
Jun 26, 2020
4528763
Merge remote-tracking branch 'origin/develop' into nm/serverless
Jun 26, 2020
b839fd5
Update CHANGELOG.md
Jun 26, 2020
c6dae22
Update cvat-ui version.
Jun 26, 2020
aa53394
Update nuclio version.
Jun 26, 2020
039b872
Implement serverless/tensorflow/faster_rcnn_inception_v2_coco
Jun 28, 2020
b3d9cac
Merge remote-tracking branch 'origin/develop' into nm/serverless
Jun 29, 2020
e41f65c
Add information how to install nuclio platform and run serverless fun…
Jun 30, 2020
6a23ccf
Add installation instructions for serverless functions.
Jun 30, 2020
573e3d1
Merge remote-tracking branch 'origin/develop' into nm/serverless
Jun 30, 2020
39fe337
Merge remote-tracking branch 'origin/develop' into nm/serverless
Jul 1, 2020
f7e12b0
Merge remote-tracking branch 'origin/develop' into nm/serverless
Jul 1, 2020
8c47af7
Update OpenVINO files which are responsible for loading network
Jul 3, 2020
d982a18
relocated functions
Jul 3, 2020
7722ffd
Update dextr function.
Jul 3, 2020
2ab1030
Update faster_rcnn function from omz
Jul 3, 2020
3622e81
Fix OpenVINO Mask-RCNN
Jul 4, 2020
89165cd
Fix YOLO v3 serverless function.
Jul 4, 2020
6906b63
Dummy serverless functions for a couple of more OpenVINO models.
Jul 5, 2020
38004a1
Protected lambda manager views by correct permissions.
Jul 6, 2020
ecb8a72
Fix name of Faster RCNN from Tensorflow.
Jul 6, 2020
9b0faf6
Implement Mask RCNN via Tensorflow serverless function.
Jul 8, 2020
b4bc20f
Minor client changes (#1847)
bsekachev Jul 8, 2020
11dc24b
Merge remote-tracking branch 'origin/develop' into nm/serverless
Jul 9, 2020
a072105
Merge remote-tracking branch 'origin/develop' into nm/serverless
Jul 13, 2020
aba0f84
Merge branch 'nm/serverless' of https://github.com/opencv/cvat into n…
Jul 13, 2020
bd70a0b
Add reid serverless function (no support in lambda manager).
Jul 14, 2020
e95660c
Fix contribution guide.
Jul 14, 2020
a8cca9c
Fix person-reidentification-retail-300 and implement text-detection-0004
Jul 14, 2020
5fdedd3
Add semantic-segmentation-adas-0001
Jul 14, 2020
e2bd562
Merge remote-tracking branch 'origin/develop' into nm/serverless
Jul 14, 2020
34f48ce
Merge remote-tracking branch 'origin/develop' into nm/serverless
Jul 15, 2020
e7501b1
Moving model management to cvat-core (#1905)
bsekachev Jul 15, 2020
7e78ccb
Merge remote-tracking branch 'origin/develop' into nm/serverless
Jul 16, 2020
aee568b
Remove duplicated files for OpenVINO serverless functions.
Jul 17, 2020
889365f
Updated CHANGELOG.md
Jul 17, 2020
e26f47e
Remove outdated code.
Jul 17, 2020
90f8e13
Running dextr via lambda manager (#1912)
bsekachev Jul 17, 2020
fcb5790
Merge remote-tracking branch 'origin/develop' into nm/serverless
Jul 19, 2020
f36df30
Deleted outdated migration.
Jul 19, 2020
98e5398
Add name for DEXTR function.
Jul 20, 2020
6461b10
Merge remote-tracking branch 'origin/develop' into nm/serverless
Jul 20, 2020
52b1f4d
Fix restart policy for serverless functions.
Jul 20, 2020
44b5d01
Fix openvino serverless functions for images with alpha channel
Jul 20, 2020
1525ce2
Add more tensorflow serverless functions into deploy.sh
Jul 21, 2020
be8d06c
Use ID instead of name for DEXTR (#1926)
bsekachev Jul 22, 2020
0684cae
Update DEXTR function
Jul 24, 2020
e5b8f2a
Merge branch 'nm/serverless' of https://github.com/opencv/cvat into n…
Jul 24, 2020
5352a94
Merge remote-tracking branch 'origin/develop' into nm/serverless
Jul 24, 2020
a1096a1
Merge remote-tracking branch 'origin/develop' into nm/serverless
Jul 24, 2020
2ff7395
Added source "auto" inside lambda manager for automatic annotation.
Jul 24, 2020
2bad1de
Customize payload (depends on type of lambda function).
Jul 24, 2020
8f60d54
First working version of REID (Server only).
Jul 25, 2020
05762ad
Fix codacy warnings
Jul 25, 2020
0c6e033
Avoid exception during migration (workaround)
Jul 25, 2020
1973a07
Add siammask serverless function (it doesn't work, need to serialize …
Jul 27, 2020
d32201e
Run ReID from UI (#1949)
bsekachev Jul 28, 2020
44df570
Merge branch 'develop' into nm/serverless
nmanovic Jul 28, 2020
cc081bd
Merge remote-tracking branch 'origin/develop' into nm/serverless
Jul 29, 2020
ced2746
Removed reid route in installation.md
Jul 29, 2020
4210645
Fix a command to get lena image in CONTRIBUTION guide.
Jul 29, 2020
6dcb354
Fix typo and crash in case a polygon is a line.
Jul 29, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .gitattributes
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@ LICENSE text
*.conf text
*.mimetypes text
*.sh text eol=lf
components/openvino/eula.cfg text eol=lf

*.avi binary
*.bmp binary
Expand Down
2 changes: 1 addition & 1 deletion .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
/datumaro/ @zhiltsov-max
/cvat/apps/dataset_manager/ @zhiltsov-max

# Advanced components (e.g. OpenVINO)
# Advanced components (e.g. analytics)
/components/ @azhavoro

# Infrastructure
Expand Down
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@
/.env
/keys
/logs
/components/openvino/*.tgz
/profiles
/ssh/*
!/ssh/README.md
Expand Down
6 changes: 3 additions & 3 deletions .pylintrc
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ persistent=yes

# List of plugins (as comma separated values of python modules names) to load,
# usually to register additional checkers.
load-plugins=
load-plugins=pylint_django

# Use multiple processes to speed up Pylint.
jobs=1
Expand Down Expand Up @@ -66,8 +66,8 @@ enable= E0001,E0100,E0101,E0102,E0103,E0104,E0105,E0106,E0107,E0110,
W0122,W0124,W0150,W0199,W0221,W0222,W0233,W0404,W0410,W0601,
W0602,W0604,W0611,W0612,W0622,W0623,W0702,W0705,W0711,W1300,
W1301,W1302,W1303,,W1305,W1306,W1307
R0102,R0201,R0202,R0203
R0102,R0202,R0203


# Disable the message, report, category or checker with the given id(s). You
# can either give multiple identifiers separated by comma (,) or put this
Expand Down
14 changes: 7 additions & 7 deletions .vscode/launch.json
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
"request": "launch",
"stopOnEntry": false,
"justMyCode": false,
"pythonPath": "${config:python.pythonPath}",
"pythonPath": "${command:python.interpreterPath}",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if this is the right change for this PR.

"program": "${workspaceRoot}/manage.py",
"args": [
"runserver",
Expand Down Expand Up @@ -58,7 +58,7 @@
"request": "launch",
"stopOnEntry": false,
"justMyCode": false,
"pythonPath": "${config:python.pythonPath}",
"pythonPath": "${command:python.interpreterPath}",
"program": "${workspaceRoot}/manage.py",
"args": [
"rqworker",
Expand All @@ -77,7 +77,7 @@
"request": "launch",
"stopOnEntry": false,
"justMyCode": false,
"pythonPath": "${config:python.pythonPath}",
"pythonPath": "${command:python.interpreterPath}",
"program": "${workspaceRoot}/manage.py",
"args": [
"rqscheduler",
Expand All @@ -93,7 +93,7 @@
"request": "launch",
"justMyCode": false,
"stopOnEntry": false,
"pythonPath": "${config:python.pythonPath}",
"pythonPath":"${command:python.interpreterPath}",
"program": "${workspaceRoot}/manage.py",
"args": [
"rqworker",
Expand All @@ -112,7 +112,7 @@
"request": "launch",
"justMyCode": false,
"stopOnEntry": false,
"pythonPath": "${config:python.pythonPath}",
"pythonPath": "${command:python.interpreterPath}",
"program": "${workspaceRoot}/manage.py",
"args": [
"update_git_states"
Expand All @@ -128,7 +128,7 @@
"request": "launch",
"justMyCode": false,
"stopOnEntry": false,
"pythonPath": "${config:python.pythonPath}",
"pythonPath": "${command:python.interpreterPath}",
"program": "${workspaceRoot}/manage.py",
"args": [
"migrate"
Expand All @@ -144,7 +144,7 @@
"request": "launch",
"justMyCode": false,
"stopOnEntry": false,
"pythonPath": "${config:python.pythonPath}",
"pythonPath": "${command:python.interpreterPath}",
"program": "${workspaceRoot}/manage.py",
"args": [
"test",
Expand Down
10 changes: 3 additions & 7 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,20 +6,22 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [1.1.0-beta] - Unreleased
### Added
- DL models as serverless functions (<https://github.com/opencv/cvat/pull/1767>)
- Source type support for tags, shapes and tracks (<https://github.com/opencv/cvat/pull/1192>)
- Source type support for CVAT Dumper/Loader (<https://github.com/opencv/cvat/pull/1192>)
- Intelligent polygon editing (<https://github.com/opencv/cvat/pull/1921>)
- python cli over https (<https://github.com/opencv/cvat/pull/1942>)

### Changed
- Smaller object details (<https://github.com/opencv/cvat/pull/1877>)
- It is impossible to submit a DL model in OpenVINO format using UI. Now you can deploy new models on the server using serverless functions (<https://github.com/opencv/cvat/pull/1767>)
- Files and folders under share path are now alphabetically sorted

### Deprecated
-

### Removed
-
- Removed OpenVINO and CUDA components because they are not necessary anymore (<https://github.com/opencv/cvat/pull/1767>)

### Fixed
- Some objects aren't shown on canvas sometimes. For example after propagation on of objects is invisible (<https://github.com/opencv/cvat/pull/1834>)
Expand Down Expand Up @@ -81,12 +83,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Colorized object items in the side panel (<https://github.com/opencv/cvat/pull/1753>)
- [Datumaro] Annotation-less files are not generated anymore in COCO format, unless tasks explicitly requested (<https://github.com/opencv/cvat/pull/1799>)

### Deprecated
-

### Removed
-

### Fixed
- Problem with exported frame stepped image task (<https://github.com/opencv/cvat/issues/1613>)
- Fixed dataset filter item representation for imageless dataset items (<https://github.com/opencv/cvat/pull/1593>)
Expand Down
Loading