CVAT 3D - Milestone-4 #2891

manasars · 2021-03-01T17:06:54Z

CVAT-3D-Milestone4 : Implement initial cuboid placement in 3D View and selected cuboid adjustment in Top, Side and Front views.

Changes include:
Implemented initial cuboid placement in 3D View.
Implemented display of top, front, and side view of the active cuboid.
Implemented adjusting the borders of views to allow changing the width and height of a specific view.
Implemented centre axis for PCD rotation.
Fixed the UTF-8 encoding exception which occurred while creating zip file in MACOS.

Test Cases:
Manual Unit testing done locally.
Existing test cases work as expected.
System Test cases will be shared.

[x ] I submit my changes into the develop branch

[ x] I submit my code changes under the same MIT License that covers the project.

…o cvat-3D-M4

coveralls · 2021-03-01T18:09:13Z

Coverage increased (+0.04%) to 75.257% when pulling e53501f on manasars:cvat-3D-M4 into 4f7b1f9 on openvinotoolkit:develop.

cvat-ui/src/components/annotation-page/standard3D-workspace/styles.scss

cvat-ui/src/components/annotation-page/canvas/canvas-wrapper3D.tsx

cvat-canvas3d/src/typescript/canvas3dView.ts

cvat-ui/package.json

bsekachev · 2021-03-12T08:49:22Z

Hi guys, here are my comments about the patch. I will check the code additionally.

N shortcut does not work to draw a shape
Shift+N shortcut does not work to redraw a shape
I cannot up/down cuboids.

Weren't these features implemented yet?

As I understand we decided that cuboid track is not applicable for 3D, right? Why do we still have options between a shape and a track when create a cuboid? Also we have options (from rectangle, from 4 points) which are also inapplicable here I believe.
Looks like keys: U I O J K L do not work.
Text is not fitted for side views when they are thin (screenshot below)
I noticed strange behavior when cuboid and view are moved during I actually do not move the cursor (attached gif below, second half)
When I start drawing performance is awful (a lot of freezes, according to chrome profiler each mouse movement call 250ms of computations). I have Core i7 8th generation CPU. A screenshot also attached. Are you planning to work on it?

Also when I open a task, in console I have some warnings:

Warning: Failed prop type: Invalid prop `height` of type `string` supplied to `Resizable`, expected `number`.
    in Resizable (created by ResizableBox)
    in ResizableBox (created by CanvasWrapperComponent)
    in div (created by CanvasWrapperComponent)
    in main (created by Basic)
    in Basic (created by Content)
    in Content (created by CanvasWrapperComponent)
    in CanvasWrapperComponent (created by ConnectFunction)
    in ConnectFunction (created by StandardWorkspace3DComponent)
    in section (created by BasicLayout)
    in BasicLayout (created by Layout)
    in Layout (created by StandardWorkspace3DComponent)
    in StandardWorkspace3DComponent (created by AnnotationPageComponent)
    in main (created by Basic)
    in Basic (created by Content)
    in Content (created by AnnotationPageComponent)
    in section (created by BasicLayout)
    in BasicLayout (created by Layout)
    in Layout (created by AnnotationPageComponent)
    in AnnotationPageComponent (created by ConnectFunction)
    in ConnectFunction (created by Context.Consumer)
    in withRouter(Connect(AnnotationPageComponent)) (created by Context.Consumer)
    in Route (created by CVATApplication)
    in Switch (created by CVATApplication)
    in GlobalHotKeys (created by CVATApplication)
    in main (created by Basic)
    in Basic (created by Content)
    in Content (created by CVATApplication)
    in section (created by BasicLayout)
    in BasicLayout (created by Layout)
    in Layout (created by CVATApplication)
    in GlobalErrorBoundary (created by ConnectFunction)
    in ConnectFunction (created by CVATApplication)
    in CVATApplication (created by Context.Consumer)
    in withRouter(CVATApplication) (created by ConnectFunction)
    in ConnectFunction
    in Router (created by BrowserRouter)
    in BrowserRouter
    in Provider

bsekachev · 2021-03-12T08:57:22Z

Implemented centre axis for PCD rotation.

I am not sure it is in the center of .pcd (gif attached). Please, correct me if I am wrong.

bsekachev · 2021-03-12T09:01:59Z

And also I have some issues with context image (It is not a part of this milestone, but we expect everything should be already worked there).

We have two test archives:
https://github.com/openvinotoolkit/cvat/blob/develop/cvat/apps/engine/tests/assets/test_pointcloud_pcd.zip
https://github.com/openvinotoolkit/cvat/blob/develop/cvat/apps/engine/tests/assets/test_velodyne_points.zip

Both of them contains .pcd and .png. But context image can be opened only in the second. Could you please comment?
On the second we also see that context image overfitted its container. So, it also looks like a bug.

…o cvat-3D-M4

…ne between views

…o cvat-3D-M4

manasars · 2021-03-17T07:56:03Z

Hi @bsekachev ,

The cuboid track and Drawing method have now been removed, it was not pushed earlier since we discussed during Milestone-5 features discussion.
U I O J K L keys work with the combination of Alt, since we have conflicting keys as discussed in Milestone3.
The text for views has been changed to Top, Side and Front as per SOW, so the fit issue will not occur now.
The strange behaviour noticed while moving cuboid and scene has been fixed.
The performance of this particular PCD(kitchen.zip) is similar in supervise.ly as well. We also noticed that this PCD does not load properly in supervise.ly and looks like the pcd is very dense, we checked with other PCD files and performance seems to be fine, if any further tuning required we will look into it in the upcoming milestones.
PFA performance comparison for the kitchen.zip w.r.t supervise.ly

7. The context image fit issue and test archive file context-image load issue have been fixed.
8. The short cut key Shift+N , N was missed from our end, let us know if it is fine to push the change along with next Milestone.

bsekachev · 2021-03-17T08:15:44Z

@manasars

The cuboid track and Drawing method have now been removed, it was not pushed earlier since we discussed during Milestone-5 features discussion.

But are you planning to remove them in the future?

U I O J K L keys work with the combination of Alt, since we have conflicting keys as discussed in Milestone3.

What about adding a tooltip because it is not clear that I should use 'Alt' together with these buttons. What is more, I see you use Button element to display shortcuts. IMHO buttons are assumed to do some actions, but they do not. Can we add the same actions for these button as shortcuts do?

The text for views has been changed to Top, Side and Front as per SOW, so the fit issue will not occur now.
The strange behaviour noticed while moving cuboid and scene has been fixed.
The context image fit issue and test archive file context-image load issue have been fixed.

Thanks, I'll check it one more time. @dvkruchinin could you please check if now you can write a tests for context image?

The short cut key Shift+N , N was missed from our end, let us know if it is fine to push the change along with next Milestone.

Please, discuss your plans with @nmanovic .

PFA performance comparison for the kitchen.zip w.r.t supervise.ly

As I see CVAT cuboid placing is 3x slower than in supervise.ly. I believe it should be accelerated. @nmanovic , what do you think?

cvat-ui/src/reducers/interfaces.ts

...i/src/containers/annotation-page/standard-workspace/controls-side-bar/draw-shape-popover.tsx

cvat-canvas3d/src/typescript/canvas3d.ts

cvat-canvas3d/src/typescript/canvas3dController.ts

cvat-canvas3d/src/typescript/canvas3dModel.ts

cvat-ui/src/components/annotation-page/canvas/canvas-wrapper3D.tsx

dvkruchinin · 2021-03-17T08:41:22Z

@bsekachev

could you please check if now you can write a tests for context image?

Sure. I`ll check.

cvat-ui/src/components/annotation-page/canvas/canvas-wrapper3D.tsx

...i/src/components/annotation-page/standard-workspace/controls-side-bar/draw-shape-popover.tsx

cvat-canvas3d/src/typescript/canvas3d.ts

cvat-ui/src/components/annotation-page/canvas/canvas-wrapper3D.tsx

…o cvat-3D-M4

bsekachev · 2021-03-19T09:59:26Z

Hi @manasars

I see CI tests were failed now. @dvkruchinin has setup some screenshot tests for the feature.
There is a test where two screenshots are compared (between and after wheeling TOP/SIDE/FRONT views).
The reason why the test failed is that both screenshots are the same (they are expected to be changed after wheeling).

Screenshot before:

Screenshot after:

You can find these screenshots pressing build detais and reviewing artifacts:

As you can see the both screenshot are equal and they are empty (no any points seen) what is unexpected.
Personally I tried to create a task with this file cvat/apps/engine/tests/assets/test_pointcloud_pcd.zip and all the views except of perspective are empty.

I've already reported this bug in the previous milestone, but nobody was able to reproduce it. Some time after it disappeared itself and now appeared again. Also we can see it is reproducible on CI.

UPD:

Uploaded the same file on supervise.ly (all the points are more noticeable there):

manasars · 2021-03-19T12:04:37Z

Hi @manasars

I see CI tests were failed now. @dvkruchinin has setup some screenshot tests for the feature.
There is a test where two screenshots are compared (between and after wheeling TOP/SIDE/FRONT views).
The reason why the test failed is that both screenshots are the same (they are expected to be changed after wheeling).

Hi @bsekachev ,
I downloaded the screenshot from the build artefact and was able to see the side and front views.PFB attachment:

The points are not very bright but are visible.
Also when we ran the cypress locally, I get as test cases as passed:

Also, when I upload the zip file locally, am able to see the pcd in all views:

We are not yet sure on what is causing the issue in the build, trying to replicate in other systems.

bsekachev · 2021-03-19T12:29:31Z

and was able to see the side and front views

Agree, that it is visible on CI screenshots (different .PCD is used on CI), but the object is much less visible on these views with file I've specified above when it could be noticeable on supervise.ly without any issues. I think we should start from fixing it.
Also I think black and blue are not enough contrast (supervise.ly used black/white as default).

In CVAT current implementation visibility apparently depends on points density somehow.
@dvkruchinin reported that increasing the density helps to pass these tests.

Probably one more issue is that the code in this patch affected somehow TOP view area and on screenshot we can't see any objects (as if it is displaced out of viewport)

Screenshot before wheel (develop):

Screenshot before wheel (this branch)

Important note: These screenshots are done in headless mode;
So, if the tests are successful for you, please, try additionally headless mode.

dvkruchinin · 2021-03-19T13:59:07Z

Hi @manasars ,

In addition to the message @bsekachev .
There is a feeling that the display of the point cloud on TOP does not work correctly (maybe I'm wrong).

This is a point cloud in the form of a cube in the amount of 100,000 points. But for TOP VIEW, it looks like the camera is farther away than on the SIDE/FRONT. (The picture was taken on the develop branch).
cy.get('.cvat-canvas3d-fullsize').first().screenshot('id_canvas3d_container')

And as wrote above @bsekachev Getting screenshots is different between the this branch and the develop branch.
This branch:

cy.get('#canvas3d-container').screenshot('id_canvas3d_container')
In this case id_canvas3d_container' is just the name of the image.

manasars · 2021-03-22T10:33:38Z

Hi @dvkruchinin ,

In Milestone-4 we implemented resizing of views due to which the div has changed and delta differs, hence difference in screenshots between develop and this branch.

bsekachev · 2021-03-22T10:45:30Z

Hi @manasars

Let me rephrase. It is expected, that the object is visible and centered in all the additional views regardless on availableness resizing feature. Also object looks smaller on top view (as @dvkruchinin mentioned), what is probably because of wrong camera position setup.

If you disagree, please explain your point of view.

manasars · 2021-03-22T12:35:30Z

Hi @manasars

Let me rephrase. It is expected, that the object is visible and centered in all the additional views regardless on availableness resizing feature. Also object looks smaller on top view (as @dvkruchinin mentioned), what is probably because of wrong camera position setup.

If you disagree, please explain your point of view.

Hi @bsekachev ,

Yes agreed, the top view positioning needs to be adjusted, this came up as a part of adjusting the camera to support PCDs with extreme coordinates, we are working on the fix.

…o cvat-3D-M4

bsekachev · 2021-03-23T10:59:27Z

@manasars

Please, merge the latest develop. We expect CI should be fixed.

…o cvat-3D-M4

bsekachev · 2021-03-23T11:16:18Z

kubernetes-templates/03_database_deployment.yml

+          imagePullPolicy: 'IfNotPresent'
          env:
-          - name: POSTGRES_DB
-            valueFrom:
-              secretKeyRef:
-                name: cvat-postgres-secret
-                key: POSTGRES_DB
-          - name: POSTGRES_USER
-            valueFrom:
-              secretKeyRef:
-                name: cvat-postgres-secret
-                key: POSTGRES_USER
-          - name: POSTGRES_PASSWORD
-            valueFrom:
-              secretKeyRef:
-                name: cvat-postgres-secret
-                key: POSTGRES_PASSWORD
+            - name: POSTGRES_DB
+              valueFrom:
+                secretKeyRef:
+                  name: cvat-postgres-secret
+                  key: POSTGRES_DB
+            - name: POSTGRES_USER
+              valueFrom:
+                secretKeyRef:
+                  name: cvat-postgres-secret
+                  key: POSTGRES_USER
+            - name: POSTGRES_PASSWORD
+              valueFrom:
+                secretKeyRef:
+                  name: cvat-postgres-secret
+                  key: POSTGRES_PASSWORD
          ports:
-          - containerPort: 5432
+            - containerPort: 5432


Have you done these changes?

bsekachev · 2021-03-23T11:16:22Z

kubernetes-templates/03_redis_deployment.yml

+        - image: redis:4.0.5-alpine
+          name: cvat-redis
+          imagePullPolicy: Always
+          ports:
+            - containerPort: 6379
+          resources:
+            limits:
+              cpu: '0.1'


Have you done these changes?

bsekachev · 2021-03-23T11:16:28Z

kubernetes-templates/04_cvat_backend_deployment.yml

+            - name: DJANGO_MODWSGI_EXTRA_ARGS
+              value: ''
+            - name: UI_PORT
+              value: '80'
+            - name: UI_HOST
+              value: 'cvat-frontend-service'
+            - name: ALLOWED_HOSTS
+              value: '*'
+            - name: CVAT_REDIS_HOST
+              value: 'cvat-redis-service'
+            - name: CVAT_POSTGRES_HOST
+              value: 'cvat-postgres-service'
+            - name: CVAT_POSTGRES_USER
+              valueFrom:
+                secretKeyRef:
+                  name: cvat-postgres-secret
+                  key: POSTGRES_USER
+            - name: CVAT_POSTGRES_DBNAME
+              valueFrom:
+                secretKeyRef:
+                  name: cvat-postgres-secret
+                  key: POSTGRES_DB
+            - name: CVAT_POSTGRES_PASSWORD
+              valueFrom:
+                secretKeyRef:
+                  name: cvat-postgres-secret
+                  key: POSTGRES_PASSWORD
          ports:
-          - containerPort: 8080
+            - containerPort: 8080
          volumeMounts:
-          - mountPath: /home/django/data
-            name: cvat-backend-data
-            subPath: data
-          - mountPath: /home/django/keys
-            name: cvat-backend-data
-            subPath: keys
-          - mountPath: /home/django/logs
-            name: cvat-backend-data
-            subPath: logs
-          - mountPath: /home/django/models
-            name: cvat-backend-data
-            subPath: models
+            - mountPath: /home/django/data
+              name: cvat-backend-data
+              subPath: data
+            - mountPath: /home/django/keys
+              name: cvat-backend-data
+              subPath: keys
+            - mountPath: /home/django/logs
+              name: cvat-backend-data
+              subPath: logs
+            - mountPath: /home/django/models
+              name: cvat-backend-data
+              subPath: models
      initContainers:
        - name: user-data-permission-fix
          image: busybox
-          command: ["/bin/chmod", "-R", "777", "/home/django"]
+          command: ['/bin/chmod', '-R', '777', '/home/django']
          volumeMounts:
-          - mountPath: /home/django/data
-            name: cvat-backend-data
-            subPath: data
-          - mountPath: /home/django/keys
-            name: cvat-backend-data
-            subPath: keys
-          - mountPath: /home/django/logs
-            name: cvat-backend-data
-            subPath: logs
-          - mountPath: /home/django/models
-            name: cvat-backend-data
-            subPath: models
+            - mountPath: /home/django/data
+              name: cvat-backend-data
+              subPath: data
+            - mountPath: /home/django/keys
+              name: cvat-backend-data
+              subPath: keys
+            - mountPath: /home/django/logs
+              name: cvat-backend-data
+              subPath: logs
+            - mountPath: /home/django/models
+              name: cvat-backend-data
+              subPath: models


Have you done these changes?

Reverted code

bsekachev · 2021-03-23T11:17:05Z

kubernetes-templates/04_cvat_frontend_deployment.yml

@@ -25,5 +25,5 @@ spec:
          image: openvino/cvat_ui:v1.2.0
          imagePullPolicy: Always
          ports:
-          - containerPort: 80
+            - containerPort: 80


Please, look at kubernetes-templates/*, I believe you didn't change these files.

Looks like pre-commit hook autocorrected the files, we shall revert.

manasars · 2021-03-23T12:13:59Z

We shall be pushing again, the kubernetes reversion has not happened yet

manasars · 2021-03-23T13:52:05Z

Hi @bsekachev , we have merged with latest develop code and the CI build is successful.

cdp added 4 commits March 1, 2021 19:46

CVAT 3D Annotation - Added initial cuboid placement in all views

5e71fbf

Fixed MACOS issue for upload of zip files

92da287

Merge branch 'develop' of https://github.com/openvinotoolkit/cvat int…

c98cad1

…o cvat-3D-M4

Fixed camera axis centre issue

c7ba5aa

manasars requested review from bsekachev and nmanovic as code owners March 1, 2021 17:06

Fixed ESLint import issues

dfe22f4

vnishukov reviewed Mar 10, 2021

View reviewed changes

cvat-ui/src/components/annotation-page/standard3D-workspace/styles.scss Outdated Show resolved Hide resolved

vnishukov suggested changes Mar 10, 2021

View reviewed changes

vnishukov assigned ActiveChooN and bsekachev Mar 10, 2021

bsekachev reviewed Mar 12, 2021

View reviewed changes

cvat-ui/package.json Outdated Show resolved Hide resolved

cdp added 8 commits March 15, 2021 09:24

Merge branch 'develop' of https://github.com/openvinotoolkit/cvat int…

8aec49d

…o cvat-3D-M4

Merge branch 'develop' of https://github.com/openvinotoolkit/cvat int…

7f48c11

…o cvat-3D-M4

Merge branch 'develop' of https://github.com/openvinotoolkit/cvat int…

4daf516

…o cvat-3D-M4

Fixed context image fit issue and resizing possible on entire grey li…

3902822

…ne between views

Fixed the multiple interection point issue

cd70249

Fixed Naming convention as per SOW

7ba93ad

Merge branch 'develop' of https://github.com/openvinotoolkit/cvat int…

55ba179

…o cvat-3D-M4

Trigger notification

c065389

bsekachev reviewed Mar 17, 2021

View reviewed changes

cvat-ui/src/components/annotation-page/canvas/canvas-wrapper3D.tsx Outdated Show resolved Hide resolved

cvat-ui/src/components/annotation-page/canvas/canvas-wrapper3D.tsx Outdated Show resolved Hide resolved

bsekachev reviewed Mar 17, 2021

View reviewed changes

cvat-ui/src/components/annotation-page/canvas/canvas-wrapper3D.tsx Show resolved Hide resolved

...i/src/components/annotation-page/standard-workspace/controls-side-bar/draw-shape-popover.tsx Outdated Show resolved Hide resolved

bsekachev reviewed Mar 18, 2021

View reviewed changes

cvat-canvas3d/src/typescript/canvas3d.ts Show resolved Hide resolved

cvat-ui/src/components/annotation-page/canvas/canvas-wrapper3D.tsx Outdated Show resolved Hide resolved

cvat-ui/src/components/annotation-page/canvas/canvas-wrapper3D.tsx Show resolved Hide resolved

cdp added 4 commits March 18, 2021 15:11

Merge branch 'develop' of https://github.com/openvinotoolkit/cvat int…

85aae86

…o cvat-3D-M4

Fixed minor issues and review changes

ebcea6a

Merge branch 'develop' of https://github.com/openvinotoolkit/cvat int…

44fb534

…o cvat-3D-M4

Merged dev code, updated changelog and minor fixes

567546a

manasars requested a review from azhavoro as a code owner March 19, 2021 07:14

Merge branch 'develop' of https://github.com/openvinotoolkit/cvat int…

ea43164

…o cvat-3D-M4

dvkruchinin mentioned this pull request Mar 19, 2021

Cypress test/pcd adaptation for cvat3d m4. #2981

Merged

8 tasks

Merged Latest develop code

85dfc0d

dvkruchinin mentioned this pull request Mar 22, 2021

Cypress test. Canvas 3d-m4. Check resize views. #2995

Merged

8 tasks

bsekachev self-requested a review March 22, 2021 14:11

cdp added 3 commits March 23, 2021 10:34

Fixed camera positioning issue in Top View

103ebfa

Merged latest develop code

c8d5cf6

Merge branch 'develop' of https://github.com/openvinotoolkit/cvat int…

0060ebf

…o cvat-3D-M4

Merge branch 'develop' of https://github.com/openvinotoolkit/cvat int…

1463c0d

…o cvat-3D-M4

bsekachev reviewed Mar 23, 2021

View reviewed changes

Reverted kubernetes auto-corrected code

b71c183

Reverted kubernetes code

e53501f

bsekachev approved these changes Mar 23, 2021

View reviewed changes

bsekachev merged commit 538bc93 into cvat-ai:develop Mar 23, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CVAT 3D - Milestone-4 #2891

CVAT 3D - Milestone-4 #2891

manasars commented Mar 1, 2021

coveralls commented Mar 1, 2021 •

edited

Loading

bsekachev commented Mar 12, 2021

bsekachev commented Mar 12, 2021

bsekachev commented Mar 12, 2021

manasars commented Mar 17, 2021

bsekachev commented Mar 17, 2021 •

edited

Loading

dvkruchinin commented Mar 17, 2021

bsekachev commented Mar 19, 2021 •

edited

Loading

manasars commented Mar 19, 2021 •

edited

Loading

bsekachev commented Mar 19, 2021

dvkruchinin commented Mar 19, 2021 •

edited

Loading

manasars commented Mar 22, 2021

bsekachev commented Mar 22, 2021

manasars commented Mar 22, 2021

bsekachev commented Mar 23, 2021

bsekachev Mar 23, 2021

manasars Mar 23, 2021

bsekachev Mar 23, 2021

manasars Mar 23, 2021

bsekachev Mar 23, 2021

manasars Mar 23, 2021

bsekachev Mar 23, 2021

manasars Mar 23, 2021

manasars commented Mar 23, 2021

manasars commented Mar 23, 2021

CVAT 3D - Milestone-4 #2891

CVAT 3D - Milestone-4 #2891

Conversation

manasars commented Mar 1, 2021

coveralls commented Mar 1, 2021 • edited Loading

bsekachev commented Mar 12, 2021

bsekachev commented Mar 12, 2021

bsekachev commented Mar 12, 2021

manasars commented Mar 17, 2021

bsekachev commented Mar 17, 2021 • edited Loading

dvkruchinin commented Mar 17, 2021

bsekachev commented Mar 19, 2021 • edited Loading

manasars commented Mar 19, 2021 • edited Loading

bsekachev commented Mar 19, 2021

dvkruchinin commented Mar 19, 2021 • edited Loading

manasars commented Mar 22, 2021

bsekachev commented Mar 22, 2021

manasars commented Mar 22, 2021

bsekachev commented Mar 23, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

manasars commented Mar 23, 2021

manasars commented Mar 23, 2021

coveralls commented Mar 1, 2021 •

edited

Loading

bsekachev commented Mar 17, 2021 •

edited

Loading

bsekachev commented Mar 19, 2021 •

edited

Loading

manasars commented Mar 19, 2021 •

edited

Loading

dvkruchinin commented Mar 19, 2021 •

edited

Loading