Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CVAT-3D-M3 #2768

Merged
merged 82 commits into from
Feb 20, 2021
Merged

CVAT-3D-M3 #2768

merged 82 commits into from
Feb 20, 2021

Conversation

manasars
Copy link
Contributor

@manasars manasars commented Feb 4, 2021

CVAT-3D-Milestone3 : Visualize 3D point cloud spaces in 3D View, Top View, Side View and Front View

Changes include:
Implemented 4 views in the main interface - 3D View, Top View, Front View, Side View.
Implemented 3D view controls functionality (Rotation + Zoom In/Out + Movement)
Implemented controls for Top View,Side View and Front View.(Zoom In/Zoom Out)

Test Cases:
Manual Unit testing done locally.
Existing test cases work as expected.
System Test cases will be shared.

[x ] I submit my changes into the develop branch

[ x] I submit my code changes under the same MIT License that covers the project.
Feel free to contact the maintainers if that's a concern.

cdp added 30 commits December 7, 2020 13:11
…d created a relatedfiles table for PCD to Image mapping.
…ration Scripts and Dimension attribute for MpegChunk Writers
@bsekachev
Copy link
Member

@ActiveChooN could you please also review the PR this week?

@bsekachev
Copy link
Member

I used this file for test: https://sourceforge.net/projects/pointclouds/files/PCD%20datasets/kitchen.zip/download
In my case front, side and top are black.

Screenshot from 2021-02-10 14-20-19

When I go to another frame 3D is blinking sometimes

Peek 2021-02-10 14-25

IMO it would be better if rotation axis was in center of point cloud. Now it is not, but if example tool uses the same approach it is okay.
Peek 2021-02-10 14-29

@bsekachev
Copy link
Member

@manasars

Please, resolve all the static linter issues in affected files.
Now I see a lot. About ten in cvat-ui/src/components/annotation-page/canvas/canvas-wrapper3D.tsx and some in cvat-canvas3d/src/typescript/*.ts

@manasars
Copy link
Contributor Author

manasars commented Feb 10, 2021

Hi @bsekachev ,
We used the same zip file uploaded in link,

  1. We are able to see all the views.
    Screen Shot 2021-02-10 at 7 20 19 PM
  2. In this pcd file, all the points fall in same quadrant and hence the rotation axis appears as away from centre of point cloud.
  3. The origin we have currently specified is (0,0,0) for all pcd files, we can change the code to display based on point cloud centre for different pcd files.

@bsekachev
Copy link
Member

@manasars

We are able to see all the views.

I pulled the latest changes, did npm ci and still the same result.
@azhavoro @ActiveChooN @vnishukov Guys, could you please try the same scenario? Maybe it is only my side issue.

The origin we have currently specified is (0,0,0) for all pcd files, we can change the code to display based on point cloud centre for different pcd files.

Let's ask for others. @vnishukov @nmanovic what do you think?

What about blinking? Were you able to reproduce?

@manasars
Copy link
Contributor Author

@manasars

Found some strange behaviour. I'm using hotkeys to move around perspective view first. Then I zoom in it hitting Alt+I couple of times. When the maximum zoom is reached, I quickly hit Alt+I again couple of times (3-5). And then I have black background in perspective view and it's freeze, I can't zoom out neither using hotkeys/arrows neither mouse wheel. Top, Side and Front view are still able to zoom in/out as expected. Please see attached video

Now when we completely zoom-in to the maximum distance, the black screen appears since the points are not present in that instance, but on zoom-out/mouse movements the perspective view can be adjusted.

@manasars
Copy link
Contributor Author

What about blinking? Were you able to reproduce?
Yes the blinking is happening at times, we shall look into the issue.

@vnishukov
Copy link
Contributor

@bsekachev

I pulled the latest changes, did npm ci and still the same result.
@azhavoro @ActiveChooN @vnishukov Guys, could you please try the same scenario? Maybe it is only my side issue.

I have checked with the kitchen.zip. I can see top, side and front

image

@vnishukov
Copy link
Contributor

@bsekachev, @manasars, @nmanovic

Let's ask for others. @vnishukov @nmanovic what do you think?

I think that axis should be in centre for each pcd

@vnishukov
Copy link
Contributor

vnishukov commented Feb 15, 2021

@manasars

What about blinking? Were you able to reproduce?
Yes the blinking is happening at times, we shall look into the issue.

I can confirm blinking as well

@bsekachev
Copy link
Member

@vnishukov

I think that axis should be in centre for each pcd

In this case, please, keep on track this change

Copy link
Member

@bsekachev bsekachev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@bsekachev
Copy link
Member

@vnishukov Please, give your approve if you believe we can merge the PR and I'll do it

@bsekachev bsekachev merged commit 20e997d into cvat-ai:develop Feb 20, 2021
kenu pushed a commit to kenu/cvat that referenced this pull request Feb 23, 2021
* CVAT-3D Updated the Mime Types with Bin Support, added dependency of open3D

* CVAT-3D Added additional column as Dimension for engine_task table and created a relatedfiles table for PCD to Image mapping.

* Added Support for 3D file Upload in BIN and PCD.

* Added Dimension attribute defaulting to 2D for importer and exporter.

* Added props passing for dimension attribute, filtering of import, Migration Scripts and Dimension attribute for MpegChunk Writers

* Modified code as per review comments

* Updated Unit test cases for 3D task creation

* Refactored Dimension Enum in UI and backend code

* Resolving conflicts

* Updated Unit Test Case

* Refactored TaskDimension to DimensionType, Simplified usage of Dimension accross classes

* Removing manually created test files

* Removing old pcd mime-type mapping

* Added test files generated by synthetic data using open3d

* Merged with develop branch latest changes

* Added libraries required for open3d

* Added files

* Added synthethic pcd,bin and img test files

* Modified test file name

* Trigger travis ci

* Modified test case to ignore 3D preview images

* Trigger notification

* Deleting DS Store files

* Modified test cases as per review comments

* Checking pre-commit hook

* Fixed Lint issues - precommit hook verification

* Added changes for CVAT-3D Milestone2 changes - Frame Navigation, photo context hide and show

* Modified changes

* Added canvas3D for 3D Perspective

* Added missing files

* Added code to get image context for 3D view

* Codacy check for stylesheet

* Modified frame navigantion for 3D View

* Modified style for context-image

* Trigger notification

* Added Support for 3D file Upload in BIN and PCD.

* Added props passing for dimension attribute, filtering of import, Migration Scripts and Dimension attribute for MpegChunk Writers

* Modified code as per review comments

* Refactored Dimension Enum in UI and backend code

* Merged with develop branch latest changes

* Added files

* Added changes for CVAT-3D Milestone2 changes - Frame Navigation, photo context hide and show

* Modified changes

* Added canvas3D for 3D Perspective

* Added missing files

* Added code to get image context for 3D view

* Codacy check for stylesheet

* Modified frame navigantion for 3D View

* Modified style for context-image

* Changed cvat-data lint issues

* Modified to use opencv as per review comments

* Removed unwanted imports

* Fixed css and added usage of hooks

* Merged Develop branch code

* Removed unused data structures

* Removed unused data structures

* Refactored unused data structures

* Added three js dependency in cvat ui package-lock.json

* Merged develop branch code and refactored code

* Fixed snyk issue

* Modified Camera Icon in photo-context

* Update icons.tsx

* Remove unused svg file

* Modified changelog file

* Added changes for CVAT-3D Milestone3

* Added missing line as per codacy check

Co-authored-by: cdp <cdp123>
@kgloveyou
Copy link

kgloveyou commented Feb 24, 2021

How to use this feature? When i upload local pcd files and submit the task,it threw a error in the console, the content was ' Could not create the task on the server. PIL.UnidentifiedImageError: cannot identify image file '/home/django/data/data/5/raw/Rf3.pcd'.'

@bsekachev
Copy link
Member

bsekachev commented Feb 24, 2021

How to use this feature? When i upload local pcd files and submit the task,it threw a error in the console, the content was ' Could not create the task on the server. PIL.UnidentifiedImageError: cannot identify image file '/home/django/data/data/5/raw/Rf3.pcd'.'

@kgloveyou
Try to make a .zip archive with .pcd files and use it when creating a task.

@kgloveyou
Copy link

@bsekachev
Thank You.

@bsekachev bsekachev mentioned this pull request Mar 19, 2021
@SaihholdChiu
Copy link

How to use this feature? When i upload local pcd files and submit the task,it threw a error in the console, the content was ' Could not create the task on the server. PIL.UnidentifiedImageError: cannot identify image file '/home/django/data/data/5/raw/Rf3.pcd'.'

@kgloveyou
Try to make a .zip archive with .pcd files and use it when creating a task.

i used same data above https://sourceforge.net/projects/pointclouds/files/PCD%20datasets/kitchen.zip/download
and put them in a .zip file, it also showed same issue.

@kgloveyou
Copy link

kgloveyou commented Mar 22, 2021

How to use this feature? When i upload local pcd files and submit the task,it threw a error in the console, the content was ' Could not create the task on the server. PIL.UnidentifiedImageError: cannot identify image file '/home/django/data/data/5/raw/Rf3.pcd'.'

@kgloveyou
Try to make a .zip archive with .pcd files and use it when creating a task.

I uploaded a .zip archive,but in the annotation interface , I got a error: (404 Not Found )http://127.0.0.1:8080/api/v1/tasks/1/data?quality=original&type=context_image&number=0,and the response was "No context image related to the frame"

@bsekachev
Copy link
Member

@kgloveyou

If you believe it is a bug, please submit corresponding issue, fill out issue template and provide steps to reproduce.
In my case I was able to create 3D task using .zip archive without context related images and no any errors on annotation interface.

@SaihholdChiu

The same comment about your issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants