Robot vision with YARP & OpenCV Assignment

Prerequisites

By now, you should be familiar with playing around with images processing such as:

• Getting images from streams
• Smoothing Images
• Eroding and Dilating (Morphology transformation)
• Basic thresholding
• Hough Circle Transform
• Template Matching
• Finding contours in your image
• Image Moments
• Point Polygon Test

documentation & examples on image filtering can be taken from here & here or from the tutorials.

more info & examples on

Assignment

As an assignment, we would like you to complete a module that employs what we have seen so far in order to extract the closest blob from a stream of disparity images. This will be useful for you when you will do the machine learning course.

You should be able to accomplish the following tasks:

1. Get a stream of images from the yarpdataplayer: disparity and rgb.
2. Apply image processing techniques to make the disparity image cleaner for processing. (GaussianBlur, erode, dilate)
3. Retrieve the maximum value and its position.
4. Apply thresholding on that point to remove unwanted background
5. Find the contour of the closest object with its moment and mass center
6. Draw it on the disparity image
7. Get its bounding box (ROI-Region of Interest)
8. Create a cropped image containing the RGB image of the ROI
9. Fill in a YARP bottle as a list
10. Finally, stream the resulting image out.

The outcome should look something like the animation below:

Most of the points reported above have been already addressed in the tutorials, so you need to fill in the missing gaps highlighted by the comment // FILL IN THE CODE.

Once done, you can test your code in two ways:

1. Manually: running the yarpmanager scripts provided from within app/scripts.
2. Automatically: running the script test.sh in the smoke-test directory. Take into account these important points:
   1. The smoke test basically will automatically compare the content of the streamed bounding box historgrams in hsv at various frames with the base solution.
   2. The test will pass if an overall comparison of the five histograms is above 90%.

IMPORTANT Notes

You will need to run the dataset dataDisparity in:

$ wget http://www.icub.org/download/software/datasetplayer-demo/dataDisparity.zip
$ unzip dataDisparity.zip

Finally, create the env var DATASETS_PATH pointing to the location where you've unzipped the archive.

Grading

Here's the score map:

Requirements	Points
Percentage >60	1
Percentage >60 and < 70	5
Percentage >70 and < 80	10
Percentage >80 and < 90	15
Percentage >90	20
Maximum score	20 🏆

How to complete the assignment