Open Frameworks and OpenCV

OpenFrameworks Lections

5. OpenFrameworks and OpenCV

lectures and announcements: www.uralvision.blogspot.com issues for projects and programming: [email protected]

USU / IMM spring 2011

Analysis and change of image In openFrameworks you can edit ofImage in a such way: 1. Get image bytes unsigned char * data = image.getPixels (); int w = image.width; int h = image.height; // These are not copied, and delete do not have to! 2. Edit build a new image data1 unsigned char * data1 = new unsigned char * [w * h * 3]; .... Fill data1, using data 3. Set image bytes image.setFromPixels (data1, w, h, OF_IMAGE_COLOR);

// OF_IMAGE_COLOR - 3-byte color. // OF_IMAGE_GRAYSCALE - 1-byte gray. // OF_IMAGE_COLOR_ALPHA - 4-byte, color + transparency.

Analysis and change of image Byte processing - it is usually inconvenient. Better use of special tools. One of the most convenient for today - the library OpenCV. In openFrameworks have addon - ofxOpenCv. To date (April 2011) he works withOpenCV 1.x. This version of OpenCV is inconvenient fact that has only a C-interface. Therefore, we study a more convenient OpenCV 2.x and we use it, without the addon ofxOpenCv.

OpenFrameworks and OpenCV In OpenCV image given by the class Mat. How to convert ofImage in Mat and back to the example of 3-channel images:

From openFrameworks in OpenCV ofImage image; //-let it is, 3-channel ... Mat imageCV (cv:: Size (image.width, image.height), CV_8UC3, image.getPixels ()); From OpenCV in openFrameworks Mat imageCV; //-the way it is, 3-channel ... image.setFromPixels ((unsigned char *) IplImage (imageCV). imageData, imageCV.size (). width, imageCV.size (). height, OF_IMAGE_COLOR);

Introduction to OpenCV - What is OpenCV - The first project to OpenCV - Class Mat - Image processing functions

What is OpenCV

"Open Computer Vision Library" Open library with a set of functions for processing, analysis and image recognition, C / C + +.

What is OpenCV 2000 - First alpha version, support for Intel, C-interface 2006 - Version 1.0 2008 - Support Willow Garage (lab. Robotics) 2009 - version 2.0, classes in C + + 2010 - version 2.2, realized work with the GPU

The first project to OpenCV 1. Creating a Project We assume that Microsoft Visual C + + 2008 Express Edition and OpenCV 2.1 is already installed. 1. Run VS2008

2. Create a console project File - New - Project - Win32 Console Application, in the Name enter Project1, click OK. 3. Set up the path Alt + F7 - opens the project properties Configuration Properties - C / C + + - General - Additional Include Directories, where we put the value "C: \ Program Files \ OpenCV2.1 \ include \ opencv"; Linker - General - Additional Library Directories, where we put the value of C: \ Program Files \ OpenCV2.1 \ lib \

Linker - Input - Additional Dependencies cv210.lib cvaux210.lib cxcore210.lib cxts210.lib highgui210.lib for Release, cv210d.lib cvaux210d.lib cxcore210d.lib cxts210.lib highgui210d.lib for Debug

The first project to OpenCV 2. Reading the image and display it on screen 1. Preparing the input data: file http://www.fitseniors.org/wp-content/uploads/2008/04/green_apple.jpg write in C: \ green_apple.jpg 2. Writing in Project1.cpp: # Include "stdafx.h" # Include "cv.h" # Include "highgui.h" using namespace cv; int main (int argc, const char ** argv) { Mat image = imread ("C:\\green_apple.jpg");// Load image from disk imshow ("image", image); // Show image waitKey (0); // Wait for keystroke return 0; } 3. Press F7 - compilation, F5 - run. The program will show the image in the window and by pressing any key will complete its work.

The first project to OpenCV 3. Linear operations on images Replace the text in the main from the previous for example: int main (int argc, const char ** argv) { Mat image = imread ("C:\\green_apple.jpg"); // Image1 pixel by pixel is equal to 0.3 * image Mat image1 = 0.3 * image; imshow ("image", image); imshow ("image1", image1); waitKey (0); return 0; }

The first project to OpenCV 4. Work with rectangular subdomains image Replace the text in the main from the previous example to: int main (int argc, const char ** argv) { Mat image = imread ("C:\\green_apple.jpg"); // Cut of the picture Rect rect = Rect (100, 100, 200, 200); // Rectangle cut Mat image3; image (rect). copyTo (image3); // Copy of the image imshow ("image3", image3); // Change the part of the picture inside the picture image (rect) *= 2; imshow ("image changed", image); waitKey (0); return 0; }

Mat Class Mat - Base class for storing images OpenCV.

Mat Class Single-and multi-channel images The image is a matrix of pixels. Each pixel can store some data. If the pixel stores the vector data, the dimension vector is number of image channels. 1-channel image - also called the half-tone 3-channel images - typically consist of three components (Red, Green, Blue).

Also, OpenCV can be used 2 - and 4-channel image.

Mat Class Creating images 1) Let the picture without some type of Mat imageEmpty; 2) Image w x h pixels, the values ​0 .. 255 (8Umeans "unsigned 8 bit",C1means "a channel"): int w = 150; int h = 100; Mat imageGray (cv:: Size ( w, h ) CV_8UC1 );

Mat Class Creating images 3) 1-channel with the floating-point values (32F means "float 32 bit"):

Mat imageFloat (cv:: Size (w, h), CV_32FC1 ); 4) 3-channel image with values ​0 .. 255 for each channel: Mat imageRGB (cv:: Size (w, h), CV_8UC3 );

Mat Class Memory management 1. Memory for the image stands out and is automatically cleared

That is, OpenCV itself creates the image of the desired size and type, if this image is an output parameter of a function: Mat imageFloat; imageGray.convertTo (imageFloat, CV_32FC1, 1.0 / 255.0); - Here OpenCV itself allocate imageFloat. It is important that if your image is already the right size, there are no operations on memory allocation is performed.

2. Assignment operator shall not copy the data (as does the std:: vector), and not by copying pointers, and using mechanism of the reference count.

Mat Class Memory management The mechanism of the reference count (In STL is a shared_ptr, in Java it is all signposts) works like this: { Mat A (cv:: Size (100, 100), CV_8UC1); // Allocate memory for the image, and the memories, // This memory is a single image. { Mat B = A; // Here the memory for the image does not stand out, but simply // Data in B point to the same area in memory. // Therefore, if we change B, then changed, and A. // Reference count increased by an image, was equal to 2. } // Here B came out of scope, the reference count is decreased, // And became equal to 1. } // Here A came out of scope, the reference count becomes equal to 0, // And the memory allocated to it are automatically cleared.

Mat Class Memory management Since the operation Mat B = A; does not copy the image A to B, then in order to create a copy of the image for subsequent independent use, you must use explicit commands copyTo andclone: image1.copyTo (image2); image3 = image1.clone ();

Mat Class Memory management Outcome: 1) an assignment Mat B = A; is very fast, and does not copy the data and adjusts the pointers in a special way to them. This allows you to transfer Mat in the function directly, without pointers and references. This will not cause unwanted copying Mat the stack (as it would stalal std:: vector). Although, of course, const Mat & will be transmitted still faster. 2) to copy the images to use explicit commands copyToand clone.

Mat Class Per-pixel access to images In OpenCV has several ways of per-pixel access to images. They vary in the degree of security (typing and go beyond the border), the speed and convenience. Wherever possible, you should try to avoid direct references to the pixels, but instead use the functions of OpenCV, since they usually work faster and the code more understandable.

Mat Class Per-pixel access to images One way to access the pixels for images that have known the type - the use of theat. For single-channel images 0 ... 255 it is: // Get values int value = imageGray.at (y, x); // Set the values imageGray.at (y, x) = 100; Note that x and y in the call are swapped.

Mat Class Conversion types Note In the derivation of the on-screen images with floating-point OpenCV means we must bear in mind that they are displayed on the assumption that their values ​lie in [0,1]. Therefore, when converting 8-bit images in an image float to do the transformation - the multiplication by 1.0 / 255.0. To convert images of different types of bit mode (float and unsigned char) used a class member convertTo. In its second argument - the type of the image.

imageGray.convertTo (imageFloat, CV_32FC1, 1.0 / 255.0); The number of channels input and output must match!

Mat Class Conversion types For converting different color spaces using the function cvtColor. If necessary, it can change the number of channels. For example, the conversion of 3-channel RGB-image to grayscale: cvtColor (inputRGB, outputGray, CV_BGR2GRAY); on the contrary: cvtColor (inputGray, outputRGB, CV_GRAY2BGR);

Mat Class Partition of the channels Function split divides the multi-channel image into channels. Functionmergestitches together a single-image multi-channel. voidsplit(Const Mat &mtx// Original color image vector &mv// Result set is 1-channel // Images ) voidmerge(Const vector &mv// Initial set of 1-channel // Images Mat &dst// The resulting color // Image ) Most often they are used to separately to each color image processing, as well as for various manipulations of the channels.

Mat Class Partition of the channels int main (int argc, const char ** argv) { Mat image = imread ("C:\\green_apple.jpg"); // Split the original image into three channels // - Channels [0], channels [1], channels [2]

vector channels; split (image, channels);

// Show the channels in separate windows // Note that the red channel - 2, not 0.

imshow ("Red", channels [2]); imshow ("Green", channels [1]); imshow ("Blue", channels [0]); waitKey (0); return 0; }

Image processing functions Smoothing

Original image

The image, smoothed box 11 x 11

FunctionGaussianBlurperforms image smoothing Gaussian filter. Most often, the smoothing is applied to remove small noise on the image for subsequent image analysis. Is done by using a filter of small size.

http://www.innocentenglish.com/funny-pics/best-pics/stairs-sidewalk-art.jpg

Image processing functions Threshold

Functionthresholdperforms threshold processing of the image.

Most often it is used to highlight objects of interest pixels in the image.

http://www.svi.nl/wikiimg/SeedAndThreshold_02.png

Image processing functions Fill areas

FunctionfloodFillprovides a fill area, starting from a pixel (x, y), with specified boundaries shutdown using a 4 - or 8 - adjacency pixels. It is important: It spoils the original image - as it fills. Most often it is used to highlight areas identified by the threshold processing, for subsequent analysis. http://upload.wikimedia.org/wikipedia/commons/thumb/5/5e/Square_4_connectivity.svg/300px-Square_4_connectivity.svg.png http://tunginobi.spheredev.org/images/flood_fill_ss_01.png

Image processing functions Isolation circuits

The contour of the object - this is the line representing the edge of the object's shape. Underline the contour points -Sobel, Leased-line circuit -Canny. Application 1. Recognition. Along the contour can often determine the type of object that we observe. 2. Dimension. With the circuit can accurately estimate the size of the object of their rotation, and location. http://howto.nicubunu.ro/gears/gears_16.png http://cvpr.uni-muenster.de/research/rack/index.html

Sample project on OpenCV: Search for a billiard ball

Problem The image of billiard field to find the coordinates of the centers of billiard balls. Algorithm:

1. Threshold find bright pixels. 2. Analysis of areas. The method of casting we find connected regions, Among them we find such dimensions that allow it balls.

1. Threshold Problem - the image of billiard field highlight the pixels that are not field (shooting conditions such that the field - dark) Mat image = imread ("C:\\billiard.png"); // load the input image imshow ("Input image", image); vector planes; split (image, planes); Mat gray = 0.299 * planes [2] + 0.587 * planes [1] + 0.114 * planes [0]; double thresh = 50.0; // The threshold is chosen empirically threshold(Gray, gray, 50.0, 255.0, CV_THRESH_BINARY); imshow ("Threshold", gray);

threshold - an example of application

Please note: We have identified just pixels "not" field. To find the coordinates of the centers of balls and cue position - requires further processing.

2. Analysis areas floodFill- Allocation of connected regions

morphological operations dilate - dilation(*) erode - Erosion(*)

floodFill - description Function floodFillprovides a fill area, starting from a pixel (x, y), with specified boundaries shutdown using a 4 - or 8 - adjacency pixels.

Important: it spoils the original image - as it fills. 1. Most often it is used to highlight areas identified by the threshold processing, for subsequent analysis. 2. It can also be used to remove small noise on the binary image (in contrast to the "erosion + dilation" - do not spoil the boundaries of larger areas). 3.If enhance overall box found in the area of ​1 pixel on all sides and make the fill, the way you can eliminate the internal hole in the area.

floodFill - a list of options Announcement and description of the parameter list: int floodFill(Mat & image, Point seed, Scalar newVal, Rect * rect= 0 Scalar loDiff= Scalar (), Scalar upDiff= Scalar (), int flags= 4) image - The input image, 1 - or 3-channel, 8 or 32-bit. seed - Pixel, from which to start pouring rect - Bounding box found by the field loDiff, UpDiff - allowable difference with its neighbors (Or - with embryonic pixel, if flags | = FLOODFILL_FIXED_RANGE) that is, a new pixel must satisfy valueNew value - loDiff