OpenCV and Visual c++ Programming in image processing

OpenCV, MATLAB or AForge ??

Image processing is the process of manipulating image data in order to make it suitable for  computer vision applications or to make it suitable to present it to humans. For example, changing brightness or contrast is a image processing task task which make the image image visually pleasi ple asing ng for hum humans ans or sui suitab table le for fur furthe therr pro proces cessin sing g for a cer certai tain n com comput puter er vis vision ion application. Computer vision which go beyond image processing, helps to obtain relevant information from images and make decisions based on that information. In other words, computer vision is making the computer see as humans do. Basic steps for a typical computer vision application as follows. 1. Image acuisition !. Image manipulation ". $.

#btaining re relevant in information %ecision making

If you are new to computer vision, you may be wondering where to start. First you have to understand the basic principles of image processing and computer vision. visio n. Then you have to choose a suita suitable ble language to devel develop op your compu computer ter vision visio n appli applicatio cation. n. Some of the most popul popular ar methods are using OpenCV with C/C++,, MATLAB C/C++ MATLAB and  and AForge AForge.. If you don't really know why you would choose one over the other, here is my explanation. !T"!# is the most easiest and the ine$cient way to process images and %pen& is the most e$cient and hardest way to process images. !Forge has (ualities in between %pen& and !T"!#. !T"!#. %pen& has become hardest only because there is no proper documentation and error handling handling codes. #ut %pen& has lots of basic inbuilt inbuilt image processing processing functions so that those who want to learn computer vision can develop their applications through proper proper understanding about what they do. So, I think that it is worthy to learn computer vision with %pen&. Therefore in this blog, it is presented basic image processing functions and computer vision applications with line by line explanations.

Read & Display Image Read Image from File and Displa )ere I am going to explain how to read an image from a *le and display the content using %pen& library functions. First of all, open your &++ I- and create crea te a new proect. proect. /ou /ou have to con*gure con*gure your new proect in order to use %pen& library functions. If you have not con*gured the proect for %pen& yet, please refer to Installing 0 &on*guring with isual Studio. 111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 11111111111111111111111 !in"l#de $open"%&/highg#i/highg#i'hpp$ !in"l#de (iostream) using namespace c! using namespace s"d! in" main# main# in"  in" argc argc,, cons" c$ar%% arg  ' Ma" img ( imread# imread#)My*ic+*-) )My*ic+*-),, CV.LOAD.IMA-/.01C2A1-/D! 33read CV.LOAD.IMA-/.01C2A1-/D! 33read "$e image da"a in "$e 4ile )My*ic+*-) and s"ore i" in 5img5 i4  #img+emp"y# 33c$ec6  #img+emp"y# 33c$ec6 7$e"$er "$e image is loaded or no" ' cou" 88 )/rror 9 Image canno" :e loaded++;;) 88 loaded++;;)  88 endl! //sys"em# sys"em#)pause) )pause)! ! 337ai"  337ai" 4or a 6ey press re"urn   namedindo7#)Myindo7) namedindo7#)Myindo7),, CV.I1DO.A0TO@I/! 33crea"e CV.I1DO.A0TO@I/! 33crea"e a 7indo7 7i"$ "$e "$e name )Myindo7)   ims$o7#)Myindo7) ims$o7#)Myindo7),, img! 33display img! 33display "$e image 7$ic$ 7$ic$ is s"ored in "$e 5img5 5img5 in "$e )Myindo7) 7indo7  

7ai"ey#! 337ai" in4ini"e "ime 4or a 6eypress 7ai"ey#! 337ai"

 

des"royindo7#)Myindo7)! des"royindo7#)Myindo7) ! 33des"roy  33des"roy "$e 7indo7 7i"$ "$e name, )Myindo7) re"urn !

>

111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 1111111111111111111111

#efore you run this program, put any image *le 2y3ic.4356 into the folder where your c++ *le is.%therwise you have to change the *rst argument of imread26 function and give the absolute path to your image *le.  /ou can download this %pen&  /ou %pen& visual c++ proect proect from here here.. 2The downloaded *le is a compressed .rar folder. folder. So, you have to extract it using 7inrar 7inrar or  or other suitable software6

Load & Diplay Image from File

*planation "et's review the above %pen& code line by line !in"l#de $stdaf'h$ If you are developing your application in isual Studio, don't forget to put this line above the code. .

•

•

!in"l#de $open"%&/highg#i/highg#i'hpp$ imread#, namedindo7# imread#, namedindo7#,, ims$o7# and 7ai"ey# functions are declared in the above header file. &o you must include it. 'e are using Ma" data structure in the above program. It is declared head ader er *l *le. e. Th Then en wh why y do don' n'tt we include this8 this8 It's in $open"%&/"ore/"ore'hpp$ he

because $open"%&/highg#i/highg#i'hpp$ header *le include that header *le inside it. So, we don't need to include it again in our program. •

using namespace c!  (ll the data structures and functions in )openc3core3core+$pp) and )openc3$ig$gui3$ig$gui+$pp) are declared inside c namespace. &o, we have to add the above line in the top of our program. #therwise we have to append )c995 specifier before each #penC* functions and data structures. +e.g  cv--at, cv--imread+/ , etc/. If you are not familiar with namespaces, please refer this article.

,!in"l#de (iostream), and )using namespace s"d5 are added because we are using )cou"5 to display some strings in the console. 0his is basic C and you should be familiar  with this. •

Ma"

img ( int .ags9C*23#(%2I(452C#3#6

imread# const

string0 -lename,

Mat is a data structure to store images in a matrix. It is declared in )openc3core3core+$pp) header file. imread0 is a function declared in $open"%&/highg#i/highg#i'hpp$ header *le. It loads an image from a *le and stores it in at data structure.

•

•

o

o

o o

o

 (rguments of imread+/ function -lename  location of the file. If you 7ust give the filename only, that image should be in the same folder as your C file. #therwise you have to give the full path to your  image. 4lags  0here are four possible inputs

&:"%!:I!5-:; bits, no. channels9@ &:"%!:I!5-:&%"%? = image=depth98, no. of channels9A &:"%!:I!5-:! and !??. If it is $, each pixel can have a value between > to 1? +1111 in binary/. ;ere is a simple model of a image with imagedepth of @ bits. 5ach small box represents a pixel. &o, each box may contain a value bet ween > to !??.

o

;ere is some properties of the following image.  Imagedepth @ bit

o

1 channel + &o, this is a grayscale image /

o

0he height is $ pixel

o

0he width is ? pixels

o

0he resolution of this image is $x?. 0his is a grayscale image +black and white image/ because this image has no color content. If the value of this pixel is higher, it will be shown more brighter. If the value is low, it will be shown more darker.

Grayscale Image with image depth of 8

Following image is a simple model of a color image. Color image should consist of at least "

planes 6ed, 4reen and Blue. (ny color can be created using a particular combination of  these " colors. (ny pixel is a combination of three " values. +!??, >, >/ represent pure red. +>, !??, >/ represent pure green. +!??, >, !??/ represents pure violate. In the same way, you can create many color. Imagedepth is !$ because each pixel is represented with @ x " bits +@ bits from each channel/.

o

;ere is some properties of the following image.  Imagedepth !$ bit

o

" channels + &o, this is a color image /

o

0he height is $ pixel

o

0he width is ? pixels

o

0he resolution of this image is $x?.

R, G, B planes of a color image

In the above model, top left pixel is +!", !"1, $/. It will be shown as a greenish color  because the green value+!"1/ of that pixel is larger than the red+!"/ and blue+$/ value. •

i4  #img+emp"y# If imread+/ function fails to load the image, )img5 will not be loaded any data. 0herefore 5img+emp"y#) should return true. It)s a good practice to check whether the image is loaded successfully and if not exit the program. #therwise your program will crash when executing ims$o7#function.

•

:ool Ma"99emp"y#

 This function returns true, if atBBdata99  

ims$o7#)MyVideo), 4rame! 33s$o7 "$e 4rame in )MyVideo) 7indo7

i4 #7ai"ey#G ((  337ai" 4or 5esc5 6ey press 4or G ms+ I4 5esc5 6ey is pressed, :rea6 loop ' cou" 88 )esc 6ey is pressed :y user) 88 endl! :rea6! > > re"urn ! >

111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111  /ou can download this %pen& visual c++ proect from here. 2The downloaded *le is a compressed .rar folder. So, you have to extract it using 7inrar or other suitable software6 This is how I pla a %ideo #sing m OpenCV program

2ew OpenCV f#n"tions whi"h are not fo#nd earlier are eplained here •

VideoCap"ure99VideoCap"ure#cons" s"ring& 4ilename

 This is one of few constructors available in ideo&apture class. This constructor open the video *le and initialiEes the ideo&apture obect for reading the video stream from the speci*ed *le.

 The destructor of this class will deallocated any associated memory with this obect. Therefore you don't need to deallocate memory explicitly in your program. •

:ool VideoCap"ure99IsOpened# If the previous call to *ideoCapture constructor is successful, this method will return true. #therwise it will return false. It is essential to check that whether the *ideoCapture initialiDe successfully. If it is unsuccessful, program should be exited. #therwise when you try to read a frame from the *ideoCapture ob7ect, your program will crash.

•

:ool VideoCap"ure99se"#in" propId, dou:le alue 9ou can change some properties of *ideoCapture ob7ect. If it is successful, this method will return true. #therwise it will return false. 9ou should try to change some properties of the video stream in your code. In my code, I have shown you how to change the position of the video, by changing the CV3CA737RO737O53M5*C property.

o

:arameters in" propID < 0his argument specify the property you are going to change. 0here are many options for this argument. &ome of them are listed here.



CV3CA737RO737O53M5*C  current position of the video in milliseconds











o

•

CV3CA737RO737O53FRAM*5  current position of the video in frames CV3CA737RO73FRAM*31IDT  width of the frame of the video stream CV3CA737RO73FRAM*3*I9T  height of the frame of the video stream CV3CA737RO73F75  frame rate 2frames per second6 CV3CA737RO73FO4RCC = four character code of codec dou:le alue < 0his is the new value you are going to assign to the property, specified by the propI% dou:leVideoCap"ure99ge"#in" propId 0his function returns the value of the property which is specified by propId+ 9ou should try to obtain some properties of the video stream in your code. In my code, I have shown you how to obtain the frame rate +frames per second/ of the video, by using the CV.CA*.*RO*.F*@ argument.

o





:arameters  in" propID < 0his argument specify the property you are going to obtain. 0here are many options for this argument. &ome of them are listed here.

CV3CA737RO737O53M5*C  current position of the video in milliseconds CV3CA737RO737O53FRAM*5  current position of the video in frames



CV3CA737RO73FRAM*31IDT  width of the frame of the video stream CV3CA737RO73FRAM*3*I9T  height of the frame of the video stream



CV3CA737RO73F75  frame rate +frames per second/





•

CV3CA737RO73FO4RCC = four character code of codec :ool VideoCap"ure99read#Ma"& image!

 The function grabs the next frame from the video, decodes it and stores it in the 'image, variable. Inside this function, ideo&aptureBBgrap26 and ideo&aptureBBretrieve26 will be called. If you want, you can use these two functions instead of ideo&aptureBBread26 function. If the operation successful, it will return true. %therwise it will return false. •

7ai"ey#@

 The function waits for AC milliseconds. If a key was pressed before the speci*ed time, it returns the !S&II value of the pressed key. If that value is HM 2!S&II value of 'esc' key is HM6, the program will execute inside the if block. If no key is pressed during that ACms, the function returns =@ program will continue the while loop. •

VideoCap"ure99NVideoCap"ure#

estructor of ideo&apture obect will destroy any associated memory of  that particular obect. This destructor will be called implicitly on exit of the main method of the above program. 5#mmar !t *rst, this program captures a video from a *le. Then the program enters into a in*nite loop. In that loop, it grabs frames from the captured video se(uentially, decodes it, shows it in a window and waits for @ milliseconds. If the video *le has no more frames or if the user presses the 'esc' key, the program will break the in*nite loop. 2ote< ;sing waiteint0 function is very important because imshowstring;= MAT0 function need time to paint the image in the window and 7ai"ey#in" will give that necessary time.

Capt#re Video From Camera  The main diGerence of the following program from the above program is the argument to the ideo&apture constructor. )ere you ust need to give the index of your camera to the constructor of the ideo&apture obect instead of  the *lename in the above program. )ere is the sample %pen& code.

111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 !in"l#de $open"%&/highg#i/highg#i'hpp$ !in"l#de (iostream) #sing namespa"e "%> #sing namespa"e std> int mainint arg"= "harE arg%G0 H VideoCapt#re "ap0> // open the %ideo "amera no'  if   "ap'isOpened00 // if not s#""ess= eit program H "o#t (( $Cannot open the %ideo "am$  (( endl> ret#rn J> K do#le d1idth  "ap'getCV3CA737RO73FRAM*31IDT0> //get the width of frames of the %ideo do#le deight  "ap'getCV3CA737RO73FRAM*3*I9T0> //get the height of frames of the %ideo "o#t (( $Frame siNe < $ (( d1idth (( $  $ (( deight (( endl>   named1indow$MVideo$=CV31I2DO13A4TO5I6*0> //"reate a window "alled $MVideo$ while J0 H Mat frame> ool 5#""ess  "ap'readframe0> // read a new frame from %ideo if  5#""ess0 //if not s#""ess= rea: loop H "o#t (( $Cannot read a frame from %ideo stream$  (( endl> rea: > K  

imshow$MVideo$= frame0> //show the frame in $MVideo$ window

if  waite@0  &0 //wait for ,es", :e press for @ms' If ,es", :e is pressed= rea: loop H "o#t (( $es" :e is pressed  #ser$ (( endl> rea: > K K

ret#rn > K 111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111  /ou can download this %pen& visual c++ proect from here. 2The downloaded *le is a compressed .rar folder. So, you have to extract it using 7inrar or other suitable software6 T$is is $o7 I cap"ure mysel4 7i"$ my 7e:cam using "$e OpenCV program

•

2ew OpenCV f#n"tions whi"h are not fo#nd earlier are eplained here VideoCapt#re //spe"if the compression #alit

ool 5#""ess  imwrite$D in the above sample program. #ut higher the value, it will take longer time to write the image 

:84

format

 9ou have to puch2back CV3IM1RIT*37293COM7R*55IO2 first and then a number between > and  +higher is the better compression, but slower/.

 The image format is chosen depending on the *le name extension. %nly images with > bit or @ bit unsigned single channel or A channel + C*2@EC1, C*2@EC", C*2@&C1, C*2@&C", &:@;&@, &:@;&A6 with 'B9R' channel order, can be saved. If the depth or channel order of the image is diGerent, use ,Mat // open the %ideo "amera no'  if "ap'isOpened00 // if not s#""ess= eit program H "o#t (( $*RROR< Cannot open the %ideo -le$  (( endl> ret#rn J> K  named1indow$MVideo$=CV31I2DO13A4TO5I6*0>  //"reate a window "alled $MVideo$ do#le d1idth  "ap'getCV3CA737RO73FRAM*31IDT0> //get the width of frames of the %ideo do#le deight  "ap'getCV3CA737RO73FRAM*3*I9T0> //get the height of frames of the %ideo "o#t (( $Frame 5iNe  $ (( d1idth (( $$ (( deight (( endl> 5iNe frame5iNestati"3"ast(int)d1idth0= stati"3"ast(int)deight00>  Video1riter oVideo1riter $D ret#rn J> K while J0 H Mat frame> ool 5#""ess  "ap'readframe0> // read a new frame from %ideo if 5#""ess0 //if not s#""ess= rea: loop H "o#t (( $*RROR< Cannot read a frame from %ideo -le$  (( endl> rea: > K  

oVideo1riter'writeframe0> //writer the frame into the -le

 

imshow$MVideo$= frame0> //show the frame in $MVideo$ window

if waiteJ0  &0 //wait for ,es", :e press for @ms' If ,es", :e is pressed= rea: loop H "o#t (( $es" :e is pressed  #ser$ (( endl> rea: > K K ret#rn > K 111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 1111111  /ou can download this %pen& visual c++ proect from here. 2The downloaded *le is a compressed .rar folder. So, you have to extract it using 7inrar or other suitable software6

ere is how I re"ord m %ideo from m we"am and sa%e to a -le in D dri%e

•

2ew OpenCV f#n"tions whi"h are not fo#nd earlier are eplained here 5iNe frame5iNestati"3"ast(int)d1idth0= stati"3"ast(int)deight00

&reate a 5iNe obect with a given width and height. )ere I have cast the width and height to integers because they are originally double values and the 5iNe constructor does not accept double values as its parameters. •

o

o













Video1riter