Image and Video Processing with MATLAB.pdf

Image and Video Processing Using MATLAB

April 12, 2011 Rochester, NY

© 2011 The MathWorks, Inc.1

Agenda

9:00 a.m.

Welcome and MathWorks Overview

9:15 a.m.

Image Processing with MATLAB

10:30 a.m.

Break

10:45 a.m.

Video Processing with MATLAB

11:30 a.m.

System Design and Hardware Targeting

12:00 p.m.

Summary and Wrap-up

2

From MathWorks 

Sr. Account Manager – Ed Donovan – [email protected] – 508-647-4251



Application Engineer – Stephanie Kwan

3

Agenda

9:00 a.m.

Welcome and MathWorks Overview

9:15 a.m.

Image Processing with MATLAB

10:30 a.m.

Break

10:45 a.m.

Video Processing with MATLAB

11:30 a.m.

System Design and Hardware Targeting

12:00 p.m.

Summary and Wrap-up

4

Where is image processing used?

5

Common Image Processing Tasks

       

Perform segmentation Deblur and remove noise Measure image features Perform edge detection Apply morphological operations Enhance image contrast Perform image registration Apply transforms

6

Common Image Processing Challenges

     

Reading and writing to various file formats Visualizing images and intermediate results Testing algorithms with what-if scenarios Identifying causes of algorithm failure Processing large images with limited memory Executing algorithms faster

7

Technical Computing Tasks Access

Explore and Discover

Files Data Analysis and Modeling

Software

Algorithm Development

Share Reporting and Documentation

Outputs for Design

Code and Applications

Hardware

Application Development

Deployment

Automate 8

MATLAB – A Platform for Technical Computing Access

Explore and Discover

Files Data Analysis and Modeling

Software

Algorithm Development

Share Reporting and Documentation

Outputs for Design

Code and Applications

Hardware

Application Development

Deployment

Automate 9

Core MathWorks Products

The leading environment for technical computing 

  



The industry-standard, high-level programming language for algorithm development Numeric computation Data analysis and visualization Toolboxes for signal and image processing, statistics, optimization, symbolic math, and other areas Foundation of MathWorks products

10

Image Processing Toolbox Perform image processing, analysis, visualization, and algorithm development 

  

  

Image analysis Image enhancement Spatial transformation Image registration Morphological operations ROI-based processing Image display and exploration

11

Demo: Camera Pipeline From Sensor Data to Image

Scene

Image

12

Demo: Camera Pipeline

1. Noise Reduction 2. Demosaic

3. Tone Mapping 4. White Balance 5. Gamma Correction

13

Image Acquisition Toolbox 

Acquire images and video directly into MATLAB and Simulink



Configure device properties



Perform background acquisition



Synchronize multimodal devices



Configure, acquire, and preview live video data using a graphical interface

14

Image Acquisition Toolbox Hardware Support 

Industry standard support: – Frame grabbers 

Analog



Camera Link

– DCAM compatible FireWire (IIDC 1394) – GigE Vision – Common OS webcam interfaces 

Operating system support: – Windows – Linux – Macintosh 15

Image Acquisition Toolbox Hardware Support 

Manufacturers include: – Allied Vision Technologies

– Basler – Baumer – DALSA – FLIR

– Hamamatsu – Lumenera – Matrox Imaging – National Instruments

– PixeLINK – Point Grey – Qimaging – Sony



See Supported Hardware Pages for more information 16

Where is Large Imagery Used?       

Satellite imagery Aerial surveys Super-resolution Image sequences and stacks Volumetric data Multispectral and hyperspectral Mosaics and panoramic imagery

17

Common Large Image Challenges

 

“Out of memory” errors Slow processing

18

Some Steps to Handle Large Images 

Try memory command to see available space



Preallocate space for variables Minimize numerical data types Load small portions of an image Visualize only what is necessary Incorporate more hardware resources

   

Can lead to complex code 19

blockproc and rsetwrite



Block processing – Automatically divides an image into blocks for processing – Reduces memory usage – Processes arbitrarily large images



Reduced resolution data set – Avoids memory demands in visualizing large images

20

Demo: Cell Segmentation with blockproc

21

Improving Performance 

MATLAB – Preallocate space for variables – Identify bottlenecks with Profiler – Vectorize code



Image Processing Toolbox – Many toolbox functions are faster as of R2010a



Parallel Computing Toolbox – Utilize multicore processors, clusters, and GPUs

22

Parallel Computing Toolbox Going Beyond Serial MATLAB Applications

Worker

Worker Worker

TOOLBOXES

BLOCKSETS

Worker Worker

Worker Worker

Worker

23

Parallel Computing enables you to …

Larger Compute Pool

Speed up Computations

Larger Memory Pool

Work with Large Data 11

26

41

12

27

42

13

28

43

14

29

44

15

30

45

16

31

46

17

32

47

17

33

48

19

34

49

20

35

50

21

36

51

22

37

52

24

Processes

Distributing Tasks (Task Parallel)

Time

Time 25

Demo: Camera Pipeline and Parallel Computing

26

Parallel Computing Toolbox Support for GPUs 

Automatically use GPU’s with supported built-in MATLAB functions – conv2, filter2



Run supported MATLAB code on the GPU – arrayfun

 

Create kernals from existing CUDA code and PTX files Run kernals on the GPU from MATLAB

27

Why would you want to use a GPU?  

Speed up execution of computationally intensive simulations For example: – Performance: A\b with Double Precision

28

Why use MATLAB for Image Processing?

  

  



Read and write many image file formats Visualize and explore images interactively Connect directly to cameras and frame grabbers Use a large library of built-in functions Quickly build custom image processing algorithms Block-process large images to avoid memory issues Process images faster with multiple cores and clusters

29

Agenda

9:00 a.m.

Welcome and MathWorks Overview

9:15 a.m.

Image Processing with MATLAB

10:30 a.m.

Break

10:45 a.m.

Video Processing with MATLAB

11:30 a.m.

System Design and Hardware Targeting

12:00 p.m.

Summary and Wrap-up

30

Agenda

9:00 a.m.

Welcome and MathWorks Overview

9:15 a.m.

Image Processing with MATLAB

10:30 a.m.

Break

10:45 a.m.

Video Processing with MATLAB

11:30 a.m.

System Design and Hardware Targeting

12:00 p.m.

Summary and Wrap-up

31

Common Video Processing Tasks

  

  

Convert between color spaces Enhance video contrast Perform segmentation De-interlace video frames Calculate running statistics Overlay text and graphics

32

Common Video Processing Challenges

     

Reading and writing to various file formats Managing buffers and intermediate storage Testing video algorithms with test cases Accessing standard video algorithms Visualizing results and intermediate steps Displaying text and graphics

33

Demo: Working with Video in MATLAB 

Read in and write out data using VideoReader and VideoWriter



What happens if the video file is very long or the frame size is large?

34

System Objects  

MATLAB objects that represent algorithms and I/O capabilities that process data frame by frame Instantiate and configure –



Execute within a loop –



reader = vision.MultimediaFileReader(‘viptraffic.avi’) step(reader)

Standard API – – –

reset isDone close

35

Video Processing in MATLAB myVid = VideoReader(‘myvideofile.avi’); numFrames = myVid.NumberOfFrames; numIter = 10; opticalFlowIn = zeros([size(currentFrame) 5]); opticalFlowOutput = zeros([size(currentFrame) numFrames]); i = 1; Explicit state management while i 400 MATLAB operators and functions • > 200 System objects for • Signal processing • Communications • Computer vision

52

Demo: Camera Pipeline Revisited 

Generate C code from MATLAB code

53

Targeting DSPs and FPGAs MATLAB and Simulink • Algorithm development • Debugging and profiling • System design Fixed-Point Modeling Generate code

Verify design Link to Embedded Software

Integrated Development Environment • Compiler • Build automation tools • Debugger

DSP/ FPGA 54

Demo: Camera Pipeline System Design From Sensor Data to Image

Scene

Image

55

Why use MATLAB and Simulink for embedded hardware design?       

Automatically convert MATLAB to C code Reduce the number of versions of code Simplify the design process Reuse MATLAB and legacy code in models Convert floating-point algorithms to fixed point Optimize algorithms for DSPs and FPGAs Test and validate an algorithm’s implementation

56

Agenda

9:00 a.m.

Welcome and MathWorks Overview

9:15 a.m.

Image Processing with MATLAB

10:30 a.m.

Break

10:45 a.m.

Video Processing with MATLAB

11:30 a.m.

System Design and Hardware Targeting

12:00 p.m.

Summary and Wrap-up

57

What You’ve Seen Today MATLAB for image and video processing  Image Processing Toolbox – Visualize, analyze, and process images 

Image Acquisition Toolbox – Test image processing algorithms with live data



Computer Vision System Toolbox – Visualize, analyze, and process video – Motion estimation, tracking, and stereo vision



Parallel Computing Toolbox – Process images faster using multiple core and clusters



MATLAB Coder – Convert MATLAB code to C/C++ code 58

Doheny Eye Institute Develops Next-Generation of Retinal Prosthesis with MathWorks Tools

Challenge Develop next-generation, higher-resolution retinal prostheses

Solution

Illustration of a retinal prosthesis prototype.

Use MathWorks tools to develop, simulate, and automatically generate code for real-time image processing algorithms

“With Video and Image Processing

Results

Blockset and Target Support

 Development time reduced from months to weeks  DSP deployment streamlined  Patient testing improved

Package, we rapidly prototype our image and video processing algorithms on the DM642 board. This

can save me days or weeks of time.” Neha Parikh Doheny Eye Institute

59

Given Imaging Develops Camera-in-aCapsule Using MATLAB to Improve the Diagnosis of Gastrointestinal Disorders

Challenge Create an alternative to endoscopy and other invasive gastrointestinal imaging procedures

Solution

The PillCam SB 2 video capsule.

Use MATLAB and companion toolboxes to develop and implement a swallowable video capsule

Results  Fast, efficient development  Easy access to precise diagnostic information  Improved patient care

“With MATLAB, we simulated the intended system and fine-tuned it at the early stages of implementation, enabling us to develop critical engineering programs that met requirements on the first iteration.” Rafi Nave Given Imaging

Link to user story

60

Additional Articles and User Stories

Newsletter: Robot Soccer

User Story: Dutch Epilepsy Clinics Foundation

Newsletter: JPL Mars rover

Newsletter: Parking assistant at BMW 61

From a Leading Textbook Author ... "I have used a number of commercial image processing packages over the years, and prefer the MathWorks Image Processing Toolbox for several reasons: the wide variety of functions it provides, the user’s ability to write additional functions with minimal effort, the quality of the software, and the high level of support." Rafael C. Gonzalez Professor Emeritus University of Tennessee 62

Support and Community

63

Consulting Services Accelerating return on investment A global team of experts supporting every stage of tool and process integration

Continuous Improvement

Process and Technology Automation Process and Technology Standardization Full Application Deployment

Process Assessment Component Deployment Advisory Services

Jumpstart Migration Planning

Research

Advanced Engineering

Product Engineering Teams

Supplier Involvement 64

Training Services Exploit the full potential of MathWorks products

Flexible delivery options: 

  

Public training available worldwide Onsite training with standard or customized courses Web-based training with live, interactive instructor-led courses Self-paced interactive online training

More than 30 course offerings: 



Introductory and intermediate training on MATLAB, Simulink, Stateflow, code generation, and Polyspace products Specialized courses in control design, signal processing, parallel computing, code generation, communications, financial analysis, and other areas 65

MATLAB Central 

 

Community for MATLAB and Simulink users Over 1 million visits per month File Exchange –

– –



Newsgroup – –



Upload/download access to free files including MATLAB code, Simulink models, and documents Ability to rate files, comment, and ask questions More than 12,500 contributed files, 300 submissions per month, 50,000 downloads per month

Web forum for technical discussions about MathWorks products More than 300 posts per day

Blogs – –

Based on February 2011

Commentary from engineers who design, build, and support MathWorks products Open conversation at blogs.mathworks.com 66

Connections Program More than 400 add-on products and services that complement and extend MathWorks products: 

  

Specialized third-party toolboxes for MATLAB Interfaces to third-party software and hardware products Specialized training courses and consulting services System integrators and suppliers that incorporate MathWorks products

67

Book Program More than 1200 books for educational and professional use, in 26 languages      

Controls Signal processing Image processing Biosciences Communications Mechanical engineering

     

Mathematics Aerospace engineering Environmental sciences Chemistry Finance Electronics

68

Technical Support

Resources 



Over 100 support engineers – All with MS degrees (EE, ME, CS) – Local support in North America, Europe, and Asia Comprehensive, product-specific Web support resources

High customer satisfaction   

95% of calls answered within three minutes 70% of issues resolved within 24 hours 80% of customers surveyed rate satisfaction at 80–100%

 







69

For More Information   

Experiment with product by downloading a trial Peruse videos, webinars, user stories, and demos online Contact us – Talk to a sales representative to get answers to your questions – Discuss your projects with MathWorks applications engineers

70