1 Introduction ........................................................ 1 1.1 Highlights ...................................................... 1 1.2 From Drawings to Electronic Images .............................. 2 1.3 Geometric Measurements: Gauging and Counting .................... 3 1.3.1 Size Distribution of Pigment Particles .................... 4 1.3.2 Gas Bubble Size Distributions ............................. 4 1.3.3 In Situ Microscopy of Cells in Bioreactors ................ 6 1.4 Radiometric Measurements: Revealing the Invisible ............... 8 1.4.1 Fluorescence Measurements of Concentration Fields ......... 8 1.4.2 Thermography for Botany .................................. 11 1.4.3 Imaging of Short Ocean Wind Waves ........................ 12 1.4.4 SAR Imaging for Planetology and Earth Sciences ........... 15 1.4.5 X-Ray Astronomy with ROSAT ............................... 19 1.4.6 Spectroscopic Imaging for Atmospheric Sciences ........... 19 1.5 Depth Measurements: Exploring 3-D Space ........................ 21 1.5.1 Optical Surface Profiling ................................ 21 1.5.2 3-D Retina Imaging ....................................... 24 1.5.3 Distribution of Chromosomes in Cell Nuclei ............... 25 1.5.4 X-Ray and Magnetic Resonance 3-D Imaging ................. 25 1.6 Velocity Measurements: Exploring Dynamic Processes ............. 27 1.6.1 Particle Tracking Velocimetry ............................ 27 1.6.2 3-D Flow Tomography ...................................... 28 1.6.3 Motor Proteins ........................................... 30 2 Tasks and Tools .................................................... 33 2.1 Highlights ..................................................... 33 2.2 Basic Concepts ................................................. 34 2.2.1 Goals for Applications of Image Processing ............... 34 2.2.2 Measuring versus Recognizing ............................. 36 2.2.3 Signals and Uncertainty .................................. 38 2.2.4 Representation and Algorithms ............................ 39 2.2.5 Models ................................................... 41 2.2.6 Hierarchy of Image Processing Tasks ...................... 42 2.3 Tools .......................................................... 45 2.3.1 Overview ................................................. 45 2.3.2 Camera and Frame Grabber ................................. 45 2.3.3 Computer ................................................. 46 2.3.4 Software and Algorithms .................................. 50 I From Objects to Images 3 Quantitative Visualization ......................................... 55 3.1 Highlights ..................................................... 55 3.2 Task ........................................................... 56 3.3 Concepts ....................................................... 58 3.3.1 Electromagnetic Waves .................................... 58 3.3.2 Particle Radiation ....................................... 63 3.3.3 Acoustic Waves ........................................... 64 3.3.4 Radiometric Terms ........................................ 64 3.3.5 Photometric Terms ........................................ 67 3.3.6 Surface-Related Interactions of Radiation with Matter .... 70 3.3.7 Volume-Related Interactions of Radiation with Matter ..... 76 3.4 Procedures ..................................................... 82 3.4.1 Introduction ............................................. 82 3.4.2 Types of Illumination .................................... 82 3.4.3 Illumination Techniques for Geometric Measurements ....... 84 3.4.4 Illumination Techniques for Depth Measurements ........... 86 3.4.5 Illumination Techniques for Surface Slope Measurements ... 88 3.4.6 Color and Multi-Spectral Imaging ......................... 96 3.4.7 Human Color Vision ...................................... 100 3.4.8 Thermal Imaging ......................................... 103 3.4.9 Imaging of Chemical Species and Material Properties ..... 106 3.5 Advanced Reference Material ................................... 108 3.5.1 Classification of Radiation ............................. 108 3.5.2 Radiation Sources ....................................... 110 3.5.3 Human Vision ............................................ 113 3.5.4 Selected Optical Properties ............................. 114 3.5.5 Further References ...................................... 116 4 Image Formation ................................................... 119 4.1 Highlights .................................................... 119 4.2 Task .......................................................... 120 4.3 Concepts ...................................................... 122 4.3.1 Coordinate Systems ...................................... 122 4.3.2 Geometrical Optics ...................................... 125 4.3.3 Wave Optics ............................................. 137 4.3.4 Radiometry of Imaging ................................... 140 4.3.5 Linear System Theory .................................... 143 4.4 Procedures .................................................... 147 4.4.1 Geometry of Imaging ..................................... 147 4.4.2 Stereo Imaging .......................................... 154 4.4.3 Confocal Laser Scanning Microscopy ...................... 159 4.4.4 Tomography .............................................. 161 4.5 Advanced Reference Material ................................... 163 4.5.1 Data of Optical Systems for CCD Imaging ................. 163 4.5.2 Optical Design .......................................... 166 4.5.3 Further References ...................................... 166 5 Imaging Sensors ................................................... 169 5.1 Highlights .................................................... 169 5.2 Task .......................................................... 169 5.3 Concepts ...................................................... 170 5.3.1 Overview ................................................ 170 5.3.2 Detector Performance .................................... 171 5.3.3 Quantum Detectors ....................................... 176 5.3.4 Thermal Detectors ....................................... 176 5.3.5 Imaging Detectors ....................................... 177 5.3.6 Television Video Standards .............................. 180 5.3.7 CCD Sensor Architectures ................................ 181 5.4 Procedures .................................................... 185 5.4.1 Measuring Performance Parameters of Imaging Sensors ..... 185 5.4.2 Sensor and Camera Selection ............................. 189 5.4.3 Spectral Sensitivity .................................... 191 5.4.4 Artifacts and Operation Errors .......................... 192 5.5 Advanced Reference Material ................................... 197 5.5.1 Basic Properties of Imaging Sensors ..................... 197 5.5.2 Standard Video Signals; Timing and Signal Forms ......... 199 5.5.3 Color Video Signals ..................................... 201 5.5.4 Cameras and Connectors .................................. 204 5.5.5 Further References ...................................... 205 6 Digitalization and Quantization ................................... 207 6.1 Highlights .................................................... 207 6.2 Task .......................................................... 207 6.3 Concepts ...................................................... 208 6.3.1 Digital Images .......................................... 208 6.3.2 The Sampling Theorem .................................... 213 6.3.3 Sampling Theorem in xt Space ............................ 217 6.3.4 Reconstruction from Sampling ............................ 218 6.3.5 Sampling and Subpixel Accurate Gauging .................. 220 6.3.6 Quantization ............................................ 221 6.4 Procedures .................................................... 226 6.4.1 The Transfer Function of an Image Acquisition System .... 226 6.4.2 Quality Control of Quantization ......................... 228 6.5 Advanced Reference Material ................................... 230 6.5.1 Evolution of Image Acquisition Hardware ................. 230 6.5.2 Analog Video Input ...................................... 232 6.5.3 Digital Video Input ..................................... 234 6.5.4 Real-Time Image Processing .............................. 236 6.5.5 Further References ...................................... 238 II Handling and Enhancing Images 7 Pixels ............................................................ 241 7.1 Highlights .................................................... 241 7.2 Task .......................................................... 242 7.3 Concepts ...................................................... 243 7.3.1 Random Variables and Probability Density Functions ...... 243 7.3.2 Functions of Random Variables ........................... 246 7.3.3 Multiple Random Variables and Error Propagation ......... 247 7.3.4 Homogenous Point Operations ............................. 251 7.3.5 Inhomogeneous Point Operations .......................... 252 7.3.6 Point Operations with Multichannel Images ............... 253 7.4 Procedures ................................................... 255 7.4.1 Gray Value Evaluation and Interactive Manipulation ...... 255 7.4.2 Correction of Inhomogeneous Illumination ................ 259 7.4.3 Radiometric Calibration ................................. 262 7.4.4 Noise Variance Equalization ............................. 263 7.4.5 Histogram Equalization .................................. 264 7.4.6 Noise Reduction by Image Averaging ...................... 265 7.4.7 Windowing ............................................... 266 7.5 Advanced Reference Material ................................... 267 8 Geometry .......................................................... 269 8.1 Highlights .................................................... 269 8.2 Task .......................................................... 270 8.3 Concepts ...................................................... 271 8.3.1 Geometric Transformations ............................... 271 8.3.2 Interpolation ........................................... 274 8.4 Procedures .................................................... 285 8.4.1 Scaling ................................................. 286 8.4.2 Translation ............................................. 288 8.4.3 Rotation ................................................ 288 8.4.4 Affine and Perspective Transforms ....................... 290 8.5 Advanced Reference Material ................................... 291 9 Restoration and Reconstruction .................................... 293 9.1 Highlights .................................................... 293 9.2 Task .......................................................... 294 9.3 Concepts ...................................................... 294 9.3.1 Types of Image Distortions .............................. 294 9.3.2 Defocusing and Lens Aberrations ......................... 296 9.3.3 Velocity Smearing ....................................... 297 9.3.4 Inverse Filtering ....................................... 297 9.3.5 Model-based Restoration ................................. 299 9.3.6 Radon Transform and Fourier Slice Theorem ............... 300 9.4 Procedures .................................................... 302 9.4.1 Reconstruction of Depth Maps from Focus Series .......... 302 9.4.2 3-D Reconstruction by Inverse Filtering ................. 304 9.4.3 Filtered Backprojection ................................. 308 9.5 Advanced Reference Material ................................... 311 III From Images to Features 10 Neighborhoods ..................................................... 315 10.1 Highlights ................................................... 315 10.2 Task ......................................................... 316 10.3 Concepts ..................................................... 317 10.3.1 Masks ................................................ 317 10.3.2 Operators ............................................ 319 10.3.3 Convolution .......................................... 319 10.3.4 Point Spread Function ................................ 321 10.3.5 Transfer Function .................................... 323 10.3.6 General Properties of Convolution Operators .......... 325 10.3.7 Error Propagation with Filtering ..................... 329 10.3.8 Recursive Convolution ................................ 331 10.3.9 Rank-Value Filters ................................... 337 10.3.10 Strategies for Adaptive Filtering .................... 337 10.4 Procedures ................................................... 340 10.4.1 Filter Design Criteria ............................... 340 10.4.2 Filter Design by Windowing ........................... 341 10.4.3 Recursive Filters for Image Processing ............... 344 10.4.4 Design by Filter Cascading ........................... 345 10.4.5 Efficient Computation of Neighborhood Operations ..... 347 10.4.6 Filtering at Image Borders ........................... 350 10.4.7 Test Patterns ........................................ 352 10.5 Advanced Reference Material .................................. 353 11 Regions ........................................................... 355 11.1 Highlights ................................................... 355 11.2 Task ......................................................... 356 11.3 Concepts ..................................................... 358 11.3.1 General Properties of Averaging Filters ............... 358 11.3.2 Weighted Averaging .................................... 362 11.3.3 Controlled Averaging .................................. 362 11.3.4 Steerable Averaging ................................... 365 11.3.5 Averaging in Multichannel Images ...................... 366 11.4 Procedures ................................................... 368 11.4.1 Box Filters ........................................... 368 11.4.2 Binomial Filters ...................................... 374 11.4.3 Cascaded Multistep Filters ............................ 377 11.4.4 Cascaded Multigrid Filters ............................ 380 11.4.5 Recursive Smoothing ................................... 380 11.4.6 Inhomogeneous and Anisotropic Diffusion ............... 381 11.4.7 Steerable Directional Smoothing ....................... 384 11.5 Advanced Reference Material .................................. 387 12 Edges and Lines ................................................... 391 12.1 Highlights ................................................... 391 12.2 Task ......................................................... 391 12.3 Concepts ..................................................... 392 12.3.1 Edge Models ........................................... 392 12.3.2 Principal Methods for Edge Detection .................. 394 12.3.3 General Properties .................................... 397 12.3.4 Edges in Multichannel Images .......................... 399 12.3.5 Regularized Edge Detection ............................ 401 12.4 Procedures ................................................... 403 12.4.1 First-Order Derivation ................................ 403 12.4.2 Second-Order Derivation ............................... 410 12.4.3 Regularized Edge Detectors ............................ 413 12.4.4 LoG and DoG Filter .................................... 415 12.4.5 Optimized Regularized Edge Detectors .................. 416 12.5 Advanced Reference Material .................................. 417 13 Orientation and Velocity .......................................... 419 13.1 Highlights ................................................... 419 13.2 Task ......................................................... 420 13.3 Concepts ..................................................... 421 13.3.1 Simple Neighborhoods .................................. 421 13.3.2 Classification of Local Structures .................... 425 13.3.3 First-Order Tensor Representation ..................... 428 13.4 Procedures ................................................... 430 13.4.1 Set of Directional Quadrature Filters ................. 430 13.4.2 2-D Tensor Method ..................................... 433 13.4.3 Motion Analysis in Space-Time Images .................. 439 13.5 Advanced Reference Material .................................. 442 14 Scale and Texture ................................................. 443 14.1 Highlights ................................................... 443 14.2 Task ......................................................... 444 14.3 Concepts ..................................................... 446 14.3.1 What Is Texture? ...................................... 446 14.3.2 The Wave Number Domain ................................ 450 14.3.3 Hierarchy of Scales ................................... 451 14.3.4 Gaussian Pyramid ...................................... 454 14.3.5 Laplacian Pyramid ..................................... 457 14.3.6 Directio-Pyramidal Decomposition ...................... 459 14.3.7 Phase and Local Wave Number ........................... 460 14.4 Procedures ................................................... 465 14.4.1 Texture Energy ........................................ 465 14.4.2 Phase Determination ................................... 467 14.4.3 Local Wave Number ..................................... 469 14.5 Advanced Reference Material .................................. 472 IV From Features to Objects 15 Segmentation ...................................................... 475 15.1 Highlights ................................................... 475 15.2 Task ......................................................... 475 15.3 Concepts ..................................................... 476 15.3.1 Pixel-Based Segmentation .............................. 476 15.3.2 Region-Based Segmentation ............................. 477 15.3.3 Edge-Based Segmentation ............................... 478 15.3.4 Model-Based Segmentation .............................. 478 15.4 Procedures ................................................... 480 15.4.1 Global Thresholding ................................... 480 15.4.2 Pyramid Linking ....................................... 481 15.4.3 Orientation-Based Fast Hough Transformation ........... 484 15.5 Advanced Reference Material .................................. 485 16 Size and Shape .................................................... 487 16.1 Highlights ................................................... 487 16.2 Task ......................................................... 487 16.3 Concepts ..................................................... 488 16.3.1 Morphological Operators ............................... 488 16.3.2 Run-Length Code ....................................... 493 16.3.3 Chain Code ............................................ 494 16.3.4 Fourier Descriptors ................................... 496 16.3.5 Moments ............................................... 499 16.4 Procedures ................................................... 501 16.4.1 Object Shape Manipulation ............................. 501 16.4.2 Extraction of Object Boundaries ....................... 503 16.4.3 Basic Shape Parameters ................................ 504 16.4.4 Scale and Rotation Invariant Shape Parameters ......... 506 16.5 Advanced Reference Material .................................. 507 17 Classification .................................................... 509 17.1 Highlights ................................................... 509 17.2 Task ......................................................... 509 17.3 Concepts ..................................................... 510 17.3.1 Statistical Decision Theory ........................... 510 17.3.2 Model Optimization and Validation ..................... 511 17.4 Procedures ................................................... 513 17.4.1 Linear Discriminant Analysis (LDA) .................... 513 17.4.2 Quadratic Discriminant Analysis (QDA) ................. 516 17.4.3 k-Nearest Neighbors (k-NN) ............................ 517 17.4.4 Cross-Validation ...................................... 518 17.5 Advanced Reference Material .................................. 519 V Appendices A Notation ........................................................... 523 A.1 General ........................................................ 523 A.2 Image Operators ................................................ 524 A.3 Alphabetical List of Symbols and Constants ..................... 525 B Mathematical Toolbox ............................................... 529 B.1 Matrix Algebra ................................................. 529 B.1.1 Vectors and Matrices ..................................... 529 B.1.2 Operations with Vectors and Matrices ..................... 529 B.1.3 Types of Matrices ........................................ 530 B.2 Least-Squares Solution of Linear Equation Systems .............. 530 B.3 Fourier Transform .............................................. 532 B.3.1 Definition ............................................... 532 B.3.2 Properties of the Fourier Transform ...................... 533 B.3.3 Important Fourier Transform Pairs ........................ 534 B.4 Discrete Fourier Transform (DFT) ............................... 534 B.4.1 Definition ............................................... 534 B.4.2 Important Properties ..................................... 535 B.4.3 Important Transform Pairs ................................ 535 B.5 Suggested Further Readings ..................................... 536 C Glossary ........................................................... 537 Bibliography ....................................................... 569 Index .............................................................. 576 D Color Plates ....................................................... 585