19.3 Image Differencing, Motion Segmentation and Filtering Techniques

19.3.1 Consecutive Image Differencing Techniques

Seyler, A.J.,
Real-Time Recording of Television Frame Difference Areas,
PIEEE(51), No. 3, March 1963, pp. 478-480. BibRef 6303

Seyler, A.J.,
Statistics of Television Frame Differences,
PIEEE(53), No. 12, December 1965, pp. 2127-2128. BibRef 6512

Candy, J.C., Franke, M.A., Haskell, R.G., Mounts, F.W.,
Transmitting Television As Clusters of Frame-to-Frame Differences,
Bell System Tech.(50), No. 6, July/August 1971, pp. 1889-1919. BibRef 7107

Onoe, M., Hamano, N., Ohba, K.,
Computer Analysis of Traffic Flow Observed by Subtractive Television,
CGIP(2), 1973, pp. 377-392. BibRef 7300

Onoe, M., Saito, M.,
Automatic Threshold Setting for the Sequential Similarity Detection Algorithm,
TC(25), 1976, pp. 1052-1053. (Or 24 in 1975??) BibRef 7600

Nagel, H.H.,
Formation of an Object Concept by Analysis of Systematic Time Variations in the Optically Perceptible Environment,
CGIP(7), No. 2, April 1978, pp. 149-194.
Elsevier DOI Motion, Differencing. Find simple objects that are moving smoothly in front of a contrasting background. This is the basic original paper for differencing for object recognition. BibRef 7804

Nagel, H.H.,
Representation of Moving Rigid Objects Based on Visual Observations,
Computer(14), No. 8, August 1981, pp. 29-39. Basic outline of the Hamburg work - derive 3-D descriptions from a sequence of 2-D images - derive a series of possible 3-D objects from sets of 2-D (each 2-D gives a partial 3-D object). BibRef 8108

Hsu, Y.Z., Nagel, H.H., and Rekers, G.,
New Likelihood Test Methods for Change Detection in Image Sequences,
CVGIP(26), No. 1, April 1984, pp. 73-106.
Elsevier DOI The image is modeled as patches with intensity determined by a polynomial of the pixel coordinates. The difference between successive images is computed using the modeled images. This eliminates much of the noise associated with straight forward differencing. BibRef 8404

Yalamanchili, S., Martin, W.N., Aggarwal, J.K.,
Extraction of Moving Object Descriptions via Differencing,
CGIP(18), No. 2, February 1982, pp. 188-201.
Elsevier DOI BibRef 8202
Earlier:
Differencing Operations for the Segmentation of Moving Objects in Dynamic Scenes,
ICPR80(1239-1242). BibRef

Yalamanchili, S., Aggarwal, J.K.,
Motion and Image Differencing,
PRIP81(211-216). BibRef 8100

Yoda, H.[Haruo], Motoike, J.[Jun],
Visual information processing apparatus,
US_Patent4,346,405, 08/24/1982.
HTML Version. BibRef 8208
Earlier:
Image data processor,
US_Patent4,254,400, 03/03/1981.
HTML Version. Frame to frame change detection. BibRef

Jain, R.C.[Ramesh C.],
Difference and Accumulative Difference Pictures in Dynamic Scene Analysis,
IVC(2), No. 2, May 1984, pp. 99-108.
Elsevier DOI BibRef 8405

Knoll, T.F., Delp, E.J.,
Adaptive Gray Scale Mapping to Reduce Registration Noise in Difference Images,
CVGIP(33), No. 2, February 1986, pp. 129-137.
Elsevier DOI BibRef 8602

Egawa, A.[Akira],
Image blur display device,
US_Patent5,060,007, Oct 22, 1991
WWW Link. BibRef 9110

Lo, T.K.[Thomas K.], Sacks, J.M.[Jack M.], Banh, N.D.[Nam D.],
Segmentation method for use against moving objects,
US_Patent5,109,435, 04/28/1992.
HTML Version. Based on background. BibRef 9204

Lo, T.K.[Thomas K.], Sacks, J.M.[Jack M.], Banh, N.D.[Nam D.],
Signal processing for autonomous acquisition of objects in cluttered background,
US_Patent4,937,878, Jun 26, 1990
WWW Link. BibRef 9006

Aschwanden, F.[Felix], Bart, T.E.[Theodor E.],
T.V. motion detector with false alarm immunity,
US_Patent4,894,716, Jan 16, 1990
WWW Link. BibRef 9001

Banh, N.D.[Nam D.], Lo, T.K.[Thomas K.], Holthaus, K.D.[Kelly D.], Sacks, J.M.[Jack M.],
Moving target detection method using two-frame subtraction and a two quadrant multiplier,
US_Patent5,150,426, 09/22/1992.
HTML Version. BibRef 9209

Abe, S.[Shozo],
Apparatus for extracting/combining change region in image corresponding to moving object,
US_Patent5,099,324, 03/24/1992.
HTML Version. BibRef 9203

Abe, S.[Shozo], Togashi, Y.[Yuichi], Ohata, H.[Hajime],
Moving object detection apparatus and method,
US_Patent5,177,794, Jan 5, 1993
WWW Link. BibRef 9301
And: A1, only: US_Patent5,134,472, Jul 28, 1992
WWW Link. BibRef

Kitazato, N.[Naohisa],
Motion detecting circuit for video image,
US_Patent5,173,771, Dec 22, 1992
WWW Link. BibRef 9212

Westberg, L.,
Hierarchical Contour-Based Segmentation of Dynamic Scenes,
PAMI(14), No. 9, September 1992, pp. 946-952.
IEEE DOI Assume one coherent moving object on the background, use pyramid based technique and boundaries. Build on temporal frame differences, detect, object, background, boundary regions. BibRef 9209

Bergen, J.R., Burt, P.J., Hingorani, R., and Peleg, S.,
A Three-Frame Algorithm for Estimating Two-Component Image Motion,
PAMI(14), No. 9, September 1992, pp. 886-896.
IEEE DOI BibRef 9209
Earlier:
Computing Two Motions from Three Frames,
ICCV90(27-32).
IEEE DOI Motion, Three frames. Two components are background motion and single object motion. Find the background motion and use it to detect the other moving objects by simple differencing. BibRef

Burt, P.J., Hingorani, R., and Kolczynski, R.J.,
Mechanisms for Isolating Component Patterns in the Sequential Analysis of Multiple Motion,
Motion91(187-193). Region by region motion estimation to find single motions, use for stabilization. BibRef 9100

Burt, P.J.[Peter J.],
Video technique for indicating moving objects from a movable platform,
US_Patent5,473,364, Dec 5, 1995
WWW Link. BibRef 9512

Burt, P.J., Bergen, J.R., Hingorani, R., Kolczynski, R.J., Lee, W.A., Leung, A., Lubin, J., and Shvaytser, H.,
Object Tracking with a Moving Camera,
Motion89(2-12). Image differencing with global tracking to get moving objects separately. BibRef 8900

Sauer, K.[Ken], Jones, C.[Coleen],
Bayesian Block-Wise Segmentation of Interframe Differences in Video Sequences,
GMIP(55), No. 2, March 1993, pp. 129-yy. BibRef 9303

Rathi, R.P.[Rajendra P.],
Method and apparatus for monitoring traffic flow,
US_Patent5,296,852, 03/22/1994.
HTML Version. Vehicle where difference between image and reference exceeds threshold. BibRef 9403

Hong, S.P.[Sam P.],
Motion detecting apparatus,
US_Patent5,339,104, Aug 16, 1994
WWW Link. BibRef 9408

Bichsel, M.[Martin],
Segmenting Simply Connected Moving-Objects In A Static Scene,
PAMI(16), No. 11, November 1994, pp. 1138-1142.
IEEE DOI BibRef 9411
Earlier:
Illumination Invariant Motion Segmentation of Simple Connected Objects,
BMVC94(xx-yy).
PDF File. 9409
Uses object-background probability and connectedness. BibRef

Ransford, G.A.[Gary A.], Cambridge, V.J.[Vivien J.],
Digital data registration and differencing compression system,
US_Patent5,490,221, Feb 6, 1996
WWW Link. BibRef 9602

Florent, R.[Raoul],
Device for the detection of objects in a sequence of images,
US_Patent5,583,947, 12/10/1996.
HTML Version. BibRef 9612
Earlier:
Method and device for use in detecting moving targets,
US_Patent5,406,501, 04/11/1995,
HTML Version. differences of registered images. BibRef

Fan, J.P., Wang, R., Zhang, L.M., Xing, D.J., Gan, F.X.,
Image Sequence Segmentation Based on 2D Temporal Entropic Thresholding,
PRL(17), No. 10, September 2 1996, pp. 1101-1107. Frame Difference Contrast, Local Variance Contrast. BibRef 9609

Ohki, M.[Mitsuharu], Igarashi, K.[Katsuji],
Motion detecting apparatus,
US_Patent5,586,202, Dec 17, 1996
WWW Link. BibRef 9612

Lee, S.M.[Sang M.], Jeong, J.H.[Joo H.], Ahn, C.T.[Chie T.],
Image signal transmitting system using image frames differences,
US_Patent5,612,744, Mar 18, 1997
WWW Link. BibRef 9703

Ghali, A., Daemi, M.F., Alkhateeb, K.A.,
Information-Based Image Dissimilarity Measure,
OptEng(37), No. 3, March 1998, pp. 808-812. 9804
BibRef

Ghali, A., Daemi, M.F., Mansour, M.,
Image Structural Information Assessment,
PRL(19), No. 5-6, April 1998, pp. 447-453. 9808
BibRef

Ghali, A., Daemi, M.F.,
Information-based shape description with scale, translation and rotation invariance,
ICIP96(III: 611-614).
IEEE DOI 9610
BibRef
And:
Recognition Information,
ICPR96(I: 544-548).
IEEE DOI 9608
(CIMI, UK) BibRef

Morimura, A.[Atsushi], Azuma, T.[Takeo],
Motion and disparity estimation method, image synthesis method, and apparatus for implementing same methods,
US_Patent5,768,404, Jun 16, 1998
WWW Link. BibRef 9806
And: US_Patent6,215,899, Apr 10, 2001
WWW Link. BibRef

Azuma, T.[Takeo], Nobori, K.[Kunio], Uomori, K.[Kenya], Morimura, A.[Atsushi],
Image signal coding method, image signal coding apparatus and storage medium,
US_Patent7,016,411, Mar 21, 2006
WWW Link. Coding a sptite image using depth image and image. BibRef 0603

Ng, H.L.[Hak-Leong],
Apparatus and method for detecting motion in a video signal,
US_Patent5,731,832, Mar 24, 1998
WWW Link. BibRef 9803

Yoon, S.C., Ratakonda, K., Ahuja, N.,
Low Bit-Rate Video Coding with Implicit Multiscale Segmentation,
CirSysVideo(9), No. 7, October 1999, pp. 1115.
IEEE Top Reference.
See also Lossless image compression with multiscale segmentation. BibRef 9910

Ratakonda, K., Yoon, S.C., and Ahuja, N.,
Coding the Displaced Frame Difference for Video Compression,
ICIP97(I: 353-356).
IEEE DOI BibRef 9700

Ratakonda, K.[Krishna], Ahuja, N.,
Segmentation Based Reversible Image Compression,
ICIP96(I: 81-84).
IEEE DOI BibRef 9600

Yoon, S.C., Ratakonda, K., and Ahuja, N.,
Region-Based Video Coding Using a Multiscale Image Segmentation,
ICIP97(II: 510-513).
IEEE DOI BibRef 9700

Le Gouzouguec, A.[Anne], Schlossers, C.[Christophe],
Procedure and device for detecting the movement of a target and their applications,
US_Patent5,883,969, Mar 16, 1999
WWW Link. Use differential iamge. BibRef 9903

Meyer, M.[Michael], Hoetter, M.[Michael], Rottmann, F.[Frank],
Method of detecting moving objects in chronologically successive images,
US_Patent6,069,918, May 30, 2000
WWW Link. BibRef 0005

Luo, J.B.[Jie-Bo], Gray, R.T.[Robert T.],
Method and system for locating objects in an image,
US_Patent6,072,893, Jun 6, 2000
WWW Link. BibRef 0006

Ito, W.[Wataru], Ueda, H.[Hirotada], Okada, T.[Toshimichi],
Method and system monitoring video image by updating template image,
US_Patent6,108,033, Aug 22, 2000
WWW Link. BibRef 0008

Pucker, II, L.G.[Leonard G.], Sofge, D.B.[David B.],
Device for and method of detecting motion in an image,
US_Patent6,298,144, Oct 2, 2001
WWW Link. BibRef 0110

Sawhney, H.S.[Harpreet S.], Guo, Y.L.[Yan-Lin], Kumar, R.[Rakesh],
Independent Motion Detection in 3D Scenes,
PAMI(22), No. 10, October 2000, pp. 1191-1199.
IEEE DOI 0011
BibRef
Earlier: Add A3: Asmuth, J., ICCV99(612-619).
IEEE DOI Tracking for surveillance. An image differencing method. BibRef

Rafanelli, G.L.[Gerard L.], Mount, S.B.[Susan B.], Johnson, S.K.[Stephen K.], Sperka, M.A.[Marilyn A.], Jensen, E.B.[Eric B.], Rehfield, M.J.[Mark J.],
Moving object and transient event detection using rotation strip aperture image measurements,
US_Patent6,192,322, Feb 20, 2001
WWW Link. BibRef 0102

Lanini, N.[Nicola],
Method for tracking moving object by means of specific characteristics,
US_Patent6,496,592, Dec 17, 2002
WWW Link. BibRef 0212

Niemann, J.C.[James C.],
Intelligent image recording system and method,
US_Patent6,591,006, Jul 8, 2003
WWW Link. BibRef 0307

Sigel, K.[Kirk], de Angelis, D.[Douglas], Ciholas, M.[Mike],
Camera with object recognition/data output,
US_Patent6,545,705, Apr 8, 2003
WWW Link. BibRef 0304

Kondo, T.[Tetsujiro], Tatehira, Y.S.[Yasu-Shi], Uchida, M.[Masashi], Asakura, N.[Nobuyuki], Morimura, T.[Takuo], Ando, K.[Kazutaka], Nakaya, H.[Hideo], Watanabe, T.[Tsutomu], Inoue, S.[Satoshi], Niitsuma, W.[Wataru],
Motion determining apparatus, method thereof, and picture information converting apparatus,
US_Patent6,597,737, Jul 22, 2003
WWW Link. BibRef 0307

Chen, H.P.[Hsiao-Ping],
Method for detecting moving objects by comparing video images,
US_Patent6,954,225, Oct 11, 2005
WWW Link. BibRef 0510

Hui, K.C.[Ko-Cheung], Siu, W.C.[Wan-Chi],
Extended Analysis of Motion-Compensated Frame Difference for Block-Based Motion Prediction Error,
IP(16), No. 5, May 2007, pp. 1232-1245.
IEEE DOI 0704
BibRef

Krishna, M.T.G.[M. T. Gopala], Ravishankar, M., Babu, D.R.R.[D.R. Ramesh], Aradhya, V.N.M.[V.N. Manjunath],
SiMOR: Single Moving Object Recognition,
IJIS(20), No. 1, 2011.
DOI Link 1103
Pre publication info, check exact issue, page reference. BibRef

Uttam, S., Goodman, N.A., Neifeld, M.A.,
Feature-Specific Difference Imaging,
IP(21), No. 2, February 2012, pp. 638-652.
IEEE DOI 1201
BibRef

Jampana, P.[Phanindra], Shah, S.[Sirish],
An image differencing method for interface level detection in separation cells,
MVA(23), No. 2, March 2012, pp. 283-298.
WWW Link. 1202
BibRef

Lissner, I., Preiss, J., Urban, P., Lichtenauer, M.S., Zolliker, P.,
Image-Difference Prediction: From Grayscale to Color,
IP(22), No. 2, February 2013, pp. 435-446.
IEEE DOI 1302
BibRef

Le Moan, S., Urban, P.,
Image-Difference Prediction: From Color to Spectral,
IP(23), No. 5, May 2014, pp. 2058-2068.
IEEE DOI 1405
hyperspectral imaging BibRef

Sowmyayani, S., Rani, P.A.J.[P. Arockia Jansi],
Frame differencing-based segmentation for low bit rate video codec using H.264,
IJCVR(6), No. 1-2, 2016, pp. 41-53.
DOI Link 1601
BibRef

Goshvarpour, A.[Atefeh], Abbasi, A.[Ataollah], Sengar, S.S.[Sandeep Singh], Mukhopadhyay, S.[Susanta],
Moving object detection based on frame difference and W4,
SIViP(11), No. 7, October 2017, pp. 1357-1364.
WWW Link. 1708
BibRef

Fan, H.Q.[Hong-Qi], Kucner, T.P.[Tomasz Piotr], Magnusson, M.[Martin], Li, T.C.[Tian-Cheng], Lilienthal, A.J.[Achim J.],
A Dual PHD Filter for Effective Occupancy Filtering in a Highly Dynamic Environment,
ITS(19), No. 9, September 2018, pp. 2977-2993.
IEEE DOI 1809
DPHD: Dual Probability Hypothesis Density. Vehicle dynamics, Uncertainty, Computational modeling, Monitoring, Adaptation models, Bayes methods, Radio frequency, Mobile robot, random finite set BibRef

Zhao, Q.[Qi], Asif, M.S.[M. Salman], Ma, Z.[Zhan],
DNeRV: Modeling Inherent Dynamics via Difference Neural Representation for Videos,
CVPR23(2031-2040)
IEEE DOI 2309
BibRef

Singh, R.[Rajhans], Shukla, A.[Ankita], Turaga, P.[Pavan],
Polynomial Implicit Neural Representations for Large Diverse Datasets,
CVPR23(2041-2051)
IEEE DOI 2309
BibRef

Parger, M.[Mathias], Tang, C.C.[Cheng-Cheng], Twigg, C.D.[Christopher D.], Keskin, C.[Cem], Wang, R.[Robert], Steinberger, M.[Markus],
DeltaCNN: End-to-End CNN Inference of Sparse Frame Differences in Videos,
CVPR22(12487-12496)
IEEE DOI 2210
Redundancy, Memory management, Graphics processing units, Coherence, Streaming media, Hardware, Real-time systems, Machine learning BibRef

Ellenfeld, M.[Marc], Moosbauer, S.[Sebastian], Cardenes, R.[Ruben], Klauck, U.[Ulrich], Teutsch, M.[Michael],
Deep Fusion of Appearance and Frame Differencing for Motion Segmentation,
PBVS21(4334-4344)
IEEE DOI 2109
Image segmentation, Fuses, Motion segmentation, Cameras BibRef

Luo, X., Jia, K., Liu, P., Xiong, D., Tian, X.,
Improved Three-Frame-Difference Algorithm for Infrared Moving Target,
ICIVC20(108-112)
IEEE DOI 2009
History, Object detection, Image resolution, Monitoring, Optical flow, Gaussian mixture model, infrared, history location data BibRef

Yoon, J.S.[Jae Shin], Rameau, F.[Francois], Kim, J.[Junsik], Lee, S.J.[Seok-Ju], Shin, S.H.[Seung-Hak], Kweon, I.S.[In So],
Pixel-Level Matching for Video Object Segmentation Using Convolutional Neural Networks,
ICCV17(2186-2195)
IEEE DOI 1802
convolution, data compression, feature extraction, image matching, image segmentation, image sequences, neural nets, video coding, Visualization BibRef

Jayaraman, D.[Dinesh], Grauman, K.[Kristen],
Slow and Steady Feature Analysis: Higher Order Temporal Coherence in Video,
CVPR16(3852-3861)
IEEE DOI 1612
BibRef

Taylor, B.[Brian], Karasev, V.[Vasiliy], Soattoc, S.[Stefano],
Causal video object segmentation from persistence of occlusions,
CVPR15(4268-4276)
IEEE DOI 1510
BibRef

Rekik, W., Le Hegarat-Mascle, S., Andre, C., Kallel, A., Reynaud, R., Ben Hamidd, A.,
Object reconstruction in an image based on belief function representation,
ICIP14(1633-1637)
IEEE DOI 1502
Feature extraction BibRef

Yang, M.Y.[Michael Ying], Rosenhahn, B.[Bodo],
Video segmentation with joint object and trajectory labeling,
WACV14(831-838)
IEEE DOI 1406
Computer vision BibRef

Scheuermann, B.[Björn], Gkoutelitsas, S.[Sotirios], Rosenhahn, B.[Bodo],
Multi-sensor Fusion Using Dempster's Theory of Evidence for Video Segmentation,
CIARP13(II:431-438).
Springer DOI 1311
BibRef

Dragon, R.[Ralf], Rosenhahn, B.[Bodo], Ostermann, J.[Jörn],
Multi-scale Clustering of Frame-to-Frame Correspondences for Motion Segmentation,
ECCV12(II: 445-458).
Springer DOI 1210
BibRef

Zhang, B.F.[Bao-Feng], Zhou, J.[Jie], Zhu, J.C.[Jun-Chao],
Research on three image difference algorithm,
IASP10(603-606).
IEEE DOI 1004
Rather than motion detection with a pair, use 3 images. BibRef

Wang, J.M.[Jung-Ming], Cherng, S.[Shen], Fuh, C.S.[Chiou-Shann], Chen, S.W.[Sei-Wang],
Foreground Object Detection Using Two Successive Images,
AVSBS08(301-306).
IEEE DOI 0809

See also Commentary Paper on Foreground Object Detection Using Two Successive Images. BibRef

Salgian, A.[Andrea],
Commentary Paper on 'Foreground Object Detection Using Two Successive Images',
AVSBS08(307-308).
IEEE DOI 0809

See also Foreground Object Detection Using Two Successive Images. BibRef

Lee, M.J.[Michelle J.], Lee, A.S.[Alexander S.], Lee, D.K.[D. Kyungsuk], Lee, S.Y.[Soo-Young],
Video Representation with Dynamic Features from Multi-Frame Frame-Difference Images,
Motion07(28-28).
IEEE DOI 0702
BibRef

Migliore, D.A.[Davide A.], Matteucci, M.[Matteo], Naccari, M.[Matteo],
A revaluation of frame difference in fast and robust motion detection,
VSSN06(215-218).
WWW Link. 0701
BibRef

Archetti, F.[Francesco], Manfredotti, C.E.[Cristina E.], Messina, V.[Vincenzina], Sorrenti, D.G.[Domenico G.],
Foreground-to-Ghost Discrimination in Single-Difference Pre-processing,
ACIVS06(263-274).
Springer DOI 0609
frame differencing, false foregrounds. BibRef

Sangi, P., Heikkila, J., Silven, O.,
Motion analysis using frame differences with spatial gradient measures,
ICPR04(IV: 733-736).
IEEE DOI 0409
BibRef

Caplier, A., Bonnaud, L., Chassery, J.M.,
Robust Fast Extraction of Video Objects Combining Frame Differences and Adaptive Reference Image,
ICIP01(II: 785-788).
IEEE DOI 0108
BibRef

Kurianski, A.[Adam], Nieniewski, M.[Mariusz],
Hidden MRF detection of motion of objects with uniform brightness,
CIAP95(656-662).
Springer DOI 9509
BibRef

Shio, A., and Sklansky, J.,
Segmentation of People in Motion,
Motion91(325-332). Get background via a mode filter over the sequence, then differences between frame and background to get the moving people. BibRef 9100

Bhat, K.S.[Kiran S.], Saptharishi, M.[Mahesh], Khosla, P.K.[Pradeep K.],
Motion Detection and Segmentation Using Image Mosaics,
ICME00(WP6). 0007
BibRef

Singer, S., and Huberman, B.A.,
Concurrent, Fault Tolerant Detection of 2-D Motion,
Draft1990. Doesn't say much more than a detector array with differences and tracking the differences of neighbors. BibRef 9000

Chapter on Motion Analysis -- Low-Level, Image Level Analysis, Mosaic Generation, Super Resolution, Shape from Motion continues in
Event Camera .