Neural Computation, Vol 9, 777-804, Copyright © 1997 by The MIT Press

ARTICLES

SEEMORE: combining color, shape, and texture histogramming in a neurally inspired approach to visual object recognition

BW Mel
Department of Biomedical Engineering, University of Southern California, Los Angeles 90089, USA.

Severe architectural and timing constraints within the primate visual system support the conjecture that the early phase of object recognition in the brain is based on a feedforward feature-extraction hierarchy. To assess the plausibility of this conjecture in an engineering context, a difficult three-dimensional object recognition domain was developed to challenge a pure feedforward, receptive-field- based recognition model called SEEMORE. SEEMORE is based on 102 viewpoint-invariant nonlinear filters that as a group are sensitive to contour, texture, and color cues. The visual domains consists of 100 real objects of many different types, including rigid (shovel), nonrigid (telephone cord), and statistical (maple leaf cluster) objects and photographs of complex scenes. Objects were individually presented in color video images under normal room lighting conditions. Based on 12 to 36 training views, SEEMORE was required to recognize unnormalized test views of objects that could vary in position, orientation in the image plane and in depth, and scale (factor of 2); for nonrigid objects, recognition was also tested under gross shape deformations. Correct classification performance on a test set consisting of 600 novel object views was 97 percent (chance was 1 percent) and was comparable for the subset of 15 nonrigid objects. Performance was also measured under a variety of image degradation conditions, including partial occlusion, limited clutter, color shift, and additive noise. Generalization behavior and classification errors illustrated the emergence of several striking natural shape categories that are not explicitly encoded in the dimensions of the feature space. It is concluded that in the light of the vast hardware resources available in the ventral stream of the primate visual system relative to those exercised here, the appealingly simple feature-space conjecture remains worthy of serious consideration as a neurobiological model.

This article has been cited by other articles:

				M. Kouh and T. Poggio A Canonical Neural Circuit for Cortical Nonlinear Operations Neural Comput., June 1, 2008; 20(6): 1427 - 1451. [Abstract] [Full Text] [PDF]

				C. Cadieu, M. Kouh, A. Pasupathy, C. E. Connor, M. Riesenhuber, and T. Poggio A Model of V4 Shape Selectivity and Invariance J Neurophysiol, September 1, 2007; 98(3): 1733 - 1750. [Abstract] [Full Text] [PDF]

				T. Serre, A. Oliva, and T. Poggio A feedforward architecture accounts for rapid categorization PNAS, April 10, 2007; 104(15): 6424 - 6429. [Abstract] [Full Text] [PDF]

				J. S. Cant and M. A. Goodale Attention to Form or Surface Properties Modulates Different Regions of Human Occipitotemporal Cortex Cereb Cortex, March 1, 2007; 17(3): 713 - 731. [Abstract] [Full Text] [PDF]

				C. Eckes, J. Triesch, and C. v. d. Malsburg Analysis of cluttered scenes using an elastic matching approach for stereo images. Neural Comput., June 1, 2006; 18(6): 1441 - 1471. [Abstract] [Full Text] [PDF]

				A. B. A. Graf*, F. A. Wichmann, H. H. Bulthoff, and B. Scholkopf Classification of Faces in Man and Machine Neural Comput., January 1, 2005; 18(1): 143 - 165. [Abstract] [Full Text] [PDF]

				H. Wersing and E. Korner Learning Optimized Features for Hierarchical Models of Invariant Object Recognition Neural Comput., July 1, 2003; 15(7): 1559 - 1588. [Abstract] [Full Text] [PDF]

				J. J. DiCarlo and J. H. R. Maunsell Anterior Inferotemporal Neurons of Monkeys Engaged in Object Recognition Can be Highly Sensitive to Object Retinal Position J Neurophysiol, June 1, 2003; 89(6): 3264 - 3278. [Abstract] [Full Text] [PDF]

				J. Fiser and R. N. Aslin From the Cover: Statistical learning of new visual feature combinations by infants PNAS, November 26, 2002; 99(24): 15822 - 15826. [Abstract] [Full Text] [PDF]

				S. O. Murray, D. Kersten, B. A. Olshausen, P. Schrater, and D. L. Woods Shape perception reduces activity in human primary visual cortex PNAS, November 12, 2002; 99(23): 15164 - 15169. [Abstract] [Full Text] [PDF]

				D. Roth, M.-H. Yang, and N. Ahuja Learning to Recognize Three-Dimensional Objects Neural Comput., May 1, 2002; 14(5): 1071 - 1103. [Abstract] [Full Text]

				B. W. Mel and J. Fiser Minimizing Binding Errors Using Learned Conjunctive Features Neural Comput., April 1, 2000; 12(4): 731 - 762. [Abstract] [Full Text]

				B. W. Mel and J. Fiser Minimizing Binding Errors Using Learned Conjunctive Features Neural Comput., February 1, 2000; 12(2): 247 - 278. [Abstract] [Full Text]