Publications by Enver Sangineto

We address the problem of automatic extraction of foreground objects from videos. The goal is to provide a method for unsupervised collection of samples which can be further used for object detection training without any human intervention. We use the well known Selective Search approach to produce an initial still-image based segmentation of the video frames. This initial set of proposals is pruned and temporally extended using optical flow and transductive learning. Specifically, we propose to use Dense Trajectories in order to robustly match and track candidate boxes over different frames. The obtained box tracks are used to collect samples for unsupervised training of track-specific detectors. Finally, the detectors are run on the videos to extract the final tubes. The combination of appearance-based static ”objectness” (Selective Search), motion information (Dense Trajectories) and transductive learning (detectors are forced to ”overfit” on the unsupervised data used for training) makes the proposed approach extremely robust. We outperform state-of-the-art systems by a large margin on common benchmarks used for tube proposal evaluation.

The current state-of-the-art in video classification is based on Bag-of-Words using local visual descriptors. Most commonly these are histogram of oriented gradients (HOG), histogram of optical flow (HOF) and motion boundary histograms (MBH) descriptors. While such approach is very powerful for classification, it is also computationally expensive. This paper addresses the problem of computational efficiency. Specifically: (1) We propose several speed-ups for densely sampled HOG, HOF and MBH descriptors and release Matlab code; (2) We investigate the trade-off between accuracy and computational efficiency of descriptors in terms of frame sampling rate and type of Optical Flow method; (3) We investigate the trade-off between accuracy and computational efficiency for computing the feature vocabulary, using and comparing most of the commonly adopted vector quantization techniques: k-means, hierarchical k-means, Random Forests, Fisher Vectors and VLAD.

Two of the most important state-of-the-art challenges in face recognition are: dealing with image acquisition conditions very different between the gallery and the probe set and dealing with large datasets of individuals. In this paper we face both aspects presenting a method which is able to work in “real life” scenarios, in which face images are differently illuminated, can be partially occluded or can show different facial expressions or noise levels. Our proposed system has been tested with datasets of 1000 different individuals, showing performances usually obtained with much smaller gallery sets and much better images. The approach we propose is based on SIFT descriptors, which are known to be robust to different illumination conditions and noise levels. SIFTs are used to automatically detect face regions (mouth area, eye area, etc.). Such regions are then independently compared with the corresponding regions of the gallery images for computing a similarity-based renking of the system’s database. © 2010 Taylor & Francis Group, LLC.