Remco Veltkamp:
Multimedia Retrieval Algorithmics

After text retrieval, the next waves in web searching and multimedia retrieval are the search for and delivery of images, music, video, and 3D scenes. Not only the perceptual and cognitive aspects, but also many of the algorithmic and performance aspects are still badly understood.

We will discuss a number of perceptual issues in visual and auditory sensing, in particular the Gestalt rules. This is followed by a discussion on multimedia retrieval based on matching of Gestalt, or shape information, within images, notated music, and three-dimensional scenes. One relevant issue is the design of dissimilarity measures (distance functions) that have desired properties. Another aspect is the development of algorithms that can compute or approximate these distances efficiently. Indexing data structures and search algorithms are necessary to make the search more efficient than sequential browsing through large collections.

Apart from provable properties of individual algorithms, the experimental verification of the performance of a complete retrieval system is important to analyze merits and drawbacks of certain approaches, and to compare various techniques. We will discuss a number of performance measures, and look at a number of benchmarks in various multimedia retrieval domains.