Partners:
IDIAP
EPFL
HEVs


Context and Goals:

IM2.IP encompasses work from the former "Leading House" IP and the IM2.DS Database IP. Its purpose is to support the different research projects to build on each other's systems and data to produce integrated results.

Activities of this IP are in four main areas: corpus collection, the media file server, database, and browsing


Corpus Collection:

An initial set of scripted meeting were collected at IDIAP, and a larger, more realistic corpus is being constructed in collaboration with the AMI project. Approximately 100 meeting recordings are planned, along with speech transcriptions and other annotations.


Multimodal Media File Server:

The MMM server is a repository for recorded media data. Media can be retrieved for off-line processing or retrieved as streaming media by playback by media browsers. It is used to store and distribute IM2 media and annotation data for common data corpora. The project requires hardware resources for large disks, ongoing administration (e.g. data updates), and also software development to support browsing and playback tools.


Annotation Database:

Annotations from diverse recognition processes are inserted into an SQL Database so that they can be retrieved in a consistent manner by browsers and other programs that need to combine multiple annotation streams. Work in this area falls into two categories: research, and development.

Database Research Schema Evolution Research, by Hassina Bounif, PhD student Annotation Query Research, by Oleksandr Drutskyy, PhD student

Database Development Protocol/API specification is defined as a collaborative effort led by IDIAP but aiming for consensus between IDIAP, UniGE, EPFL, and Fribourg on a system that can be used by all. An initial implementation has been developed by HEVS


Meeting browser:

Work on meeting browsers includes a multi-phase, multi-pronged implementation effort, both using the web platform, and also a Java-based framework called JFerret.

In addition, a Browser Evaluation Test (BET) is being applied based on "observations of interest" as determined by independent human observers, and browser performance is measured by testing human subjects' ability to find the maximum number of observations of interest in the minimum amount of time.