Semi-automatic semantic video indexing and retrieval

Abstract

As the use of digital video increases, so does the need to provide effective management and access to such data. To achieve this we can create annotations that describe the content. This can be done manually, automatically or assisted by a professional indexer. Manual annotation of video is a very time-consuming task and different indexers are likely to use different terminology, which leads to inconsistencies. Low-level features, such as color and texture, can be automatically extracted from video data without user intervention. However, they are not sufficient to index video at a higher-level. Semantic video indexing is a promising approach to enable semi-automatic video annotation and semantic video retrieval via keywords. In order to express semantics, we need to use prior knowledge about the domain of the video in the indexing process. Semantic information cannot be extracted automatically; human intervention is unavoidable. For this reason, we should see indexing as a collaborative process between the user and the system and design interactive indexing systems. We have designed and developed an innovative end-to-end semantic video indexing and search tool. Our tool can automatically extract keywords from the video and uses ontologies to represent domain knowledge in the system. The indexed video is semantically enhanced by mapping the extracted keywords to a set of ontology concepts organized in a hierarchy. We have also developed a search tool that allows users to efficiently search and retrieve the indexed video presentations.