Abstract: This paper proposes a novel virtual Content-Aware Network (CAN) layer as a part of a full layered architecture, focused, but not limited to, on multimedia distribution with Quality of Services (QoS) assurance. The overall system is based on a flexible cooperation between providers, operators and end-users, enabling users to access the offered multimedia services in various contexts and also to become private content providers. The paper introduces the main concepts and architecture for the main virtual network layer (i.e., CAN), exposing its role and interfaces among overall system layers. This work is a part of the starting effort inside of a new European FP7 ICT research project, ALICANTE.

Abstract: This paper introduces a prototype test-bed for triggering sensory effects like light, wind, or vibration when presenting audiovisual resources, e.g., a video, to users. The ISO/IEC MPEG is currently standardizing the Sensory Effect Description Language (SEDL) for describing such effects. This language is briefly described in the paper and the testbed that is destined to evaluate the quality of the multimedia experience of users is presented. It consists of a video annotation tool for sensory effects, a corresponding simulation tool, and a real test system. Initial experiments and results on determining the color of light effects from the video content are reported.

Abstract: Virtual worlds (often referred to as 3D3C for 3D visualization & navigation and the 3C’s of Community, Creation and Commerce) integrate existing and emerging (media) technologies (e.g. instant messaging, video, 3D, VR, AI, chat, voice, etc.) that allow for the support of existing and the development of new kinds of networked services. The emergence of virtual worlds as platforms for networked services is recognized by businesses as an important enabler as it offers the power to reshape the way companies interact with their environments (markets, customers, suppliers, creators, stakeholders, etc.) in a fashion comparable to the Internet and to allow for the development of new (breakthrough) business models, services, applications and devices. Each virtual world however has a different culture and audience making use of these specific worlds for a variety of reasons. These differences in existing Metaverses permit users to have unique experiences. In order to bridge these differences in existing and emerging Metaverses a standardized framework is required, i.e., MPEG-V Media Context and Control (ISO/IEC 23005), that will provide a lower entry level to (multiple) virtual worlds both for the provider of goods and services as well as the user. The aim of this paper is to provide an overview of MPEG-V and its intended standardization areas. Additionally, a review about MPEG-V’s most advanced part – Sensory Information – is given.

Abstract: Nowadays, mobile devices have implemented several transmission technologies which enable access to the Internet and increase the bit rate for data exchange. Despite modern mobile processors and high-resolution displays, mobile devices will never reach the stage of a powerful notebook or desktop system (for example, due to the fact of battery powered CPUs or just concerning the smallsized displays). Due to these limitations, the deliverable content for these devices should be adapted based on their capabilities including a variety of aspects (e.g., from terminal to network characteristics). These capabilities should be described in an interoperable way. In practice, however, there are many standards available and a common mapping model between these standards is not in place. Therefore, in this paper we describe such a mapping model and its implementation aspects. In particular, we focus on the whole delivery context (i.e., terminal capabilities, network characteristics, user preferences, etc.) and investigated the two most prominent state-of-the-art description schemes, namely User Agent Profile (UAProf) and Usage Environment Description (UED).

Abstract: We present a new approach for video browsing using visualization of motion direction and motion intensity statistics by color and brightness variations. Statistics are collected from motion vectors of H.264/AVC encoded video streams, so full video decoding is not required. By interpreting visualized motion patterns of video segments, users are able to quickly identify scenes similar to a prototype scene or identify potential scenes of interest. We give some examples of motion patterns with different semantic value, including camera zooms, hill jumps of ski-jumpers, and the repeated appearance of a news speaker. In a user study we show that certain scenes of interest can be found significantly faster using our video browsing tool than using a video player with VCR-like controls.

Abstract: We present a video browsing tool that uses a novel and powerful visualization technique of video motion. The tool provides an interactive navigation index that allows users to quickly and easily recognize content semantics like scenes with fast/slow motion (in general or according to a specific direction), scenes showing still/moving objects in front of a still/moving background, camera pans, or camera zooms. Moreover, the visualization facilitates identification of similar segments in a video. A first user study has shown encouraging results.

Abstract: A new approach for interactive video browsing is described. The novelty of the proposed approach is the flexible concept of interactive navigation summaries. Similar to time sliders, commonly used with standard soft video players, navigation summaries allow random access to a video. In addition, they also provide abstract visualizations of the content at a user-defined level of detail and, thus, quickly communicate content characteristics to the user. Navigation summaries can provide visual information about both low-level features but even high-level features. The concept fully integrates the user, who knows best which navigation summary at which level of detail could be most beneficial for his/her current video browsing task, and provide him/her a flexible set of navigation means. A first user study has shown that our approach can significantly outperform standard soft video players - the state-of-the art “poor man’s” video browsing tool.

Abstract: Commercial blocks provide no extra value for video indexing, retrieval, archiving, or summarization of TV broadcasts. Therefore, automatic detection of commercial blocks is an important topic in the domain of multimedia information systems. We present a commercial detection approach which is based on logo detection performed in the compressed domain. The novelty of our approach is that by taking advantage of advanced features of the H.264/AVC coding, it is both significantly faster and more exact than existing approaches working directly on compressed data. Our approach enables removal of commercials in a fraction of real-time while achieving an average recall of 97.33% with an average precision of 99.31%. Moreover, due to its run-time performance, our approach can also be employed on low performance devices, for instance DVB recorders.

Abstract: We describe an approach for viewing any large, detail-rich picture on a small display by generating a video from the image, as taken by a virtual camera moving across it at varying distance. Our main innovation is the ability to build the virtual camera's motion from a textual description of a picture, e.g., a museum caption, so that relevance and ordering of image regions are determined by co-analyzing image annotations and natural language text. Furthermore, our system arranges the resulting presentation such that it is synchronized with an audio track generated from the text by use of a text-to-speech system.

Abstract: In this paper, we present an approach for presenting large, feature-rich pictures on small displays by generating an animation and subsequently a video from the image, as it could be taken by a virtual camera moving across the image. Our main innovation is the ability to build the virtual camera's motion upon a textual description of a picture, as from a museum caption, so that relevance and ordering of image regions is determined by co-analyzing image annotations and text. Furthermore, our system can arrange the resulting presentation in a way that it is synchronized with an audio track generated from the text by use of a text-to-speech system.

Abstract: Currently, much research aims at coping with the shortcomings in multimedia consumption that may exist in a user's current context, e.g., due to the absence of appropriate devices at many locations, a lack of capabilities of mobile devices, restricted access to content, or non-personalized user interfaces. Recently, solutions to specific problems have been emerging, e.g., wireless access to multimedia repositories over standardized interfaces; however, due to usability restrictions the user has to spend much effort to or is even incapable of fulfilling his/her demands. The vision of user-centric multimedia places the user in the center of multimedia services to support his/her multimedia consumption intelligently, dealing with the aforementioned issues while minimizing required work. Essential features of such a vision are comprehensive context awareness, personalized user interfaces, and multimedia content adaptation. These aspects are addressed in this paper as major challenges toward a user-centric multimedia framework.

Abstract: MPEG-7 is an extensive multimedia metadata standard covering a huge number of aspects of metadata. However, as with most metadata standards details of usage and application of the standards are – at least partially – open to interpretation. In case of MPEG-7storage and transmission of high level metadata on concept level are defined but retrieval methods are not proposed. So if for instance a user annotates photos using the MPEG-7 semantic description scheme, there are no standardized ways to retrieve the photos based on the annotation. In this paper we propose metrics for retrieval based on the MPEG-7 semantic description scheme and evaluate them in a digital photo retrieval scenario.

Abstract: The increasing availability of short, unstructured video clips on the Web has generated an unprecedented need to organize, index, annotate and retrieve video contents to make them useful to potential viewers. This paper presents a novel, simple, and easy-to-use tool to benchmark different low level features for video summarization based on keyframe extraction. Moreover, it shows the usefulness of the benchmarking tool by developing hypothesis for a chosen domain through an exploratory study. It discusses the results of exploratory studies involving users and their judgment of what makes the summary generated by the tool a good one.

Abstract: Results of internet searches are typically presented as lists. When searching for digital photos different search result presentations however offer different benefits. If users are primarily interested in the visual content of images a thumbnail grid may be more appropriate than a list. For people searching photos taken at a specific place image metadata in the result presentation is of interest too. In this paper we present an application which monitors a user's behavior while searching for digital photos and classifies the user's intention. Based on the intention, the result is adapted to support the user in an optimal way.

Abstract: Search queries are typically interpreted as specification of information need of a user. Typically the search query is either interpreted as is or based on the context of a user, being for instance a user profile, his/her previously undertaken searches or any other background information. The actual intent of the user – the goal s/he wants to achieve with information retrieval – is an important part of a user’s context. In this paper we present the results of an exploratory study on the interplay between the goals of users and their search behavior in multimedia retrieval.

Abstract: This paper presents a novel approach that combines both in-network, application-layer adaptation and network-layer traffic control of scalable video streams based on the H.264/SVC standard. In the IPTV/VoD scenario considered, an intercepting RTSP/RTP proxy performs admission control of the requested video, based on the signaled scalability information, and decides whether the content can be streamed without changes or in an adapted version. The proxy configures the network layer appropriately in order to separate the video stream from besteffort traffic on the same link. Rather than performing fixed bandwidth allocation, our proxy approach uses the Hierarchical Token Bucket (HTB) queuing discipline to allow for borrowing bandwidth between traffic classes. In that setting, two different allocation policies are introduced. The Hard Reservation Policy (HRP) performs admission control and adaptation on the video streams and does not modify video bandwidth allocation after admission. In contrast, the Flexible Borrowing Policy (FBP) restricts the admission control to the base layer of the SVC stream. The packets carrying MGS enhancement layer data are marked with priorities by the proxy and are handled at the network layer by a priority-based queuing mechanism. Both a qualitative comparison and an experimental evaluation of the two policies are given.

Abstract: One of the most active research topics in the field of video signal processing is scalable video coding (SVC). The recently published extension of the H.264/AVC video coding standard introduces scalability features by employing a layered encoding of the video stream. In our work we investigated the usage of this scalable extension of H.264/AVC for in-network multimedia adaptation. We developed an RTSP/RTP-based proxy which exploits the layered encoding of the video and can perform real-time video adaptation on an inexpensive off-the-shelf WiFi router. This is achieved by applying a stateful, packet-based adaptation approach that keeps the computational costs at a minimum. With that approach it is possible to simultaneously adapt multiple video streams to varying network conditions or to the capabilities of the consumers' end-devices. In our demonstration we show the streaming of two scalable video streams from a server to a client and the in-network adaptation of the video at the WiFi router. The adaptation can be controlled interactively in the temporal, spatial and SNR domains.

Abstract: The streaming of multimedia content over Peer-to-Peer (P2P) networks is nowadays a well appreciated concept, as it helps distributing content to a great number of users and additionally reduces the server costs for providing the content. As the users of P2P networks often have different bandwidth connections and terminals, the same content is usually provided in different qualities. Although such a provision of the same content in different qualities helps to satisfy all users, it makes the sharing process less efficient. Users that are interested in the content in a specific quality can only exchange pieces with those users that are interested in the same content and the same quality. Thus, layered video coding, which provides different qualities within one bitstream, is especially well suited for P2P distribution. If the layered content is provided once in the best quality, all peers interested in this content can at least exchange the base layer, plus the enhancement layers they are interested in with those peers that have them available.

Abstract: This paper presents an image retrieval suite called img(Rummager) which brings into effect a number of new as well as state of the art descriptors. The application can execute an image search based on a query image, either from XML-based index files, or directly from a folder containing image files, extracting the comparison features in real time. In addition the img(Rummager) application can execute a hybrid search of images from the application server, combining keyword information and visual similarity. Also img(Rummager) supports easy retrieval evaluation based on the normalized modified retrieval rank (NMRR) and average precision (AP).

Abstract: Compact Composite Descriptors (CCD) are global image features capturing both, color and texture characteristics, at the same time in a very compact representation. In this paper we propose a combination of two recently introduced CCDs (CEDD and FCTH) into a Joint Composite Descriptor (JCD). We further present a method for descriptor selection to approach the best ANMRR that would result from CEDD and FCTH. With our approach the most appropriate descriptor in terms of maximization of information content can be found on a per image basis without knowledge of the data set as a whole. Experiments conducted on three known benchmarking image databases demonstrate the effectiveness of the proposed technique.