Abstract: Medical case retrieval (MCR) is defined as a multimedia retrieval problem, where the document collection consists of medical case descriptions that pertain to particular diseases, patients' histories, or other entities of biomedical knowledge. Case descriptions are multimedia documents containing textual and visual modalities (images). A query may consist of a textual description of patient's symptoms and related diagnostic images. This thesis proposes and evaluates methods that aim at improving MCR effectiveness over the baseline of fulltext retrieval. We hypothesize that this objective can be achieved by utilizing controlled vocabularies of biomedical concepts for query expansion and concept-based retrieval. The latter represents case descriptions and queries as vectors of biomedical concepts, which may be generated automatically from textual and/or visual modalities by concept mapping algorithms. We propose a multimodal retrieval framework for MCR by late fusion of text-based retrieval (including query expansion) and concept-based retrieval and show that retrieval effectiveness can be improved by 49% using linear fusion of practical component retrieval systems. The potential of further improvement is experimentally estimated as a 166% increase of effectiveness over fulltext retrieval using query-adaptive fusion of ideal component retrieval systems. Additional contributions of this thesis include the proposal and comparative evaluation of methods for concept mapping, query and document expansion, and automatic classification and separation of compound figures found in case descriptions.

Abstract: Real-time entertainment (mainly audio/video streaming) is responsible for the largest traffic share in today's networks. Social and entertainment platforms such as YouTube, Netflix and Facebook provide a tremendous amount of multimedia content to their global customers via the Internet. With the ever growing popularity of these services, the Internet is struggling to suffice the continuously increasing requirements demanded by applications. In particular, the demands go far beyond the intent of the Internet's original design. Architectural and legacy design choices lead to issues, the solutions to which are neither efficient nor elegant. One approach to tackle these challenges is Information-Centric Networking (ICN), a new concept for today's Internet. The idea is to base the network's principal communication model on the most important item, namely the content to be transferred. This novel concept provides significant opportunities to enhance networking. In this thesis we investigate how ICN can be used as an enabler for effective multimedia dissemination. As a first step we analyse the technology's characteristic capabilities and their potential benefits for content distribution in future networks. We develop an analytical model taking account of the main building blocks (network-inherent caching, multi-path forwarding) and compare the obtained upper bound to the current state of ICN considering the scenario of pull-based adaptive multimedia streaming. The results show that there exists a significant gap between the promised and the realized performance, largely caused by ineffective Interest forwarding strategies. Therefore, we design and implement a novel probability-based forwarding strategy named Stochastic Adaptive Forwarding (SAF), which provides effective multi-path forwarding, identifies unknown cached content replicas and deals with local topology changes without guidance from the routing plane. The results indicate that SAF brings ICN one step closer towards effective content distribution. In particular, we show that it is important to consider context information in the forwarding plane. This includes content characteristics and application demands. SAF is the first strategy that takes account of context information that can be supplied by the network operators. Furthermore, this work provides a framework for a testbed that can be used by researchers to readily deploy an ICN-based testbed. This allows researchers to conduct experiments on physical hardware providing deeper insights on proposed algorithms than network simulations or analytical methods could ever do. We use the testbed to validate our results concerning multimedia delivery in ICN, and conduct network emulations investigating the performance of SAF and its competitors. Furthermore, we compare the results of network emulations to the findings obtained from simulations to assess their validity. Both simulations and emulations show that our SAF approach provides a significant step towards effective multimedia content distribution in ICN.

Abstract: Multimedia content is omnipresent in our life. Thus, one can consume content through various distribution channels such as a DVD, Blu-Ray, or the Internet. Recently, 3D video gained more and more importance and a lot of movies presented in cinemas are 3D. Currently, research on additional constituents such as light and scent effects for further enhancing the viewing experience is conducted. As this research is taken up by more and more researchers and companies, the Moving Picture Experts Group (MPEG) ratified the MPEG-V standard, referred to as Media Context and Control, which allows the annotation of multimedia content with additional effects (e.g., light, wind, vibration) and render these effects synchronized to the multimedia content. Due to this fairly new research area, there are only a few subjective quality assessments evaluating such effects. Moreover, standardized assessment methods cannot be used as originally developed since they are optimized for audio-visual quality evaluations. Thus, this work lists and describes existing subjective quality assessment methods suitable for conducting assessments comprising multimedia content, especially videos, enriched by sensory effects (i.e., light, wind, and vibration). As there is a lack of suitable software for rendering sensory effects, this work introduces a multimedia player for playing multimedia content accompanied by sensory effects. Moreover, in this work, we performed four subjective quality assessments answering the following questions: (1) Do sensory effects enhance the viewing experience for different genres? (2) Do sensory effects have an influence on the perceived video quality? (3) Do light effects enhance the viewing experience for Web videos? (4) Do sensory effects have an impact on the perceived emotions while watching a video? Therefore, this work presents these subjective quality assessments including a detailed description of the assessments and their results. Moreover, this work introduces a dataset consisting of video sequences annotated with sensory effects for conducting subjective quality assessments. Finally, some recommendations for performing assessments comprising sensory effects which have been extracted from the conducted subjective quality assessments are given.

Abstract: How to build better multimedia information systems? Management and organization of multimedia data has become easier thanks to the wide availability of metadata as well as advances in content-based image retrieval (CBIR); these advances, however, do not address what matters the most: the actual users of multimedia information systems. The goals and aims of users, i.e., their intentions, need to be put into focus in some creative way. This talk introduces the very notion of user intentions in multimedia information systems and discusses challenges and approaches in this new topic.

Abstract: On TV screens, PCs, tablets, and mobile phones, video streaming has become a constant companion in our daily lives. For every video, we expect high visual quality, free from distortions, that is adjusted to the device at hand. But how can streaming systems cope with the increasing network traffic, the subsequent network congestions, and the different characteristics of end-user terminals? This thesis covers approaches for distributed adaptation of scalable video resources in media delivery. Scalable video resources consist of several layers that enable various spatial resolutions, frame rates, or qualities of a content. By dropping some of these layers, the video can be adjusted to the available bandwidth or to a specific end-user terminal. The adaptation can be performed on the sender side, on the receiver side, and on one or more network nodes. Scalable media coding can also help to reduce bandwidth requirements in multicast scenarios (e.g., for IPTV). One popular realization of scalable media coding is the Scalable Video Coding (SVC) standard. This thesis consists of three main parts, addressing various challenges towards efficient SVC adaptation. The first part of this thesis focuses on the encoding of SVC. In order to enable efficient adaptation, the configuration of layers has to be carefully chosen at encoding time. Thus, the performances of various encoding configurations and encoder implementations are evaluated. Furthermore, encoding guidelines for SVC are developed, which are aligned with recommendations of industry streaming solutions. The evaluation results of the developed SVC encoding guidelines suggest that quality scalability should be preferred over spatial scalability for adaptive streaming scenarios. Different resolutions for supporting device classes should rather be provided as separate SVC streams. The second part of this thesis deals with the fact that scalable media formats, such as SVC, are still not widely adopted neither on the sender side nor on the end-user terminal. In order to enable the deployment of SVC for network transmission and to improve the support for streaming to heterogeneous devices, the concept of SVC tunneling is introduced in this thesis. The video is transcoded to SVC at the sender side and then transcoded back to another video format at the receiver side at an advanced home-gateway. However, the transcoding between video formats has a negative impact on the video quality. The trade-off between quality loss and bandwidth efficiency of SVC tunneling is evaluated. SVC tunneling with quality layers enables bandwidth savings at moderate quality loss (approx. 2.5 dB) compared to streaming separate non-scalable representations of the same qualities. In the third part of this thesis, adaptation techniques for content-aware networks are investigated. In content-aware networks, some network nodes are capable to dynamically adapt video streams in reaction to varying network loads. With the increasing adoption of HTTP streaming, adaptation at the client side becomes a main factor for the viewing experience. The switch between two representations (e.g., different bitrates) of a video can disrupt that viewing experience. To reduce the effect of an abrupt quality change, the approach of a smooth transition between representations is developed and evaluated. A subjective user study indicates that this approach can indeed improve the overall viewing quality. Finally, the findings of the previous parts are integrated in an adaptive end-to-end SVC streaming system. Evaluations of this streaming system show that the developed adaptation framework significantly improves the video quality under packet loss (by up to 6 dB) compared to non-adaptive streaming.

Abstract: Internet video streaming is a hot topic in multimedia systems. A large variety of devices (computers, mobile phones, TVs, etc.) are connected to the Internet via wired or wireless networks and are capable of receiving and decoding HD video content. To enable new services like HD video streaming (e.g., online video rental), the Internet’s infrastructure was enhanced. But the Internet is still a best-effort network, which does not implement quality-of-service or admission control, resulting in time-varying bandwidth and packet delay, packet loss and network congestion. Because video streaming accounts for a considerable amount of the Internet’s traffic, video streaming needs additionally to be congestion-aware, to avoid a congestion collapse of the Internet. The Transmission Control Protocol (TCP) can adapt to changing network conditions and is currently the de facto standard protocol for congestion-aware and reliable data transmission in the Internet. This fact gave TCP-based video streaming a huge momentum. Consequently, this thesis investigates TCP-based adaptive video streaming for the Internet. The main goal is to provide a solution for congestion-aware video streaming, while still being able to achieve a reasonable performance in error-prone networks. To complement existing work on congestion-aware adaptive streaming, this thesis makes six contributions. (1) The baseline performance of TCP-based adaptive streaming is identified by means of an evaluation of different adaptive streaming approaches. The results represent a reference for further investigations. (2) An investigation on the influence of TCP’s behavior in presence of packet loss on the video streaming performance. (3) To overcome the shortcomings of TCP-based video streaming (single TCP connections fail to deliver a good performance in case of packet loss), a new approach to video streaming based on multiple request-response streams was introduced. The novelty of this system is that it is able to make use of multiple HTTP-based request-response streams while still providing TCP-friendliness. (4) A performance model of the HTTP-based request-response streams was developed, to estimate the influence of the system parameters and the network characteristics on the throughput performance. (5) A comprehensive evaluation of the HTTP-based request-response streams under diverse network conditions was conducted, to validate the model’s estimations. Additionally, the TCP-friendliness was evaluated, showing that request-response streaming systems can be configured to achieve TCP-friendliness. (6) A cellular network with high bandwidth fluctuations and RTTs was used to investigate the performance of the request-response streaming system in a mobile video streaming scenario. The results indicate that the streaming system can make good use of the available bandwidth, while the number of quality switches is kept low. While aggregating multiple TCP connections to improve the TCP streaming performance is quite common, usually the improvement comes at the cost of high deployment effort. By placing the streaming logic at the client, request-response streams can avoid this complexity. Additionally, this client-driven approach responds faster to changing network conditions and enables easy recovery from connection stalls or aborts, because the control loop is at the client. To improve the network efficiency and the scalability in terms of number of clients served, HTTP-based request-response streams can utilize HTTP proxies and caches.

Abstract: This thesis discusses three major issues that arise in the context of non-sequential usage of multimedia content, i.e. a usage, where users only access content that is interesting for them. These issues are (1) semantically meaningful segmentation of videos, (2) composition of new video streams with content from different sources and (3) non-sequential presentation of multimedia content. A semantically meaningful segmentation of videos can be achieved by partitioning a video into scenes. This thesis gives a comprehensive survey of scene segmentation approaches, which were published in the last decade. The presented approaches are categorized based on the underlying mechanisms used for the segmentation. The characteristics that are common for each category as well as the strengths and weaknesses of the presented algorithms are stated. Additionally, an own scene segmentation approach for sports videos with special properties is introduced. Scenes are extracted based on recurring patterns in the motion information of a video stream. Furthermore, different approaches in the context of real-life events are presented for the composition of new video streams based on content from multiple sources. Community-contributed photos and videos are used to generate video summaries of social events. The evaluation shows that by using content provided by a crowd of people a new and richer view of an event can be created. This thesis introduces a new concept for this emerging view, which is called ``The Vision of Crowds''. The presentation of such newly, composed video streams is described with a simple but powerful formalism. It provides a great flexibility in defining the temporal and spatial arrangement of content. Additionally, a video browsing application for the hierarchical, non-sequential exploration of video content is introduced. It is able to interpret the formal description of compositions and can be adapted for different purposes with plug-ins.

Abstract: This thesis investigates mechanisms and applications for in-network adaptation of scalable video bit streams based on the recent H.264/Scalable Video Coding (SVC) standard. In-network adaptation refers to the adaptation of a video stream by a network element during the stream's transport through the network. The advantages of performing adaptation directly in the network are the availability of local monitoring data and a higher responsiveness according to the current networking conditions. In contrast to previous work in this field, this thesis focuses on the feasibility and realization of in-network adaptation on existing home router platforms. In this context this thesis addresses the following six research objectives. Initially, the relevant transport mechanisms for H.264/SVC and their implications on in-network adaptation (1) were analysed. In the context of this work three different Linux-based router platforms which cover a representative range of residential router devices were used as a basis for further studies and evaluations. In general these platforms can be characterized by rather modest processing capabilities and networking performance. The hardware limitations were identified and quantified in evaluations (2) using both different benchmarks and real network traffic. The offered processing power and memory throughput are roughly 10 to 100 times lower than those of a modern desktop PC. Although their application-layer networking performance is not that low, all platforms fail in fully utilizing their nominal link capacities of 100 and 1000 Mbps, respectively. Based on the known limitations the thesis proposes a stateful, packet-based adaptation mechanism for adapting scalable video bit streams (3). The approach utilizes the RTP payload format for H.264/SVC and represents a light-weight approach for in-network adaptation on the application layer. It further meets the important requirements towards a media-aware network element (MANE) to be signaling aware and to operate statefully. The mechanism was integrated in a proxy service which was deployed on all of the three platforms to prove its feasibility. Experimental evaluations with different video bit streams in standard-definition quality demonstrate the scalability of the approach (4). The results indicate that the proxy service is able to adapt up to 16 concurrent video streams depending on the platform and video bit stream. On two of the three evaluated platforms the proposed approach even allows to handle and to adapt video streams in high-definition quality at bit rates around 15 Mbps. In addition to the proposed H.264/SVC-specific adaptation mechanism, also the applicability of generic metadata-driven adaptation on home router platforms was investigated. In particular, a proof-of-concept study of an XML-metadata-driven approach based on the MPEG-21 generic Bitstream Syntax Description (gBSD) was conducted on the platforms (5). In contrast to former evaluations that have been done on PC-based platforms, the obtained results indicate that the use of this generic adaptation cannot be recommended on such resource limited network devices. The benefits of using in-network adaptation on home router platforms are finally demonstrated in the context of high-definition streaming over IEEE 802.11 wireless networks (6). Monitoring information regarding the queueing delay, which is obviously available exclusively on the router, is used to control the adaptation of the video according to the varying throughput of the wireless link. This allows to react timely to changing conditions particularly in the case of mobile clients.