Abstract: This chapter introduces the concept of Sensory Experience which aims to define the Quality of Experience (QoE) going beyond audio-visual content. In particular, we show how to utilize sensory effects such as ambient light, scent, wind, or vibration as additional dimensions contributing to the quality of the user experience. Therefore, we utilize a standardized representation format for sensory effects that are attached to traditional multimedia resources such as audio, video, and image contents. Sensory effects are rendered on special devices (e.g., fans, lights, motion chair, scent emitter) in synchronization with the traditional multimedia resources and shall stimulate also other senses than hearing and seeing with the intention to increase the Quality of Experience (QoE), in this context referred to as Sensory Experience.

Abstract: Lately, 3D is gaining momentum in cinemas and home environments. However, 2D and 3D video content only stimulates senses like hearing and seeing. In this paper we focus on a more enhanced level of entertainment by presenting a 4D multimedia player and a corresponding demonstration setup, which stimulates further senses such as haptics using the MPEG-V: Media Context and Control standard. The presented demonstration setup uses stereoscopic 3D and sensory devices, i.e., fans, vibration panels and lights. The combination of conventional 3D content with tailored sensory effects allows us to further enhance the viewing experience of the users.

Abstract: Multimedia content is omnipresent in our life. Thus, one can consume content through various distribution channels such as a DVD, Blu-Ray, or the Internet. Recently, 3D video gained more and more importance and a lot of movies presented in cinemas are 3D. Currently, research on additional constituents such as light and scent effects for further enhancing the viewing experience is conducted. As this research is taken up by more and more researchers and companies, the Moving Picture Experts Group (MPEG) ratified the MPEG-V standard, referred to as Media Context and Control, which allows the annotation of multimedia content with additional effects (e.g., light, wind, vibration) and render these effects synchronized to the multimedia content. Due to this fairly new research area, there are only a few subjective quality assessments evaluating such effects. Moreover, standardized assessment methods cannot be used as originally developed since they are optimized for audio-visual quality evaluations. Thus, this work lists and describes existing subjective quality assessment methods suitable for conducting assessments comprising multimedia content, especially videos, enriched by sensory effects (i.e., light, wind, and vibration). As there is a lack of suitable software for rendering sensory effects, this work introduces a multimedia player for playing multimedia content accompanied by sensory effects. Moreover, in this work, we performed four subjective quality assessments answering the following questions: (1) Do sensory effects enhance the viewing experience for different genres? (2) Do sensory effects have an influence on the perceived video quality? (3) Do light effects enhance the viewing experience for Web videos? (4) Do sensory effects have an impact on the perceived emotions while watching a video? Therefore, this work presents these subjective quality assessments including a detailed description of the assessments and their results. Moreover, this work introduces a dataset consisting of video sequences annotated with sensory effects for conducting subjective quality assessments. Finally, some recommendations for performing assessments comprising sensory effects which have been extracted from the conducted subjective quality assessments are given.

Abstract: This paper provides an overview of our research conducted in the area of Sensory Experience including our implementations using MPEG-V Part 3 entitled ”Sensory Information”. MPEG-V Part 3 introduces Sensory Experience as a tool to increase the Quality of Experience by annotating traditional multimedia data with sensory effects. These sensory effects are rendered on special devices like fans, vibration chairs, ambient lights, scent disposers, water sprayers, or heating/cooling devices stimulating senses beyond the traditional ones. The paper's main focus is on the end-to-end aspects including the generation, transmission, and synchronized rendering of sensory effects with the traditional multimedia data taking movie clips as an example. Therefore, we present in this paper an open source tool chain that provides a complete end-to-end sensory effect generation and consumption framework. Furthermore, we summarize results from various subjective quality assessments conducted in this area. Finally, we point out research challenges that may encourage further research within this emerging domain.

Abstract: Enriching multimedia with additional effects such as olfaction, light, wind, or vibration is gaining more and more momentum in both research and industry. Hence, there is the need to determine the influence of individual effects on the Quality of Experience (QoE). In this paper, we present a subjective quality assessment using the MPEG-V standard to annotate video sequences with individual sensory effects (i.e., wind, light, and vibration) and all combinations thereof. Based on the results we derive a utility model for sensory experience that accounts for the assessed sensory effects. Finally, we provide an example instantiation of the utility model and validate it against current and past results of our subjective quality assessments conducted so far.

Abstract: Selecting and adopting the appropriate assessment method for conducting subjective quality assessments is a challenging task. The method decides whether the assessment is successful in delivering the correct answers to previously set up hypotheses. Therefore, in this paper we provide recommendations on test methods used in the domain of Sensory Experience. The proposed test methods comprise single stimulus and double stimulus methods. These test methods were used in previous studies and are presented in combination with the results of the subjective quality assessments with which they were used. Furthermore, we briefly outline our test setup, test design, and test content for assessing Sensory Experience which have been validated through conducted assessments.

Abstract: Preparing and conducting subjective quality assessments is a time consuming and expensive task. Therefore, we present a Web-based evaluation framework which aims on reducing the time needed for planning and designing a subjective quality assessment. The presented framework can be used for both crowdsourced and laboratory experiments. It should ease the task of designing a subjective quality assessment by providing a flexible framework. The framework has proven its applicability and flexibility to design and conduct assessments in the past and is available as open source.

Abstract: Dynamic Adaptive Streaming over HTTP (DASH) is an ISO/IEC MPEG standard which enables the convenient and smooth transportation of multimedia data to heterogeneous end devices over networks with variable bandwidth conditions. This kind of streaming technology is mainly used with HTTP 1.0 and 1.1 respectively, which both have some drawbacks. Therefore, the IETF has started the development of HTTP 2.0, which is based on Google’s SPDY proposal and already supported by several major companies, e.g., Facebook, Twitter, Akamai, Mozilla and obviously Google. Furthermore, Content Centric Networking (CCN) is another novel approach for future networks that is considered as an revolutionary approach compared to HTTP 2.0. The CCN communication paradigm is completely different and does not rely on direct connections between hosts, it rather focuses on the content. This paper demonstrates DASH with HTTP 2.0/SPDY and CCN using our universal libdash library. Moreover, different mechanisms of DASH will be shown that can be used to provide on-demand and live content in an efficient and comfortable way.

Abstract: Multimedia delivery over the Hypertext Transfer Protocol (HTTP) is currently very popular and with MPEGs' Dynamic Adaptive Streaming over HTTP (DASH) a standard is available to provide interoperability and enable large-scale deployments using existing infrastructures (servers, proxies, caches, etc.). This paper identifies some issue when multiple DASH clients compete for a bandwidth bottleneck when transparent proxy caches are deployed. Therefore, we propose a fair share adaptation scheme to be included within the client which – through experimental results – achieve a more efficient utilization of the bottleneck bandwidth and less quality switches.

Abstract: MPEG Dynamic Adaptive Streaming over HTTP (DASH) is a new streaming standard that has been recently ratified as an international standard (IS). In comparison to other streaming systems, e.g., HTTP progressive download, DASH is able to handle varying bandwidth conditions providing smooth streaming. Furthermore, it enables NAT and Firewall traversal, flexible and scalable deployment as well as reduced infrastructure costs due to the reuse of existing Internet infrastructure components, e.g., proxies, caches, and Content Distribution Networks (CDN). Recently, the Hypertext Transfer Protocol Bis (httpbis) working group of the IETF has officially started the development of HTTP 2.0. Initially three major proposals have been submitted to the IETF i.e., Googles' SPDY, Microsofts' HTTP Speed+Mobility and Network-Friendly HTTP Upgrade, but SPDY has been chosen as working draft for HTTP 2.0. In this paper we implemented MPEG-DASH over HTTP 2.0 (i.e., SPDY), demonstrating its potential benefits and drawbacks. Moreover, several experimental evaluations have been performed that compare HTTP 2.0 with HTTP 1.1 and HTTP 1.0 in the context of DASH. In particular, the protocol overhead, the performance for different round trip times, and DASH with HTTP 2.0 in a lab test scenario has been evaluated in detail.

Abstract: In this paper, we present our implementation and evaluation of Dynamic Adaptive Streaming over Content centric networking (DASC) which implements MPEG Dynamic Adaptive Streaming over HTTP (DASH) utilizing a Content Centric Networking (CCN) naming scheme to identify content segments in a CCN network. In particular, video segments formatted according to MPEG-DASH are available in different quality levels but instead of HTTP, CCN is used for referencing and delivery. Based on the conditions of the network, the DASC client issues interests for segments achieving the best throughput. Due to segment caching within the network, subsequent requests for the same content can be served quicker. As a result, the quality of the video a user receives progressively improves, effectively overcoming bottlenecks in the network. We present two sets of experiments to evaluate the performance of DASC showing that throughput indeed improves. However, the generated overhead is relatively large and the adaptation strategy used for DASH that assumes an end-to-end connection could be revised for the hop-by-hop architecture of CCN.

Abstract: This paper presents the usage of CCN, which is a candidate for the next-generation Internet, in combination with the new Dynamic Adaptive Streaming over HTTP (DASH) standard, which was recently ratified by ISO/IEC MPEG. In contrast to the Internet Protocol, which is mainly based on the host-to-host connection paradigm originated in the 1970s, Content Centric Networking (CCN) focuses on the content itself, instead of its location. Considering the dominance of multimedia traffic in todays' Internet, the streaming performance of DASH over CCN as well as the problems introduced by this combination is worth to be investigated in detail. Therefore, we evaluate the protocol overhead introduced by the usage of CCN compared to the HTTP versions 1.0 and 1.1. Furthermore, the performance of DASH over CCN under different network conditions is compared to the performance of HTTP 1.0/1.1. Our results showed that although CCN comes together with higher protocol overhead than HTTP 1.0/1.1 as well as a prototype implementation, it can definitely compete with HTTP 1.0 in media streaming. Based on the evaluation results, problems as well as improvement possibilities are identified, which are the basis for future work in this area.

Abstract: This paper presents the usage of Content Centric Networking (CCN) for adaptive multimedia streaming in mobile environments, leveraging the recent ISO/IEC MPEG Dynamic Adaptive Streaming over HTTP (DASH) standard. The performance of DASH over CCN is evaluated using real-world mobile bandwidth traces and compared to previous evaluations of different DASH-based as well as proprietary systems. As there are no client-server connections in CCN, it offers the possibility to transfer data from multiple sources as well as over multiple links in parallel, which is definitely an important feature, e.g., for mobile devices offering multiple network links. This functionality is used and evaluated in this paper in combination with DASH, making it possible to dynamically choose the best performing link for media streaming, which is a clear advantage over DASH using HTTP and the TCP/IP protocol stack. The evaluation therefore investigates DASH over CCN in two scenarios using synthetic and real-world mobile bandwidth traces respectively, showing a significantly better performance than conventional DASH using only one connection.

Abstract: The delivery of multimedia content over HTTP and on top of existing Internet infrastructures is becoming the preferred method within heterogeneous environment. The basic design principle is having an intelligent client which selects given and applicable media representations by issuing HTTP requests for individual segments based on the users' context and current conditions. Typically, this client behavior differs between implementations of the same kind and for the objective evaluations thereof appropriate datasets are needed. This paper presents a distributed dataset for the recently published MPEG-DASH standard which is mirrored at different sites across Europe, namely Klagenfurt, Paris, and Prague. A client implementation may choose to request segments from these sites and dynamically switch to a different location, e.g., in case the one currently used causes any issues. Thus, this distributed DASH dataset can be used for real-world evaluations enabling the simulation of switching between different content delivery networks.

Abstract: MPEG-M is a suite of ISO/IEC standards (ISO/IEC 23006) that has been developed under the auspices of Moving Picture Experts Group (MPEG). MPEG-M, also known as Multimedia Service Platform Technologies (MSPT), facilitates a collection of multimedia middleware APIs and elementary services as well as service aggregation so that service providers can offer users a plethora of innovative services by extending current IPTV technology toward the seamless integration of personal content creation and distribution, e-commerce, social networks and Internet distribution of digital media.

Abstract: Scalable Video Coding (SVC) in media streaming enables dynamic adaptation based on device capabilities and network conditions. In this paper, we investigate deployment options of SVC for Dynamic Adaptive Streaming over HTTP (DASH) with a special focus on scalability options, which are relevant for dynamic adaptation, especially in wireless and mobile environments. We evaluate the performance of SVC with respect to spatial and quality scalability options and compare it to non-scalable Advanced Video Coding (AVC). Performance evaluations are performed for various encoder implementations with high-definition (1080p) content. We show that a hybrid approach with multiple independent SVC bitstreams can have advantages in storage requirements at comparable rate-distortion performance.

Abstract: On TV screens, PCs, tablets, and mobile phones, video streaming has become a constant companion in our daily lives. For every video, we expect high visual quality, free from distortions, that is adjusted to the device at hand. But how can streaming systems cope with the increasing network traffic, the subsequent network congestions, and the different characteristics of end-user terminals? This thesis covers approaches for distributed adaptation of scalable video resources in media delivery. Scalable video resources consist of several layers that enable various spatial resolutions, frame rates, or qualities of a content. By dropping some of these layers, the video can be adjusted to the available bandwidth or to a specific end-user terminal. The adaptation can be performed on the sender side, on the receiver side, and on one or more network nodes. Scalable media coding can also help to reduce bandwidth requirements in multicast scenarios (e.g., for IPTV). One popular realization of scalable media coding is the Scalable Video Coding (SVC) standard. This thesis consists of three main parts, addressing various challenges towards efficient SVC adaptation. The first part of this thesis focuses on the encoding of SVC. In order to enable efficient adaptation, the configuration of layers has to be carefully chosen at encoding time. Thus, the performances of various encoding configurations and encoder implementations are evaluated. Furthermore, encoding guidelines for SVC are developed, which are aligned with recommendations of industry streaming solutions. The evaluation results of the developed SVC encoding guidelines suggest that quality scalability should be preferred over spatial scalability for adaptive streaming scenarios. Different resolutions for supporting device classes should rather be provided as separate SVC streams. The second part of this thesis deals with the fact that scalable media formats, such as SVC, are still not widely adopted neither on the sender side nor on the end-user terminal. In order to enable the deployment of SVC for network transmission and to improve the support for streaming to heterogeneous devices, the concept of SVC tunneling is introduced in this thesis. The video is transcoded to SVC at the sender side and then transcoded back to another video format at the receiver side at an advanced home-gateway. However, the transcoding between video formats has a negative impact on the video quality. The trade-off between quality loss and bandwidth efficiency of SVC tunneling is evaluated. SVC tunneling with quality layers enables bandwidth savings at moderate quality loss (approx. 2.5 dB) compared to streaming separate non-scalable representations of the same qualities. In the third part of this thesis, adaptation techniques for content-aware networks are investigated. In content-aware networks, some network nodes are capable to dynamically adapt video streams in reaction to varying network loads. With the increasing adoption of HTTP streaming, adaptation at the client side becomes a main factor for the viewing experience. The switch between two representations (e.g., different bitrates) of a video can disrupt that viewing experience. To reduce the effect of an abrupt quality change, the approach of a smooth transition between representations is developed and evaluated. A subjective user study indicates that this approach can indeed improve the overall viewing quality. Finally, the findings of the previous parts are integrated in an adaptive end-to-end SVC streaming system. Evaluations of this streaming system show that the developed adaptation framework significantly improves the video quality under packet loss (by up to 6 dB) compared to non-adaptive streaming.

Abstract: When an adaptive media streaming system has to switch from one representation of the content to another, the switch causes viewer distraction. We introduce the concept of representation switch smoothing for alleviating the distraction and improving the overall quality of experience. As adaptive HTTP streaming systems typically deploy video buffers on the client side, the adaptation decision is known far enough ahead of playout time to perform a seamless transition between quality representations. We discuss implementation considerations for an adaptive HTTP streaming system with scalable video coding, present a subjective evaluation of the proposed approach, and identify factors that influence how smooth transitions are perceived.

Abstract: Scalability within media coding allows for content adaptation towards heterogeneous user contexts and enables in-network adaptation. However, there is no straightforward solution how to encode the content in a scalable way while maximizing rate-distortion performance. In this paper we provide encoding guidelines for scalable video coding based on a survey of media streaming industry solutions and a comprehensive performance evaluation using four state of the art scalable video codecs with a focus on high-definition content (1080p).

Abstract: Scalable Video Coding (SVC) in media streaming enables dynamic adaptation based on device capabilities and network conditions. In this paper, we investigate deployment options of SVC for Dynamic Adaptive Streaming over HTTP (DASH) with a special focus on scalability options, which are relevant for dynamic adaptation, especially in wireless and mobile environments. We establish encoding recommendations and evaluate the performance of SVC with respect to spatial and quality scalability options and compare it to non-scalable Advanced Video Coding (AVC). Performance evaluations are performed for various encoder implementations with high-definition (1080p) content. We show that a hybrid approach with multiple independent SVC bitstreams can have advantages in storage requirements at comparable rate-distortion performance.

Abstract: Given that multimedia services are becoming increasingly popular, they are expected to play a dominant role for the Future Internet. In this context, it is essential that Content-Aware Networking (CAN) architectures, as envisaged in the frame of the Future Internet, explicitly address the efficient delivery and processing of multimedia content. This article proposes adopting a content-aware approach into the network infrastructure, thus making it capable of identifying, processing, and manipulating (i.e., adapting, caching, etc.) media streams and objects in real time towards Quality of Service/Experience (QoS/QoE) maximization. Our proposal is built upon the exploitation of scalable media coding technologies within such a content-aware networking environment and is discussed based on four representative use cases for media delivery (unicast, multicast, peer-to-peer, and adaptive HTTP streaming) and with respect to a selection of CAN challenges, specifically flow processing, caching/buffering, and QoS/QoE management.

Abstract: Multimedia (especially video) services constitute a dominant and ever increasing portion of the global Internet traffic, while they are expected to also play a major role in the Future Internet scene. In order to address this reality in the networking domain, a promising perspective is to gradually shift from the current, service-unaware, best-effort nature of IP networks into a network logic which is service-aware – and, in specific, media-aware. This chapter discusses how media-awareness can be introduced in the networking domain in a way which is both feasible and scalable, leveraging at the same time state-of-the-art technologies in video representations, such as Scalable Video Coding (SVC) and Dynamic Adaptive Streaming over HTTP (DASH).