Abstract: MPEG-M is a suite of ISO/IEC standards (ISO/IEC 23006) that has been developed under the auspices of Moving Picture Experts Group (MPEG). MPEG-M, also known as Multimedia Service Platform Technologies (MSPT), facilitates a collection of multimedia middleware APIs and elementary services as well as service aggregation so that service providers can offer users a plethora of innovative services by extending current IPTV technology toward the seamless integration of personal content creation and distribution, e-commerce, social networks and Internet distribution of digital media.

Abstract: Scalable Video Coding (SVC) in media streaming enables dynamic adaptation based on device capabilities and network conditions. In this paper, we investigate deployment options of SVC for Dynamic Adaptive Streaming over HTTP (DASH) with a special focus on scalability options, which are relevant for dynamic adaptation, especially in wireless and mobile environments. We evaluate the performance of SVC with respect to spatial and quality scalability options and compare it to non-scalable Advanced Video Coding (AVC). Performance evaluations are performed for various encoder implementations with high-definition (1080p) content. We show that a hybrid approach with multiple independent SVC bitstreams can have advantages in storage requirements at comparable rate-distortion performance.

Abstract: On TV screens, PCs, tablets, and mobile phones, video streaming has become a constant companion in our daily lives. For every video, we expect high visual quality, free from distortions, that is adjusted to the device at hand. But how can streaming systems cope with the increasing network traffic, the subsequent network congestions, and the different characteristics of end-user terminals? This thesis covers approaches for distributed adaptation of scalable video resources in media delivery. Scalable video resources consist of several layers that enable various spatial resolutions, frame rates, or qualities of a content. By dropping some of these layers, the video can be adjusted to the available bandwidth or to a specific end-user terminal. The adaptation can be performed on the sender side, on the receiver side, and on one or more network nodes. Scalable media coding can also help to reduce bandwidth requirements in multicast scenarios (e.g., for IPTV). One popular realization of scalable media coding is the Scalable Video Coding (SVC) standard. This thesis consists of three main parts, addressing various challenges towards efficient SVC adaptation. The first part of this thesis focuses on the encoding of SVC. In order to enable efficient adaptation, the configuration of layers has to be carefully chosen at encoding time. Thus, the performances of various encoding configurations and encoder implementations are evaluated. Furthermore, encoding guidelines for SVC are developed, which are aligned with recommendations of industry streaming solutions. The evaluation results of the developed SVC encoding guidelines suggest that quality scalability should be preferred over spatial scalability for adaptive streaming scenarios. Different resolutions for supporting device classes should rather be provided as separate SVC streams. The second part of this thesis deals with the fact that scalable media formats, such as SVC, are still not widely adopted neither on the sender side nor on the end-user terminal. In order to enable the deployment of SVC for network transmission and to improve the support for streaming to heterogeneous devices, the concept of SVC tunneling is introduced in this thesis. The video is transcoded to SVC at the sender side and then transcoded back to another video format at the receiver side at an advanced home-gateway. However, the transcoding between video formats has a negative impact on the video quality. The trade-off between quality loss and bandwidth efficiency of SVC tunneling is evaluated. SVC tunneling with quality layers enables bandwidth savings at moderate quality loss (approx. 2.5 dB) compared to streaming separate non-scalable representations of the same qualities. In the third part of this thesis, adaptation techniques for content-aware networks are investigated. In content-aware networks, some network nodes are capable to dynamically adapt video streams in reaction to varying network loads. With the increasing adoption of HTTP streaming, adaptation at the client side becomes a main factor for the viewing experience. The switch between two representations (e.g., different bitrates) of a video can disrupt that viewing experience. To reduce the effect of an abrupt quality change, the approach of a smooth transition between representations is developed and evaluated. A subjective user study indicates that this approach can indeed improve the overall viewing quality. Finally, the findings of the previous parts are integrated in an adaptive end-to-end SVC streaming system. Evaluations of this streaming system show that the developed adaptation framework significantly improves the video quality under packet loss (by up to 6 dB) compared to non-adaptive streaming.

Abstract: When an adaptive media streaming system has to switch from one representation of the content to another, the switch causes viewer distraction. We introduce the concept of representation switch smoothing for alleviating the distraction and improving the overall quality of experience. As adaptive HTTP streaming systems typically deploy video buffers on the client side, the adaptation decision is known far enough ahead of playout time to perform a seamless transition between quality representations. We discuss implementation considerations for an adaptive HTTP streaming system with scalable video coding, present a subjective evaluation of the proposed approach, and identify factors that influence how smooth transitions are perceived.

Abstract: Scalability within media coding allows for content adaptation towards heterogeneous user contexts and enables in-network adaptation. However, there is no straightforward solution how to encode the content in a scalable way while maximizing rate-distortion performance. In this paper we provide encoding guidelines for scalable video coding based on a survey of media streaming industry solutions and a comprehensive performance evaluation using four state of the art scalable video codecs with a focus on high-definition content (1080p).

Abstract: Scalable Video Coding (SVC) in media streaming enables dynamic adaptation based on device capabilities and network conditions. In this paper, we investigate deployment options of SVC for Dynamic Adaptive Streaming over HTTP (DASH) with a special focus on scalability options, which are relevant for dynamic adaptation, especially in wireless and mobile environments. We establish encoding recommendations and evaluate the performance of SVC with respect to spatial and quality scalability options and compare it to non-scalable Advanced Video Coding (AVC). Performance evaluations are performed for various encoder implementations with high-definition (1080p) content. We show that a hybrid approach with multiple independent SVC bitstreams can have advantages in storage requirements at comparable rate-distortion performance.

Abstract: Given that multimedia services are becoming increasingly popular, they are expected to play a dominant role for the Future Internet. In this context, it is essential that Content-Aware Networking (CAN) architectures, as envisaged in the frame of the Future Internet, explicitly address the efficient delivery and processing of multimedia content. This article proposes adopting a content-aware approach into the network infrastructure, thus making it capable of identifying, processing, and manipulating (i.e., adapting, caching, etc.) media streams and objects in real time towards Quality of Service/Experience (QoS/QoE) maximization. Our proposal is built upon the exploitation of scalable media coding technologies within such a content-aware networking environment and is discussed based on four representative use cases for media delivery (unicast, multicast, peer-to-peer, and adaptive HTTP streaming) and with respect to a selection of CAN challenges, specifically flow processing, caching/buffering, and QoS/QoE management.

Abstract: Multimedia (especially video) services constitute a dominant and ever increasing portion of the global Internet traffic, while they are expected to also play a major role in the Future Internet scene. In order to address this reality in the networking domain, a promising perspective is to gradually shift from the current, service-unaware, best-effort nature of IP networks into a network logic which is service-aware – and, in specific, media-aware. This chapter discusses how media-awareness can be introduced in the networking domain in a way which is both feasible and scalable, leveraging at the same time state-of-the-art technologies in video representations, such as Scalable Video Coding (SVC) and Dynamic Adaptive Streaming over HTTP (DASH).

Abstract: This paper shows how adaptive streaming and on-device caching can be used to provide an always available video service. A DASH client has been extended to deal with periods of zero network connectivity, and seamlessly works in conjunction with CCN to provide local storage that is intelligently updated to provide an improved quality of experience.

Abstract: Dynamic Adaptive Streaming over HTTP (DASH) is referred to as a multimedia streaming standard to deliver high quality multimedia content over the Internet using conventional HTTP Web servers. As a fundamental feature, it enables automatic switching of quality levels according to network conditions, user requirements, and expectations. Currently, the proposed adaptation schemes for HTTP streaming mostly rely on throughput measurements and/or buffer-related metrics, such as buffer exhaustion and levels. In this paper, we propose to enhance the DASH adaptation logic by feeding it with additional information from our evaluation of the users' perception approximating the user- perceived quality of video playback. The proposed model aims at conveniently combining TCP-, buffer-, and media content-related metrics as well as user requirements and expectations to be used as an input for the DASH adaptation logic. Experiments have demonstrated that the chosen model enhances the capability of the adaptation logic to select the optimal video quality level. Finally, we integrated all our findings into a real DASH system with QoE monitoring capabilities.

Abstract: Additional constituents for the representation of multimedia content gained more and more attention. For example, the amount of cinemas equipped with additional devices (e.g., ambient light, vibrating seats, wind generators, water sprayers, heater/coolers) that stimulate senses going beyond audition and vision increases. On the content side the MPEG-V standard specifies – among others – Sensory Effect Metadata (SEM) which provides means to describe sensory effects such as wind, vibration, light, etc. to be attached to audio-visual content and, thus, offering an enhanced and immersive experience for the user. However, there is a lack of a common set of test content allowing for various subjective user studies and verification across different test sites. In this paper we provide our dataset comprising a number of videos from different genres enriched with MPEG-V compliant Sensory Effect Metadata descriptions. Furthermore, we describe possible test setups using off-the-shelf hardware for conducting subjective quality assessments.

Abstract: More and more content in various formats becomes available via the WorldWideWeb (WWW). Currently availableWeb browsers are able to access and interpret these contents (i.e., Web videos, text, image, and audio). These contents stimulate only senses like audition or vision. Recently, it has been proposed to stimulate also other senses while consuming multimedia content, through so-called sensory effects. These sensory effects aim to enhance the ambient experience by providing effects such as light, wind, vibration, etc. The effects are represented as Sensory Effect Metadata (SEM) description which is associated to multimedia content and is rendered on devices like fans, vibration chairs, or lamps. In this paper we present two subjective quality assessments which comprise sensory effects, such as light, in the area of the WWW and their results achieved. The first assessment evaluates the influence of light effects on the Quality of Experience (QoE). The second assessment measures the impact of different settings for the color calculation on the viewing experience. Furthermore, we describe a Web browser plug-in for Mozilla Firefox which is able to render such sensory effects that are provided via the WWW.

Abstract: Multimedia content is increasingly used in every area of our life. Still, each type of content only stimulates the visual and/or the hearing system. Thus, the user experience depends only on those two stimuli. In this paper we introduce a standard which offers the possibility to add additional effects to multimedia content. Furthermore, we present a multimedia player and a Web browser plug-in which uses this standard to stimulate further senses by using additional sensory effects (i.e., wind, vibration, and light) to enhance the user experience resulting in a unique, worthwhile sensory experience.

Abstract: This paper describes a toolset for the authoring, simulating, and rendering of multimedia content annotated with Sensory Effect Metadata (SEM) descriptions as specified in Part 3 of the MPEG V standard. This part of MPEG-V standardizes the description of sensory effects (e.g., light, wind) in order to be rendered on sensory devices (e.g., fans, vibration chairs) aiming at generating a sensory experience stimulating possibly all human senses. Our implementation comprises a toolset to author sensory effects associated with multimedia content and the simulation thereof. Furthermore, it includes a library, a standalone player, and a Web browser plug-in which enables the playback and rendering of sensory effects on off-the-shelf rendering devices and in various contexts. All software modules are available under the GNU General Public License (GPL) v3 and the GNU Lesser General Public License (LGPL) v3 respectively.

Abstract: In this tutorial we present dynamic adaptive streaming over HTTP ranging from content creation to consumption. It particular, it provides an overview of the recently ratified MPEG-DASH standard, how to create content to be delivered using DASH, its consumption, and the evaluation thereof with respect to competing industry solutions. The tutorial can be roughly clustered into three parts. In part I we will provide an introduction to DASH, part II covers content creation, delivery, and consumption, and, finally, part III deals with the evaluation of existing (open source) MPEG-DASH implementations compared to state-of-art deployed industry solutions.

Abstract: This paper introduces the concept of sensory experience by utilizing sensory effects such as wind or lighting as another dimension which contributes to the quality of the user experience. In particular, we utilize a representation format for sensory effects that are attached to traditional multimedia resources such as audio, video, and image contents. Sensory effects (e.g., wind, lighting, explosion, heat, cold) are rendered on special devices (e.g., fans, ambient lights, motion chair, air condition) in synchronization with the traditional multimedia resources and shall stimulate other senses than audition and vision (e.g., mechanoreception, equilibrioception, thermoreception), with the intention to increase the users Quality of Experience (QoE). In particular, the paper provides a comprehensive introduction into the concept of sensory experience, its assessment in terms of the QoE, and related standardization and implementation efforts. Finally, we will highlight open issues and research challenges including future work.

Abstract: The design of interactive systems, especially in distributed research projects, is a challenging process in which many concepts are developed with successful outcomes but also with dissatisfying results. In order to structure and relay knowledge about good or bad approaches, design patterns are a well-known instrument in research and development. Due to the condition that a design pattern must be easy to read, different stakeholders in the system engineering and design process are able to understand the described concepts without the need of specific expert knowledge . In distributed projects, application design knowledge may be scattered and documented in different manners. This means, before we can start formulating patterns, we need to discover and gather the available and partially concealed design knowledge. Since these fragments of knowledge may not always be accurately formulated for being used as design patterns, we seek for a collaborative method for collecting and formulating early findings together with established design knowledge. In this paper we present a concept of an evolutionary process for capturing, formulating, refining and validating design patterns. Our approach aims at involving as many stakeholders as possible in order to shape a pattern language over a project’s lifetime in a collaborative process allowing facile participation. We implement our approach in the scope of the EU research project BRIDGE that aims at supporting inter-agency collaboration during emergency response. We close with a discussion of the current state and envisioned next steps in order to foster our considerations.

Abstract: Multimedia is ubiquitously available online with large amounts of video increasingly consumed through Web sites such as YouTube or Google Video. However, online multimedia typically limits users to visual/auditory stimulus, with onscreen visual media accompanied by audio. The recent introduction of MPEG-V proposed multi-sensory user experiences in multimedia environments, such as enriching video content with so-called sensory effects like wind, vibration, light, etc. In MPEG-V, these sensory effects are represented as Sensory Effect Metadata (SEM), which is additionally associated to the multimedia content. This paper presents three user studies that utilize the sensory effects framework of MPEG-V, investigating the emotional response of users and enhancement of Quality of Experience (QoE) of Web video sequences from a range of genres with and without sensory effects. In particular, the user studies were conducted in Austria and Australia to investigate whether geography and cultural differences affect users’ elicited emotional responses and QoE.

Abstract: Nowadays video is an important part of the Web and Web sites like YouTube, Hulu, etc. count millions of users consuming their content every day. However, these Web sites mainly use media players based on proprietary browser plug-ins (i.e., Adobe Flash) and do not leverage adaptive streaming systems. This paper presents a seamless integration of the recent MPEG standard on Dynamic Adaptive Streaming over HTTP (DASH) in the Web using the HTML5 video element. Therefore, we present DASHJS, a JavaScript-based MPEG-DASH client which adopts the Media Source API of Google’s Chrome browser to present a flexible and potentially browser independent DASH client. Furthermore, we present the integration of WebM based media segments in DASH giving a detailed description of the used container format structure and a corresponding Media Presentation Description (MPD). Our preliminary evaluation demonstrates the bandwidth adaption capabilities to show the effectiveness of the system.

Abstract: We evaluate the performance of several routing protocols for mobile ad-hoc networks (MANETs) in an emergency response scenario. The simulated scenario uses a disaster area mobility model and a wireless shadowing model to represent realistic first responder movements in a hybrid indoor/outdoor environment. The resulting scenario imposes some challenges on the MANET routing protocols such as intermittent connectivity and network partitions. The simulation results show that nodes have diverse connectivity characteristics which are challenging for state-of-the-art MANET routing protocols.