Abstract: This book provides a comprehensive examination of the use of MPEG-2, MPEG-4, MPEG-7, MPEG-21, and MPEG-A standards, providing a detailed reference to their application. In this book, the authors address five leading MPEG standards, focusing not only on the standards themselves, but specifically upon their application (e.g. for broadcasting media, personalised advertising and news, multimedia collaboration, digital rights management, resource adaptation, digital home systems, and so on); including MPEG cross-breed applications. In the evolving digital multimedia landscape, this book provides comprehensive coverage of the key MPEG standards used for generation and storage, distribution and dissemination, and delivery of multimedia data to various platforms within a wide variety of application domains. It considers how these MPEG standards may be used, the context of their use, and how supporting and complementary technologies and the standards interact and add value to each other. Key Features: * Integrates the application of five popular MPEG standards (MPEG-2, MPEG-4, MPEG-7, MPEG-21, and MPEG-A) into one single volume, including MPEG cross-breed applications * Up-to-date coverage of the field based on the latest versions of the five MPEG standards * Opening chapter provides overviews of each of the five MPEG standards * Contributions from leading MPEG experts worldwide * Includes an accompanying website with supporting material (www.wiley.com/go/angelides_mpeg) This book provides an invaluable reference for researchers, practitioners, CTOs, design engineers, and developers. Postgraduate students taking MSc, MRes, MPhil and PhD courses in computer science and engineering, IT consultants, and system developers in the telecoms, broadcasting and publishing sectors will also find this book of interest.

Abstract: The term Universal Multimedia Experience (UME) has gained momentum and is well recognized within the research community. As this approach puts the user into the center stage, additional complexity is added to the overall quality assessment problem which calls for a scientific framework to capture, measure, quantify, judge, and explain the user experience. In previous work we have proposed the annotation of multimedia content with sensory effect metadata that can be used to stimulate also other senses than vision or audition. In this paper we report first results obtained from subjective tests in the area of sensory effects attached to traditional multimedia presentations such as movies that shall lead to an enhanced, unique, and worthwhile user experience.

Abstract: Multimedia is now situated in everyday's life. One can get multimedia content via television, Internet, DVD, etc. One problem with this type of content is that there is only the audio/video content without any extra experience. This extra experience should be realized by using Sensory Effects like wind, light, etc. There are devices which were developed especially for enriching movies or computer games with such effects. MPEG Representation of Sensory Effects (RoSE) is a standardization process which offers a possibility to annotate the content of a video with sensory effects. The resulting description can be used to enrich the video experience. In this book an annotation tool and a simulator for MPEG RoSE are introduced due to the lack of an easy way to create MPEG RoSE compliant descriptions. The annotation tool provides a solution for creating sensory effect descriptions for movies. These descriptions can be simulated by the simulator or shipped with the movie for enriching the experience at one's home. Furthermore, the book gives an overview of Universal Multimedia Experience (UME) and MPEG RoSE. It also describes some companies and devices which already offer sensory effects.

Abstract: In previous and related work sensory effects are presented as a tool for increasing the user experience of multimedia presentations by stimulating also other senses than vision or audition. In this paper we primarily investigated the relationship of the Quality of Experience (QoE) due to various video bit-rates of multimedia contents annotated with sensory effects (e.g., wind, vibration, light). Therefore, we defined a subjective quality assessment methodology based on standardized methods. The paper describes the test environment, its setup, and conditions in detail. Furthermore, we experimented with a novel voting device that allows for continuous voting feedback during a sequence in addition to the overall quality voting at the end of each sequence. The results obtained from the subjective quality assessment are presented and discussed thoroughly. In anticipation of the results we can report an improvement of the quality of the multimedia experience thanks to the sensory effects.

Abstract: In the past years the amount of multimedia content on the Internet or in home networks has been drastically increasing. Instead of buying traditional media (such as CDs or DVDs) users tend to buy online media. This leads to the difficulty of managing the content (e.g., movies, images). A vast amount of tools for content management exists but they are mainly focusing on one type of content (e.g., only images). Furthermore, most of the available tools are not configurable to the user’s preferences and cannot be accessed by different devices (e.g., TV, computer, mobile phone) in the home network. In this paper we present a UPnP A/V-based system for managing and sharing audio/visual content in home environments which is configurable to the user’s preferences. Furthermore, the paper depicts how this system can be used to improve the user experience by using MPEG-V.

Abstract: The World Wide Web (WWW) is one of the main entry points to access and consume Internet content in various forms. In particular, the Web browser is used to access different types of media (i.e., text, image, audio, and video) and on some platforms is the only way to access the vast amount of information on the Web. Recently, it has been proposed to stimulate also other senses than vision or audition while consuming multimedia content through so- called sensory effects, with the aim to increase the user’s Quality of Experience (QoE). The effects are represented as Sensory Effects Metadata (SEM) which is associated to traditional multimedia content and is rendered (synchronized with the media) on sensory devices like fans, vibration chairs, lamps, etc. In this paper we provide a principal investigation of whether the sensory effects are ready for the WWW and, in anticipation of the result, we propose how to embed sensory effect metadata within Web content and the synchronized rendering thereof.

Abstract: In this paper we describe a metadata solution for a Peer-to-Peer (P2P) content distribution system termed NextShare. We outline the key motivating factors for our approach, detail the overall generic architecture we have developed and present the workflow for delivering metadata through Peer-to-Peer based content distribution. The paper also presents the metadata model we have developed and we describe in detail how all the content can be packetized and distributed using NextShare. Finally, a description of the core and optional metadata attributes which may be utilized within the system is provided.

Abstract: MPEG has developed various technologies for multimedia transport, such as MPEG-2 Transport Stream (TS) and ISO Media Base File Format. These technologies have been widely accepted and heavily used by various industries and applications, such as digital broadcasting, audio and video transport over the Internet, mobile phones, etc. In recent years, the Internet has become an important channel for the delivery of multimedia. As HTTP is widely used on the Internet, it has recently been used extensively for the delivery of multimedia content. However, there is no standard for HTTP- based streaming of MPEG media. MPEG intends to standardize a solution that addresses this need. This paper provides an overview of the recent Call of Proposals (CfP) for HTTP Streaming of MPEG Media, a new work item within ISO/IEC MPEG. In particular, it will provide an overview of existing systems and the outcome of the evaluation of the aforementioned CfP which has been conducted during the 93rd MPEG meeting in July 2010.

Abstract: This document provides an overview of the MPEG Extensible Middleware (MXM), one of ISO/IEC MPEG’s latest achievements, defining an architecture and corresponding application programming interfaces (APIs) which enable accelerated media business developments. The paper describes the vision behind MXM, its architecture, and a high level overview of the API. Additionally, example MXM applications are given.

Abstract: Editor’s Note: Media applications are becoming increasingly complex. They handle many data formats, run across multiple platforms, and support a wide range of functions. This article describes a standardized set of protocols and APIs that provides efficient access to individual system components, enables rapid deployment of new applications, and improves portability. —Anthony Vetro

Abstract: The demand for access to advanced, distributed media resources is nowadays omnipresent due to the availability of Internet connectivity almost anywhere, anytime, and with a huge amount of different devices. This calls for rethinking of the current Internet architecture by making the network aware of which content is actually transported. This paper introduces Scalable Video Coding (SVC) as a tool for Content-Aware Networks (CANs) which is currently researched as part of the EU FP7 ALICANTE project. The architecture of ALICANTE with respect to SVC and CAN is reviewed, use cases are described, and, finally, research challenges and open issues are discussed.

Abstract: The aim of the MPEG‐21 standard, the so‐called Multimedia Framework, is to enable transparent and augmented use of multimedia resources across a wide range of networks, devices, user preferences, and communities, notably for trading (of bits). As such, it provides an important step in MPEG's standards evolution, i.e., the transaction of Digital Items among Users. This paper provides an overview of applications making use specifically of MPEG‐21 Digital Items and a more in‐depth presentation of a few selected applications in research and practice.

Abstract: Nowadays, mobile devices have implemented several transmission technologies which enable access to the Internet and increase the bit rate for data exchange. Despite modern mobile processors and high-resolution displays, mobile devices will never reach the stage of a powerful notebook or desktop system (for example, due to the fact of battery powered CPUs or just concerning the small-sized displays). Due to these limitations, the deliverable content for these devices should be adapted based on their capabilities including a variety of aspects (e.g., from terminal to network characteristics). These capabilities should be described in an interoperable way. In practice, however, there are many standards available and a common mapping model between these standards is not in place. Therefore, in this paper we describe such a mapping model and its implementation aspects. In particular, we focus on the whole delivery context (i.e., terminal capabilities, net- work characteristics, user preferences, etc.) and investigated the two most prominent state-of-the-art description schemes, namely User Agent Profile (UAProf) and Usage Environment Description (UED).

Abstract: In this tutorial we present immersive future media technologies ranging from 3D video to sensory experiences. The former targets stereo and multi-view video technologies whereas the latter aims at stimulating other senses than vision or audition enabling an advanced user experiences through sensory effects.

Abstract: This paper introduces the piece selection problem that arises when streaming layered video content over peer-to-peer networks. The piece selection algorithm decides periodically which pieces to request from other peers (network nodes) for download. The main goal of the piece selection algorithm is to provide the best possible quality for the available bandwidth. Our recommended solution approaches are related to the typical problems and solutions in the knapsack problem.

Abstract: The Scalable Video Coding (SVC) extension of the H.264/AVC (AVC) video coding standard features spatial, quality and temporal scalability. Backwards compatibility with legacy decoding devices is maintained through an H.264/AVC compliant base layer, which represents the lowest quality of an SVC bit-stream. However, it is often desirable to also provide the higher quality layers to legacy H.264/AVC devices. This is achieved by a process commonly known as “bit-stream rewriting”, which allows for an efficient SVC to AVC conversion by exploiting the similarities of the two codecs. This paper introduces an improved version of the existing JSVM reference software rewriter (JSVM-rewriter). The improvements include a better run-time performance through parallel processing, as well as applicability in streaming scenarios. A detailed evaluation provides performance measurements for the improved rewriter and compares it to the existing JSVM-rewriter. The evaluation shows that notable performance improvements can be achieved using the presented approach. The paper concludes on how the rewriter could be further improved.

Abstract: This paper compares handhelds based on the iPhone and Android operating systems in multimedia streaming scenarios. We simulate typical Internet network impairments, i.e. packet delay and packet loss, and evaluate their effects on the end devices. Additional evaluations include bandwidth overhead in icted by the different streaming approaches and traffic shape and fairness when both handhelds consume media simultaneously. Based on the quantitative evaluation, both approaches show weaknesses and strengths. A final qualitative discussion points out additional advantages for the streaming approach implemented in the iPhone operating system.

Abstract: H.264/SVC enables runtime-efficient scalability in the spatial, temporal and fidelity dimension. Existing adaptation mechanisms facilitate this to automatically adapt the H.264/ SVC stream to the current usage environment without any user interaction. This paper argues that the Quality of Experience (QoE) of the end user can be enhanced by enabling him to manually adjust the adaptation if he wishes to do so. An approach which enables this is presented and evaluated. It is shown that by facilitating this approach an increased QoE is provided compared to automatic adaptation approaches. Finally, future work indicates the next steps in order to implement this approach.

Abstract: This paper proposes a novel combination of arti- ficial intelligence planning and other techniques for improv- ing decision-making in the context of multi-step multime- dia content adaptation. In particular, it describes a method that allows decision-making (selecting the adaptation to perform) in situations where third-party pluggable multi- media conversion modules are involved and the multime- dia adaptation planner does not know their exact adapta- tion capabilities. In this approach, the multimedia adapta- tion planner module is only responsible for a part of the required decisions; the pluggable modules make additional decisions based on different criteria. We demonstrate that partial decision-making is not only attainable, but also in- troduces advantages with respect to a system in which these conversion modules are not capable of providing additional decisions. This means that transferring decisions from the multi-step multimedia adaptation planner to the pluggable conversion modules increases the flexibility of the adapta- tion. Moreover, by allowing conversion modules to be only partially described, the range of problems that these modules can address increases, while significantly decreasing both the description length of the adaptation capabilities and the planning decision time. Finally, we specify the conditions under which knowing the partial adaptation capabilities of a set of conversion modules will be enough to compute a proper adaptation plan.

Abstract: Recent work in TCP video streaming indicates that multimedia streaming via TCP provides satisfactory performance when the achievable TCP throughput is approximately twice the media bit rate. However, these conditions may not be achievable on the Internet, e.g., when the delivery path offers insufficient bandwidth or becomes congested due to competing traffic. Therefore, adaptive streaming for videos over TCP is required and a number of rate-control algorithms for video streaming have been proposed and evaluated in the literature. In this paper, we evaluate and compare three existing rate-control algorithms for TCP streaming in terms of the (PSNR) quality of the delivered video and in terms of the timeliness of delivery. The contribution of the paper is that, to the best of our knowledge, this is the first evaluation of TCP-based streaming in an Internet-like setting making use of the scalability features of the H.264/SVC video codec. Two simple bandwidth estimation algorithms and a priority-/deadline-driven approach are described to adapt the bit rates of, and transmit, the H.264/SVC video in a rate-distortion optimal manner. The results indicate that the three algorithms perform robustly in terms of video quality and timely delivery, both on under-provisioned links and in case of competing TCP flows. The priority-/deadline-driven technique is even more stable in terms of packet delays and jitter; thus, client buffers can be dimensioned more easily.

Abstract: TCP-based video streaming encounters difficulties in unreliable networks with unanticipated packet loss. In combination with high round trip times, the effective throughput deteriorates rapidly and TCP connection resets or stalls may occur. In this paper, we propose a client-driven video transmission scheme which utilizes multiple HTTP/TCP streams. The scheme is largely insensitive to unanticipated packet loss and thereby reduces throughput fluctuations. Since it is based on HTTP, the scheme can easily be deployed in existing network infrastructures. It fosters scalability on the server side by shifting complexity from the server to the clients. Certain features of request-response schemes allow maintaining fairness, despite of using multiple HTTP streams. Making use of TCP, the scheme inherently adapts to congested network links.

Abstract: This thesis investigates mechanisms and applications for in-network adaptation of scalable video bit streams based on the recent H.264/Scalable Video Coding (SVC) standard. In-network adaptation refers to the adaptation of a video stream by a network element during the stream's transport through the network. The advantages of performing adaptation directly in the network are the availability of local monitoring data and a higher responsiveness according to the current networking conditions. In contrast to previous work in this field, this thesis focuses on the feasibility and realization of in-network adaptation on existing home router platforms. In this context this thesis addresses the following six research objectives. Initially, the relevant transport mechanisms for H.264/SVC and their implications on in-network adaptation (1) were analysed. In the context of this work three different Linux-based router platforms which cover a representative range of residential router devices were used as a basis for further studies and evaluations. In general these platforms can be characterized by rather modest processing capabilities and networking performance. The hardware limitations were identified and quantified in evaluations (2) using both different benchmarks and real network traffic. The offered processing power and memory throughput are roughly 10 to 100 times lower than those of a modern desktop PC. Although their application-layer networking performance is not that low, all platforms fail in fully utilizing their nominal link capacities of 100 and 1000 Mbps, respectively. Based on the known limitations the thesis proposes a stateful, packet-based adaptation mechanism for adapting scalable video bit streams (3). The approach utilizes the RTP payload format for H.264/SVC and represents a light-weight approach for in-network adaptation on the application layer. It further meets the important requirements towards a media-aware network element (MANE) to be signaling aware and to operate statefully. The mechanism was integrated in a proxy service which was deployed on all of the three platforms to prove its feasibility. Experimental evaluations with different video bit streams in standard-definition quality demonstrate the scalability of the approach (4). The results indicate that the proxy service is able to adapt up to 16 concurrent video streams depending on the platform and video bit stream. On two of the three evaluated platforms the proposed approach even allows to handle and to adapt video streams in high-definition quality at bit rates around 15 Mbps. In addition to the proposed H.264/SVC-specific adaptation mechanism, also the applicability of generic metadata-driven adaptation on home router platforms was investigated. In particular, a proof-of-concept study of an XML-metadata-driven approach based on the MPEG-21 generic Bitstream Syntax Description (gBSD) was conducted on the platforms (5). In contrast to former evaluations that have been done on PC-based platforms, the obtained results indicate that the use of this generic adaptation cannot be recommended on such resource limited network devices. The benefits of using in-network adaptation on home router platforms are finally demonstrated in the context of high-definition streaming over IEEE 802.11 wireless networks (6). Monitoring information regarding the queueing delay, which is obviously available exclusively on the router, is used to control the adaptation of the video according to the varying throughput of the wireless link. This allows to react timely to changing conditions particularly in the case of mobile clients.

Abstract: The distribution of layered content over peer-to-peer networks becomes more important today as the users are consuming the content on terminals with various display capabilities and different network connections. For single-layer content distribution, the piece-picking algorithm only needs to ensure that content pieces are downloaded in time for display. When layered content is distributed over a peer-to-peer network, the piece-picking algorithm needs to be modified to ensure that the best possible quality is displayed while all desired pieces still have to be received before their deadline expires. In this paper, the piece-picking problem for layered content is analyzed and a number of piece-picking algorithms for layered content based on the solutions for the knapsack problem are presented. Furthermore, an evaluation of these algorithms is performed and possible applications are discussed.

Abstract: Peer-to-Peer systems are nowadays a very popular solution for multimedia distribution, as they provide significant cost benefits compared with traditional server-client distribution. Additionally, the distribution of scalable content enables the consumption of the content in a quality suited for the available bandwidth and the capabilities of the end-user devices. Thus, the distribution of scalable content over Peer-to-Peer networks is a very actual research topic. This paper presents an architecture for the distribution of scalable content in a fully distributed Peer-to-Peer network. The architectural description includes how the scalable layers of the content are mapped to the pieces distributed in the Peer-to-Peer system and detailed descriptions of the producer- and consumer-site architecture of the system. The presented system is to our knowledge the first open-source Peer-to-Peer network with full Scalable Video Coding support.