Abstract: HTTP streaming has gained significant attraction in the last few years. Currently many commercial as well as standardized streaming systems are already offering adaptive streaming. In most cases, the adaptation is achieved by switching between separately encoded video streams in different qualities. In contrast to that, this paper focuses on the applicability of scalable video coding based on the H.264/SVC standard for adaptive HTTP streaming. Recent work has already highlighted the conceptual advantages like better cache utilization, fine-grained bit rate scalability, and lower storage requirements. This paper discusses the actual realization and design options for implementing priority streaming using the ISO Base Media File Format (BMFF). We propose three different strategies for organizing the scalable video bit stream that consider both the possibilities as well as limitations of the ISO BMFF. The proposed strategies are discussed and evaluated both conceptually and quantitatively. For that purpose, we provide a detailed analysis based on modeling both the overhead of the file format and the HTTP encapsulation. The results for all three priority streaming strategies show that the limitations of the ISO BMFF result in a high relative overhead in the case of low bit rate content. However, when applied to high quality content, priority streaming of H.264/SVC can be implemented at a very low cost. Depending on the number of layers and the offered scalability dimensions, different strategies should be chosen to minimize the overhead. Based on the analytical model and the discussion, this paper provides guidance for selecting the most efficient strategy.

Abstract: Adaptive video streaming based on TCP/HTTP is becoming popular because of its ability to adapt to changing network conditions. We present an in-depth experimental analysis of the use of HTTP-based request-response streams for video streaming. In this scheme, video fragments are fetched by a client from the server, in smaller units called chunks, potentially via multiple parallel HTT P requests (TCP connections). A model for the achievable throughput is formulated.The model is validated by a broad range of streaming experiments, including an evaluation of TCP-friendliness. Our findings include that request-response streams are able to scale with the available bandwidth by increasing the chunk size or the number of concurrent streams. Several combinations of system parameters exhibiting TCP-friendliness are presented. We also evaluate the video streaming performance in terms of video quality in the presence of packet loss. Multiple request-response streams are able to maintain satisfactory performance, while a single TCP connection deteriorates rapidly with increasing packet loss. The results provide experimental evidence that HTTP-based request-response streams are a good alternative to classical TCP streaming

Abstract: Wireless router platforms based on the Linux operating system are becoming popular in consumers' home networks. The transmission of multimedia data or their use as media-aware network elements imposes high traffic and computational loads on these devices. Thus, it is interesting to evaluate the networking and processing capabilities of such home router platforms in order to assess their usefulness for improved multimedia services such as in-network H.264/SVC video stream adaptation. This paper presents a performance evaluation of three home router platforms representative for low-end, mid-range, and high-end devices. The scope of the evaluation is the performance of the Linux networking stack on these routers; results for both application-layer (TCP and UDP) transmission and kernel-level (UDP) traffic routing are given. The results show that both TCP and UDP throughputs are significantly below (less than half of) the outgoing (wired) links' nominal capacities and depend very much on the sizes of the transmitted data blocks. This clearly indicates that the networking performance is limited by the platforms' processing capabilities and the lack of mechanisms that offload networking tasks from the CPUs. This behaviour cannot be observed on today's PC systems and has to be considered when deploying multimedia services on these network devices. Furthermore, a detailed analysis of the Linux networking stack reveals that the performance is heavily impacted by the netfilter code, even when no packet filtering or network address translation is being performed. Considerable performance gains can be achieved when this netfilter code is bypassed.

Abstract: In this paper, we present an approach for in-network adaptation of H.264/SVC in the context of 802.11 wireless networks. It builds upon our previous work on an adaptive RTSP/RTP proxy which allows to adapt video streams on Linux-based home router platforms. The proposed approach tackles the throughput variations that occur as a consequence of the physical rate adaptation in 802.11 equipment caused by the mobility of clients. By combining monitoring information available exclusively on the wireless router with the ability to adapt scalable video streams on-the-fly, the proposed in-network adaptation approach allows to quickly adjust the video bit rate to the current link conditions. Instead of reacting on packet loss, our approach uses an increase in queueing delay at the router to detect phases of throughput degradation. This allows a higher responsiveness compared to traditional end-to-end approaches that rely solely on RTCP feedback. The behavior of our novel approach was evaluated in several mobility scenarios in an experimental test bed. The results obtained by streaming and adapting high-definition content clearly demonstrate the feasibility and benefits of this approach.

Abstract: This chapter covers the topic of making use of scalable video content in streaming frameworks and applications. Specifically, the recent standard H.264/SVC, i.e., the scalable extension of the widely used H.264/AVC coding scheme, and its deployment for adaptive streaming, the combined activities of content adaptation and streaming, are considered. H.264/SVC is regarded as a promising candidate to enable applications to cope with bandwidth variations in networks and heterogeneous usage environments, mainly diverse end device capabilities and constraints. The relevant coding and transport principles of H.264/SVC are reviewed first. Subsequently, an overview of H.264/SVC applications is given. The chapter then focuses on presenting architectural/implementation options and applications of H.264/SVC for adaptive streaming, emphasizing the aspect of where, i.e., on which network node and on which layer in the networking stack, in the video delivery path the content adaptation can take place; also, methods of content adaptation are covered. This pragmatic perspective is seen as complementing more general discussions of scalable video adaptation issues in the existing literature.

Abstract: Recent work in TCP video streaming indicates that multimedia streaming via TCP provides satisfactory performance when the achievable TCP throughput is approximately twice the media bit rate. However, these conditions may not be achievable on the Internet, e.g., when the delivery path offers insufficient bandwidth or becomes congested due to competing traffic. Therefore, adaptive streaming for videos over TCP is required and a number of rate-control algorithms for video streaming have been proposed and evaluated in the literature. In this paper, we evaluate and compare three existing rate-control algorithms for TCP streaming in terms of the (PSNR) quality of the delivered video and in terms of the timeliness of delivery. The contribution of the paper is that, to the best of our knowledge, this is the first evaluation of TCP-based streaming in an Internet-like setting making use of the scalability features of the H.264/SVC video codec. Two simple bandwidth estimation algorithms and a priority-/deadline-driven approach are described to adapt the bit rates of, and transmit, the H.264/SVC video in a rate-distortion optimal manner. The results indicate that the three algorithms perform robustly in terms of video quality and timely delivery, both on under-provisioned links and in case of competing TCP flows. The priority-/deadline-driven technique is even more stable in terms of packet delays and jitter; thus, client buffers can be dimensioned more easily.

Abstract: TCP-based video streaming encounters difficulties in unreliable networks with unanticipated packet loss. In combination with high round trip times, the effective throughput deteriorates rapidly and TCP connection resets or stalls may occur. In this paper, we propose a client-driven video transmission scheme which utilizes multiple HTTP/TCP streams. The scheme is largely insensitive to unanticipated packet loss and thereby reduces throughput fluctuations. Since it is based on HTTP, the scheme can easily be deployed in existing network infrastructures. It fosters scalability on the server side by shifting complexity from the server to the clients. Certain features of request-response schemes allow maintaining fairness, despite of using multiple HTTP streams. Making use of TCP, the scheme inherently adapts to congested network links.

Abstract: This thesis investigates mechanisms and applications for in-network adaptation of scalable video bit streams based on the recent H.264/Scalable Video Coding (SVC) standard. In-network adaptation refers to the adaptation of a video stream by a network element during the stream's transport through the network. The advantages of performing adaptation directly in the network are the availability of local monitoring data and a higher responsiveness according to the current networking conditions. In contrast to previous work in this field, this thesis focuses on the feasibility and realization of in-network adaptation on existing home router platforms. In this context this thesis addresses the following six research objectives. Initially, the relevant transport mechanisms for H.264/SVC and their implications on in-network adaptation (1) were analysed. In the context of this work three different Linux-based router platforms which cover a representative range of residential router devices were used as a basis for further studies and evaluations. In general these platforms can be characterized by rather modest processing capabilities and networking performance. The hardware limitations were identified and quantified in evaluations (2) using both different benchmarks and real network traffic. The offered processing power and memory throughput are roughly 10 to 100 times lower than those of a modern desktop PC. Although their application-layer networking performance is not that low, all platforms fail in fully utilizing their nominal link capacities of 100 and 1000 Mbps, respectively. Based on the known limitations the thesis proposes a stateful, packet-based adaptation mechanism for adapting scalable video bit streams (3). The approach utilizes the RTP payload format for H.264/SVC and represents a light-weight approach for in-network adaptation on the application layer. It further meets the important requirements towards a media-aware network element (MANE) to be signaling aware and to operate statefully. The mechanism was integrated in a proxy service which was deployed on all of the three platforms to prove its feasibility. Experimental evaluations with different video bit streams in standard-definition quality demonstrate the scalability of the approach (4). The results indicate that the proxy service is able to adapt up to 16 concurrent video streams depending on the platform and video bit stream. On two of the three evaluated platforms the proposed approach even allows to handle and to adapt video streams in high-definition quality at bit rates around 15 Mbps. In addition to the proposed H.264/SVC-specific adaptation mechanism, also the applicability of generic metadata-driven adaptation on home router platforms was investigated. In particular, a proof-of-concept study of an XML-metadata-driven approach based on the MPEG-21 generic Bitstream Syntax Description (gBSD) was conducted on the platforms (5). In contrast to former evaluations that have been done on PC-based platforms, the obtained results indicate that the use of this generic adaptation cannot be recommended on such resource limited network devices. The benefits of using in-network adaptation on home router platforms are finally demonstrated in the context of high-definition streaming over IEEE 802.11 wireless networks (6). Monitoring information regarding the queueing delay, which is obviously available exclusively on the router, is used to control the adaptation of the video according to the varying throughput of the wireless link. This allows to react timely to changing conditions particularly in the case of mobile clients.

Abstract: This paper presents a novel approach that combines both in-network, application-layer adaptation and network-layer traffic control of scalable video streams based on the H.264/SVC standard. In the IPTV/VoD scenario considered, an intercepting RTSP/RTP proxy performs admission control of the requested video, based on the signaled scalability information, and decides whether the content can be streamed without changes or in an adapted version. The proxy configures the network layer appropriately in order to separate the video stream from besteffort traffic on the same link. Rather than performing fixed bandwidth allocation, our proxy approach uses the Hierarchical Token Bucket (HTB) queuing discipline to allow for borrowing bandwidth between traffic classes. In that setting, two different allocation policies are introduced. The Hard Reservation Policy (HRP) performs admission control and adaptation on the video streams and does not modify video bandwidth allocation after admission. In contrast, the Flexible Borrowing Policy (FBP) restricts the admission control to the base layer of the SVC stream. The packets carrying MGS enhancement layer data are marked with priorities by the proxy and are handled at the network layer by a priority-based queuing mechanism. Both a qualitative comparison and an experimental evaluation of the two policies are given.

Abstract: One of the most active research topics in the field of video signal processing is scalable video coding (SVC). The recently published extension of the H.264/AVC video coding standard introduces scalability features by employing a layered encoding of the video stream. In our work we investigated the usage of this scalable extension of H.264/AVC for in-network multimedia adaptation. We developed an RTSP/RTP-based proxy which exploits the layered encoding of the video and can perform real-time video adaptation on an inexpensive off-the-shelf WiFi router. This is achieved by applying a stateful, packet-based adaptation approach that keeps the computational costs at a minimum. With that approach it is possible to simultaneously adapt multiple video streams to varying network conditions or to the capabilities of the consumers' end-devices. In our demonstration we show the streaming of two scalable video streams from a server to a client and the in-network adaptation of the video at the WiFi router. The adaptation can be controlled interactively in the temporal, spatial and SNR domains.

Abstract: End-to-end support for Quality of Service (QoS) has been broadly discussed in the literature. Many technologies have been proposed, each focusing on specific aspects for providing QoS guarantees to the end user. However, the integrated management of the end-to-end chain preserving QoS in heterogeneous environments is still a significant issue and insufficiently addressed to date. In this paper we propose an integrated management supervisor that takes into account the requirements from all stakeholders along the multimedia content delivery chain and provides an end-to-end management solution enabling QoS to the end user. This architecture and the subsystems that can be distributed along the end-to-end chain are detailed in this paper.

Abstract: One of the future visions of multimedia networking is the provision of multimedia content at a variety of quality and price levels. Of the many approaches to this issue, one of the most predominant techniques is the concept of Perceived Quality of Service (PQoS), which extends the traditional engineering-based QoS concept to the perceptual satisfaction that the user receives from the reception of multimedia content. In this context, PQoS monitoring is becoming crucial to media service providers (SPs) for providing not only quantified PQoS-based services, but also service assurance based on multimedia content adaptation across heterogeneous networks. This work proposes a novel cross-layer monitoring architecture that utilizes a new Network QoS (NQoS) to PQoS mapping framework at the application level. The resulting QoS monitoring should allow the content delivery system to take sophisticated actions for real time media content adaptation, and aims to provide perceived service performance verification with respect to the QoS guarantees that have been specified in contractual agreements between providers and end-users. A subsequent performance evaluation of the proposed model conducted using a real test-bed environment demonstrates both the accuracy and feasibility of the network level measurements, the NQoS to PQoS mapping and the overall feasibility of the proposed end-to-end monitoring solution.

Abstract: The amount of audiovisual data available on the Internet and thus of multimedia communication over today's networks is increasing at a rapid pace. Despite the availability of specific media transport protocols like RTP, most content providers make use of the well-established and reliable TCP protocol to deliver audiovisual content over the Internet. The reason is that TCP-based data delivery in general is much less complicated for the clients to be served and over today's networks traversed (including proxies and firewalls), than making use of UDP-based RTP connections. However, in case of network bandwidth fluctuations and packet losses, TCP-based media delivery may lead to annoying jerky playback at the client side, due to retransmissions and late arrival of media data. This papers deals with TCP-based perceptual QoS improvement mechanisms for increasing the media experience for the consumer under unstable network conditions. Our approach is based on media content adaptation (transcoding) to fit the actual network bandwidth continuously monitored by the sender. The proposed mechanisms are applied at the application level at the server side, leaving the existing TCP implementation untouched and therefore enabling transparent use of existing media players. An evaluation of a realistic use case is presented which underlines the efficency of our approach.

Abstract: Supporting transparent delivery and convenient use of multimedia content across a wide range of networks and devices is still a challenging task within the multimedia research community; Universal Multimedia Access (UMA) is a vision that has been pursued for quite some time. In multimedia frameworks, content adaptation is the core concept to make progress toward this goal. Most media adaptation engines targeting UMA scale the content w.r.t. terminal capabilities and network resource constraints and do not sufficiently consider end user preferences or even the utility of the adapted content for the user. Based on our previous work and the support of the MPEG-21 framework, we present a transparent solution to provide a content utility-aware adaptation decision for such utility-unaware multimedia frameworks. The idea is to outsource the challenging utility-aware adaptation decision taking task, which takes many factors into consideration and leads to a complex optimization problem. A realistic use case is adopted to show how related external multimedia frameworks can easily integrate and use our proposed adaptation decision taking Web Service.

Abstract: This paper explores design options and evaluates implementations of in-network, RTP/RTSP based adaptation MANEs (Media Aware Network Elements) for H.264/SVC content streaming. The obvious technique to be employed by such an adaptation MANE is to perform SVC specific bitstream extraction or truncation. Another mechanism that can be used is description (metadata) driven, coding format independent adaptation based on generic Bitstream Syntax Descriptions (gBSD), as specified within MPEG-21 Digital Item Adaptation (DIA). Adaptation MANE architectures for both approaches are developed and presented, implemented in end-to-end streaming/adaptation prototype systems, and experimentally evaluated and compared. For the gBSD based solution, open issues like the granularity of bitstream descriptions and of bitstream adaptation, metadata overhead, metadata packetization and transport options, and error resilience in case of metadata losses, are addressed. The experimental results indicate that a simple SVC specific adaptation MANE does clearly outperform the gBSD based adaptation variants. Yet, the conceptual advantages of the description driven approach, like coding format independence and flexibility, may outweigh the performance drawbacks in specific applications.

Abstract: This paper presents a cross-layer model—formulated using interoperable description formats—for the adaptation of scalable H.264/MPEG-4 AVC (i.e., SVC) content in a video streaming system operating on aWireless LANaccess network without QoS mechanisms.SVCcontent adaptation on the server takes place on the application layer using an adaptation process compliant with the MPEG-21 Digital Item Adaptation (DIA) standard, based on input comprised of MPEG-21 DIA descriptions of content and usage environment parameters. The latter descriptions integrate information from different layers, e.g., device characteristics and packet loss rate, in an attempt to increase the interoperability of this cross-layer model, thus making it applicable to other models. For the sake of deriving model parameters, performance measurements from two wireless access point models were taken in account. Throughout the investigation it emerged that the behavior of the system strongly depends on the access point. Therefore, we investigated the use of end-to-end-based rate control algorithms for steering the content adaptation. Simulations of rate adaptation algorithms were subsequently performed, leading to the conclusion that a TFRC-based adaptation technique (TCP-Friendly Rate Control) performs quite well in adapting to limited bandwidth and varying network conditions. In the paper we demonstrate howTFRC-based content adaptation can be realized using MPEG-21 tools.

Abstract: Recent advances in video coding technology like the scalable extension of the MPEG-4 AVC/H.264 video coding standard pave the way for computationally cheap adaptation of video content. In this paper we present our work on a lightweight RTSP/RTP proxy that enables in-network stream processing. Based on an off-the-shelf wireless router that runs a Linux-based firmware we demonstrate that the video adaptation can be performed on-the-fly directly on a network device. The paper covers design and implementation details of the proxy as well as a discussion about the actual adaptation of the SVC stream. Based on experimental evaluations we show that our approach can handle a reasonable number of concurrent sessions for a typical home deployment scenario. Furthermore, the paper covers possible applications in which adaptation on the network device can be beneficial.

Abstract: The adaptation of multimedia resources is a common method to enable the transport and consumption of audio-visual content in constrained environments. An important aspect in this field is adaptation decision-taking, which aims to find adaptation parameters that maximize the quality for the consumer while considering the constraints of the networks and terminals involved. In this paper we focus on improving the adaptation of audio-visual content by maximizing the perceived quality. This can be realized by using a multimedia quality model and content-related metadata. We present an approach to derive this content-related metadata from subjective tests and use it for adaptation decision-taking within the MPEG-21 multimedia framework.

Abstract: The MPEG-21 standard defines a framework for the interoperable delivery and consumption of multimedia content. Within this framework the adaptation of content plays a vital role in order to support a variety of terminals and to overcome the limitations of the heterogeneous access networks. In most cases the multimedia content can be adapted by applying different adaptation operations that result in certain characteristics of the content. Therefore, an instance within the framework has to decide which adaptation operations have to be performed to achieve a satisfactory result. This process is known as adaptation decision-taking and makes extensive use of metadata describing the possible adaptation operations, the usage environment of the consumer, and constraints concerning the adaptation. Based on this metadata a mathematical optimization problem can be formulated and its solution yields the optimal parameters for the adaptation operations. However, the metadata is represented in XML resulting in a verbose and inefficient encoding. In this paper, an architecture for an Adaptation Decision-Taking Engine (ADTE) is introduced. The ADTE operates both on XML metadata and on metadata encoded with MPEG's Binary Format for Metadata (BiM) enabling an efficient metadata processing by separating the problem extraction from the actual optimization step. Furthermore, several optimization algorithms which are suitable for scalable multimedia formats are reviewed and extended where it was appropriate

Abstract: The adaptation of multimedia resources is a common method to enable the transport and consumption of audio-visual content in constrained environments. An important aspect in this field is adaptation decision-taking, which aims to find adaptation parameters that maximize the quality for the consumer while considering the constraints of the networks and terminals involved. In this paper we focus on improving the adaptation of audio-visual content by maximizing the perceived quality. This can be realized by using a multimedia quality model and content-related metadata. We present an approach to derive this content-related metadata from subjective tests and use it for adaptation decision-taking within the MPEG-21 multimedia framework.

Abstract: The MPEG-21 standard defines a framework for the delivery and consumption of multimedia content. Within this framework the adaptation of content plays a vital role in order to support a variety of terminals and to overcome the limitations of the heterogeneous delivery networks. In most cases the multimedia content can be adapted by applying different adaptation operations that result in certain properties of the content. Therefore, an instance within the framework has to decide which adaptation operations have to be performed to achieve a satisfactory result. This process is known as adaptation decision taking and makes extensively use of metadata describing the possible adaptation operations, the usage environment of the consumer, and constraints concerning the adaptation. Based on this metadata a mathematical optimization problem can be formulated and its solution yields the optimal parameters for the adaptation operations. However, this generic approach enables a very generic class of optimization problems that are difficult to solve. Furthermore, the metadata is represented in XML resulting in a very verbose and ineffcient encoding. In this thesis an architecture for an Adaptation Decision Taking Engine (ADTE) is introduced. The ADTE operates both on XML metadata and on metadata encoded with the Binary Format for Metadata (BiM) enabling an effcient metadata processing. Concerning the solving of the optimization problem three different algorithms, ranging from a simple generate and test approach to the sophisticated Mesh Adaptive Direct Search, were implemented and evaluated.

Abstract: XML-based metadata for digital media is becoming increasingly important, as a consequence also calling for efficient encoding and compression schemes for the storage and transport of this metadata. Moreover, support for streaming the XML metadata in conjunction with the media data is highly desirable. Such support is provided, for instance, by MPEG's Binary Format for Metadata (BiM) encoding approach, which facilitates fragmenting, delivering, and accessing the metadata in so- called Access Units (AUs). In this paper, we present a quantitative evaluation of existing XML metadata compression and encoding techniques, reaching from widely used state-of-the-art data compression algorithms to sophisticated XML-aware encoding schemes. The comparison is based on compressing MPEG-21 generic Bitstream Syntax Descriptions (gBSDs) which can grow to non-negligible sizes. The main conclusion from this investigation is that in terms of pure compression efficiency on XML files, the BiM approach (exemplified by the MPEG reference software as well as a commercial version thereof) is comparable – in terms of performance – with traditional data or specific XML compression tools. However, when XML metadata have to be fragmented, compressed, and streamed in such fragments, the results indicate that the BiM approach is superior to the other schemes.