Abstract: More and more data are structured, stored, and sent over a networ using the Extensible Markup Language (XML) language. There are, however, concerns about the verbosity of XML in such a way that it may restrain further adoption of the language, especially when exchanging XML-based data over heterogeneous networks, and when it is used within constrained (mobile) devices. Therefore, alternative (binary) serialization formats of the XML data become relevant in order to reduce this overhead. However, using binary-encoded XML should not introduce interoperability issues with existing applications nor add additional complexity to new applications. On top of that, it should have a clear cost reduction over the current plain-text serialization format. A first technology is developed within the ISO/IEC Moving Picture Experts Group, namely the Binary MPEG Format for XML. It provides good compression efficiency, ability to (partially) update existingXMLtrees, and facilitates random access into, and manipulation of, the binary-encoded bit stream. Another technique is based on the Abstract Syntax Notation One specification with the Packed Encoding Rules created by the ITU-T. This paper evaluates both techniques as alternative XML serialization formats and introduces a solution for the interoperability concerns. This solution and the alternative serialization formats are validated against two real-life use cases in terms of processing speed and cost reduction. The efficiency of the alternative serialization formats are compared to a classic plain text compression technique, in particular ZIP compression.

Abstract: More and more data are structured, stored, and sent over a networ using the Extensible Markup Language (XML) language. There are, however, concerns about the verbosity of XML in such a way that it may restrain further adoption of the language, especially when exchanging XML-based data over heterogeneous networks, and when it is used within constrained (mobile) devices. Therefore, alternative (binary) serialization formats of the XML data become relevant in order to reduce this overhead. However, using binary-encoded XML should not introduce interoperability issues with existing applications nor add additional complexity to new applications. On top of that, it should have a clear cost reduction over the current plain-text serialization format. A first technology is developed within the ISO/IEC Moving Picture Experts Group, namely the Binary MPEG Format for XML. It provides good compression efficiency, ability to (partially) update existingXMLtrees, and facilitates random access into, and manipulation of, the binary-encoded bit stream. Another technique is based on the Abstract Syntax Notation One specification with the Packed Encoding Rules created by the ITU-T. This paper evaluates both techniques as alternative XML serialization formats and introduces a solution for the interoperability concerns. This solution and the alternative serialization formats are validated against two real-life use cases in terms of processing speed and cost reduction. The efficiency of the alternative serialization formats are compared to a classic plain text compression technique, in particular ZIP compression.

Abstract: Over the last decade, a wide spectrum of (multi­media) content has become available to an increasing number of users who desire to access it through various devices and over heterogeneous networks. Interoperability is the key for enabling transparent and augmented use of (multimedia) content across a wide range of networks and devices. Standardization efforts within the Moving Picture Experts Group (MPEG), in particular MPEG-7 and MPEG-21, aim to provide appropriate tools for achieving this goal of Universal Multimedia Access (UMA). This tutorial provides, in the first place, the concepts of UMA and corresponding MPEG-7 metadata tools built to support these concepts. Subsequently, the vision, an overview, and the state of the art of the emerging MPEG-21 Multimedia Framework are given. Finally, MPEG-21 Digital Item Adaptation (DIA) tools which implement the “Terminal and Networks Characteristics” key element within the whole framework are illustrated in detail. The goal of MPEG-21 DIA is to achieve interoperable transparent access to (distributed) advanced multimedia content by shielding users from network and terminal installation, configuration, management and implementation issues.

Abstract: XML-based metadata is widely adopted across the different communities and plenty of commercial and open source tools for processing and transforming are available on the market. However, all of these tools have one thing in common: they operate on plain text encoded metadata which may become a burden in constrained and streaming environments, i.e., when metadata needs to be processed together with multimedia content on the fly. In this paper we present an efficient approach for transforming such kind of metadata which are encoded using MPEG's Binary Format for Metadata (BiM) without additional en-/decoding overheads, i.e., within the binary domain. Therefore, we have developed an event-based push parser for BiM encoded metadata which transforms the metadata by a limited set of processing instructions – based on traditional XML transformation techniques - operating on bit patterns instead of cost-intensive string comparisons.

Abstract: Digital Item Adaptation (DIA) has been recently standardized as Part 7 of the MPEG-21 Multimedia Framework. This standard specifies tools enabling interoperable communication and adaptation of so-called Digital Items. The adaptation process becomes ever more difficult due to the heterogeneity of terminals and networks utilizing different types of multimedia contents encoded in various coding formats. Other aspects are the users� preferences and accessibility characteristics as well as the natural environment in which the content is consumed. This article describes how to use the tools within DIA in order to build a device and coding format independent adaptation module enabling interoperable multimedia communication.

Abstract: XML-based metadata for digital media is becoming increasingly important, as a consequence also calling for efficient encoding and compression schemes for the storage and transport of this metadata. Moreover, support for streaming the XML metadata in conjunction with the media data is highly desirable. Such support is provided, for instance, by MPEG's Binary Format for Metadata (BiM) encoding approach, which facilitates fragmenting, delivering, and accessing the metadata in so- called Access Units (AUs). In this paper, we present a quantitative evaluation of existing XML metadata compression and encoding techniques, reaching from widely used state-of-the-art data compression algorithms to sophisticated XML-aware encoding schemes. The comparison is based on compressing MPEG-21 generic Bitstream Syntax Descriptions (gBSDs) which can grow to non-negligible sizes. The main conclusion from this investigation is that in terms of pure compression efficiency on XML files, the BiM approach (exemplified by the MPEG reference software as well as a commercial version thereof) is comparable – in terms of performance – with traditional data or specific XML compression tools. However, when XML metadata have to be fragmented, compressed, and streamed in such fragments, the results indicate that the BiM approach is superior to the other schemes.

Abstract: Multimedia is gaining ever more importance on the Internet. This increases the need for intelligent and efficient video caches. A promising approach to improve caching efficiency is to adapt videos. With the availability of MPEG-4 it is possible to develop a standard compliant proxy cache that allows fast and efficient adaptation. We propose a modular design for an adaptive MPEG-4 video proxy that supports efficient full and partial video caching in combination with filtering options that are driven by the terminal capabilities of the client. We use the native scalability operations provided by MPEG-4, the MPEG-7 standard to describe the scalability options for a video and the emerging MPEG-21 standard to describe the terminal capabilities. We restrict ourselves to full video caching. The combination of adaptation with MPEG-4, MPEG-7 and client terminal capabilities is to the best of our knowledge unique and will increase the quality of service for end users. Key words: Adaptation, MPEG-4, MPEG-7, MPEG-21, adaptive proxy, caching.

Abstract: The information revolution of the last decade has resulted in a phenomenal increase in the quantity of multimedia content available to an increasing number of different users with different preferences who access it through a plethora of devices and over heterogeneous networks. In order to address the amount of different content types, MPEG-21 Digital Item Adaptation (DIA) introduces interoperable description tools which enable coding format independent adaptation. Bandwidthefficient transport of the content to terminals with different capabilities and through a variety of access networks with various characteristics requires adaptation facilities not only on the server but also within the network. In this paper we present transport mechanisms for MPEG-21-based metadata enabling generic adaptation within the network. Three different transport mechanisms for delivering this metadata in conjunction with the corresponding multimedia content are evaluated and a payload format for the transport of this metadata is presented. Furthermore, we performed measurements which demonstrate the bandwidth benefits of our distributed adaptation approach compared to server-centric adaptation in a multicast scenario. Finally, we applied various encoding formats for the metadata which further reduces the metadata overhead.

Abstract: The growing demand for multimedia information by different types of users equipped with a large variety of devices and connecting through different kinds of networks results in an increasing amount of different multimedia formats. Research is currently concentrating on the adaptation of the contents in order to provide Universal Multimedia Access (UMA) for the content consumer. But this does not solve the problem of the content provider, who still has to signal this variety of different multimedia formats to the consumer. In this contribution, we show a way to stream any type of multimedia format based on generic hint information. This hint information is based on a generic bit stream syntax description (gBSD) which is used for format-independent content adaptation within the MPEG-21 Multimedia Framework. Ultimately, this can lead to a frame-work which allows generic streaming and generic adaptation anywhere in the network.

Abstract: In this paper, we present a prototype application that receives Digital Video Broadcast (DVB) TV streams on a PC, extracts semantic and syntactic metadata from the MPEG-2 Transport Stream (TS), maps them to MPEG-7 compliant descriptors, and packs these metadata and the associated multimedia content into standard compliant MPEG-21 Digital Items (DIs). In this form, i.e., as DIs, the DVB content can be searched and accessed by clients such as PCs and PDAs, and can be transported over IP networks. The DVB server can be used as a live source of multimedia content to demonstrate multimedia streaming (e.g., via RTP) and adaptation to diverse devices.

Abstract: Video streaming systems in best effort networks have to somehow cope with dynamically changing bandwidth. Various scalable video codecs allow intra-stream adaptation by use of temporal, spatial, or quality (SNR) scalability; optimizations for finer grained scalability are available as layered coding and FGS techniques. However, if there is no scalable video stream at hand, stream switching among pre-encoded stream versions of different bitrates and qualities allows at least coarse-grained adaptation. Those different approaches compete to be the most efficient solution for adaptive video streaming. However, this paper will show that the efficacy is significantly increased by combining those approaches. As will be discussed, the combination of coarse-grained stream switching and temporal intra-stream adaptation offers better visual results and more stable client buffer behavior than the denoted approaches used separately.

Abstract: The diversity of end-terminal and access network capabilities as well as the dynamic nature of wireless connections pose significant challenges to providers of multimedia streaming services. In this paper, we present a system based on MPEG-21 Digital Item Adaptation (DIA) technologies that automatically adapts scalable multimedia resources, like upcoming MPEG-21 Scalable Video Coding (SVC) streams, in a generic and transparent way to the user and session context. This context includes terminal and network capabilities as well as user characteristics. A server side adaptation engine reacts to context changes by dynamic decision taking and accordingly modified bitstream adaptation. Furthermore, novel concepts are presented that facilitate multimedia adaptation in a distributed fashion along the delivery path.

Abstract: The seamless access to rich multimedia content on any device and over an network, usually known as Universal Multimedia Access, requires interoperable description tools and adaptation techniques to be developed. To address the latter issue, MPEG-21 Digital Item Adaptation (DIA) introduces the Bitstream Syntax Description (BSD) framework, which provides tools for adapting multimedia content in a generic (i.e., coding format independent) way. The basic idea is to use the eXtensible Markup Language (XML) to describe the high-level structure of a binary media bitstream, to transform its description (e.g., by means of eXtensible Stylesheet Language Transformations, XSLT), and to construct the adapted media bitstream from the transformed description. This paper presents how this basic BSD framework, initially developed for non-streamed content and suffering from inherent limitations and high memory consumption of XML-related technologies such as XSLT, can be advanced and efficiently implemented in a streaming environment and on resource-constrained devices. Two different attempts to solve the inherent problems are described. The first approach proposes an architecture based on the streamed processing of SAX (Simple Application Programming Interface for XML) events and adopts STX (Streaming Transformations for XML) as an alternative to XSLT, whereas the second approach breaks a BSD up into well-formed fragments called Process Units (PUs) that can be processed individually by a standard XSLT processor. The current status of our work as well as directions for future research are given.

Abstract: In multimedia applications, XML is being increasingly used to represent metadata; examples are MPEG-7 multimedia description schemes and MPEG-21 usage environment descriptions. As with the media data, the size of, or the overhead induced by, the XML metadata is important, particularly when used on constrained mobile devices. Therefore, compression (binary encoding) of the XML data becomes relevant to reduce this overhead. Within the MPEG-7 standardization effort, a Binary Format for Metadata (BiM) was developed, ´providing good compression efficiency and facilitating random access into, and manipulation of, the binary encoded bit stream. However, using binary encoded XML should not introduce interoperability issues with existing applications, nor add additional complexity to new applications. In this paper we investigate a solution for this issue by handling the binary encoded XML data by the XML parser. As such, applications do not need to be aware of the type of encoding of the XML data. In this paper, we introduce such an XML parser and evaluate its usability in different scenarios. We measure the memory requirements and compare the processing speed of parsing binary encoded XML to plain text XML.

Abstract: Adaptation in multimedia systems is usually restricted to defensive, reactive media adaptation (often called stream-level adaptation). We argue that offensive, proactive, system-level adaptation deserves not less attention. If a distributed multimedia system cares for overall, end-to-end quality of service then it should provide a meaningful combination of both. We introduce an adaptive multimedia server (ADMS) and a supporting middleware which implement offensive adaptation based on a lean, flexible architecture. The measured costs and benefits of the offensive adaptation process are presented. We introduce an intelligent video proxy (QBIX), which implements defensive adaptation. The cost/benefit measurements of QBIX are presented elsewhere. We show the benefits of the integration of QBIX in ADMS. Offensive adaptation is used to find an optimal, user-friendly configuration dynamically for ADMS, and defensive adaptation is added to take usage environment (network and terminal) constraints into account.

Abstract: Digital Item Adaptation (DIA) ist der jüngste Standard im Rahmen des ISO/IEC MPEG-21 Multimedia Frameworks. Dieser Standard spezifiziert Deskriptoren und Konzepte (Tools) für die interoperable Kommunikation und ggf. Adaption von komplexen digitalen multimedialen Objekten, sog. Digital Items. Schwierigkeiten der Kommunikation und ggf. die Notwendigkeit der Adaption multimedialer Inhalte entstehen durch die Heterogenität und Beschränkungen der durchlaufenen Netzwerke (z.B. im heutigen Internet) und der benutzten Endgeräte sowie durch unterschiedliche Präferenzen und Profile der Nutzer. Dieser Artikel beschreibt eine endgeräte- und codierformat-unabhängige Komponente zur Adaption von Digital Items an verschiedene mobile Endgeräte und Netzcharakteristika.

Abstract: Due to the increasing availability of audio/visual data on the Internet, proxy caching is gaining on importance as a performance factor. This increase is accompanied by a diversification in the end terminals, which calls for media gateways and filters. An adaptive proxy is presented which performs (1) caching, (2) filtering and (3) media gateway functionality in one. The proxy can perform media adaptation -- using transcoding -- on its own. A cost model is presented which incorporates user requirements, terminal capabilities and video variations in one formula. Based on this model, the proxy acts as a general broker of different user requirements and of different video variations. This is a first step towards "What You Need is What You Get" (WYNIWYG) video services, which deliver videos to users in exactly that quality what they need and what they want to pay for. The MPEG-7 and MPEG-21 standards enables this in an interoperable way. A detailed evaluation based on a series of measurements is provided.

Abstract: The rapid evolution in the hardware sector brought up various (mobile) end user devices like PDAs or cell phones on which online multimedia content can be consumed. Due to different capabilities of these devices as well as individual user preferences, the original multimedia resources have to be adapted in order to fit the specific devices’ constraints and to meet the users’ requirements. Given the high variety of possible adaptation operations both on the format as well as the content level, an intelligent multimedia server must be able to integrate multiple existing and specialized adaptation tools. In this paper, we demonstrate how the usage of modular software components and declarative descriptions of component behavior enables us to develop extensible multimedia adaptation systems. The precise semantics of the utilized functionality description mechanism as well as the defined vocabulary from existing and emerging multimedia standards also allows us to automatically assemble adaptation chains that are executed on a given resource involving multiple, externally provided software components.

Abstract: This paper introduces the principal approach and describes the basic architecture and current implementation of the knowledge-based multimedia adaptation framework we are currently developing. The framework can be used in Universal Multimedia Access scenarios, where multimedia content has to be adapted to specific usage environment parameters (network and client device capabilities, user preferences). Using knowledge-based techniques (state-space planning), the framework automatically computes an adaptation plan, i.e., a sequence of media conversion operations, to transform the multimedia resources to meet the client's requirements or constraints. The system takes as input standards-compliant descriptions of the content (using MPEG-7 metadata) and of the target usage environment (using MPEG-21 Digital Item Adaptation metadata) to derive start and goal states for the planning process, respectively. Furthermore, declarative descriptions of the conversion operations (such as available via software library functions) enable existing adaptation algorithms to be invoked without requiring programming effort. A running example in the paper illustrates the descriptors and techniques employed by the knowledge-based media adaptation system.

Abstract: Modern video streaming servers should adapt, and switch quality levels of, the streamed data according to precise and timely feedback about the network conditions, and should also incorporate selective retransmissions of important reference frames (I- and P-VOPs). This paper evaluates two recent IETF Internet Drafts on RTP extensions for immediate feedback and retransmission and shows, in conjunction with temporal video adaptation, how a substantial visual quality gain can be achieved by using those extensions (up to 4.4 dB PSNR under lossy conditions).

Abstract: If videos are streamed over heterogeneous networks like the Internet, severe bandwidth fluctuations can emerge which hamper seamless transmission to the end user. To overcome this issue, a video's quality and, as a consequence, its bandwidth requirements can be reduced. Quality reduction in the temporal domain (i.e., frame dropping) turned out to be a promising approach because it is fast and easy to perform. In this paper, we present different approaches for temporal video adaptation and we investigate their performance in terms of the achieved visual quality when applied on several videos. The results show that our QCTVA approach (quality controlled temporal video adaptation), based on PSNR evaluation of frames, yields superior quality.

Abstract: Abstract. Universal Multimedia Access (UMA), where users can consume any multimedia resource anywhere at any time, is the driving vision of ongoing ISO/IEC Moving Picture Experts Group (MPEG) standardization efforts. In that context, intelligent adaptation means that before resources are sent over the network, they are prepared according to the client’s device capabilities, the network conditions, or even the user’s content preferences. In this paper, we argue that Semantic Web Services can serve as a key enabling technology to achieve the goals of UMA. As the standards evolve, more and more specialized software tools will be available that provide specific functionalities for adapting the media in different dimensions. When the functionality of such tools is described declaratively with the means of Semantic Web Services technology, intelligent adaptation network nodes can be developed, capable of automatically composing multi-step adaptation sequences and dynamically integrating such services available on the Web. This paper describes the architecture and a prototypical implementation of an intelligent adaptation node that supports automatic, knowledge-based service composition which is made possible by the shared domain ontology defined in MPEG metadata standards.

Abstract: Multimedia content is becoming increasingly important in many areas not only for pure entertainment but also for commercial or educational purposes like, e.g., distance learning or online training. In parallel, the rapid evolution in the hardware sector brought up various new (mobile) end user devices like pocket PCs or mobile phones that are capable of displaying such content. Due to the different capabilities and usage environments of these devices, the basic multimedia content has to be adapted in order to fit the specific devices' capabilities and requirements, whereby such transformations typically include changes in the display size or quality adaptation. Based on the capabilities of the target device that can be expressed using recent multimedia standards like MPEG-21, these adaptation steps are typically carried out by the video server or a proxy node before the data is transferred to the client. In this paper, we present a software framework and implementation of such a multimedia server add-on that advances state-of-the-art technology in two ways. First, the framework supports the integration of various (already existing) multimedia transformation tools based on declarative interface and semantic capability descriptions in a way comparable to Semantic Web Services approaches. Second, by using the components' capability descriptions and the usage environment of the end user device, we employ a knowledge-based planning approach for dynamically constructing and executing the needed transformation program for a specific multi-media content request.

Abstract: In order to enable transparent and augmented use of multimedia resources across a wide range of networks and devices, the forthcoming MPEG-21 standard aims at integrating the various existing technologies for delivery and consumption of digital content in a common multimedia framework. Digital Item Adaptation is one of the core concepts of the framework that will support the adaptation of multimedia resources to device capabilities, underlying network characteristics, or the user’s preferences. The scope of the standardization, however, is limited to the definition of description tools and does not deal with the internals of the adaptation process itself. In this paper, we first discuss the requirements of the resource adaptation component of an adaptation engine. These requirements include, for instance, openness for the integration of external multimedia transforming tools as well as intelligent decision taking when determining the set of required adaptation steps. We also present a prototype of a simple video resource adaptation engine that completely relies on descriptions of the resource itself (MPEG-7), the usage environment of the resource (MPEG-21), as well as declarative descriptions of the transformation tools. The prototype employs a knowledge-based engine for finding and executing the needed adaptation sequences.

Abstract: In multimedia applications, XML is being increasingly used to represent metadata; examples are MPEG-7 multimedia description schemes and MPEG-21 usage environment descriptions. As with the media data, the size of, or the overhead induced by, the XML metadata is important, particularly when used on constrained mobile devices. Therefore, compression (binary encoding) of the XML data becomes relevant to reduce this overhead. Within the MPEG-7 standardization effort, a Binary Format for Metadata (BiM) was developed, providing good compression efficiency and facilitating random access into, and manipulation of, the binary encoded bit stream. In order to support processing of metadata streams in the binary domain and making this task for client applications as simple as possible, we are developing a universal parser for handling both plain text and binary encoded XML-based metadata. The parser exposes a single interface making it transparent for the application whether a plain text or a binary XML document is being processed. As part of this effort, this paper provides a detailed study of five existing XML parser models and evaluates their applicability to serve as a model for parsing binary XML data, encoded using the BiM codec. Additionally, the parser models are investigated against important usage scenarios enabled by BiM, such as dynamic updates of XML data. From the five models, two are rejected and one is only applicable for domain specific applications. Of the remaining two, one model is proposed as preferred model because of different advantages over the other model.