Abstract: In order to enable transparent and convenient use of multimedia content across a wide range of networks and devices, content adaptation is an important issue within multimedia frameworks. The so called Digital Item Adaptation (DIA) standard is one of the core concepts of the MPEG-21 framework that will support the adaptation of multimedia resources according to device capabilities, underlying network characteristics, and user preferences. Most multimedia adaptation engines for providing Universal Multimedia Access (UMA) scale the content with respect to terminal capabilities and resource constraints. This paper focuses on the cross-modal adaptation decision taking process considering the user environment and terminal capabilities as well as resource limitations on the server, network, and client side. This approach represents a step toward increased Universal Multimedia Experience (UME). Based on four different algorithms for solving this optimization process, we present an evaluation of results gained by running their implementations on different test networks.

Abstract: In order to enable transparent and augmented use of multimedia content across a wide range of networks and devices, content adaptation is an important issue within multimedia frameworks. In this paper, we present a prototype application that receives Digital Video Broadcast (DVB) TV streams on a PC, transcodes the streams on the fly according to the individual User requirements and packs the adapted content together with available metadata into a standard compliant MPEG-21 Digital Item (DI). In this form, the framework enables the live Universal Multimedia Access (UMA) scenario where the DVB content can be transparently accessed by clients such as PCs and PDAs, anytime and anywhere.

Abstract: Content adaptation is an important issue of multimedia frameworks in order to achieve Universal Multimedia Access (UMA), that is, to enable consumption of individual multimedia content independently of the given resource limitations, terminal capabilities and user preferences. The Digital Item Adaptation (DIA) standard, one of the core specifications of the MPEG-21 framework, supports content adaptation considering a wide range of networks, devices and user preferences. Most adaptive multimedia frameworks enabling this UMA do not consider utility aspects in their adaptation decisions. This paper focuses on a generic multi-modal utility model for DIA, enabling increased multimedia experience to the client. Our proposed model is able to take the semantic and the perceptual features of the content as well as the clients individual utility aspects into consideration. Based on a detailed analysis of these constraints we will show how the model reacts on individual input data. Finally we will discuss results of the multi-modal decision taking process according to a few use case scenarios as well.

Abstract: This paper focuses on an approach for real-time metal extraction of x-ray images taken from modernx-ray machines like C-arms. Such machines are used for vessel diagnostics, surgical interventions, as well as cardiology, neurology and orthopedic examinations. They are very fast in taking images from different angles. For this reason, manual adjustment of contrast is infeasible and automatic adjustment algorithms have been applied to try to select the optimal radiation dose for contrast adjustment. Problems occur when metallic objects, e.g., a prosthesis or a screw, are in the absorption area of interest. In this case, the automatic adjustment mostly fails because the dark, metallic objects lead the algorithm to overdose the x-ray tube. This outshining effect results in overexposed images and bad contrast. To overcome this limitation, metallic objects have to be detected and extracted from images that are taken as input for the adjustment algorithm.In this paper, we present a real-time solution for extracting metallic objects of x-ray images. We will explore the characteristic features of metallic objects in x-ray images and their distinction from bone fragments which form the basis to find a successful way for object segmentation and classification. Subsequently, we will present our edge based real-time approach for successful and fast automatic segmentation and classification of metallic objects. Finally, experimental results on the effectiveness and performance of our approach based on a vast amount of input image data sets will be presented.

Abstract: The adaptation of multimedia resources is a common method to enable the transport and consumption of audio-visual content in constrained environments. An important aspect in this field is adaptation decision-taking, which aims to find adaptation parameters that maximize the quality for the consumer while considering the constraints of the networks and terminals involved. In this paper we focus on improving the adaptation of audio-visual content by maximizing the perceived quality. This can be realized by using a multimedia quality model and content-related metadata. We present an approach to derive this content-related metadata from subjective tests and use it for adaptation decision-taking within the MPEG-21 multimedia framework.

Abstract: The delivery and adaptation of multimedia content in dis tributed and heterogeneous environments requires flexible control and management mechanisms in terminals and in control entities inside the network. In the near future, it is important to reach interoperability between the IETF ap proaches on multimedia session establishment and control and the MPEG-21 efforts for multimedia streaming and adaptation to bring advanced multimedia service provi sioning and adaptation services towards the customer. MPEG-21 Digital Item Adaptation (DIA) provides norma tive descriptions for supporting adaptation of multimedia content, but does not define interactions with transport and control mechanisms. On the other hand, the IETF standardization efforts on multimedia session control pro vide the necessary transport (e.g. RTP) and control mechanisms (SDP/SDPng). We thus bridge the gap be tween those approaches by creating a converged XML model that enables the integration of session management and negotiation protocols (e.g. SIP or Megaco) inspired by the XML formats of MPEG-21 DIA and SDPng. We also present preliminary implementation results of the con verged model along with concepts and implementation of network-based content adaptation mechanisms through media gateways that enable flexible multimedia manage ment for heterogeneous consumer terminals.

Abstract: Over the last decade, a wide spectrum of (multi­media) content has become available to an increasing number of users who desire to access it through various devices and over heterogeneous networks. Interoperability is the key for enabling transparent and augmented use of (multimedia) content across a wide range of networks and devices. Standardization efforts within the Moving Picture Experts Group (MPEG), in particular MPEG-7 and MPEG-21, aim to provide appropriate tools for achieving this goal of Universal Multimedia Access (UMA). This tutorial provides, in the first place, the concepts of UMA and corresponding MPEG-7 metadata tools built to support these concepts. Subsequently, the vision, an overview, and the state of the art of the emerging MPEG-21 Multimedia Framework are given. Finally, MPEG-21 Digital Item Adaptation (DIA) tools which implement the “Terminal and Networks Characteristics” key element within the whole framework are illustrated in detail. The goal of MPEG-21 DIA is to achieve interoperable transparent access to (distributed) advanced multimedia content by shielding users from network and terminal installation, configuration, management and implementation issues.

Abstract: XML-based metadata is widely adopted across the different communities and plenty of commercial and open source tools for processing and transforming are available on the market. However, all of these tools have one thing in common: they operate on plain text encoded metadata which may become a burden in constrained and streaming environments, i.e., when metadata needs to be processed together with multimedia content on the fly. In this paper we present an efficient approach for transforming such kind of metadata which are encoded using MPEG's Binary Format for Metadata (BiM) without additional en-/decoding overheads, i.e., within the binary domain. Therefore, we have developed an event-based push parser for BiM encoded metadata which transforms the metadata by a limited set of processing instructions – based on traditional XML transformation techniques - operating on bit patterns instead of cost-intensive string comparisons.

Abstract: XML-based metadata is widely adopted across the different communities and plenty of commercial and open source tools for processing and transforming are available on the market. However, all of these tools have the same requirement: they operate on plain text encoded metadata which may become a burden especially in constrained and streaming environments, e.g., when metadata needs to be processed together with multimedia content which is available in a highly efficient, binary representation format. In this paper we present techniques for transforming such kind of metadata which is encoded using the well known MPEG-7 Systems Binary Format for Metadata (BiM) without additional en-/decoding overheads, i.e., within the binary domain. As such it enables us to process both the multimedia data as well as the metadata within its compressed domain, e.g., for metadata-driven adaptation purposes within intermediary network nodes which are becoming increasingly popular in the multimedia community as well as in the XML community.

Abstract: XML-based metadata for digital media is becoming increasingly important, as a consequence also calling for efficient encoding and compression schemes for the storage and transport of this metadata. Moreover, support for streaming the XML metadata in conjunction with the media data is highly desirable. Such support is provided, for instance, by MPEG's Binary Format for Metadata (BiM) encoding approach, which facilitates fragmenting, delivering, and accessing the metadata in so- called Access Units (AUs). In this paper, we present a quantitative evaluation of existing XML metadata compression and encoding techniques, reaching from widely used state-of-the-art data compression algorithms to sophisticated XML-aware encoding schemes. The comparison is based on compressing MPEG-21 generic Bitstream Syntax Descriptions (gBSDs) which can grow to non-negligible sizes. The main conclusion from this investigation is that in terms of pure compression efficiency on XML files, the BiM approach (exemplified by the MPEG reference software as well as a commercial version thereof) is comparable – in terms of performance – with traditional data or specific XML compression tools. However, when XML metadata have to be fragmented, compressed, and streamed in such fragments, the results indicate that the BiM approach is superior to the other schemes.

Abstract: Universal Multimedia Access (UMA) has become a driving concept behind a significant amount of research activities. One of MPEG’s (Moving Pictures Experts Group) responses to UMA is MPEG-21 Digital Item Adaptation (DIA). In this paper we present how tools as specified within DIA (i.e., normative XML-based description formats) are applied in streaming and constrained environments enabling piece-wise multimedia content adaptation including the adaptation decision- taking process and the actual resource adaptation in a coding format-independent way. Additionally, we demonstrate how the metadata overhead imposed by DIA tools can be reduced by means of appropriate metadata encoding tools.

Abstract: The information revolution of the last decade has resulted in a phenomenal increase in the quantity of multimedia content available to an increasing number of different users with different preferences who access it through a plethora of devices and over heterogeneous networks. In order to address the amount of different content types, MPEG-21 Digital Item Adaptation (DIA) introduces interoperable description tools which enable coding format independent adaptation. Bandwidthefficient transport of the content to terminals with different capabilities and through a variety of access networks with various characteristics requires adaptation facilities not only on the server but also within the network. In this paper we present transport mechanisms for MPEG-21-based metadata enabling generic adaptation within the network. Three different transport mechanisms for delivering this metadata in conjunction with the corresponding multimedia content are evaluated and a payload format for the transport of this metadata is presented. Furthermore, we performed measurements which demonstrate the bandwidth benefits of our distributed adaptation approach compared to server-centric adaptation in a multicast scenario. Finally, we applied various encoding formats for the metadata which further reduces the metadata overhead.

Abstract: The growing demand for multimedia information by different types of users equipped with a large variety of devices and connecting through different kinds of networks results in an increasing amount of different multimedia formats. Research is currently concentrating on the adaptation of the contents in order to provide Universal Multimedia Access (UMA) for the content consumer. But this does not solve the problem of the content provider, who still has to signal this variety of different multimedia formats to the consumer. In this contribution, we show a way to stream any type of multimedia format based on generic hint information. This hint information is based on a generic bit stream syntax description (gBSD) which is used for format-independent content adaptation within the MPEG-21 Multimedia Framework. Ultimately, this can lead to a frame-work which allows generic streaming and generic adaptation anywhere in the network.

Abstract: In this paper, we present a prototype application that receives Digital Video Broadcast (DVB) TV streams on a PC, extracts semantic and syntactic metadata from the MPEG-2 Transport Stream (TS), maps them to MPEG-7 compliant descriptors, and packs these metadata and the associated multimedia content into standard compliant MPEG-21 Digital Items (DIs). In this form, i.e., as DIs, the DVB content can be searched and accessed by clients such as PCs and PDAs, and can be transported over IP networks. The DVB server can be used as a live source of multimedia content to demonstrate multimedia streaming (e.g., via RTP) and adaptation to diverse devices.

Abstract: Video streaming systems in best effort networks have to somehow cope with dynamically changing bandwidth. Various scalable video codecs allow intra-stream adaptation by use of temporal, spatial, or quality (SNR) scalability; optimizations for finer grained scalability are available as layered coding and FGS techniques. However, if there is no scalable video stream at hand, stream switching among pre-encoded stream versions of different bitrates and qualities allows at least coarse-grained adaptation. Those different approaches compete to be the most efficient solution for adaptive video streaming. However, this paper will show that the efficacy is significantly increased by combining those approaches. As will be discussed, the combination of coarse-grained stream switching and temporal intra-stream adaptation offers better visual results and more stable client buffer behavior than the denoted approaches used separately.

Abstract: The diversity of end-terminal and access network capabilities as well as the dynamic nature of wireless connections pose significant challenges to providers of multimedia streaming services. In this paper, we present a system based on MPEG-21 Digital Item Adaptation (DIA) technologies that automatically adapts scalable multimedia resources, like upcoming MPEG-21 Scalable Video Coding (SVC) streams, in a generic and transparent way to the user and session context. This context includes terminal and network capabilities as well as user characteristics. A server side adaptation engine reacts to context changes by dynamic decision taking and accordingly modified bitstream adaptation. Furthermore, novel concepts are presented that facilitate multimedia adaptation in a distributed fashion along the delivery path.

Abstract: In multimedia applications, XML is being increasingly used to represent metadata; examples are MPEG-7 multimedia description schemes and MPEG-21 usage environment descriptions. As with the media data, the size of, or the overhead induced by, the XML metadata is important, particularly when used on constrained mobile devices. Therefore, compression (binary encoding) of the XML data becomes relevant to reduce this overhead. Within the MPEG-7 standardization effort, a Binary Format for Metadata (BiM) was developed, ´providing good compression efficiency and facilitating random access into, and manipulation of, the binary encoded bit stream. However, using binary encoded XML should not introduce interoperability issues with existing applications, nor add additional complexity to new applications. In this paper we investigate a solution for this issue by handling the binary encoded XML data by the XML parser. As such, applications do not need to be aware of the type of encoding of the XML data. In this paper, we introduce such an XML parser and evaluate its usability in different scenarios. We measure the memory requirements and compare the processing speed of parsing binary encoded XML to plain text XML.

Abstract: Digital Item Adaptation (DIA) ist der jüngste Standard im Rahmen des ISO/IEC MPEG-21 Multimedia Frameworks. Dieser Standard spezifiziert Deskriptoren und Konzepte (Tools) für die interoperable Kommunikation und ggf. Adaption von komplexen digitalen multimedialen Objekten, sog. Digital Items. Schwierigkeiten der Kommunikation und ggf. die Notwendigkeit der Adaption multimedialer Inhalte entstehen durch die Heterogenität und Beschränkungen der durchlaufenen Netzwerke (z.B. im heutigen Internet) und der benutzten Endgeräte sowie durch unterschiedliche Präferenzen und Profile der Nutzer. Dieser Artikel beschreibt eine endgeräte- und codierformat-unabhängige Komponente zur Adaption von Digital Items an verschiedene mobile Endgeräte und Netzcharakteristika.

Abstract: The rapid evolution in the hardware sector brought up various (mobile) end user devices like PDAs or cell phones on which online multimedia content can be consumed. Due to different capabilities of these devices as well as individual user preferences, the original multimedia resources have to be adapted in order to fit the specific devices’ constraints and to meet the users’ requirements. Given the high variety of possible adaptation operations both on the format as well as the content level, an intelligent multimedia server must be able to integrate multiple existing and specialized adaptation tools. In this paper, we demonstrate how the usage of modular software components and declarative descriptions of component behavior enables us to develop extensible multimedia adaptation systems. The precise semantics of the utilized functionality description mechanism as well as the defined vocabulary from existing and emerging multimedia standards also allows us to automatically assemble adaptation chains that are executed on a given resource involving multiple, externally provided software components.

Abstract: This paper introduces the principal approach and describes the basic architecture and current implementation of the knowledge-based multimedia adaptation framework we are currently developing. The framework can be used in Universal Multimedia Access scenarios, where multimedia content has to be adapted to specific usage environment parameters (network and client device capabilities, user preferences). Using knowledge-based techniques (state-space planning), the framework automatically computes an adaptation plan, i.e., a sequence of media conversion operations, to transform the multimedia resources to meet the client's requirements or constraints. The system takes as input standards-compliant descriptions of the content (using MPEG-7 metadata) and of the target usage environment (using MPEG-21 Digital Item Adaptation metadata) to derive start and goal states for the planning process, respectively. Furthermore, declarative descriptions of the conversion operations (such as available via software library functions) enable existing adaptation algorithms to be invoked without requiring programming effort. A running example in the paper illustrates the descriptors and techniques employed by the knowledge-based media adaptation system.

Abstract: Modern video streaming servers should adapt, and switch quality levels of, the streamed data according to precise and timely feedback about the network conditions, and should also incorporate selective retransmissions of important reference frames (I- and P-VOPs). This paper evaluates two recent IETF Internet Drafts on RTP extensions for immediate feedback and retransmission and shows, in conjunction with temporal video adaptation, how a substantial visual quality gain can be achieved by using those extensions (up to 4.4 dB PSNR under lossy conditions).

Abstract: If videos are streamed over heterogeneous networks like the Internet, severe bandwidth fluctuations can emerge which hamper seamless transmission to the end user. To overcome this issue, a video's quality and, as a consequence, its bandwidth requirements can be reduced. Quality reduction in the temporal domain (i.e., frame dropping) turned out to be a promising approach because it is fast and easy to perform. In this paper, we present different approaches for temporal video adaptation and we investigate their performance in terms of the achieved visual quality when applied on several videos. The results show that our QCTVA approach (quality controlled temporal video adaptation), based on PSNR evaluation of frames, yields superior quality.

Abstract: Abstract. Universal Multimedia Access (UMA), where users can consume any multimedia resource anywhere at any time, is the driving vision of ongoing ISO/IEC Moving Picture Experts Group (MPEG) standardization efforts. In that context, intelligent adaptation means that before resources are sent over the network, they are prepared according to the client’s device capabilities, the network conditions, or even the user’s content preferences. In this paper, we argue that Semantic Web Services can serve as a key enabling technology to achieve the goals of UMA. As the standards evolve, more and more specialized software tools will be available that provide specific functionalities for adapting the media in different dimensions. When the functionality of such tools is described declaratively with the means of Semantic Web Services technology, intelligent adaptation network nodes can be developed, capable of automatically composing multi-step adaptation sequences and dynamically integrating such services available on the Web. This paper describes the architecture and a prototypical implementation of an intelligent adaptation node that supports automatic, knowledge-based service composition which is made possible by the shared domain ontology defined in MPEG metadata standards.

Abstract: Multimedia content is becoming increasingly important in many areas not only for pure entertainment but also for commercial or educational purposes like, e.g., distance learning or online training. In parallel, the rapid evolution in the hardware sector brought up various new (mobile) end user devices like pocket PCs or mobile phones that are capable of displaying such content. Due to the different capabilities and usage environments of these devices, the basic multimedia content has to be adapted in order to fit the specific devices' capabilities and requirements, whereby such transformations typically include changes in the display size or quality adaptation. Based on the capabilities of the target device that can be expressed using recent multimedia standards like MPEG-21, these adaptation steps are typically carried out by the video server or a proxy node before the data is transferred to the client. In this paper, we present a software framework and implementation of such a multimedia server add-on that advances state-of-the-art technology in two ways. First, the framework supports the integration of various (already existing) multimedia transformation tools based on declarative interface and semantic capability descriptions in a way comparable to Semantic Web Services approaches. Second, by using the components' capability descriptions and the usage environment of the end user device, we employ a knowledge-based planning approach for dynamically constructing and executing the needed transformation program for a specific multi-media content request.

Abstract: In order to enable transparent and augmented use of multimedia resources across a wide range of networks and devices, the forthcoming MPEG-21 standard aims at integrating the various existing technologies for delivery and consumption of digital content in a common multimedia framework. Digital Item Adaptation is one of the core concepts of the framework that will support the adaptation of multimedia resources to device capabilities, underlying network characteristics, or the user’s preferences. The scope of the standardization, however, is limited to the definition of description tools and does not deal with the internals of the adaptation process itself. In this paper, we first discuss the requirements of the resource adaptation component of an adaptation engine. These requirements include, for instance, openness for the integration of external multimedia transforming tools as well as intelligent decision taking when determining the set of required adaptation steps. We also present a prototype of a simple video resource adaptation engine that completely relies on descriptions of the resource itself (MPEG-7), the usage environment of the resource (MPEG-21), as well as declarative descriptions of the transformation tools. The prototype employs a knowledge-based engine for finding and executing the needed adaptation sequences.