Abstract: If videos are streamed over heterogeneous networks like the Internet, severe bandwidth fluctuations can emerge which hamper seamless transmission to the end user. To overcome this issue, a video's quality and, as a consequence, its bandwidth requirements can be reduced. Quality reduction in the temporal domain (i.e., frame dropping) turned out to be a promising approach because it is fast and easy to perform. In this paper, we present different approaches for temporal video adaptation and we investigate their performance in terms of the achieved visual quality when applied on several videos. The results show that our QCTVA approach (quality controlled temporal video adaptation), based on PSNR evaluation of frames, yields superior quality.

Abstract: Abstract. Universal Multimedia Access (UMA), where users can consume any multimedia resource anywhere at any time, is the driving vision of ongoing ISO/IEC Moving Picture Experts Group (MPEG) standardization efforts. In that context, intelligent adaptation means that before resources are sent over the network, they are prepared according to the client’s device capabilities, the network conditions, or even the user’s content preferences. In this paper, we argue that Semantic Web Services can serve as a key enabling technology to achieve the goals of UMA. As the standards evolve, more and more specialized software tools will be available that provide specific functionalities for adapting the media in different dimensions. When the functionality of such tools is described declaratively with the means of Semantic Web Services technology, intelligent adaptation network nodes can be developed, capable of automatically composing multi-step adaptation sequences and dynamically integrating such services available on the Web. This paper describes the architecture and a prototypical implementation of an intelligent adaptation node that supports automatic, knowledge-based service composition which is made possible by the shared domain ontology defined in MPEG metadata standards.

Abstract: Multimedia content is becoming increasingly important in many areas not only for pure entertainment but also for commercial or educational purposes like, e.g., distance learning or online training. In parallel, the rapid evolution in the hardware sector brought up various new (mobile) end user devices like pocket PCs or mobile phones that are capable of displaying such content. Due to the different capabilities and usage environments of these devices, the basic multimedia content has to be adapted in order to fit the specific devices' capabilities and requirements, whereby such transformations typically include changes in the display size or quality adaptation. Based on the capabilities of the target device that can be expressed using recent multimedia standards like MPEG-21, these adaptation steps are typically carried out by the video server or a proxy node before the data is transferred to the client. In this paper, we present a software framework and implementation of such a multimedia server add-on that advances state-of-the-art technology in two ways. First, the framework supports the integration of various (already existing) multimedia transformation tools based on declarative interface and semantic capability descriptions in a way comparable to Semantic Web Services approaches. Second, by using the components' capability descriptions and the usage environment of the end user device, we employ a knowledge-based planning approach for dynamically constructing and executing the needed transformation program for a specific multi-media content request.

Abstract: In order to enable transparent and augmented use of multimedia resources across a wide range of networks and devices, the forthcoming MPEG-21 standard aims at integrating the various existing technologies for delivery and consumption of digital content in a common multimedia framework. Digital Item Adaptation is one of the core concepts of the framework that will support the adaptation of multimedia resources to device capabilities, underlying network characteristics, or the user’s preferences. The scope of the standardization, however, is limited to the definition of description tools and does not deal with the internals of the adaptation process itself. In this paper, we first discuss the requirements of the resource adaptation component of an adaptation engine. These requirements include, for instance, openness for the integration of external multimedia transforming tools as well as intelligent decision taking when determining the set of required adaptation steps. We also present a prototype of a simple video resource adaptation engine that completely relies on descriptions of the resource itself (MPEG-7), the usage environment of the resource (MPEG-21), as well as declarative descriptions of the transformation tools. The prototype employs a knowledge-based engine for finding and executing the needed adaptation sequences.

Abstract: Multimedia services typically need not only huge resources but also a fairly stable level of Quality of Services. This requires server architectures that enable continuous adaptation. The Adaptive Distributed Multimedia Server (ADMS) of the University Klagenfurt is able to dynamically add and remove nodes to the actual configuration, thus realizing the offensive adaptation approach. This paper focuses on the optimal placement of nodes for hosting certain ADMS components (the so-called data collectors, collecting and streaming stripe units of a video) in the network. We propose four different algorithms for host recommendation and compare the results gained by running their implementations on different test networks. The greedy algorithm seems to be a clear looser. Among the three other algorithms (particle swarm, linear programming and incremental) there is no single winner of the comparison, they can be applied in a smart combination.

Abstract: In multimedia applications, XML is being increasingly used to represent metadata; examples are MPEG-7 multimedia description schemes and MPEG-21 usage environment descriptions. As with the media data, the size of, or the overhead induced by, the XML metadata is important, particularly when used on constrained mobile devices. Therefore, compression (binary encoding) of the XML data becomes relevant to reduce this overhead. Within the MPEG-7 standardization effort, a Binary Format for Metadata (BiM) was developed, providing good compression efficiency and facilitating random access into, and manipulation of, the binary encoded bit stream. In order to support processing of metadata streams in the binary domain and making this task for client applications as simple as possible, we are developing a universal parser for handling both plain text and binary encoded XML-based metadata. The parser exposes a single interface making it transparent for the application whether a plain text or a binary XML document is being processed. As part of this effort, this paper provides a detailed study of five existing XML parser models and evaluates their applicability to serve as a model for parsing binary XML data, encoded using the BiM codec. Additionally, the parser models are investigated against important usage scenarios enabled by BiM, such as dynamic updates of XML data. From the five models, two are rejected and one is only applicable for domain specific applications. Of the remaining two, one model is proposed as preferred model because of different advantages over the other model.

Abstract: Adaptation is usually restricted to defensive, reactive media adaptation (often called stream-level adaptation). We argue that offensive, proactive, system-level adaptation deserves not less attention. If a distributed multimedia system cares for overall, end-to-end quality of service then it should provide a meaningful combination of both. We introduce an adaptive multimedia server (ADMS) and a supporting middleware which implement offensive adaptation based on a lean, flexible architecture. The measured costs and benefits of the offensive adaptation process are presented. We introduce an intelligent video proxy (QBIX), which implements defensive adaptation. The cost/benefit measurements of QBIX are presented elsewhere \citeSchojer03. We show the benefits of the integration of QBIX in ADMS. Offensive adaptation is used to find an optimal, user-friendly configuration dynamically for ADMS, and defensive adaptation is added to take usage environment (network and terminal) constraints into account.

Abstract: Abstract. This paper presents an adaptive distributed multimedia streaming server architecture (ADMS) which explicitly controls the serverlayout. It consists of four types of streaming server components, which all provide dedicated services in an arbitrary number of instances on an arbitrary number of server hosts. Vagabond2 is used as the underlying middleware for component adaptation. It is shown, how the CORBA-based components have to be declared in order to run on top of Vagabond2. Finally, inter-component dependencies are pointed out, which have to be taken into account during component adaptations.

Abstract: We present an adaptive distributed multimedia server architecture (ADMS) that builds upon the idea of offensive adaptivity, where the server proactively controls its layout through replication or migration of server components to recommended hosts. Proactive actions are taken when network or server resources become critical when fulfilling client demands. Recommendations are provided by a so-called “host recommender” which represents an integral part of Vagabond2 - the middleware used for component distribution. Recommendations are based on measured or estimated server and network resource availabilities. Network distance and host resource metrics - obtained from network and host resource services respectively - may be communicated as MPEG-21 DIA descriptors. Finally we evaluate our architecture in a real-world streaming scenario.

Abstract: Due to the heterogeneity of the current terminal and network infrastructures, multimedia content needs to be adapted to specific capabilities of these terminals and network devices. Furthermore, user preferences and user environment characteristics must also be taken into consideration. The problem becomes even more complex by the diversity of multimedia content types and encoding formats. In order to meet this heterogeneity and to be applicable to different coding formats, the adaptation must be performed in a generic and interoperable way. As a response to this problem and in the context of MPEG-21, we present an approach which uses XML to describe the high-level structure of a multimedia resource in a generic way, i.e., how the multimedia content is organized, for instance in layers, frames, or scenes. For this purpose, a schema for XML-based bitstream syntax descriptions (generic Bitstream Syntax Descriptions or gBSDs) has been developed. A gBSD can describe the high-level structure of a multimedia resource in a coding format independent way. Adaptation of the resource is based on elementary transformation instructions formulated with respect to the gBSDs. These instructions have been separated from the gBSDs in order to use the same descriptions for different adaptations, e.g., temporal scaling, SNR scaling, or semantic adaptations. In the MPEG-21 framework, those adaptations can be steered for instance by the network characteristics and the user preferences. As a result, it becomes possible for coding format agnostic adaptation engines to transform media bitstreams and associated descriptions to meet the requirements imposed by the network conditions, device capabilities, and user preferences.

Abstract: Due to the increasing availability and use of digital video data on the Web, video caching will be an important performance factor in the future WWW. We propose an architecture of a video proxy cache that integrates modern multimedia and communication standards. Especially we describe features of the MPEG-4 and MPEG-7 multimedia standards that can be helpful for a video proxy cache. QBIX supports real-time adaptation in the compressed and in the decompressed domain. It uses adaptation to improve the cache replacement strategies in the proxy, but also to realize media gateway functionality driven by the clients' terminal capabilities.

Abstract: Multimedia streaming is becoming ever more popular. However, the Internet does not support streaming with its high bandwidth and low latency requirements very well. The problem is that QoS guarantees cannot be given. Hence, communication partners have to deal with rapidly changing connection parameters. This requires sophisticated streaming concepts that can handle these varying conditions using adaptation techniques. Adaptation methods can be dropping layers, dropping access units or transcoding the contents. But this places specific requirements on the underlying protocol. This paper identifies and discusses these requirements and analyzes how existing protocols can meet them. Unfortunately, none of the known protocols can meet all requirements. Hence, we propose a new adaptation-aware multimedia streaming protocol that can operate as required in the given Internet environment. Furthermore, we show how this protocol can be used to carry MPEG-4 audio-visual contents.

Abstract: Multimedia streaming is becoming more and more popular. Seamless video streaming in heterogeneous networks like the Internet turns out as almost impossible due to varying network conditions - streams must be adapted to the current network QoS. Temporal scalability is one of the most reasonable adaptation techniques because it is fast and easy to perform. Today's approaches simply drop frames out of a video without spending much effort on finding an intelligent dropping behavior. This usually leads to good adaptation results in terms of bandwidth consumption but also to suboptimal video quality within the given bounds. Our approach offers analysis of video streams to achieve the qualitatively best temporal scalability. For this reason, we introduce a data structure called modification lattice which represents all frame dropping combinations within a sequence of frames. On the basis of the modification lattice, qualityestimations on frame sequences can be performed. Moreover, a heuristic for fast and efficient quality computation in a modification lattice is presented. Experimental results illustrate that temporal video adaptation based on QCTVA information leads to a better video quality compared to "usual" frame dropping approaches. Furthermore, QCTVA offers frame priority lists for videos. Based on these priorities, numerous adaptation techniques can increase their overall performance when using QCTVA.

Abstract: Broadly used Database Management Systems (DBMS) are not able to tackle the requirements of multimedia in querying, indexing and content modeling. Therefore, extenders for multimedia data types have been proposed. These extensions, however, offer only limited semantic modeling and rely on basic index structures which do not meet the whole nature of multimedia, for instance for a Nearest-Neighbor Search. In this context, the paper presents a methodology for enhancing extensible ORDBMS for multimedia data. In particular, we introduce an MPEG-7 Multimedia Data Cartridge which includes a semantically rich metadata model for multimedia content relying on the MPEG-7 standard. Furthermore, to fulfill the needs for efficient multimedia query processing, we created in this Cartridge a new indexing and query framework for various types of retrieval operations

Abstract: Bei der Indizierung von Videoarchiven haben im allgemeinen bessere Ergebnisse einen höheren Preis. Automatische Indizierung, basierend auf physikalischen Eigenschaften, kann für viele Bereiche (wie z.B. ein Grossteil der Unterhaltung) ausreichend gute Ergebnisse erzielen. Sie ist allerdings in der Genauigkeit der Fragestellungen eingeschränkt, und zwingt uns – oft unbewusste – Suchkriterien auf. Wenn es uns wichtig ist, auf fein differenzierte Fragen genaue Antworten zu erhalten (wie das z.B. in wissenschaftlich/technischen Systemen meistens der Fall ist), so sind wir auf eine zumindest teilweise manuelle Annotierung angewiesen. In einem kombinierten System – wie dem VIDEX-Modell – können die physikalischen Eigenschaften dazu benutzt werden, den manuellen Annotierungsprozess zu erleichtern. Kombinierte Systeme dieser Art bieten beide Arten der Indizierung an und kombinieren sie miteinander. Längerfristig ist zu erwarten, dass sich zu jeder Anwendung die angemessenen Indizierungs- und Suchmethoden herausstellen. Die zwei Verfahren – das automatische und das teils manuelle – sollten nicht als Gegner, sondern als Partner angesehen werden.

Abstract: Multimedia is gaining ever more importance on the Internet. This increases the need for intelligent and efficient video caches. Typical Web proxies were not designed to efficiently support the caching of videos. A promising approach to improve caching efficiency is to adapt videos. With the availability of MPEG-4 it is possible to develop a standard compliant proxy that allows fast and efficient adaptation. We propose a modular design for an adaptive MPEG-4 video proxy that supports efficient full and partial video caching in combination with filtering options that are driven by the terminal capabilities of the client. We use the native scalability operations provided by MPEG-4 and use the emerging MPEG-7 standard to describe the scalability options for a video. The proxy parses the MPEG-7 description and decides, based on this description and the terminal capabilities of the client, which adaptation step to choose. Simple MPEG-4 audio-visual streams are supported by filter operations in the compressed domain that realize several temporal scaling algorithms and color reduction. In this paper, we will restrict ourselves to full video caching. The combination of adaptation with MPEG-4, MPEG-7 and client terminal capabilities is to the best of our knowledge unique and will increase the quality of service for end users.

Abstract: Due to the future dominance of video data video caching will be an important performance factor in future networked multimedia systems. A major component of video caches is the replacement strategy. This paper presents replacement strategies for video caches that incorporate quality reduction and the use of metadata given by the content provider. The strategies are evaluated by simulation.

Abstract: Multimedia streaming becomes ever more popular. The multimedia standard MPEG-4 has been designed to support scenes of different levels of complexity and applications with low bandwidth requirements up to very high bandwidth requirements. One protocol suitable to transfer this kind of data over IP networks is the real-time transport protocol (RTP). This report describes standardized and proposed payload formats that support the transport of MPEG-4 elementary streams over RTP connections. These RTP packetization formats are compared w.r.t. their suitability for the adaptation (scaling) of the media data within the network, i.e., by advanced routers or proxy caches. This adaptation process is governed by metadata that need to be transferred and inspected in conjunction with the media streams.

Abstract: This work introduces a framework for adapting MPEG-4 intra- and inter-Elementary Streams and for encoding the results in an MPEG-7 stream to be used for resource adaptation on the delivery path to the user.

Abstract: Multimedia Database Systems (MMDBMS) organize and store multimedia data for content retrieval. The relying multimedia data models represent abstractions of media objects for querying, indexing, and so on. However, most currently available implementations reval shortcomings. Either they are limited by one kind of multimedia data supported or by the capacity of their semantic modelling. Despite the upcoming MPEG-7 standard for representing low-level and high-level features of multimedia and respective annotation and use tools (see http://www.mpeg-industry.com), we are not aware of any MMDBMS product which integrates this standard for the purpose of a more meaningful indexing and querying. In this context, our paper presents a Multimedia Data Cartridge (MDC) for demonstration at ACM Multimedia 2002 that implements an object-relational data model for the core part of the MPEG-7 standard. It is an open and extensible system realized with the Oracle Data Cartridge technology. Oracle offers with that technology a mechanism for extending the capabilities of an Oracle database (e.g., type system, query processing and indexing) for the users needs. Besides the more effective modelling of multimedia content, efficient retrieval was considered. Efficient query processing is guaranteed through new database indexing mechanisms. Innately, most database systems provide only a limited number of integrated access methods such as B-trees. Available multimedia database extension packages (e.g., DataBlades of Informix) rarely handle indexing of d-dimensional data (e.g., feature vectors with d>2) or advanced similarity search functionalities (e.g., k-NN search). These circumstances limit the use of database systems in multimedia. For overcoming this drawback, we introduce an Multimedia Indexing Framework (MIF) relying on the GiST framework. Finally, in order to use the MMDBMS we provide query and presentation interfaces which automatically adapt to their usage environment.

Abstract: This book constitutes the refereed proceedings of the 13th International Conference on Database and Export Systems Applications, DEXA 2002, held in Aix-en-Provence, France, in September 2002. The 89 revised full papers presented together with three invited papers and a position paper were carefully reviewed and selected from 241 submissions. The papers are organized in topical sections on Web, workflow, data warehouses and datamining, applications, XML, distributed systems, knowledge engineering, advanced databases, queries, information retrieval, and indexing.

Abstract: In this paper we present the design, architecture and implementation of the A2QM3 System. It provides programmers re-usable QoS-aware Control Objects to enable building a complete middleware for adaptive applications over active networks. We introduce the programming model, the system architecture, and show the parts that make this system a full-featured middleware supporting QoS-aware reliable stream-oriented communication, communication using the request/reply-based CORBA model and real-time streaming for continuous multimedia contents.

Abstract: Adaptation is becoming an increasingly important tool for resource and media management in distributed multimedia systems. Best-effort scheduling and worst-case reservation of resources are two extreme cases, none of them well suited to cope with large-scale, dynamic multimedia systems. The middle course can be met by a system which dynamically adapts its data, resource requirements, and processing components to achieve user satisfaction. Nevertheless, there is no agreement about the questions, where, when, what and who should adapt. A number of papers have been published in recent years, where adaptation is a central issue, however, in most different interpretations and generally in a somehow limited scope; e.g.,[1, 2, 8, 9, 10, 12]. A distributed multimedia system comprises several types of components, such as media servers, meta-databases, proxies, routers, clients. Also, a large number of adaptation possibilities exist, from simple frame dropping up to virtual server systems which dynamically allocate new resources on demand. The main question is, which kind of component can be best used for what kind of adaptation. In the ADMITS project (Adaptation in Distributed Multimedia IT Systems), we are seeking for answers to exactly this basic question, and to a number of related questions.

Abstract: This paper introduces a mobile agent-based infrastructure for an adaptive multimedia server enabling a dynamic migration or replication of certain multimedia applications among a set of available server nodes. It discusses the requirements from both, the server’s and the middleware’s point of view to each other and comes up with a specification and implementation of a CORBA-based interface between them.

Abstract: Elliptic curve cryptosystems allow the use of prime fields with special prime moduli that speed up the finite field arithmetic considerably. Two algorithms for reduction with respect to special moduli have been implemented in software on both a 32-bit and a 64-bit platform and compared to well-known generic modular reduction methods. Timing results for multiplications in prime fields of size between 2^191 and 2^512 are presented and discussed.