Abstract: Cataract surgery is one of the most frequently performed microscopic surgeries in the field of ophthalmology. The goal behind this kind of surgery is to replace the human eye lense with an artificial one, an intervention that is often required due to aging. The entire surgery is performed under microscopy, but co-mounted cameras allow to record and archive the procedure. Currently, the recorded videos are used in a postoperative manner for documentation and training. An additional benefit of recording cataract videos is that they enable video analytics (i.e., manual and/or automatic video content analysis) to investigate medically relevant research questions (e.g., the cause of complications). This, however, necessitates a medical multimedia information system trained and evaluated on existing data, which is currently not publicly available. In this work we provide a public video dataset of 101 cataract surgeries that were performed by four different surgeons over a period of 9 months. These surgeons are grouped into moderately experienced and highly experienced surgeons (assistant vs. senior physicians), providing the basis for experience-based video analytics. All videos have been annotated with quasi-standardized operation phases by a senior ophthalmic surgeon.

Abstract: Multimedia research is becoming more and more important for the medical domain, where an increasing number of videos and images are integrated in the daily routine of surgical and diagnostic work. While the collection of medical multimedia data is not an issue, appropriate tools for efficient use of this data are missing. This includes management and inspection of the data, visual analytics, as well as learning relevant semantics and using recognition results for optimizing surgical and diagnostic processes. The characteristics and requirements in this interesting but challenging field are different than the ones in classic multimedia domains. Therefore, this tutorial gives a general introduction to the field, provides a broad overview of specific requirements and challenges, discusses existing work and open challenges, and elaborates in detail how machine learning approaches can help in multimedia-related fields to improve the performance of surgeons/clinicians.

Abstract: In this chapter, we present Merge and Forward, an IDMS scheme for adaptive HTTP streaming as a distributed control scheme and adopting the MPEG-DASH standard as representation format. We introduce so-called IDMS sessions and describe how an unstructured peer-to-peer overlay can be created using the session information using MPEG-DASH. We objectively assess the performance of Merge and Forward with respect to convergence time (time needed until all clients hold the same reference time stamp) and scalability. After the negotiation on a reference time stamp, the clients have to synchronize their multimedia playback to the agreed reference time stamp. In order to achieve this, we propose a new adaptive media playout approach minimizing the impact of playback synchronization on the QoE. The proposed adaptive media playout is assessed subjectively using crowd sourcing. We further propose a crowd sourcing methodology for conducting subjective quality assessments in the field of IDMS by utilizing GWAP. We validate the applicability of our methodology by investigating the lower asynchronism threshold for IDMS in scenarios like online quiz games.

Abstract: We present our video search system for the Video Browser Showdown (VBS) 2018 competition. It is based on the collaborative system used in 2017, which already performed well but also revealed high potential for improvement. Hence, based on our experience we introduce several major improvements, particularly (1) a strong optimization of similarity search, (2) various improvements for concept-based search, (3) a new flexible video inspector view, and (4) extended collaboration features, as well as numerous minor adjustments and enhancements, mainly concerning the user interface and means of user interaction. Moreover, we present a spectator view that visualizes the current activity of the team members to the audience to make the competition more attractive.

Abstract: Cataract surgeries are frequently performed to correct a lens opacification of the human eye, which usually appears in the course of aging. These surgeries are conducted with the help of a microscope and are typically recorded on video for later inspection and educational purposes. However, post-hoc visual analysis of video recordings is cumbersome and time-consuming for surgeons if there is no navigation support, such as bookmarks to specific operation phases. To prepare the way for an automatic detection of operation phases in cataract surgery videos, we investigate the effectiveness of a deep convolutional neural network (CNN) to automatically assign video frames to operation phases, which can be regarded as a single-label multi-class classification problem. In absence of public datasets of cataract surgery videos, we provide a dataset of 21 videos of standardized cataract surgeries and use it to train and evaluate our CNN classifier. Experimental results display a mean F1-score of about 68% for frame-based operation phase classification, which can be further improved to 75% when considering temporal information of video frames in the CNN architecture.

Abstract: Large scale computing solutions are increasingly used in the context of Big Data platforms, where efficient scheduling algorithms play an important role in providing optimized cluster resource utilization, throughput and fairness. This paper deals with the problem of scheduling a set of jobs across a cluster of machines handling the specific use case of fair scheduling for jobs and machines with heterogeneous characteristics. Although job and cluster diversity is unprecedented, most schedulers do not provide implementations that handle multiple resource type fairness in a heterogeneous system. We propose in this paper a new scheduler called h-Fair that selects jobs for scheduling based on a global dominant resource fairness heterogeneous policy, and dispatches them on machines with similar characteristics to the resource demands using the cosine similarity. We implemented h-Fair in Apache Hadoop YARN and we compare it with the existing Fair Scheduler that uses the dominant resource fairness policy based on the Google workload trace. We show that our implementation provides better cluster resource utilization and allocates more containers when jobs and machines have heterogeneous characteristics.

Abstract: Nowadays, when we face with numerous data, when data cannot be classified into regular relational databases and new solutions are required, and when data are generated and processed rapidly, we need powerful platforms and infrastructure as support. Extracting valuable information from raw data is especially difficult considering the velocity of growing data from year to year and the fact that 80% of data is unstructured. In addition, data sources are heterogeneous (various sensors, users with different profiles, etc.) and are located in different situations or contexts. Cloud computing, which concerns large-scale interconnected systems with the main purpose of aggregation and efficient exploiting the power of widely distributed resources, represent one viable solution. Resource management and task scheduling play an essential role, in cases where one is concerned with optimized use of resources (Negru et al., 2017) [1].The goal of this special issue is to explore new directions and approaches for reasoning about advanced resource management and task scheduling methods and algorithms for Big Data platforms. The accepted papers present new results in the domain of resource management and task scheduling, Cloud platforms supporting Big Data processing, data handling and Big Data applications.

Abstract: The Medico: Multimedia for Medicine Task, running for the secondtime as part of MediaEval 2018, focuses on detecting abnormalities,diseases, anatomical landmarks and other findings in imagescaptured by medical devices in the gastrointestinal tract. The taskis described, including the use case and its challenges, the datasetwith ground truth, the required participant runs and the evaluationmetrics.

Abstract: This paper presents an open-source classification tool for image and video frame classification. The classification takes a search-based approach and relies on global and local image features. It has been shown to work with images as well as videos, and is able to perform the classification of video frames in real-time so that the output can be used while the video is recorded, playing, or streamed. OpenSea has been proven to perform comparable to state-of-the-art methods such as deep learning, at the same time performing much faster in terms of processing speed, and can be therefore seen as an easy to get and hard to beat baseline. We present a detailed description of the software, its installation and use. As a use case, we demonstrate the classification of polyps in colonoscopy videos based on a publicly available dataset. We conduct leave-one-out-cross-validation to show the potential of the software in terms of classification time and accuracy.

Abstract: Appropriate visualization of endoscopic surgery recordings has a huge potential to benefit surgical work life. For example, it enables surgeons to quickly browse medical interventions for purposes of documentation, medical research, discussion with colleagues, and training of young surgeons. Current literature on automatic action recognition for endoscopic surgery covers domains where surgeries follow a standardized pattern, such as cholecystectomy. However, there is a lack of support in domains where such standardization is not possible, such as gynecologic laparoscopy. We provide ActionVis, an interactive tool enabling surgeons to quickly browse endoscopic recordings. Our tool analyses the results of a post-processing of the recorded surgery. Information on individual frames are aggregated temporally into a set of scenes representing frequent surgical actions in gynecologic laparoscopy, which help surgeons to navigate within endoscopic recordings in this domain.

Abstract: With a rigorous long-term archival of endoscopic surgeries, vast amounts of video and image data accumulate. Surgeons are not able to spend their valuable time to manually search within endoscopic multimedia databases (EMDBs) or manually maintain links to interesting sections in order to quickly retrieve relevant surgery sections. Enabling the surgeons to quickly access the relevant surgery scenes, we utilize the fact that surgeons record external images additionally to the surgery video and aim to link them to the appropriate video sequence in the EMDB using a query-by-example approach. We propose binary Convolutional Neural Network (CNN) features off-the-shelf and compare them to several baselines: pixel-based comparison (PSNR), image structure comparison (SSIM), hand-crafted global features (CEDD and feature signatures), as well as CNN baselines Histograms of Class Confidences (HoCC) and Neural Codes (NC). For evaluation, we use 5.5 h of endoscopic video material and 69 query images selected by medical experts and compare the performance of the aforementioned image mathing methods in terms of video hit rate and distance to the true playback time stamp (PTS) for correct video predictions. Our evaluation shows that binary CNN features are compact, yet powerful image descriptors for retrieval in the endoscopic imaging domain. They are able to maintain state-of-the-art performance, while providing the benefit of low storage space requirements and hence provide the best compromise.

Abstract: With the growing hype for wearable devices recording biometric data comes the readiness to capture and combine even more personal information as a form of digital diary - lifelogging today is practiced ever more and can be categorized anywhere between an informative hobby and a life-changing experience. From an information processing point of view, analyzing the entirety of such multi-source data is immensely challenging, which is why the first Lifelog Search Challenge 2018 competition is brought into being, as to encourage the development of efficient interactive data retrieval systems. Answering this call, we present a retrieval system based on our video search system diveXplore, which has successfully been used in the Video Browser Showdown 2017 and 2018. Due to the different task definition and available data corpus, the base system was adapted and extended to this new challenge. The resulting lifeXplore system is a flexible retrieval and exploration tool that offers various easy-to-use, yet still powerful search and browsing features that have been optimized for lifelog data and for usage by novice users. Besides efficient presentation and summarization of lifelog data, it includes searchable feature maps, concept and metadata filters, similarity search and sketch search.

Abstract: The emerging ubiquity of videos in all aspects of society demands for innovative and efficient browsing and navigation mechanisms. We propose a novel visualization and interaction paradigm that replaces the traditional linear timeline with a circular timeline. The main advantages of this new concept are (1) significantly increased and dynamic navigation granularity, (2) minimized spacial distances between arbitrary points on the timeline, as well as (3) the possibility to efficiently utilize the screen space for bookmarks or other supplemental information associated with points of interest. The demonstrated prototype implementation proves the expedience of this new concept and includes additional navigation and visualization mechanisms, which altogether create a powerful video browser.

Abstract:

Abstract: The popularity of computer games and e-sports is enormously high and still growing every year. Despite the popularity computer games often rely on old technologies, especially in the field of networking. Research in networking for games is challenging due to the low availability of up-todate datasets and network traces. In order to achieve a high user satisfaction while keeping the network activity as low as possible, modern networking solutions of computer games take players’ activities as well as closeness of players in the game world into account. In this paper, we analyze the Battle Royale game mode of the online multiplayer game Fortnite, where 100 players challenge each other in a king-of-the-hill like game within a constantly contracting game world, as an example for a popular online game with demanding technical requirements. We extrapolate player movement patterns by finding player positions automatically from videos, uploaded by Fortnite players on popular streaming platforms and show, how they influence network traffic from the client to the server and vice versa. This extended abstract features the highlights of [1], which has been accepted at the NetGames 2018 event.

Abstract: When developing new approaches in networking research, one of the most important requirements is to evaluate the degree of improvement of a new approach both realistically and cost-effectively. Wireless networks and their adequate emulation play an important role in evaluation, but emulation of wireless links and networks is still difficult to handle. In this paper, we present a low-cost, fixed-network testbed able to emulate the dynamically changing conditions of wireless links caused by client mobility and physical phenomena. We extend the existing fixed-network testbed for the purpose of wireless network emulation using the Linux tools tc, iptables, and NetEm in sophisticated ways. Convenient function blocks are provided to configure wireless network topologies as well as dynamic link and mobility conditions to be emulated with modest efforts. We utilize the testbed's capabilities to investigate the influence of different mobility models on streaming SVC-encoded videos in Named Data Networking (NDN), a novel Information-Centric Networking architecture. Furthermore, we evaluate the benefits of using early loss detection mechanisms for streaming in NDN, by implementing Wireless Loss Detection and Recovery (WLDR). Our results show that the extended fixed-network testbed can precisely emulate wireless network conditions and usage. For instance, the emulation revealed that both the choice of the mobility model and the use of WLDR have a substantial influence on the resulting SVC video streaming performance.

Abstract: Recent research in the field of Information-Centric Networking (ICN) shows the need for push-based data transfer, which is not supported in current pull-based ICN architectures, such as Named Data Networking (NDN). IoT deployments as well as emergency notifications and real-time multimedia communication are well suited to be realized using the ICN principles, but experience challenges in pull-based environments. Persistent Interests (PIs) are a promising approach to introduce pushlike traffic in Interest-based ICN architectures such as NDN. In this paper, we explore the characteristics of PIs and discuss advantages and disadvantages of using them. We provide an efficient solution for preventing so-called Data loops, which are introduced by giving up NDN’s one-request-per-packet principle. Furthermore, we investigate the performance of PIs compared to classical Interests in terms of the computational complexity of forwarding and discuss possible applications of PIs.

Abstract: The speedup measures the improvement in performance when the computational resources are being scaled. The efficiency, on the other side, provides the ratio between the achieved speedup and the number of scaled computational resources (processors). Both parameters (speedup and efficiency), which are defined according to Amdahl’s Law, provide very important information about performance of a computer system with scaled resources compared with a computer system with a single processor. However, as cloud elastic services’ load is variable, apart of the scaled resources, it is vital to analyse the load in order to determine which system is more effective and efficient. Unfortunately, both the speedup and efficiency are not sufficient enough for proper modeling of cloud elastic services, as the assumptions for both the speedup and efficiency are that the system’s resources are scaled, while the load is constant. In this paper, we extend the scaling of resources and define two additional scaled systems by (i) scaling the load and (ii) scaling both the load and resources. We introduce a model to determine the efficiency for each scaled system, which can be used to compare the efficiencies of all scaled systems, regardless if they are scaled in terms of load or resources. We have evaluated the model by using Windows Azure and the experimental results confirm the theoretical analysis. Although one can argue that web services are scalable and comply with Gustafson’s Law only, we provide a taxonomy that classifies scaled systems based on the compliance with both the Amdahl’s and Gustafson’s laws.For three different scaled systems (scaled resources R, scaled load L and combination RL), we introduce a model to determine the scaling efficiency. Our model extends the current definition of efficiency according to Amdahl’s Law, which assumes scaling the resources, and not the load.

Abstract: That video games have reached the masses is well known. Moreover,game streaming and watching other people play video games is aphenomenon that has outgrown its small beginnings. Game videostreams, be it live or recorded, are viewed by millions. E-sports is theresult of organized leagues and tournaments in which players cancompete in controlled environments and viewers can experiencethe matches, discuss and criticize, just like in physical sports. In theGameStory task, taking place the first time in 2018, we approachthe game streaming and e-sports phenomena from a multimediaresearch side. We focus on the task of summarizing matches usinga specific relevant game, Counter-Strike: Global Offensive, as a casestudy. With the help of ZNIPE.tv, we provide a data set of highquality data and meta data from competitive tournaments and aimto foster research in the area of e-sports and game streaming.

Abstract: Across many domains, machine learning approaches start to compete with human experts in tasks originally considered as very difficult for automation. However, effective retrieval of general video shots still represents an issue due to their variability, complexity and insufficiency of training sets. In addition, users can face problems trying to formulate their search intents in a given query interface. Hence, many systems still rely also on interactive human-machine cooperation to boost effectiveness of the retrieval process. In this paper, we present our experience with known-item search tasks in the Video Browser Showdown competition, where participating interactive video retrieval systems mostly rely on various similarity models. We discuss the observed difficulty of known-item search tasks, categorize employed interaction components (relying on similarity models) and inspect successful interactive known-item searches from the recent iteration of the competition. Finally, open similarity search challenges for known-item search in video are presented.

Abstract: The last decade has seen innovations that make video recording, manipulation, storage and sharing easier than ever before, thus impacting many areas of life. New video retrieval scenarios emerged as well, which challenge the state-of-the-art video retrieval approaches. Despite recent advances in content analysis, video retrieval can still benefit from involving the human user in the loop. We present our experience with a class of interactive video retrieval scenarios and our methodology to stimulate the evolution of new interactive video retrieval approaches. More specifically, the Video Browser Showdown evaluation campaign is thoroughly analyzed, focusing on the years 2015-2017. Evaluation scenarios, objectives and metrics are presented, complemented by the results of the annual evaluations. The results reveal promising interactive video retrieval techniques adopted by the most successful tools and confirm assumptions about the different complexity of various types of interactive retrieval scenarios. A comparison of the interactive retrieval tools with automatic approaches (including fully automatic and manual query formulation) participating in the TRECVID 2016 Ad-hoc Video Search (AVS) task is discussed. Finally, based on the results of data analysis, a substantial revision of the evaluation methodology for the following years of the Video Browser Showdown is provided.