[13] | Klaus Schoeffmann, Bernd Münzer, Michael Riegler, Paal Halvorsen, Medical Multimedia Information Systems (MMIS), In MM ’17 Proceedings of the 2017 ACM on Multimedia Conference (Qiong Liu, Rainer Lienhart, Haohong Wang, eds.), ACM, New York, NY, USA, pp. 1957-1958, 2017.
[bib][url] [doi] [abstract]
Abstract: In hospitals all around the world, medical multimedia information systems have gained high importance over the last few years. One of the reasons is that an increasing number of interventions are performed in a minimally invasive way. These endoscopic inspections and surgeries are performed with a tiny camera -- the endoscope -- which produces a video signal that is used to control the intervention. Apart from the viewing purpose, the video signal is also used for automatic content analysis during the intervention as well as for post-surgical usage, such as communicating operation techniques, planning future interventions, and medical forensics. Another reason is video documentation, which is even enforced by law in some countries. The problem, however, is the sheer amount of unstructured medical videos that are added to the multimedia archive on a daily basis. Without proper management and a multimedia information system, the medical videos cannot be used efficiently for post-surgical scenarios. It is therefore already foreseeable that medical multimedia information systems will gain even more attraction in the next few years. In this tutorial we will introduce the audience to this challenging new field, describe the domain-specific characteristics and challenges of medical multimedia data, introduce related use cases, and talk about existing works -- contributed by the medical imaging and robotics community, but also already partly from the multimedia community -- as well as the many open issues and challenges that bear high research potential.
|
[12] | Klaus Schoeffmann, Manfred Jürgen Primus, Bernd Muenzer, Stefan Petscharnig, Christoph Karisch, Qing Xu, Wolfgang Huerst, Collaborative Feature Maps for Interactive Video Search, In MultiMedia Modeling: 23rd International Conference, MMM 2017, Reykjavik, Iceland, January 4-6, 2017, Proceedings, Part II (Laurent Amsaleg, Gylfi Þór Guðmundsson, Cathal Gurrin, Björn Þór Jónsson, Shin’ichi Satoh, eds.), Springer International Publishing, Cham, pp. 457-462, 2017.
[bib][url] [doi] [abstract]
Abstract: This extended demo paper summarizes our interface used for the Video Browser Showdown (VBS) 2017 competition, where visual and textual known-item search (KIS) tasks, as well as ad-hoc video search (AVS) tasks in a 600-h video archive need to be solved interactively. To this end, we propose a very flexible distributed video search system that combines many ideas of related work in a novel and collaborative way, such that several users can work together and explore the video archive in a complementary manner. The main interface is a perspective Feature Map, which shows keyframes of shots arranged according to a selected content similarity feature (e.g., color, motion, semantic concepts, etc.). This Feature Map is accompanied by additional views, which allow users to search and filter according to a particular content feature. For collaboration of several users we provide a cooperative heatmap that shows a synchronized view of inspection actions of all users. Moreover, we use collaborative re-ranking of shots (in specific views) based on retrieved results of other users.
|
[11] | Bernd Münzer, Manfred Jürgen Primus, Sabrina Kletz, Stefan Petscharnig, Klaus Schoeffmann, Static vs. Dynamic Content Descriptors for Video Retrieval in Laparoscopy, In IEEE International Symposium on Multimedia (ISM2017) (Kang-Ming Chang, Wen-Thong Chang, eds.), IEEE, Taichung, Taiwan, pp. 8, 2017.
[bib] [abstract]
Abstract: The domain of minimally invasive surgery has recently attracted attention from the Multimedia community due to the fact that systematic video documentation is on the rise in this medical field. The vastly growing volumes of video archives demand for effective and efficient techniques to retrieve specific information from large video collections with visually very homogeneous content. One specific challenge in this context is to retrieve scenes showing similar surgical actions, i.e., similarity search. Although this task has a high and constantly growing relevance for surgeons and other health professionals, it has rarely been investigated in the literature so far for this particular domain. In this paper, we propose and evaluate a number of both static and dynamic content descriptors for this purpose. The former only take into account individual images, while the latter consider the motion within a scene. Our experimental results show that although static descriptors achieve the highest overall performance, dynamic descriptors are much more discriminative for certain classes of surgical actions. We conclude that the two approaches have complementary strengths and further research should investigate methods to combine them.
|
[10] | Bernd Münzer, Klaus Schoeffmann, Laszlo Böszörmenyi, EndoXplore: A Web-based Video Explorer for Endoscopic Videos, In IEEE International Symposium on Multimedia (ISM2017) (Kang-Ming Chang, Wen-Thong Chang, eds.), IEEE, Taichung, Taiwan, pp. 2, 2017.
[bib] [abstract]
Abstract: The rapidly increasing volume of videos recorded in the course of endoscopic screenings and surgeries poses demanding challenges to video retrieval and browsing systems. Surgeons typically have to use standard video players to retrospectively review their procedures, which is an extremely cumbersome and time-consuming process. We present an HTML5-based video explorer that is specially tailored to this purpose and enables a time-efficient post-operative review of procedures. It incorporates various interactive browsing mechanisms as well as domain-specific content-based features based on previous research results. Preliminary interviews with surgeons indicate that this tool can considerably improve retrieval and browsing efficiency for users in the medical domain and allows surgeons to more easily and quickly revisit specific moments in recordings of their endoscopic surgeries.
|
[9] | Bernd Münzer, Manfred Jürgen Primus, Marco Hudelist, Christian Beecks, Wolfgang Hürst, Klaus Schoeffmann, When content-based video retrieval and human computation unite: Towards effective collaborative video search, In 2017 IEEE International Conference on Multimedia & Expo Workshops (ICMEW) (Yui-Lam Chan, Susanto Rahardja, eds.), IEEE, Hongkong, China, pp. 214-219, 2017.
[bib] [doi] [abstract]
Abstract: Although content-based retrieval methods achieved very good results for large-scale video collections in recent years, they still suffer from various deficiencies. On the other hand, plain human perception is a very powerful ability that still outperforms automatic methods in appropriate settings, but is very limited when it comes to large-scale data collections. In this paper, we propose to take the best from both worlds by combining an advanced content-based retrieval system featuring various query modalities with a straightforward mobile tool that is optimized for fast human perception in a sequential manner. In this collaborative system with multiple users, both subsystems benefit from each other: The results of issued queries are used to re-rank the video list on the tablet tool, which in turn notifies the retrieval tool about parts of the dataset that have already been inspected in detail and can be omitted in subsequent queries. The preliminary experiments show promising results in terms of search performance.
|
[8] | Bernd Münzer, Klaus Schoeffmann, Laszlo Böszörmenyi, Content-based processing and analysis of endoscopic images and videos: A survey, In Multimedia Tools and Applications, Springer, Berlin, Heidelberg, New York, pp. 1-40, 2017.
[bib][url] [doi] [abstract]
Abstract: In recent years, digital endoscopy has established as key technology for medical screenings and minimally invasive surgery. Since then, various research communities with manifold backgrounds have picked up on the idea of processing and automatically analyzing the inherently available video signal that is produced by the endoscopic camera. Proposed works mainly include image processing techniques, pattern recognition, machine learning methods and Computer Vision algorithms. While most contributions deal with real-time assistance at procedure time, the post-procedural processing of recorded videos is still in its infancy. Many post-processing problems are based on typical Multimedia methods like indexing, retrieval, summarization and video interaction, but have only been sparsely addressed so far for this domain. The goals of this survey are (1) to introduce this research field to a broader audience in the Multimedia community to stimulate further research, (2) to describe domain-specific characteristics of endoscopic videos that need to be addressed in a pre-processing step, and (3) to systematically bring together the very diverse research results for the first time to provide a broader overview of related research that is currently not perceived as belonging together.
|
[7] | Bernd Münzer, Klaus Schoeffmann, Laszlo Böszörmenyi, Domain-Specific Video Compression for Long-term Archiving of Endoscopic Surgery Videos, In 29th International Symposium on Computer-Based Medical Systems (CBMS'16) (B Kane, A Marshall, P Soda, eds.), IEEE, Dublin, Ireland, pp. 312-317, 2016.
[bib] [doi] |
[6] | Bernd Münzer, Klaus Schoeffmann, Laszlo Böszörmenyi, Johannes Franciscus Smulders, Jack J Jakimowicz, Investigation of the Impact of Compression on the Perceptional Quality of Laparoscopic Videos, In 27th International Symposium on Computer-Based Medical Systems (CBMS'14) (Marina Krol, ed.), IEEE, New York City, USA, pp. 153-158, 2014.
[bib] |
[5] | Bernd Münzer, Klaus Schoeffmann, Laszlo Böszörmenyi, Relevance Segmentation of Laparoscopic Videos, In IEEE International Symposium on Multimedia (ISM2013) (Anthony Y H Liao, ed.), IEEE, Anaheim, CA, USA, pp. 84-91, 2013.
[bib] [abstract]
Abstract: In recent years, it became common to record video footage of laparoscopic surgeries. This leads to large video archives that are very hard to manage. They often contain a considerable portion of completely irrelevant scenes which waste storage capacity and hamper an efficient retrieval of relevant scenes. In this paper we (1) define three classes of irrelevant segments, (2) propose visual feature extraction methods to obtain irrelevance indicators for each class and (3) present an extensible framework to detect irrelevant segments in laparoscopic videos. The framework includes a training component that learns a prediction model using nonlinear regression with a generalized logistic function and a segment composition algorithm that derives segment boundaries from the fuzzy frame classifications. The experimental results show that our method performs very good both for the classification of individual frames and the detection of segment boundaries in videos and enables considerable storage space savings.
|
[4] | Bernd Münzer, Klaus Schoeffmann, Laszlo Böszörmenyi, Improving Encoding Efficiency of Endoscopic Videos by using Circle Detection based Border Overlays, In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME) 2013 (Xenophon Zabulis, ed.), IEEE, San Jose, USA, pp. 1-4, 2013.
[bib] [doi] [abstract]
Abstract: Videos of endoscopic procedures typically feature a circular content area in the image center. This area is surrounded by a dark border that carries no relevant information but is subject to noise. Thus, a considerable proportion of the available bitrate has to be wasted to encode the border regions. We propose to superimpose the border regions with a homogenous black mask so that it can be encoded efficiently with skipped macroblocks. To determine the exact position and size of the circular content area we use an efficient circle detection algorithm. Through an evaluation with 138 videos we show that the border overlay can significantly reduce the bitrate without degrading the visual quality of the content area.
|
[3] | Bernd Münzer, Klaus Schoeffmann, Laszlo Böszörmenyi, Detection of Circular Content Area in Endoscopic Videos, In 26th International Symposium on Computer-Based Medical Systems (CBMS'13) (Paolo Soda, ed.), IEEE, Porto, Portugal, pp. 534-536, 2013.
[bib] [doi] [abstract]
Abstract: The actual content of endoscopic videos is typically limited to a circular area in the image center. This area has a dynamic position and size and is surrounded by a dark, but noisy border. In this paper we present a novel algorithm that (1) classifies which frames of an endoscopic video feature the circular content area and (2) determines its exact position and size, if present. This information is very useful for improving the performance of subsequent analysis techniques. It can also be used for more efficient video encoding and economic printing of still images in findings and reports. The evaluation shows that the proposed method is very accurate, robust and efficient in terms of runtime.
|
[2] | Marco Andrea Hudelist, Klaus Schoeffmann, David Ahlström, Evaluation of Image Browsing Interfaces for Smartphones and Tablets, In IEEE International Symposium on Multimedia (ISM2013) (Gerald Friedland, Zhu Liu, Nadine Steinmetz, eds.), IEEE, Los Alamitos, CA, USA, pp. 8, 2013.
[bib] |
[1] | Bernd Münzer, Klaus Schoeffmann, Laszlo Böszörmenyi, Detection of Circular Content Area in Endoscopic Videos for Efficient Encoding and Improved Content Analysis, Technical report, Institute of Information Technology (ITEC), Klagenfurt University, no. TR/ITEC/12/2.03, Klagenfurt, Austria, pp. 20, 2012.
[bib] [pdf] [abstract]
Abstract: The actual content of endoscopic videos is typically limited to a circular area in the center of the image due to the inherent characteristics of the camera. This area is surrounded by a dark border that fills up the remainder of the rectangular image and is subject to noise. The position and size of the circle is not standardized and usually varies over time. In this paper a robust algorithm is presented that (1) classifies which parts of an endoscopic video feature a circular content area and (2) determines its exact position and size, if present. This information is useful for improving video encoding efficiency, limiting further analysis steps to the relevant area and saving ink when printing still images on findings. Our evaluation shows that the proposed method is very fast, reliable and robust. Moreover, it indicates that by exploiting this information for video encoding a considerable bitrate reduction is possible with the same visual quality.
|