Video Dataset of 101 Cataract Surgeries

Posted on .

Our paper “Cataract-101 – Video Dataset of 101 Cataract Surgeries” has been accepted for publication at MMSys 2018 conference (Open DataSet & Software Track).

Authors: Klaus Schoeffmann, Mario Taschwer, Stephanie Sarny, Bernd Münzer, Jürgen Primus, Doris Putzgruber

Abstract:
Cataract surgery is one of the most frequently performed microscopic surgeries in the field of ophthalmology. The goal behind this kind of surgery is to replace the human eye lense with an artificial one, an intervention that is often required due to aging. The entire surgery is performed under microscopy, but co-mounted cameras allow to record and archive the procedure. Currently, the recorded videos are used in a postoperative manner for documentation and training. An additional benefit of recording cataract videos is that they enable video analytics (i.e., manual and/or automatic video content analysis) to investigate medically relevant research questions (e.g., the cause of complications). This, however, necessitates a medical multimedia information system trained and evaluated on existing data, which is currently not publicly available. In this work we provide a public video dataset of 101 cataract surgeries that were performed by four different surgeons over a period of 9 months. These surgeons are grouped into moderately experienced and highly experienced surgeons (assistant vs. senior physicians), providing the basis for experience-based video analytics. All videos have been annotated with quasi-standardized operation phases by a senior ophthalmic surgeon.

Dataset: http://www.itec.aau.at/ftp/datasets/ovid/cat-101/

Preprint PDF

DOI: https://doi.org/10.1145/3204949.3208137

Classification of Operation Phases in Cataract Surgery Videos

Posted on .

Our paper has been accepted for publication and oral presentation at MMM 2018 conference:

Title: Frame-Based Classification of Operation Phases in Cataract Surgery Videos

Authors: Manfred Jürgen Primus, Doris Putzgruber-Adamitsch, Mario Taschwer, Bernd Muenzer, Yosuf El-Shabrawi, Laszlo Boeszoermenyi and Klaus Schöffmann

Abstract: Cataract surgeries are frequently performed to correct a lens opacification of the human eye, which usually appears in the course of aging. These surgeries are conducted with the help of a microscope and are typically recorded on video for later inspection and educational purposes. However, post-hoc visual analysis of video recordings is cumbersome and time-consuming for surgeons if there is no navigation support, such as bookmarks to specific operation phases. To prepare the way for an automatic detection of operation phases in cataract surgery videos, we investigate the effectiveness of a deep convolutional neural network (CNN) to automatically assign video frames to operation phases, which can be regarded as a single-label multi-class classification problem. In absence of public datasets of cataract surgery videos, we provide a dataset of 21 videos of standardized cataract surgeries and use it to train and evaluate our CNN classifier. Experimental results display a mean F1-score of about 68% for frame-based operation phase classification, which can be further improved to 75% when considering temporal information of video frames in the CNN architecture.

Dataset: http://www.itec.aau.at/ftp/datasets/ovid/cat-21/

Preprint PDF
DOI:
https://doi.org/10.1007/978-3-319-73603-7_20

Bibtex:

@InProceedings{Primus2018,
  Title                    = {Frame-Based Classification of Operation Phases in Cataract Surgery Videos},
  Author                   = {Primus, Manfred J{\"u}ergen and Putzgruber-Adamitsch, Doris and Taschwer, Mario and M{\"u}nzer, Bernd and El-Shabrawi, Yosuf and B{\"o}sz{\"o}rmenyi, Laszlo and Schoeffmann, Klaus},
  Booktitle                = {MultiMedia Modeling},
  Year                     = {2018},

  Address                  = {Cham},
  Editor                   = {Schoeffmann, Klaus and Chalidabhongse, Thanarat H. and Ngo, Chong Wah and Aramvith, Supavadee and O'Connor, Noel E. and Ho, Yo-Sung and Gabbouj, Moncef and Elgammal, Ahmed},
  Pages                    = {241--253},
  Publisher                = {Springer International Publishing},
  ISBN                     = {978-3-319-73603-7}
}

Courses in winter term 2017

Posted on .

In the upcoming winter term (starting on October 2), I will give the following courses at AAU:

  • 620.005 UE Introduction to Computer Science (Part 1, German)
  • 620.025 UE Introduction to Computer Science (Part 2, German)
  • 621.703 PR Computer Organization (German)

Access to course material in Moodle is restricted to enrolled students, but can also be requested from me by e-mail.

PhD thesis submitted

Posted on .

My PhD thesis has been submitted on April 6 and graded as excellent (grade 1).

Title of thesis: Concept-Based and Multimodal Methods for Medical Case Retrieval

Abstract:
Medical case retrieval (MCR) is defined as a multimedia retrieval problem, where the document collection consists of medical case descriptions that pertain to particular diseases, patients’ histories, or other entities of biomedical knowledge. Case descriptions are multimedia documents containing textual and visual modalities (images). A query may consist of a textual description of patient’s symptoms and related diagnostic images. This thesis proposes and evaluates methods that aim at improving MCR effectiveness over the baseline of fulltext retrieval. We hypothesize that this objective can be achieved by utilizing controlled vocabularies of biomedical concepts for query expansion and concept-based retrieval. The latter represents case descriptions and queries as vectors of biomedical concepts, which may be generated automatically from textual and/or visual modalities by concept mapping algorithms. We propose a multimodal retrieval framework for MCR by late fusion of text-based retrieval (including query expansion) and concept-based retrieval and show that retrieval effectiveness can be improved by 49% using linear fusion of practical component retrieval systems. The potential of further improvement is experimentally estimated as a 166% increase of effectiveness over fulltext retrieval using query-adaptive fusion of ideal component retrieval systems. Additional contributions of this thesis include the proposal and comparative evaluation of methods for concept mapping, query and document expansion, and automatic classification and separation of compound figures found in case descriptions.

Keywords: multimedia information retrieval / biomedical information retrieval / biomedical concept detection / information fusion / image processing

Bibtex citation:

@PhdThesis{Taschwer2017,
Title                    = {Concept-Based and Multimodal Methods for Medical Case Retrieval},
Author                   = {Taschwer, Mario W.},
School                   = {Alpen-Adria-Universit{\"a}t Klagenfurt},
Year                     = {2017},
Address                  = {Austria},
Month                    = mar,
Url                      = {http://www.itec.aau.at/bib/files/phd-thesis-taschwer.pdf}
}

Courses in summer term 2017

Posted on .

In the upcoming summer term (starting on March 1), I will give the following courses at AAU:

  • 620.002 Introduction to Computer Science (Exercises, German)
  • 621.401 Compiler Construction (Lab, English)

Courses in winter term 2016

Posted on .

Here are the courses I give this winter term, starting on October 3:

621.702 Computer organization (lab)
621.704 Computer organization (lab)

Students access the course material through non-public Moodle. If you are not enrolled to these courses but are interested in the course material (available in German only), please drop me an e-mail.

Compound Figure Separation Journal Paper

Posted on .

We submitted extended work on compound figure separation to the MTAP Journal.

Update: The revised version of our paper has been accepted for publication on Dec 1, 2016 and published online on Dec 29, 2016.

Title: Automatic Separation of Compound Figures in Scientific Articles

Abstract:
Content-based analysis and retrieval of digital images found in scientific articles is often hindered by images consisting of multiple subfigures (compound figures). We address this problem by proposing a method (ComFig) to automatically classify and separate compound figures, which consists of two main steps: (i) a supervised compound figure classifier (ComFig classifier) discriminates between compound and non-compound figures using task-specific image features; and (ii) an image processing algorithm is applied to predicted compound images to perform compound figure separation (ComFig separation). The proposed ComFig classifier is shown to achieve state-of-the-art classification performance on a published dataset. Our ComFig separation algorithm shows superior separation accuracy on two different datasets compared to other known automatic approaches. Finally, we propose a method to evaluate the effectiveness of the ComFig chain combining classifier and separation algorithm, and use it to optimize the misclassification loss of the ComFig classifier for maximal effectiveness in the chain.

DOI: https://doi.org/10.1007/s11042-016-4237-x

Bibtex citation:

@Article{Taschwer2016a,
  Title                    = {Automatic separation of compound figures in scientific articles},
  Author                   = {Taschwer, Mario and Marques, Oge},
  Journal                  = {Multimedia Tools and Applications},
  Year                     = {2016},
  Pages                    = {1--30},
  Doi                      = {10.1007/s11042-016-4237-x},
  ISSN                     = {1573-7721}
}

Courses in summer term 2016

Posted on .

In the upcoming summer term (starting on March 1, 2016), I will give the following two courses:

  • 620.002 Introduction to computer science (exercises, German)
  • 621.401 Compiler construction (lab, English)

Course material will be provided for students in non-public Moodle. If you are not enrolled to these courses and interested in the course material, please drop me an e-mail.

Courses in winter term 2015

Posted on .

Here are the courses I give this winter term, starting this week:

620.005 Introduction to computer science (exercises)
621.704 Computer organization (lab)

Students access the course material through non-public Moodle. If you are not enrolled to these courses but are interested in the course material (available in German only), please drop me an e-mail.