[2] | Mathias Lux, Gisela Granitzer, Günter Beham, User-Centered Multimedia Retrieval Evaluation based on Empirical Research, In Multimedia Semantics. The Role of Metadata (Mathias Lux, Marc Spaniol, eds.), Springer, Berlin, Heidelberg, New York, pp. 175-194, 2008.
[bib][url] |
[1] | Mathias Lux, Gisela Dösinger, Günter Beham, Empirical Studies in Multimedia Retrieval Evaluation, In Datenbanksysteme in Business, Technologie und Web (BTW 2007) (Matthias Jarke, Thomas Seidl, Christoph Quix, David Kensche, Stefan Conrad, Erhard Rahm, Ralf Klamma, Harald Kosch, Michael Granitzer, Sven Apel, Marko Rosenm, Gunter Saake, Olaf Spinczyk, eds.), NA, NA, pp. 199-217, 2007.
[bib][url] [abstract]
Abstract: The evaluation of retrieval mechanisms for inter-method comparison is necessary in academic as well as in applied research. A major issue in every evaluation is in which way and to what extent the actual perception of the user from the target user group is integrated. Within multimedia retrieval systems the impressions and perceptions of users vary much more than in text retrieval. Empirical studies are a common tool in social science and offer a way to research the correlation between the user perception and the computed similarity between pairs of multimedia documents or a query and the set of results. This approach can be used to complement and extend current evaluation approaches. Within this contribution we summarize general methods from social science and psychology for the interested reader in the area of computer science with some knowledge about statistics. Furthermore we give two examples of undertaken empirical experiments and their outcomes. Within the first one the perception of users is investigated and compared to factors like background and gender, while in the second study metrics are tested upon their ability to reflect the notion of similarity of users. Both experiments aim to give examples and insight on how empirical studies can be used in multimedia research in general and multimedia retrieval evaluation in special.
|