Print
Univis
Search
 
FAU-Logo
Techn. Fakultät Willkommen am Institut für Informatik FAU-Logo

Iconographics

The interdisciplinary research project Iconographics is dedicated to innovative possibilities of digital image recognition for the arts and humanities. While computer vision is already often able to identify individual objects or specific artistic styles in images, the project is confronted with the open problem of also opening up the more complex image structures and contexts digitally. On the basis of a close interdisciplinary collaboration between Classical Archaeology, Christian Archaeology, Art History and the Computer Sciences, as well as joint theoretical and methodological reflection, a large number of multi-layered visual works will be analyzed, compared and contextualized. The aim is to make the complex compositional, narrative and semantic structures of these images tangible for computer vision.


Iconography and Narratology are identified as a challenging research questions for all subjects of the project. The iconography will be interpreted in its plot, temporality, and narrative logic. Due to its complex cultural structure; we selected four important scenes: (1) The Annunciation of the Lord; (2) The Adoration of the Magi; (3) The Baptism of Christ; and (4) Noli me tangere (Do not touch me).

 

Project Members: Opens internal link in current windowRonak Kosti, Opens external link in current windowPrathmesh Madhu, Opens external link in current windowVincent Christlein

Michael Baxandall‘s scheme of four stages of Mary after the sermon of Fra Roberto Caracciolo

Publications

Recognizing Characteres in Art History Using Deep Learning [SUMAC ’19, ACM]
Prathmesh Madhu, Ronak Kosti, Lara Mührenberg, Peter Bell, Andreas Maier, Vincent Christlein
Depiction of Mary (a – d) and Gabriel (e – f) as the main protagonists in the Annunciation of the Lord iconography

In the field of Art History, images of artworks and their contexts are core to understanding the underlying semantic information. However, the highly complex and sophisticated representation of these artworks makes it difficult, even for the experts, to analyze the scene. From the computer vision perspective, the task of analyzing such artworks can be divided into sub-problems by taking a bottom-up approach. In this paper, we focus on the problem of recognizing the characters in Art History. From the iconography of Annunciation of the Lord, we consider the representation of the main protagonists, Mary and Gabriel, across different artworks and styles. We investigate and present the findings of training a character classifier on features extracted from their face images. The limitations of this method, and the inherent ambiguity in the representation of Gabriel, motivated us to consider their bodies (a bigger context) to analyze in order to recognize the characters. Convolutional Neural Networks (CNN) trained on the bodies of Mary and Gabriel are able to learn person related features and ultimately improve the performance of character recognition. We introduce a new technique that generates more data with similar styles, effectively creating data in the similar domain. We present experiments and analysis on three different models and show that the model trained on domain related data gives the best performance for recognizing character. Additionally, we analyze the localized image regions for the network predictions.