National Science Foundation
Computer and Information Science and Engineering
Division of Information and Intelligent Systems (IIS)
Information Integration and Informatics (III)
This proposed project will develop a framework for organizing images that allows the specific types of relationships between those images to be represented, manipulated, highlighted, enhanced, and studied. The technical challenges involve building new representational and algorithmic systems to capture the major “longitudinal categories” that relate heterogeneous images to each other within collections. The problem of relationship between images is normally posed through registration, which is most often highly contextualized. This work will capture the steps necessary to specify registration as a metadata construction that enables a range of granularities in mapping images to each other, and heterogeneous relationship across organizational categories such as time (diachronic), multi-modal, and instances related by a semantic object. The work is interdisciplinary, with the PI and Co-PI working together to apply the results in the field of humanities to a large digitized collection of signature manuscripts, starting with the Homeric Iliad.
This study will focus on four things: major longitudinal categories that relate images; how to make different kinds of measurement possible through registration; ways to expand the granularity of the range of potential registrations; and definitions of metadata constructs that can support longitudinally-organized image collections in order for automated and semi-automated tools to emerge. There are technical barriers to building longitudinal collections of images and models. During the work these problems will be formalized and the deployed prototype will offer an example of systematically designed solutions. Image recognition and scene understanding, image registration, and image similarity metrics play a role, although this work will focus on using existing methods to study three important categories for how images in a collection relate to one another: multi-modal, diachronic, and multi-instance. Multi-modal imagery is a collection of images of essentially the same scene taken under varying conditions. Diachronic imagery is an image set taken of essentially the same scene but over a wide span of time, such as the images of a building or landmark. Multi-instance imagery is an image set that is connected semantically, like images of the pages of a story written by two completely different scribes (same story, different instances).
Papers and Links:
This material is based upon work supported by the National Science Foundation under Grant No. 0916421. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.