Sptaial Cognition Research Proposal with Anett Ragó Psychologist:
At the intersection of digital visual art and classical traditions of painting has emerged a new hybrid genre: paint mapping. Paint mapping is a sub-genre of projection mapping. It creates a new sense of reality that comes into being as the painting meets the light cast upon it. Digital animation reflects the painting’s architecture. In this way, it produces a specially integrated unity of the space, the structural relations, the light (the projected non-figurative animations), and the movement.
The object situated in the tangible space, and virtuality, looming pixels into pigments of the painting through light, are creating a new abstract context for the interpretation of both the painting and the animation. The borderline between physical and virtual presence fades, the painting becomes ‘virtual’ and gets moving, animation, projected on it, assumes a shape of ‘physical’ stimulus. In this new atmosphere and fresh experience, the tradition of visual improvisation revives.
This integration process simulates human vision. Letting participants watch the dynamic change of the artwork (by systematically adding new layers) we could follow the change of the interpretation process. Following the gaze movements with an eye-tracking technique, this experiment would add much to the event cognition literature. Not just the stimuli (the artwork) is new in this case but the possibility it gives for us to investigate how each aspect changes the quality of reception.1
Space perception is closely connected to motion and body representation. Another research project could be to follow the induced motion experience with an imaging technique (fMRI). If the space information is detected, the motory representation becomes active in the brain. This artwork, with its dynamic nature, makes possible to follow the emergence of spatial experience by adding extra layers (without showing real motion information).
We believe that our approach would be interesting for a collaboration with neuroscientists working with fMRI. Our goal is that while revealing new directions in the relationship between light art and vision, and realizing a dynamically changing artwork, also demonstrate the process of vision and the changes in interpretation of visual events.
In the Relief Method (Silver Installations) we use the projected light to give an emboss effect to the paintings, thereby the bright ‘tint’ of the painting comes from the shifted lighting of black surfaces. The sound reacted vortex dynamics of the projected lighting transforms the vision of the painting into a sense of moving, brightening relief, which indicates a three dimensional space in feelings. In this way, as a consequence of the spatial expansion, the resolution of the sense increases on the level of perception. The light’s pulsation and the reiterating circulation of the sound bring about continuity in time.
With the Interference Method, the audiovisual painting-installations, responding to the noises of the outside world and tonal environment, are novelties for our visual perception because of the interference of the sense of painting and digital marking. The installation decorates black and white traces by creating dashes of color, and in such way it reproduces some phenomena to be found in nature, as twinkling of the shellfish, or interference color that can be seen on the wings of some insects. The colors influenced by moving voices shift ceaselessly in accordance with the moiré pattern. Therefore, it brings about elementary perception at the particle’s level.
This art project is also connected to current research questions related to the audiovisual integration in the presence of the spatial information (cf. Karimpur & Hamburger, 2016; Wahn & König, 2015).2
1 Even model is (cf. Zacks & Tversky, 2001) is a general theory of understanding visual events. This influential model triggered a publication boom testing the event boundary assumption (according to which segmenting of the events is automatic and this segmentation defines the memory encoding, so the later retrieval performance) and the related concepts (cf. Radvansky & Zacks, 2014; Radvansky et al. 2017). A crucial issue is how event segmentation is influenced by the different physical aspects of the information, e.g., the arrangement of the object, the saliency of the objects and the spatial information. A well-known Hungarian film theoretician, Balint Andras Kovacs has a research regarding the letter question (Benini et al, 2016). Our research proposal could be easily integrated into this research field.
Benini, S., Svanera, M., Adami, N., Leonardi, R., & Kovács, A. B. (2016). Shot scale distribution in art films. Multimedia Tools and Applications, 75(23), 16499-16527.
Radvansky, G. A., Andrea, E. O., & Fisher, J. S. (2017). Event models and the fan effect. Memory & cognition, 45(6), 1028-1044.
Radvansky, G. A., & Zacks, J. M. (2014). Event cognition. Oxford University Press.
Zacks, J. M., & Tversky, B. (2001). Event structure in perception and conception. Psychological bulletin, 127(1), 3.
2 Karimpur, H., & Hamburger, K. (2016). Multimodal integration of spatial information: The influence of object-related factors and self-reported strategies. Frontiers in psychology, 7, 1443.
Wahn, B., & König, P. (2015). Audition and vision share spatial attentional resources, yet attentional load does not disrupt audiovisual integration. Frontiers in Psychology, 6, 1084.
