Reading hybrid texts : remarks on text/image transitions
Miniatura indisponível
Ficheiros
Data
2011
Título da revista
ISSN da revista
Título do Volume
Editora
Edições Universitárias Lusófonas
Resumo
The reading of printed materials implies the visual processing of information originated
in two distinct semiotic systems. The rapid identification of redundancy,
complementation or contradiction rhetoric strategies between the two information types
may be crucial for an adequate interpretation of bimodal materials. Hybrid texts (verbal
and visual) are particular instances of bimodal materials, where the redundant
information is often neglected while the complementary and the contradictory ones are
essential.Studies using the 504 ASL eye-tracking system while reading either additive or
exhibiting captions (Baptista, 2009) revealed fixations on the verbal material and
transitions between the written and the pictorial in a much higher number and duration
than the initially foreseen as necessary to read the verbal text. We therefore hypothesized
that confirmation strategies of the written information are taking place, by using
information available in the other semiotic system.Such eye-gaze patterns obtained from
denotative texts and pictures seem to contradict some of the scarce existing data on visual
processing of texts and images, namely cartoons (Carroll, Young and Guertain, 1992),
descriptive captions (Hegarty, 1992 a and b), and advertising images with descriptive and
explanatory texts (cf. Rayner and Rotello, 2001, who refer to a previous reading of the
whole text before looking at the image, or even Rayner, Miller and Rotello, 2008 who
refer to an earlier and longer look at the picture) and seem to consolidate findings of
Radach et al. (2003) on systematic transitions between text and image.By framing
interest areas in the printed pictorial material of non redundant hybrid texts, we have
identified the specific areas where transitions take place after fixations in the verbal text.
The way those transitions are processed brings a new interest to further research.
Descrição
Journal of Eye Tracking, Visual Cognition and Emotion
Palavras-chave
PSICOLINGUÍSTICA, PROCESSAMENTO DE INFORMAÇÃO, PSICOLOGIA, PSYCHOLINGUISTICS, INFORMATION PROCESSING, PSYCHOLOGY