Description
Z is a moving image analysis system that produces abstract representations of video sequences using a system of predefined grayscale disks.
Visually, the disks are characterized by two properties: their radial order (the number of concentric rings) and their frequency or repetition (the number of circular sections). Disks of the same order are shown on the same column and those of the same frequency are shown on the same row.
The visual content of each frame in the movie is represented by rotations and brightness changes in the various disks. Changes in the higher frequencies reflect changes in the smaller details of the source image. Changes in the lower frequencies reflect changes in the overall shape of the source image.
The following example is an analysis from the Hungarian film My Way Home (Miklos Jancso, 1964). It is best viewed in HD fullscreen.
The following example is a section of the opening sequence from The Man from London (2007, Béla Tarr and Agnes Hranitzky). It is best viewed in HD fullscreen.
The Z system describes a video sequence as changing configurations of circular shapes. This representation does not correspond to any pre-existing concepts used in film theory and criticism. For instance, it is not the case that one disk represents the amount of camera movement while another represents the camera position or angle. None of the information contained in any of the disks has a clear interpretation in terms of traditional cinematic categories. Instead, Z raises the possibility of using computer technologies to generate novel methods for the description and analysis of cinematic sequences. Additional clips can be viewed here .
An important inspiration for this project is Soviet filmmaker S. M. Eisenstein's suggestion that cinematic images are akin to musical overtones. (Einsenstein 1988, 182-5) An image is for Eisenstein the total sum of many visual overtones. The interest of Eisenstein's proposal lies in the suggestion that we ought to pay close attention to the subtle and varying micro-events of a moving image stream, but the notion of a visual overtone was not precisely defined and has not been adopted by cinema theorists. Z supplies a mathematical formalization of Eisenstein's idea, enabling a precise decomposition of an image into local harmonics, and encourages the viewer to pay close attention to the dynamic graphical properties of cinematic sequences.
Z can be appreciated at several levels. First of all, it is an abstract work of art in the tradition of visual music (Hans Richter, Viking Eggeling, Oskar Fischinger, Len Lye), where abstract patterns are generated by data from the input video sequence. Secondly, it provides a method of decomposition that sensitizes the viewer to changes in the video sequence. By observing the vicissitudes of the disks, we can learn to pay attention to the purely visual or graphical changes in the input video. Thirdly, it is a work of visual mathematics that aims to construct a bridge between the abstract world of computational procedures and the sensory world of visual representation. The technical background is explained in detail here.
Z employs a mathematical framework originally devised by physicist Frits Zernike to describe the aberrations of microscopes, telescopes and other optical systems that have circular pupils. The circular shape of the disks evokes the images seen through (e.g.) a microscope. Just as the microscope grants perceptual access to what cannot be seen by the naked eye, so the disks in Z draw attention to the often unnoticed micro-events of a moving image.
The following documentation explains the main idea informally.
Z explores the application of Zernike's framework to the analysis and synthesis of moving image sequences. The overall visual design of the system constitutes a diagram of the various procedural steps in the computation of the Zernike moments of a moving image, self-reflexively exposing the computational process on which the system depends. The graphical composition expresses the idea of a circuit, starting with the source image displayed in the lower part and ending with its reconstruction in the upper part. This visual approach affirms the aesthetic value of process-based diagrammatic representation in the context of computational art.
_____________
References
Eisenstein, S. M. 1988. "The Fourth Dimension in Cinema" In Richard Taylor, ed., Selected Works, Volume 1, 182-5. London: BFI.
Development of this project was partly funded by City University strategic research grant 7004725.