Ralf Nuhn, London, UK

From UNCAGED to Cyber-Spatialism

Keywords

UNCAGED, mixed reality, media art, Fontana, Cyber-Spatialism

Introduction

The starting point for this paper is my recent project UNCAGED, which is a series of six ‘telesymbiotic’ ¹ installations exploring interrelationships and transitions between screen-based digital environments and their immediate physical surroundings. The underlying motivation behind my approach was to ‘uncage’ screen-based realities from the confines of their digital existence and to bring the remote computer world closer to our human experience. In particular, my approach opposed the notion of immersive Virtual Reality, where the physical world is more or less excluded from the participants, and instead attempted to situate the virtual domain within the physical world.

In the first instance I regarded this project as being an expression of artistic vision led by aesthetic and formal considerations and situated within a broader physico-philosophical context. For instance, I could sense a certain relationship between my approach to the quantum physical notion of non-locality, as proposed by the physicist Niels Bohr, and its implications for the existence of an invisible reality that supports our world – or to put it in different terms, the implication that an action in one part of the world could cause an instantaneous effect in another part of the world without there being a perceivable connection. ² For me traversing the distance between the physical and the screen-based world of computers is an assertion of this idea, even though in my approach the link between the two worlds is of course only a make-believe situation.

The Theoretical Framework Behind UNCAGED

The methodological approach of my project could be best described as experimentation with and reconfiguration of existing technologies, which result in new creative inventions and designs. This ‘blue-sky research’ was led primarily by my artistic taste, personal intuition and experience. However this does not mean that I worked without any guidelines. In the following section I will attempt to outline the theoretical framework behind my approach, ranging from formal and aesthetic considerations, incorporating aspects of computer science and interaction design, to social implications.

Aesthetic and Formal Considerations

Based on some initial practical studies, I conceived a set of aesthetic and formal parameters which would underpin the further development of my work. To give a brief summary, I presumed that the work would consist of screen-based animations as well as (digitally-processed) video material and images, linked to different computer-controlled electronic devices and automated sculptures positioned around the screen. The off-screen devices would be triggered by different events happening in the virtual domain and vice versa. Thus, relationships between what happens on-screen and what happens off-screen would be established. The linkage would be based primarily on isomorphic visual and audio-visual relationships between both domains.

Further, I decided that the work should allow for audience participation through various tangible user interfaces. Even though, artistically, I was mainly interested in the perceptual interaction between on-screen and off-screen artefacts, I believed that the introduction of user interaction would increase the engagement with the exhibits and enhance the linkage between the virtual and the physical world.

From a perceptual point of view, the overall vision of my approach may be well illustrated by a certain special effect, sometimes used in films or television advertisements. I am referring to the fictional situation where someone is watching television, and all of a sudden humans, animals or objects start to come out of the television and invade the physical space of the viewer. I envisioned that my approach would to some degree ‘materialise’ the essence of this captivating special effect by creating ‘magical’ relationships between what happens on screen and what happens around the screen.

My confidence in the success of this approach was based on my belief that people have a strong liking for make-believe situations; maybe because these defy the rationality of the scientific age we are living in. I hope it will become clear from the project description, that much of what defines my approach relies on people’s fascination with irrational situations, and their willingness to use their own imagination.

Gaby Wood, when exploring the public’s fascination with Kempelen’s Chess Player automaton in the late eighteenth century, points out that the Chess Player ‘fulfilled what the historian Richard Altick has called “the public’s desire to be baffled”. It didn’t matter how many times the inventor insisted the automaton was merely an “illusion”; it was constructed during the Age of Reason, yet many found reason less appealing than enchantment.’ ³ Although Wood’s socio-psychological viewpoint is concerned with a situation of more than 200 years ago, I am inclined to argue that it is still (or maybe better, again) applicable to our ultra-technological society.

I conceived that the nature of the screen-based visuals as well as the automated sculptures and electronic devices should be fairly crude. In particular, I did not aspire to any form of high-definition realism. This choice was not only based on aesthetic considerations and personal taste, but was also an expression of my belief that the extent of engagement and immersion with a ‘system’ does not depend on the degree to which it imitates the ‘real world’. What is important, in my view, is how much it stimulates the participants’ imaginations and invites them to (playfully) engage.

There seems to be a general agreement on this point between two theoreticians who, on the whole, have rather different standpoints regarding modern technologies: the media theorist Marshall McLuhan, who regarded modern technologies as ‘extensions of men’; ⁴ and the postmodern sociologist Jean Baudrillard, who claims that we should see modern technologies rather as ‘expulsions of men’. ⁵

Baudrillard, when discussing the ‘cinematic development from silents to talkies and now to 3D and the current range of special effects’, argues that the ‘cinematographic illusion faded as the technical prowess increased […] The more we move towards the perfect definition, that useless perfection, the more the power of the illusion is lost.’ ⁶ Baudrillard illustrates his position with the following example:

To appreciate this [Baudrillard’s point made above], one only has to think of the Peking Opera and how, with the mere movement of their bodies, the old man and the girl brought to life on the stage the sheer size of the river and how, in the duel scene, the two bodies, skimming each other with their weapons yet not touching, made the darkness in which the duel took place tangible. That was total illusion – an ecstasy more physical and material than aesthetic or theatrical, precisely because all realist presence of the night or the river had been excised. Today, they would pipe tons of water into the studio, and the duel would be shot in the darkness with infra-red cameras. ⁷

McLuhan expresses a similar view on the issue of high definition in his concept of ‘hot’ and ‘cool’ media. According to him,

a hot medium is one that extends one single sense in ‘high definition’. High definition is the state of being well filled with data. A photograph is, visually ‘high definition’. A cartoon is ‘low definition’, simply because very little visual information is provided […] hot media do not leave so much to be filled in or completed by the audience. Hot media are therefore, low in participation or completion by the audience.’ ⁸

It should be obvious from the above outline that my approach would best fit within the category of ‘cool’ media, and following McLuhan, is therefore ‘high’ in terms of ‘participation or completion by the audience’.

As mentioned above, the linkage between the virtual and the physical domain would be based on visual and audio-visual relationships between both domains. With regard to the audio-visual aspect, I am particularly interested in the notion of ‘synchresis’ that was coined by the French composer-filmmaker-critic Michel Chion and which refers to the cerebral process of ‘forging an immediate and necessary relationship between something one sees and something one hears at the same time.’ ⁹

For me, the most interesting aspect of synchresis is that it ‘can even work out of thin air – that is, with images and sounds that strictly speaking have nothing to do with each other, forming monstrous yet inevitable and irresistible agglomerations in our perception.’ ¹⁰

This phenomenon is highly relevant for my work, because the sounds are created by computer-controlled electromechanical devices in the physical domain and are linked to visual events happening on the screen. Hence, the sounds and the on-screen visuals have no inherent relationship, but the linkage is solely constructed in the mind of the viewer. For instance, in a preliminary experiment which would later be developed into the installation Square Pusher (see below), I linked the movements of a bouncing on-screen square with the sound of an electromechanical hammer hitting a wooden board placed underneath the monitor screen. In this example, the user ‘throws’ an animated square on a touch-screen. Each time the square bounces off the bottom of the screen the hammer hits the wooden board. Although the bouncing movements of the square and the sounds created by the hammer have obviously no ‘real’ cause and effect relationship, I believe that a more or less inevitable linkage between the two is formed in the mind of the participant. With regard to the aim of UNCAGED to bridge the gap between the virtual and the physical world, one could argue that the vibrations and ‘live’ sounds (in contrast to recorded or synthesised sounds) synchronised to the square’s movement imply a materialisation of the virtual image and create the illusion of the square being a heavy physical object.

Human Computer Interaction

Although my interest in the notion of mixed reality was primarily inspired by the development of playful, perceptually-intriguing art installations rather than the desire to develop new technologies to be used in a scientific or commercial context, I could sense the potential relevance of my approach to issues concerning computer sciences, e.g. in the area of HCI (Human Computer Interaction). In particular, I anticipated that the practical work might be able to address some of our difficulties in engaging with computers in a meaningful and satisfying way. UNCAGED may also be viewed as a response to my research at the University of Sheffield, conducted between June 2001 and June 2002. My investigations for the University of Sheffield were based on a qualitative analysis of electro-acoustic composers ‘at work’ and demonstrated that many composers who work with computer-based systems ‘suffer’ from the distance between the physical reality and virtual computer data. For instance, my study revealed ‘a need for more direct, tactile means of seeking and manipulating sounds in composition and performance [which] was expressed by the desire for malleable interfaces that would allow for a sculptural shaping of sounds.’ ¹¹

Regarding the design of appropriate physical input interfaces for UNCAGED, I was very much stimulated by the approach taken by the Tangible Media Group at the MIT Media Laboratory. They advocate the use of everyday objects as a basis for input devices with the rationale of looking ‘towards the bounty of richly-afforded physical devices of the last few millennia and inventing ways to reapply these elements of tangible media augmented by digital technology.’ ¹²

One of the works by the Tangible Media Group comprises a bottle interface – a glass bottle with a cork. ¹³ The bottle is connected to a computer via wireless sensing technology. By means of small electromagnetic resonator tags placed around the opening of the bottle it is possible to detect when the cork is in place or removed from the bottle. The bottle interface is used in the sound installation ‘musicBottles’, this comprises three glass bottles, each stoppered with corks. Removing a cork triggers a particular soundtrack to start and replacing a cork stops the music. Conceptually, it was important for the group to maintain the coherence between the new digital meaning of the interface and its everyday functionality as a physical object. The basic ‘affordance’ ¹⁴ of a bottle is of course to store content and in the installation this content is represented by musical sound. According to the Tangible Media Group participants quickly understood the bottle metaphor and despite its simplicity most people enjoyed the interaction and often reacted emotionally to it.

For me, the approach taken by the Tangible Media Group was an assertion of my belief that user interaction with computer-based systems should be an organic extension of our interaction with the physical world, rather than a showcase for new technological possibilities that bear no resemblance to our normal interaction with the world.

Even though the final exhibits do not involve everyday objects as such, they are designed to be evocative of familiar forms of interaction and are often reminiscent of familiar games or childhood themes. For example the exhibit Blow Life (see below) includes a ‘hand interface’, which is similar to the interface of a fortune telling machine of the type often found at fun fairs. The exhibit Glitchy & Scratchy (see below) features two virtual records on a touch screen. I assumed that many people would have the urge to spin the records on the screen as a DJ might do, and certainly would have some preconceived idea of how to do so.

Further, I thought that, where appropriate, a simple push-button interface allowing participants to activate or play with an exhibit would be preferable to a more complex interface such as that of a motion capture device.

I would therefore suggest that to a great extent the user interfaces of UNCAGED aim to involve the participant(s) in a very straightforward way by appropriating well-established forms (of interaction) from the ‘physical’ world. What is new in UNCAGED is how the participants’ engagement affects the interaction between the physical and the screen-based world of the exhibits.

Social Interaction

In a public (art) exhibition context, my work also seemed to address issues regarding social interaction and was motivated in particular by recent studies by the Work, Interaction and Technology group (WIT) at King’s College London which suggest that most conventional screen-based exhibits in galleries and museums ‘not only undermine co-participation and collaboration at the exhibit itself, but remove the possibility of others seeing and making relevant sense of what people are doing elsewhere within the scene’. Their research suggests that ‘whilst interactive exhibits, in particular those relying on computing and information technologies, can often enhance an individual’s experience, they inadvertently impoverish the social interaction which can arise with and around exhibits in museums and galleries.’ ¹⁵ I was curious to find out if my ‘screen-based’ approach could evade the problem of inhibiting social interaction amongst gallery audiences because of its extension into the physical domain, and if it would make possible shared experiences amongst participants.

Description of the UNCAGED Series

To view photographs and videos of UNCAGED, please visit the project website <http://www.telesymbiosis.com> (active 5 April 2007)

UNCAGED comprises six interactive installations which are linked by a common theme: to explore interrelationships and transitions between screen-based digital environments and their immediate physical surroundings.

UNCAGED incorporates electromechanical devices and automated sculptures which interact, visually and acoustically, with computer-generated animations and video images. Most of the exhibits are reminiscent of familiar games or feature modified toys, and participants can engage playfully with the installations via touch-screens and tangible custom-made interfaces.

Blow Life is reminiscent of the fortune-telling machines often found at funfairs. Participants cover a hand image in front of a computer screen with their own hand. This activates an electric fan, mounted next to the screen, blowing an on-screen barcode like ‘grass in the wind’. After a few seconds the fan stops and participants receive their own ‘lucky number’ encrypted into a new, randomly generated barcode.

Direct video link:<http://www.telesymbiosis.com/movies/blowlife_hi.mov> (active 5 April 2007)

Bubblelabub is a screen display featuring a person blowing air into a tube. The tube is extended from the virtual image to a real glass bottle filled with water. Depending on the amount of pressure applied to a squeezable interface at the front of the exhibit, the cheeks of the on-screen person will inflate or deflate, and the amount of bubbles generated in the water bottle will vary accordingly. Contrary to the exhibit Blow Life, where the airflow of a fan triggers movements on the screen, Bubblelabub creates the illusion that air generated within the virtual domain can transfuse into the physical world.

Direct video link: <http://www.telesymbiosis.com/movies/bubblelabub_hi.mov> (active 5 April 2007)

Glitchy & Scratchy allows the user to spin two records on a touch-screen, backwards and forwards. The turntables (with vinyl record) on either side of the screen follow the spin direction and (variable) speed of the respective virtual record. Pressing the left or right button at the front of the screen repeats the most recent scratching pattern of the respective record.

Direct video link: <http://www.telesymbiosis.com/movies/glitchyandscratchy_hi.mov> (active 5 April 2007)

Square Pusher is similar to the traditional fun fair game known as Ring-the-Bell or Easy Striker. Like the traditional game, it features a bell mounted high above the ground. Participants ‘throw’ a bouncy square on a touch screen. If they hit the top centre of the screen the bell will ring, and they will earn a ‘100 point bonus’. When the square bounces off the bottom of the screen it triggers a banging sound and vibrations, which are caused by an electromechanical hammer hidden underneath the screen.

Direct video link: <http://www.telesymbiosis.com/movies/squarepusher_hi.mov> (active 5 April 2007)

PONG (telesymbiotic version) is a modified version of the renowned video game Pong. Two solenoid ‘bats’ are mounted at the edge of a computer screen, one on the left and one on the right. Using a push-button interface, player(s) trigger the bats at to keep a moving on-screen ball in play. If the ball is missed, it will disappear from the screen.

Direct video link: <http://www.telesymbiosis.com/movies/blowlife_hi.mov> (active 5 April 2007)

Not Only Jingle Bells for Two Spuikars and Other Players is an orchestra of automated acoustic instruments, most of which are based on toys, that provide ‘live’ soundtracks to various on-screen animations. On the initial screen display participants can select from four buttons. When the bottom button is selected the display will change into an interface of 39 ‘idle musicians’. Each musician is linked to a different sound in the orchestra of acoustic instruments. The musicians are animated by the touch of a finger, triggering their respective sound. With the top three buttons of the initial display, participants can select from three different ready-made clips.

Direct video link: <http://www.telesymbiosis.com/movies/jinglebells_hi.mov> (active 5 April 2007)

After UNCAGED

Throughout the early stages of the research and development phase of UNCAGED, but in particular after the work had been completed, I questioned my initial motivation, that is to ‘uncage’ computer-based realities from the confines of their digital existence and to bring the remote computer world closer to our human experience. I believe that my reservations about this rather positivistic motivation arose from two coinciding, arguably interrelated, notions.

First, my critical engagement with the work itself nourished the impression that despite the perceptual fusion between the digital and the physical world, UNCAGED actually seems to highlight the distance between the two domains. In my view, all six exhibits have an underlying absurdity which arises from the very fusion of their physical and digital components. For me, this absurdity ultimately hints at the fallacy of the initial motivation behind UNCAGED and in a wider context, questions the idea of seeking a place for meaningful human exchange and experiences in virtual worlds.

Second, temporally coinciding with, but not necessarily causally linked to, the creation of UNCAGED, my former enthusiasm for the computer as a working tool was clouded by a growing frustration with, and to put it bluntly, dislike of spending a good deal of my life (isolated) in front of the computer screen. Admittedly, in the light of my original motivation, one could argue that the very objective of UNCAGED was precisely about improving our relationship with digital technology, and that therefore UNCAGED could be regarded as a step towards overcoming my own frustration with the computer. I do believe that UNCAGED is successful in bridging the gap between the digital world and the physical world on a perceptual basis, and I feel the six installations certainly incorporate digital technology in a rather enjoyable and stimulating way. However, for me the fusion between the digital and the physical world in UNCAGED only works within the context of installation art or games. Ultimately, when applied to ‘real life’, it does not offer much hope toward making the digital world a more satisfying space to engage with.

Not surprisingly my new ‘attitude’ towards UNCAGED had a strong impact on the direction of my artistic practice. Initially, I intended to create a subsequent (to UNCAGED) body work, where the transition between the physical and the virtual world as well as the user interaction would be more seamless. I assume that it is obvious from my concerns, described above, that a development of UNCAGED in this (positivistic) direction would have been extremely pretentious. Instead my new artistic direction is not concerned with perfecting the perceptual and interactive level of UNCAGED, i.e. through technological advancement, but with exploring further the socio-philosophical issues implied in UNCAGED. In this context I am particularly interested in the writings of French theorist Jean Baudrillard. Despite the dangers of over-simplifying, even misunderstanding, his often ambiguous messages, I feel that UNCAGED, in particular in the light of my revised interpretation of the work, addresses some of the issues raised in Baudrillard’s texts.

For instance, Baudrillard differentiates between the ‘real’ as the world we perceive rationally and try to explain scientifically, and the notion of a ‘true real’ which hides behind the ‘real’ through illusion – defined as ‘the radical impossibility of a real presence of things or beings, their definite absence from themselves.’ ¹⁶ According to Baudrillard, ‘it is not, then, the real which is the opposite of simulation [virtuality] – the real is merely a particular case of that simulation – but illusion.’ ¹⁷

For Baudrillard, this illusion is vital to keep alive such notions as singularity, alterity, secret and seduction, which provide a necessary counterbalance to our, in his view, over-rationalised and (technologically) fully-realised world. ‘Against this artificial paradise of technicity and virtuality, against the attempt to build a world completely positive, rational, and true, we must save the traces of the illusory world’s definite opacity and mystery.’ ¹⁸

In Baudrilllard’s view, we are now faced with a fundamental extermination of this illusion, because through the omnipresence of virtual technologies we are moving towards a ‘hyperrealist world where any direct experiences [of the world] are replaced by televised [computerised] images.’ ¹⁹

As a very direct response to my reservations and my increasingly critical view on digital technology, I have created a series of apparently computer-controllable, networked toasters (Figs. 1 – 3). Essentially, the work aims to ridicule our obsession with computerising and automating an increasingly large part of our environment.

Fig. 1. Ralf Nuhn, Toaster mit Schnittstelle, 2004. Electric toaster, Sub-D type computer connector.

Fig. 2. Ralf Nuhn, Two networked toasters, 2004. Electric toasters, Sub-D type computer connectors, computer cable.

Fig. 3. Ralf Nuhn, Four networked toasters with hub, 2005. Electric toasters, Sub-D type computer connectors, computer cables.

Finally, my most recent project, Cyber-Spatialism, is a series of canvases in which computer connectors are inserted (Figs. 4 - 5). The work refers to Luigi Fontana’s (1899 – 1968) series of slashed canvases (Attese) and his concept of Spatialism (Concetto Spaziale), which is usually regarded as an attempt to overcome the illusionistic representation of space in painting by introducing physical space.

By substituting Fontana’s slashes with computer connectors, Cyber-Spatialism implies an extension of the canvas into cyberspace, and thus attempts to address the notion, that in today’s world physical space is increasingly being replaced by virtual space.

With regard to colour, pattern and relative dimensions, Cyber-Spatialism is closely based on Fontana’s ‘originals’ in order to make the relation between the two series more perceptible (Fig. 4 – 6).

Fig. 4. Ralf Nuhn, Cyber-Spatialism 1, 2005.Water-based paint and Sub-D type computer connectors on canvas, 27x35cm.

Fig. 5. Ralf Nuhn, Cyber-Spatialism 1, detail.

Fig. 6. Luigi Fontana, Concetto Spaziale – Attese, 1967. Water-based paint on canvas, 50x62cm.

Notes

1. The term telesymbiosis is used in biological research and literally means ‘symbiosis at a distance’. It has been adopted to describe the quasi-symbiotic relationships – between the physical world and the distant computer world – in UNCAGED. Admittedly, the term might be misleading as it is also used (synonymously with the term telepresence) in the context of Virtual Reality, where it refers to a person’s feeling of being present in a virtual or remote environment.

2. cf. McEvoy, J.P. & Zarate, O., (1999), [1996], Introducing Quantum Theory, Duxford: Icon Books Ltd.

3. Wood, G., (2002), Living Dolls, London: Faber and Faber.

4. McLuhan, M., (1994), [1964], Understanding Media, Cambridge, Massachusetts: MIT Press.

5. Baudrillard, J., (1996), [1995], The Perfect Crime, London: Verso.

6. Baudrillard, J., (1996), [1995], The Perfect Crime, London: Verso.

7. Baudrillard, J., (1996), [1995], The Perfect Crime, London: Verso.

8. McLuhan, M., (1994), [1964], Understanding Media, Cambridge, Massachusetts: MIT Press.

9. Chion, M., (1994), Audio-Vision, New York: Columbia University Press.

10. Chion, M., (1994), Audio-Vision, New York: Columbia University Press.

11. Nuhn, R., Eaglestone, B.M., Ford, N., Moore, A., Brown, G., (2002), ‘A Qualitative Analysis of Composers at Work’, Proceedings of the International Computer Music Conference, 2002, Göteborg.

12. Ishii, H. and Ullmer, B., (1997), ‘Tangible Bits: Towards Seamless Interfaces between People, Bits and Atoms’, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 1997, Atlanta, Georgia.

13. Cf. Ishii, H., Mazalek, A., Lee, J., (2001), ‘Bottles as a Minimal Interface to Access Digital Information’, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2001, Seattle, Washington.

14. Affordance is a property of an object that determines or indicates how that object can be used. Affordances may be actual physical properties, or perceived properties. The term was first introduced by psychologist James J. Gibson in 1966.

15. Heath, C. and Lehm, D., (2003), ‘Misconstruing Interaction’, Conference Proceedings: Interactive Learning in Museums of Art and Design, 2002, London.

16. Baudrillard, J., (2000), The Vital Illusion, New York: Columbia University Press.

17. Baudrillard, J., (1996), [1995], The Perfect Crime, London: Verso.

18. Baudrillard, J., (2000), The Vital Illusion, New York: Columbia University Press.

19. Baudrillard, J., (1993), [1990], The Transparency of Evil London: Verso.

 

Back to contents