Workshop 2: Jonathan Bird, Open-ended Research in the Wild

Jonathan Bird (Pervasive Interaction Lab, Open University) gave an overview of three projects which involved rapid prototyping novel technologies and testing them ‘in the wild’, rather than in a laboratory: a wearable tactile vision sensory substitution (TVSS) system; a participatory curation system for a film festival; and an interactive art installation for a music festival. Bird explained that the motivation for developing the TVSS was scientific whereas the two other projects had artistic goals. However, he argued, the development process in all three projects was very similar, suggesting that there is some common ground between art and science when they adopt an open-ended experimental approach. Bird consequently referred to the project ‘E-Sense’ (www.esenseproject.org) which promotes speculative, interdisciplinary research, combining HCI, philosophy, computer science and psychology. He explained goals of the ‘E-Sense’ project in terms of building useful sensory augmentation devices and generating novel insights into sensory, bodily and cognitive extension.

Firstly, Bird referred to the seminal research of Paul Bach-y-Rita (1969), who used a device including a camera on wheels and a chair with 400 solenoids in the back. This converted visual stimuli into vibro-tactile stimuli to enable blind research subjects to 'see'; subjects sat in the chair and learned to recognize objects and even displayed a 'looming' response when objects moved quickly towards the camera. Bird also described Tactually Guided Batting experiment by Jannson and Brabyn (1981) where Bach-y-Rita chair equipped with a static camera allowed  subjects to perform batting even when they cannot see their hand and the ball (it was crucial that subjects’ movement affects the pattern of tactile stimulation they receive).  

 

 

Consequently, Jonathan Bird presented five interesting findings from Bach-y-Rita’s TVSS Experiments:

  • subjects report their perceptual experience in Quasi-visual terms (not vibrotactile ones)
  • subjects report sensing objects out in the world, not vibrations on their body
  • subjects can only learn to perceive objects if they are able to actively move the camera and sense how the vibrations change
  • the vibrotactile stimulation can be moved to different parts of the body and subjects are still able to perceive objects
  • blind subjects did not find the TVSS system very ‘emotionally satisfying’ – they have not been widely adopted

 

 

Correspondingly, Bird referred to his own research outlining the requirements for a minimal tactile vision sensory substitution (TVSS) system such as portability (so they can demonstrate and test the system ‘in the wild’ away from the lab), open source tools and designs (to engage the DIY communities and encourage participatory design), cheap off the shelf components (so that hobbyists can afford to build one) and finally, low resolution (to reduce cost, with the capacity to increase the number of motors as required). Subsequently, Jonathan Bird described his research approach as rapid, periodic prototyping, as that based on the work of Rettig: “no matter how hard you think about it, you aren’t going to start getting it right until you put something in front of actual users and start refining your idea based on their experience with your designs” (1994). Bird also explained how his research is aimed to work as Lo-Fi prototyping with focus on usability issues that increases number of build-test-refine cycles. Bird explained how this Lo-Fi prototyping allows users to see designs as formative and give more than ‘fit and finish’ criticisms, avoiding what Schrage describes in the context of the US automobile industry clay models: “The work required to craft them made them more like untouchable works of art than malleable platforms for creative interaction” (1996). 

Jonathan Bird presented himself as a tinkerer, drawing distinctions between that activity and  that of enginnering: 

Following, Bird emphasized the importance of operationalisation of the transparent technology, understood as ‘a technology that is so well filed to, and integrated with, our own lives, biological capacities, and projects as to become … almost invisible in use’, as opposed to the opaque technology which ‘by contrast, is one that […] remains the focus of attention even during routine problem- activity’ (Clark, Natural-Born Cyborgs, 2003:37). Concomitantly, Bird described two distinct modes of tool use:

 

 

Arguing for a move towards a transparency test Bird made three claims. Firstly, Bird explained that when subjects are using the minimal TVSS in a transparent way their attention will be focused on objects in the world, not the vibrotactile interface. Secondly, he pointed out that they will not detect changes in properties of the vibration (e.g. intensity) – they will exhibit change numbness. Thirdly, Bird argued that if we change the mapping between the visual image and the vibrotactile array, the interface will become opaque and they will detect changes in the vibration. Consequently, Bird posed a question: Will there be a difference in the time taken to adapt to the new mapping when the camera is fixed and when it is head mounted?

Subsequently, Bird referred to another example of the use of motion capture and vibrotactile feedback – in this case used to teach violin bowing.  He also referred to a haptic drum kit and augmenting multitouch tables with vibrotactile feedback.

Jonathan Bird then summarised the principles of the E-Sense Research in four points. Firstly, he referred to their use of a lo-fi, periodic prototyping approach to build sensory augmentation devices that they test ‘in the wild’. Secondly, Bird explained that their devices use a variety of sensors (cameras, microphones, gyroscopes) which are mapped to vibrotactile feedback. Thirdly, he claimed that they hope to gain more insight into the conditions under which technologies become transparent. And finally, Bird pointed out that they are starting to investigate whether there are differences between static and headmounted cameras in the TVSS system with respect to the time taken to adapt to changes in the mapping.

In the second part of his presentation, Jonathan Bird gave an examples of the application of an open-ended tinkering approach to art projects. His first example was building of a participatory curation system for 60 Second Film Festival that took place  in Portsmouth in November 2004. Bird explained how over a weekend installation visitors watched videos for over 9 hours where the inbuilt system was intuitive and did not require written instructions, appealing to a wide age range. Bird described how the public watched the most established artist’s video the most but also rated some videos more highly than the curators of the film show.

Bird’s second example in the relation to the arts was building of an interactive artwork for Loop Music Festival in Brighton in August 2008. He explained how the system provided resources that participants could appropriate facilitating play (ludic design). Bird pointed out that just as in the case of his previous example, this system was intuitive and did not require written instructions making it appealing to a wide age range. He also described how groups as well as individuals played with the system for extended periods of time, emphasising the crucial role of the latency of reciprocal feedback between machine and participants. Bird also stated that this system could be installed and taken down rapidly and adapted to different size spaces which was advantageous.

In conclusion, Jonathan Bird argued that an open-ended tinkering approach can be applied for both artistic and scientific goals and experimenting ‘in the wild’ is important in order to understand how people use technologies. Bird finished his presentation by stating that facilitating public participation in artworks is one way to generate novelty: if we want a system to generate novelty then to some extent we must leave it underdetermined and allow it to be configured through interaction with its environment.

DISCUSSION

Discussion following Bird’s presentation, focused around three issues: contingency, cognition and translational research.

Firstly, a reference was made to the work of Derrida, stating that every system is always undetermined by contingency and not a single technology we are used to ends up making what is expected. Contingency was consequently considered in terms of its degree. A distinction was made between certain artists who are concerned with self expression and don’t want their artwork to be configured by different people, and  others (including researchers) who set up minimal conditions and expect human interactions.

Secondly, a problematic of how cognition starts coming into being was considered. Here, an agreement was reached that a fundamentally physical system is where cognition is happening and that relates to the systems in the world.

Thirdly, a notion of translational research (in particular in medical sphere) was discussed in terms of taking research out to the point of application, instead of finalizing it first. Concomitantly, questions relating to discourse around knowledge economy and knowledge production were raised, including debates of whether knowledge system (academy versus the public) is shifting in the 21st century.  Also, the issues of ‘unfinished knowledge’ going out into the world were considered in terms of the ‘dark side’ as elements of that move raise ethical and political questions of the world as the laboratory. Following, it was decided that our systems are undetermined because we don’t have the ‘finished knowledge’. Discussion was closed with a consideration of the relationship between  academia, public affairs and knowledge production. Here, an example of the Open University was brought forward, with claims that this institution represents a true interdisciplinarity (the public and professionals researching together).