The issues of interactivity between human and machine cross over many research topics. Among them we identify three for conceptual background before we revisit the concept of musical performance practice. These background topics include the theory of emotion, inference processes, and the mind-body problem.
The birth of the theory of emotion and its destiny in the laboratory
In the theory of emotion proposed by William James, the faculty of emotion is applied to low level mechanisms focusing on physiological changes given external stimuli (James 1884). Since then, innumerable experiments have been conducted in laboratories for understanding sensory responses of animal subjects to conditioned and unconditioned stimuli, and the method for extinction of conditioned behavioral responses (Davis 1994, Fanselow 1996). Recently, a neurophysiological basis of emotion offers an alternative scientific explanation for brain mechanisms related to emotional processes (LeDoux 1994). These are within the address of emotion as a low level function, and brain scientists acknowledge the limit of their approaches for providing a comprehensive view of emotion. Von Foerster’s theory of compatibility between external stimuli and movements in internal states of an organism offers yet another site to look at the problem from a cybernetics point of view (Von Foerster 1981). In this view the changes in sensation of an observer induces the observer’s movements, that is, the changes of the observer’s own internal states. Such changes in sensations can be caused by the movements of external stimuli when the observer has a compatible mechanism to detect or perceive the movements of external stimuli. The conceptualization between an observer and machines described in section 1.2 is indebted to this theory and Maturana’s theory of autopoiesis (Maturana 1980). The signal exchanges among interacting agents as to induce mutual state changes would enhance the emotional engagements in their interactivity.
Logical boxes shape inference processes
In the computer music community some interesting works can be found which attempt to understand musical phenomena in terms of cognitive process, suggesting as a basis the structural organization of musical elements and decision making processes with computational logic (Rowe 1993, Cope 1990). Particularly the modeling of machine listening opens a rich problem area in feature classification that is compatible to human listening (Rowe 1993). These works are mostly associated with the methods offered by the artificial intelligence field. The organizational methods of musical elements are coupled with logical processes so that listeners can observe the sound output, with the suggestion that listeners possibly reverse engineer the undergone organizational processes while listening. These approaches invoke the valuable association between listening experiences and computational processes in the realm of cognitive science. At the same time they leave a wide range of interpretations of the structural organization of the piece itself, and of the listening experience of the piece. Though such approaches do not presuppose an exclusive value in seeking or establishing obvious correlation between the piece and the experience of listening to it, the anchor of these approaches lies on inference processes both in computers and listeners.
Ontological debate on the mind-body problem is an obstacle
We state that a compositional task concerns the practice of generating listening experiences. With this statement, when considering the first approach based on the theory of emotion we know we are not aiming for scientifically-controlled listening experiences such as in conditioning rooms. Yet we are inclined to define practical ground to obtain a view for creating listening experiences by accounting not only for the listening environment also for the cognitive processes of listeners. In this attempt we face the following: composition may be defined as algorithm, and performances may proceed with algorithms, yet it is worth noting that listening experiences might never be defined algorithmically. As for approaches based on inference processes, let us suppose, as listeners in the wild, we become good at chasing the instances of logic trees while we listen. Suppose we learn more of the logical boxes so that someday our knowledge can be applied to understanding the fascinating aspects of musical phenomena in general. This does not mean we would understand how music evokes emotion or how emotion evokes music. Nor does it mean we would know how the connectivity of neurons shifts its weight inside of a virtuoso’s brain over time while she or he controls global and local dynamics along the projection of musical structure in a time-critical manner. Neither can we predict whether we would know more about music even if we can see through the brain of the performer to watch the neuronal activities. These belong to the epistemological dissatisfactions that could misguide us in our attempt to solve them. In addition, we have the burden of a history of research in fields such as cognitive science, AI, and neurophysiology. For the most part the long history of mind-body problems remains an on-going discourse among different opponents, between functionalism and reductionism, between cognitive science and neurophysiology.
Cope, D. (1990). Pattern matching as an engine for the computer music simulation of musical style. In Proceedings of the ICMC. Glasgow: International Computer Music Association, 1990.
Davis, M. (1992). The role of the amygdala in conditioned fear. The Amygdala: Neurobiological Aspects of Emotion, Memory, ad Mental Dysfunction, J. P. Aggleton, ed. NY: Wiley-Liss, Inc., 255-306.
Fanselow, M. S. (1994) Neural organization of the defense behavior system responsible for fear. Psychonomic Bulletin and Review, 1: 429-438.
James, W. (1884). What is an emotion? Mind, 9: 188-205.
LeDoux, J. E. (1994) Emotion, memory, and the brain. Scientific American, 270: 32-39.
Maturana, H. and Varela, F. (1980). Autopoiesis and Cognition: The Realization of the Living, Dordrecht: Reidel.
Rowe, R. (1993). Interactive Music Systems. 1993. Cambridge, MA: The MIT Press.
Von Foerster, H. (1981). Observing Systems, 1991. Seaside, CA: Intersystems Publications.
From my paper: “Interactivity vs. Control: Human-Machine performance basis of emotion,” Kansei, the Technology of Emotion. Proceedings of the AIMI International Workshop, A. Camurri, ed. Genoa, Associazione di Informatica Musicale Italiana, October 3-4, 1997, pp. 24-35.