Mazzola, Guerino / Noll, Thomas / Lluis-Puebla, Emilio: Perspectives in Mathematical and Computational Music Theory

a set of four parameters (i.e. their position in space and time) will be needed for each voice for polyphonic playing. The physical curve $g$ represents the transformed symbolic gesture curve $G$ .

At this point the question arises what kind of parameters will constitute the gesture spaces. Intuitively, the physical gesture space is defined by its application. For example, if we want to model a virtual piano player, the parameters will be defined by the desired complexity of the virtual scene. Typically, this includes geometric parameters of the hands playing, such as angles between finger segments, and motion parameters such as velocity and acceleration vectors of the finger tips or of the ankles. Those parameters are represented by the $a$ and $b$ axes in the physical gesture space of 1.

The situation is different for the case of the symbolic gesture space, however: there is no implicit definition what a symbolic gesture space is and we have to define one for our own purposes. Currently we are using a space close to the one given in figure 1 according to its similarity to MIDI. MIDI can be seen as a very simple way of defining symbolic gestures. From this viewpoint the “Note on”, “Note off”, and “Velocity” (which is used instead of “Loudness” in MIDI jargon) concepts are actions of several symbolic fingers playing an instrument. The symbolic gesture curve in figure 1 exactly incorporates these concepts, the note on and note off events are denoted by the arrows in the symbolic gesture space and the velocity itself is given by $dG/dt$ .

3.2 Towards Calculation of Physical Gesture Curves

One goal of the current PerformanceRubette is to be able to calculate physical gesture curves, which then can be represented by avatars in a virtual environment. Unfortunately we do not know anything about a possible direct transformation $pGesture$ . Thus, instead of defining $pGesture$ , we start out with a given symbolic gesture curve, which can for instance be obtained directly from a given MIDI file, ‘freeze’ the curve, which yields events in the score space. From here, $pScore$ can be applied, as it has been done in the past: $pScore$ is defined by performance vector fields, which are numeric results delivered by a number of analyses (e.g. melodic, harmonic, motivic, etc.). The process results in physical instrument events in the performance space. What remains, is to ‘thaw’ those events, which results in the physical gesture curve we are looking for. However, this ‘thawing’ operation is not trivial but the problem can be attacked in the symbolic domain, as shown in the following section.

4 Construction of Symbolic Gesture Curves

As we have seen in the previous section, we are left with the problem of how the vertical relationships in figure 1 can be accomplished. This section deals with the ‘freezing’ and ‘thawing’ operations between the two symbolic domains.

Figure 2:

A symbolic score, above without, below with fingering. Horizontal axis denotes onset time, vertical axis pitch.

For the discussion of the ‘thawing’ operation, consider the piano-roll like scores in figure 2. Pitch is given by the vertical axis, onset time by the horizontal axis. The five events reside inside the onset boundaries $e0$ to $e6$ . Observe that the end of