Jean Bresson and Carlos Agon
Music Representations Team
Institut de Recherche et Coordination
Acoustique/Musique
1 Place Igor Stravinsky,
75004 Paris, France
{jean.bresson, carlos.agon}@ircam.fr
Current computer-music systems deal with complex
processes related to heterogeneous musical contents
and information. In particular, the objects involved
in music composition correspond to various tem-
poral paradigms and representations. For instance,
contemporary music composers frequently deal
with time durations, or even directly with the con-
tinuous aspects of sound structures, instead of (or in
relation to) a traditional pulsed-time representation.
In these cases, however, continuous musical mate-
rial must generally be associated with discrete and
symbolic structures to allow for their meaningful
representation (Boulez 1987). The score is a general
framework for music writing in which this integra-
tion of musical material and the related issues of
time representations should be carefully considered.
Regardless of whether it concerns instrumental or
electro-acoustic music, this document is of major
importance as a formal support and a preferred place
for musicians to write, read, and think about music.
In this article, we focus on this notion of the score
in computer-aided composition (CAC) through a
presentation of related work in the visual program-
ming environment OpenMusic (Agon 1998; Assayag
et al. 1999). We introduce a new object recently
created in OpenMusic: the sheet. This new object
gathers results of CAC experiments and research
concerning the relations between programs and
score representations, the integration of sound and
signal processing in compositional processes, the
mixing of heterogeneous temporal systems, and the
general visual representation of time structures in
music notation. The sheet editor allows for accurate
rhythmic notation while integrating specific fea-
tures such as the support for new types of musical
objects and programming tools.
After a brief discussion of the question of the
score in computer music composition, we present
previous attempts at combining programs and scores
Computer Music Journal, 32:4, pp. 3147, Winter 2008
c
!2008 Massachusetts Institute of Technology.
Scores, Programs, and Time
Representation: The Sheet
Object in OpenMusic
in OpenMusic. The sheet editor is then presented. Its
main features, which are detailed next, concern the
common representation of musical objects related
to heterogeneous time systems and the capability to
develop programs from within the score. We then
examine the possible function of this new object in
the general CAC framework, and we conclude with
a concrete example with the reconstitution of an
excerpt from a score by Karlheinz Stockhausen.
Computer-Music Scores
Traditional musical categories are being decon-
structed by contemporary musical practices and
computer-composition tools. The nature and the
role of the score has thus to be considered from a
renewed point of view.
The concept of the score usually refers to a music
representation built on a notation system inherited
from the Western music tradition, which is basically
a compound graphical structure used to organize
musical events (notes) in time. More generally, it
can be seen as a place where the composer writes
music in terms of a standard and universal language
describing some musical parameters or instructions
to perform in order to produce sound. In this sense,
computer-composition tools commonly call any
document containing a list or sequence of such
parameters or commands a score. In principle,
this kind of score is not always intended to be read by
human performers and thus does not systematically
meet high-level symbolical requirements. It can
be represented with simple numerical data, then
converted to synthesis parameters or to symbolic
notation for performance.
Our position, however, is to reaffirm the need
for symbolic musical representations in computer
system scores. As mentioned previously, a score
is not a simple sequence of instructions but also a
fundamental medium for composers to think about
their music, to describe and communicate musical
Bresson and Agon 31
forms and structures according to a symbolic
representation system. It supports and informs the
underlying compositional intentions.
This notion of score can therefore be a problematic
point, particularly in the context of electro-acoustic
music (Ebbeke 1990). The questions of how an
electro-acoustic score should be notated and inter-
preted, which are the objects of various research
projects in the musicological and signal-processing
communities, are not the object of our current dis-
cussion. It is important to comment, however, that
these notions are getting closer, and even sometimes
tightly integrated into, the compositional process:
Composers now create their own instruments using
sound-synthesis languages and invent their own
notational systems. The role of the score is therefore
reinforced as a central support federating these
different domains for which adequate flexibility is
required. Our objective, for now, is thus to think
about how to represent and implement a score in
a computer-composition system, given some nota-
tional conventions and possible sound-processing
systems at hand.
A first related issue is that of the new objects
used to represent and organize the score. Indeed, as
much contemporary music precisely attempts to
emancipate itself from traditional systems, note-
based representations should be replaced by, or
completed with, other kinds of sound-description
structures and representations. Musical examples
are widespread where traditional musical objects
(chords, voices, etc.) are used in addition to other
types of heterogeneous data (sound waveforms,
envelopes, or other extra objects and symbols)
meant to be interpreted by performers or by sound-
synthesis programs.
Current score editors are generally restricted to
traditional music notation and do not allow for the
representation of these new types of objects and
structures. On the other hand, advanced sequencers
allow one to schedule musical events and to control
different kind of musical or sonic parameters, but
without real symbolic notation. Often, composers
use external graphical tools to integrate extra com-
ponents to their scores. Some score-engraving and
notation systems allow for the extension of common
notation with user-defined symbols and graphics
32
(e.g., Smith 1972; Nienhuys and Nieuwenhuizen
2003), but usually this extra score editing is done
after the composition is completed. As a result, no in-
teraction can take place within the score framework
during the actual compositional process. The ENP
system (Kuuskankare and Laurson 2006) proposes
interesting solutions to work with personalized
systems within programmable score editors. The
ability to define new graphical objects, possibly re-
lated to specific synthesis parameters, constitutes an
important and successful attempt in this direction.
Another example of a common problem is the
cross-representation and manipulation of rhyth-
mic and proportional-duration representations.
Computer-composition systems generally provide
tools for working with one or the other of these
representations by using distinct objects and struc-
tures, but they do not allow for their common and
simultaneous use. The work of Hamel (1998) is a
notable exception, which proposes a flexible score
editor allowing for the use and graphical represen-
tation of rhythmic notation within a linear time
representation system.
A second major issue concerning the notion
of score in computer-composition systems is the
reconsideration of the traditional static design of
the score in the context of the new compositional
frameworks, consisting of networks of personal
conventions, representations, and relations. This
is one of the main points addressed by computer-
aided composition environments like those derived
from the PatchWork visual programming language
(Laurson 1996), OpenMusic (Agon 1998), and PWGL
(Laurson and Kuuskankare 2002). We now discuss
the ways in which the aforementioned issues guided
the main lines in the early developments of the
OpenMusic environment.
Score Programming in OpenMusic
Programs as Potential Scores
The creative and subjective aspects of music
composition led to the evolution of CAC systems
toward programming environments rather than
ready-made programs. Programming languages are
Computer Music Journal
indeed best-suited to allow composers to express
their musical ideas using all the possibilities
provided by the computer.
The concept of potential score introduced in
Assayag (1998) relates notation to the languages
that allow for the expression and the creation of
music as a materialization of information flowing
in the system but also as a medium where formal
inventiveness lives. Through this concept, the
merging of notation with language (and in particular,
with programming languages) is suggested as the
core purpose of the CAC systems: The computer
program that leads to the creation of a musical
structure becomes a potential score representation
of a piece of music. This representation emphasizes
the semantics of the musical processes and reflects
the corresponding compositional intentions.
The principal current CAC environments started
from such considerations and enriched programming
environments with tools, features, and interfaces
allowing for the development of specific musical
models and processes. In particular, with visual
programming languages such as OpenMusic, the
musical structures can be represented, created, and
manipulated graphically in relation to the processes
from which they are derived.
Scores in Compositional Processes
In addition to its fundamental visual programming
features, the OpenMusic environment was designed
to allow for a constant interaction between the
musical data and processes by means of a variety of
interfaces and editors. In particular, the score editors
are major elements in the compositional processes
created in OpenMusic visual programs.
At the origin of this design stands the idea of
programs that would contain applications related
to each other by functional means. This idea was
inspired by the OpenDoc project by Apple Computer
(Orfali, Harkey, and Edwards 1996), the purpose of
which was to integrate heterogeneous components
inside a federating document. The role of the ap-
plication (which usually produces documents) was
reversed by centralizing multiple applications in a
document, considered as a more intuitive working
space for a computer user. This project was never ac-
tually completed, but it inspired the architecture of
OpenMusic, which extended this basis by consider-
ing the program as the federating document. In this
way the communication, relations, and interdepen-
dencies between the different components (i.e., ap-
plications) could be better and more easily dealt with
than if they were integrated in a mere document.
In OpenMusic, a program is represented in
a patch editor and constitutes a framework for
the functional integration of (musical) data and
processes. In the patch editor, all these components
are represented by graphical boxes linked to each
other by graphical connections. Some special boxes
called factories represent the musical data structures
(classes) created and processed by the program (see
Figure 1). These boxes make it possible to generate
and store musical objects and to manipulate them
by programming. The factories also represent
applications in the program: They are associated to
special editors, allowing for a manual interaction
with the musical data (Agon and Assayag 2002). In
Figure 1, for instance, the breakpoint-function (BPF)
editor allows for the manual shaping of the input
values of the process represented, while the score
editor allows for the editing and modification of the
result of this process: We see that manual changes
(additional notes, ties, etc.) have been performed on
the voice object.
By integrating the editors/factories among the
different musical objects and processes in a common
symbolic description, the patches therefore consti-
tute powerful musical representations maintaining
scores as a central part of compositional processes.
However, except within the different musical object
editors, the patch does not actually form a consistent
temporal structure and thus cannot be considered as
a real score representation. A relevant representation
should at least include a temporal dimension so that
the music represented can be mentally or concretely
unfolded in time.
Musical Data and Temporal Processes
The maquette is another type of visual program, ex-
tending the notion of patch with temporal properties.
Bresson and Agon 33
Figure 1. Programs and
editors in an OpenMusic
patch. A voice object is
created from a ratio list
converted into a rhythm
tree and from pitch values
sampled from a breakpoint
function (BPF).
A maquette editor has a horizontal ruler that repre-
sents time and a vertical ruler whose function may
be set freely (or just used as a topological landmark).
Figure 2 shows an example of a maquette editor.
The maquette contains rectangular boxes, each
representing a program (i.e., a sub-patch) intended to
produce a particular musical result. The horizontal
axis gives a temporal meaning to the graphical
properties of these boxes: The position and size
of a box on this axis determine the beginning
and ending of the musical objects created by the
corresponding programs in relation to the global
time line. From this point of view, the maquette
editor can be considered as a sequencing tool
gathering all these objects in a temporally organized
structure. Moreover, the programs in the maquette
boxes can access and modify the graphical and
temporal properties of the box to which they are
34
attached (e.g., their position and extents regarding
the horizontal or vertical axes), so that the overall
context organization and the internal object calculus
may be affected by one another.
The boxes can also be connected to each other
by functional relations, which turns the maquette
itself into a program. Examples of such relations can
involve the construction of the musical objects. For
example, in Figure 2, the chords in boxes labeled
C and D may be computed starting from some
data coming respectively from boxes A and B, or
the sound in box E synthesized from data coming
from box B. They can also concern the temporal
organization of the containing boxes (e.g., the offset
of box F is computed to make it meet with the
beginning of box D). Musical objects are thus related
in two possible ways: causal relations expressed
in the program and temporal relations resulting
Computer Music Journal
Figure 2. A maquette
editor. The boxes are
embedded programs
(patches) that compute
musical objects, integrated
in a common time
reference on the horizontal
axis. (The vertical axis
consists of arbitrary,
dimensionless units for
spatial convenience.)
Functional connections are
visible between the
inputs/outputs of some of
these programs.

from the graphical layout. Hence, there exists a
computing time, independent from the time ruler
but following the functional composition of the
maquette, and an execution time depending on the
resulting position and size of the components of
the maquette. These two types of organizations
can actually be related, because the programs in
these boxes both produce musical data and they
access and/or modify their own temporal and
graphical properties. Finally, a maquette itself can
be contained in a box, which leads to the creation of
hierarchical temporal structures.
The maquette thus integrates formal musical
constructions in an intermediate form between
the visual program and the score. Various types of
temporal organizations can be implemented in this
document: Linear and hierarchical time organiza-
tions naturally incorporate its structure, and other
functional or logical organizations are available
through the programming aspects. In addition, het-
erogeneous objects and their possibly diverse time
paradigms can be brought together in a common time
reference. (Figure 2 shows a sound file together with
rhythmice.g., box Fand proportional-timee.g.,
boxes A and Bmusical objects.)
Additional details about the maquette features
and implementation are given in Agon (1998) and
Agon et al. (1998). Recent work has also been
carried out on the maquette for the specific design
of sound-synthesis processes, making it possible,
through a special visual program attached to the
maquette, to define the semantics of the overall
structure, namely, to determine how the musical
data organized in it are to be converted into a
global musical result. Temporal structures can
thus be created with sound-synthesis parameters,

Bresson and Agon 35

Figure 3. The maquette as Left: top-level structure
an alternative score including two
representation for sound sub-maquettes, each
synthesis: reconstitution producing a sound
on an excerpt from component; middle: one of
Traiettoria . . . deviata the sub-maquettes of the
(19821984) by M. Stroppa. intermediate level,
to be converted afterward into sounds via this
user-defined process (Bresson and Agon 2006). Such
musical forms, including the functional/temporal
organization of sound-description data, constitutes
a global sound representation that one could also
compare to a sound-synthesis score (see Figure 3).
Music Notation
In addition to the function of a potential score that
can be given to a compositional process represented
by a visual program or a maquette, we now expose
additional comments concerning our particular
interest in traditional music notation and in scores
under their traditional form. A first point is that
musicians and composers are generally still willing
to adopt this conventional representation to notate
works using newly created or extended systems.
It constitutes a common landmark and means of
communication for the increasing number of musi-
36
organizing synthesis
parameters; right: contents
of a box from the
intermediate level
sub-maquette, a program
generating sound synthesis
parameters.
cal systems. Hence, some kind of relation with the
traditional score notation is generally maintained,
even by composers working with sound synthesis or
electro-acoustic means. For instance, the events or
other types of data controlling the sound-synthesis
processes are frequently computed starting from
traditional score representation data, especially for
the specification of onsets and pitch-related infor-
mation. (Figure 4 shows an example in OpenMusic.)
It might also be worth mentioning that, even if the
OpenMusic score editor is far from equaling existing
dedicated score-editing software, it includes some
notable features such as rhythmic structures (Agon,
Haddad, Assayag 2002) that provide unique possibil-
ities of computation and representation of the rhyth-
mic hierarchical constructs, as well as an accurate
rendering of polyphonic scores, including polymetric
information and individual tempo/meter variations
for each voice. The microtonal notation avail-
able in the editor also facilitates working in with
microtones (Bancquart, Andreatta, and Agon 2008).
Computer Music Journal
Figure 4. Maintaining
relations among
sound-synthesis processes
and traditional score
representation in
OpenMusic:
time-stretching parameters
computed from a rhythmic
structure.
However, the combination of this notation
aspect with advanced temporal structures remains
an unsolved problem. When using a maquette, for
example, the boxes must start and end precisely in
specific points: One of the main potentials of this
interface is to integrate tightly and precisely such
temporal information in the musical processes.
As a consequence, it is clearly observable in
Figure 2 that the notation inside the different
boxes does not accurately match the time axis. (For
example, the beginning of a voice requires some
graphical space for the clef, so that the first note
graphically starts after the actual beginning of
the corresponding box.) Therefore, the maquette
cannot stand as a real support for music notation
and still looks more like a temporal programming
interface integrating out-of-time notated musical
objects.
Sheet: Formal Structure and General Description
The sheet is a new object created in OpenMusic that
can be seen both as a document and as a musical
container. Its editor resembles a traditional score
but follows the CAC principles discussed herein
regarding integration and common representation
of musical materials and programs. The objective is
to gather in one document the characteristics of a
score as a readable support (which implies providing
a coherent, symbolic temporal representation) and
the (visual) programming features enabled by the
computer-aided composition environment.
A sheet is composed of a variable number of
tracks, each containing one or more sequenced
objects. These different tracks contents can be
created in programs (i.e., in OpenMusic patches) and
gathered and inserted into a sheet using drag-and-
drop in the editor or directly by programming in the
same patches. Figure 5 shows a sheet constructed in
a patch starting from various musical objects.
The basic functionalities of the sheet editor
are thus the creation and removal of tracks and
the addition and removal of musical objects to or
from these tracks. The musical objects can then be
moved, copied, and so on, within tracks or from one
track to another. The tracks display can be edited
(using operations such as resize, reposition, respace,
Bresson and Agon 37
Figure 5. The sheet factory
in a patch and its
associated editor.
show/hide background, etc.) to reach the desired
graphical setup for the score.
In principle, any type of object coming from the
OpenMusic framework can be inserted in a sheet
provided it has a duration: chords and sequences,
voices, MIDI files, and even curves and envelopes,
sound files, etc. Each one of the inner objects is
individually accessible through its own editor (e.g.,
score editor, BPF editor, or sound file editor).
Time Systems and Graphical Representation
One of the main features of the sheet editor is the
consistent common graphical representations of the
different objects it contains.
Nonlinearity in the Time Representation
Music notation is not linear with respect to repre-
sentation of time: there is no linear relation between
38
time (durations) and the graphical space occupied by
the symbols in the score. A typical example of this
nonlinearity is the graphical representation of the
score elements that do not take any time at all (e.g.,
clefs, measure bars, metric information, acciden-
tals). As their size cannot be adapted to the timeline
(their dimension would be zero), then the time axis
is distorted to let them fit in the representation.
In Figure 6, a temporal grid with a 500-msec
step is displayed on the score, which underlines
the constant variation of the space occupied by this
duration along the voice representation. (The display
of a regular temporal grid available in the editor, as
well as the optional display of the absolute starting
times of the internal objects, will help to assess
this space/time distortion in the different example
figures given in this article.)
This nonlinearity can be problematic when vari-
ous simultaneous objects are represented together.
In such a joint representation, the simultaneous
events must be at the exact same position in the
horizontal time axis. In addition to the display
Computer Music Journal
Figure 6. A voice in a sheet
editor with a regular
temporal grid display.
Figure 7. Two voices
aligned in a sheet editor:
mutual influence of the
different objects
representations.

the different objects contained in the score. All

Figure 6
Figure 7
constraints of individual objects, the representation
of each one must therefore take in account the
nonlinearity brought by the others.
In Figure 7, two identical voices are represented
in canon, with the second one shifted in time
(by one second in this case). Therefore, the first
sixteenth-note of the second voice starts with the
fifth sixteenth-note of the first voice. As shown in
the figure, the two voices are identical in content,
but the graphical representation has been modified
to respect the temporal alignment of the musical
events. For example, the second note of the second
voice uses almost twice the space of the second note
of the first voice: Some space was required there for
displaying the first bar line of the first voice.
Temporal/Visual Alignment
The consistency of the time representation in the
sheet editor is ensured by an algorithm that calcu-
lates a global function relating time to space, given
possible landmarks in these objects are considered
and used to compute ratios between time (dura-
tion between two successive landmarks) and space
(dimension of the corresponding graphical compo-
nents). A method is defined for each class of objects
to determine these temporal landmarks whenever
possible. With traditional score objects, for instance,
they correspond to the beginnings and endings of
notes, rests, and other symbols. Determining land-
marks in linear objects (such as a sound files,
or other continuous functions, curves, etc.) also
adds the corresponding intervals in the time/space
function. As will be shown subsequently, this may
therefore make their graphical representation better
correspond to the global score time flow.
The resulting ratios are gathered in a global
discrete function x = f[t] in which every landmark of
the score finds a position depending on the possible
display constraints of the musical object of which it
is a part (as in Figure 6), and on those of the other
objects in the score (as in Figure 7). By applying this
function when building the graphical representation
(i.e., each time a modification occurs in the score
contents), the relations between time and space are
therefore modified for every time interval and for
each individual object in the score, including those
that could commonly be represented in a linear way.
This approach deliberately prefers the rhythmic
notation and rendering, which will generally be
the main source of time/space distortions. This
somehow goes against the usual way to proceed in
such mixed cases, where rhythmic objects submit
to linear time-line proportions. The algorithm, in
that case, could just have consisted of finding the
minimal space required for the minimum time
interval contained in the score and applying the
corresponding ratio to the whole representation,
hence possibly allocating large and useless spaces to
long notes or rests when used together with short
time divisions (e.g., sixteenth-notes or shorter).
With the sheet-alignment algorithm, in contrast, all
objects are locally stretched or compressed following
the distortions of the nonlinear notation symbols in
the score and the intervals determined by their own
temporal landmarks. Inside each single interval,
if no specific constraint exists, the representation

Bresson and Agon 39

Figure 8. Mixing temporal
systems in the sheet
editor: two traditional
voices and a MIDI file.
remains linear, but each interval has a specific ratio
depending on the overall context.
Time Systems
Different musical objects that might exist in the
sheet editor may correspond to various time sys-
tems. Among the objects existing in the OpenMusic
environment, three main types are identified: pulsed
time (divisions of regular pulses, such as the case of
traditional music notation), proportional time (e.g.,
sequences of musical events specified in millisec-
onds), and (pseudo-) continuous time (e.g., signals
and curves defined by mathematical functions).
Pulsed time objects were discussed previously. They
mostly correspond to the voice objects represented
using a traditional music-notation system and are
the main source of space/time distortions.
Proportional-Time Objects
Figure 8 shows another example of a sheet editor
where the tracks now correspond to different time
systems: The first two are voices in traditional music
notation, as in the previous examples, and the third is
a MIDI file (which is actually a MIDI transcription of
the voices) where events and durations are specified
in milliseconds (proportional time). The beginnings
and endings of the MIDI events in this MIDI file are
considered as landmarks: Following our temporal
alignment algorithm, the onsets and durations of
40
Figure 9. Mixing rhythmic stretched in the
and proportional score representation to make the
notations: each note in the chord-seq notes of track 2
upper (rhythmic notation) (in proportional notation)
track lasts 1 sec. The fit between the second and
second beat of 1 sec is third beats of track 1.
the MIDI events are therefore graphically adapted
accordingly with the score notation constraints, in
order to coincide with the notes and symbols of
this score. Two successive MIDI notes of the same
duration will thus no longer necessarily have the
same graphical size.
In OpenMusic, the chord-seq object is another
proportional-time object, but represented with
music notation: Although it resembles a score
representation, each chord or note is specified
by an onset given in milliseconds, with no other
consideration for meter or pulse units. However,
unlike the notes in MIDI files (Figure 8), the notes
in chord-seq objects have non-linear properties,
because they are displayed on a score system in the
traditional way (with note heads, stems, possible
accidentals, etc.). The sheet editor, in this case, will
thus also allow for the representation of chord-seq
objects together with other types of objects and in
particular with rhythmic notation (implemented
in the voice object), as shown Figure 9. Even
though chord-seq is a proportional-time object, it
is responsible for the main time/space distortion in
this example.
Continuous-Time Objects
In addition to the previous time systems, other
objects exist that we call continuous, because
they are not composed of well-marked discrete
events (e.g., notes). This category mainly includes
Computer Music Journal
Figure 10. Mixing time
systems in the sheet editor:
a traditional voice and a
sound file. The markers in
the sound allow the user to
align its segments with the
score representation. Note
the linear representation of
the same sound file in the
audio-file editor.

curves and sound signals. With these objects, only
two important time points are known: the beginning
and the end. These points, however, allow one to
position the objects and to deduce their size in the
graphical representation.
Additional points of interest can be defined
or deduced inside these objects to refine their
representation: sub-segments, particular events not
explicitly represented in the objects, etc. Making
these points participate in the time/space function
and in the graphical-display algorithm will correctly
position them in the global time-line representation
by independently applying possible distortions to
the subsequent intervals.
In the curves and functions, the inflection points
(or control points) can be used as landmarks. In the
case of sound files, we use markers that can be set
manually or algorithmically (i.e., in visual programs)
along the time axis.
Figure 10 shows an audio file in which some
markers were regularly placed every second, along
with a voice in traditional music notation. The
beginning of the audio track corresponds to the
actual beginning of the voice (the time of the first
note of this voice), and the segments highlighted in
the audio file with the markers (in this case, of one
second each) are stretched or compressed to respect
the time spacing of the score (here with a tempo of
60 beats per minute).
Programming in the Sheet Editor
The second main feature of the sheet editor results
from its embedded programming possibilities, which
make it possible to build the objects it is made of
algorithmically from within the score and/or to set
functional relations among them. When switched
to the patch mode, the editor resembles the usual
patches of OpenMusic (i.e., a visual program editor)
in which every object is displayed with an input
and an output and can be connected to functional
call boxes and/or to other objects of the score. The
sheet editor can therefore be a framework for the
development of programs in which objects of the
score are created and processed or linked by local
functional relations. This provides the user with the
possibility of bringing to light another part of the
musical semantics of the score, on both functional
and generative levels. In Figure 11, for example, two

Bresson and Agon 41

Figure 11. Functional Figure 12. A sheet editor
relations among containing various tracks
components of the sheet and sequenced musical
by programming in the objects.
editor (patch mode).

Figure 11

voices are connected through a program that makes

one the retrograde of the other.
The ability to evaluate objects or the whole
score allows one to build or update the contents
of the score according to these types of possible
dependencies. Whatever modification would be
done on the first voice of Figure 11, evaluating the
editor will make the second one its retrograde.
Now considering again an example such as the
one in Figure 10, one might also wish that the elec-
troacoustic part of the score (the sound file) would be
related to the upper (assumed instrumental) voice.
Figures 12 and 13 show a sheet in modes normal
and patch, respectively. The sound file in this ex-
ample is computed by a program embedded within
the sheet editor, using data coming from the notes of Figure 12
the first track and from the envelope of the third one.
The representation can thus be enriched with
different objects taking part in the musical and/or
sound-synthesis processes, or simply allowing a Figure 14 is another example where the process
better reading and understanding of the score. By represented in Figure 4 is reconstituted in a sheet,
switching back to the score mode, the program- hence emphasizing again from within the score the
ming components are hidden for a clearer score functional and temporal relations between the voice
display (Figure 12). and the synthesized sound.

42 Computer Music Journal

Figure 13. Sound-synthesis
process in the sheet editor
from Figure 12 (patch
mode). The Csound box is
an abstraction of a patch
(visible at the right side of
the figure) that performs
sound synthesis starting
from a musical sequence
and two dynamic
envelopes (one global,
coming from the third
track of the score, and the
other local, applied to each
individual note).

The Sheet in the Composition Environment
The sheet has been presented as a new musical
container in OpenMusic, associated with its specific
editor. It can be created (or inspected, etc.) as a part
of a program via the corresponding factory box in
the patch editor (Figure 5). Conversely, we have
shown in the previous section how patches could be
integrated in the sheet editor.
The integration of maquettes is also an interesting
issue: Considering their dual aspect of programs and
musical objects, they can be embedded in the sheet
editor and act either as a process producing a musical
output, or as a more or less complex musical form,
made of temporal and functional structures, inserted
in one of the sheet tracks (see Figure 15). In this
case, the maquette corresponds to what we have
called a proportional time object (with events
positioned in a time axis, specified in milliseconds).
The boxes it contains (whose onsets and endings are
considered as temporal landmarks in the alignment
algorithm) are therefore stretched and compressed
in the representation to keep alignment with the
music notation.
Like patches and maquettes, the sheet also exists
in OpenMusic as a standalone persistent document.
Therefore, patches, maquettes, and sheets are now
complementary documents: The patch emphasizes
the programming aspects, the maquette emphasizes
the time-processing aspects, and the sheet empha-
sizes the score and notation aspects. These different
supports should then naturally interact with one
another in the compositional processes and models
created in the CAC environment.

Bresson and Agon 43

Figure 14. Dynamic Figure 15. A maquette in
computing of a the sheet editor.
time-stretched sound in
the sheet editor. The
resulting stretched sound
segments are displayed
accordingly with the
initial rhythmic notation.

This score was written using personalized in-

struction graphics for the performers: Two players
are to excite the tam-tam, while two other players
handle microphones around the tam-tam following
precisely notated rhythmic and gesture instructions.
In the meantime, the two remaining players handle
filters with potentiometers to control the rendering
of the microphones outputs. Figure 16 shows a part
of this score reconstituted in the sheet.
The instructions for the tam-tam players
(rhythms, dynamics, positions, etc.) were created
here using OpenMusic voice objects with empty
staves and special note heads (tracks 1 and 5). The
movements and distances of the microphones from
the tam-tam and from the points of excitation are
represented by special curves with variable thick-
nesses, created in OpenMusic using modified BPF
objects (tracks 2 and 6). Finally, the different po-
tentiometers are controlled according to the visible
break-point or B-Spline curves (on tracks 4 and 8)
and to the gray figures representing the evolutions
of the filters high and low limits (tracks 3 and
Figure 14 7). These figures correspond to a special class of
musical object (BPF-bande) specifically designed for
this example. Figure 17 shows a patch implementing
part of the sheet-construction process.
In the future, we are considering pursuing this
experiment by implementing a sound-synthesis
processprobably using the OpenMusic interface
with Modalyss physical models (Ellis, Bensoam,
and Causse 2005)which would allow us to roughly
simulate and experiment with a performance of
the piece, starting from the data contained in our
score reproduction (following the principles of the
example in Figures 13 and 14). Aside from the
compositional aspect, this example also stresses
possible musicological applications; such an
Figure 15
implementation can indeed offer a computational
model of compositional and performing processes
that would enable musicologists to simulate and
A Concluding Example analyze particular aspects of a piece.

Before concluding, we here present a current work

in progress making use of the sheet editor, in which Conclusion and Future Works
we have tried to transcribe a part of the score of
Stockhausens Mikrophonie I (1965), for tam-tam This article introduced a new editor in the Open-
and six players. Music environment, addressing various questions

44 Computer Music Journal

Figure 16. A part of
Stockhausens
Mikrophonie I score,
reconstituted in the sheet
editor.
related to the general notion of the score in computer
music composition. The sheet editor allows for the
creation of scores integrating instrumental parts
(written in traditional music notation) together with
sound signals and other data or instructions possibly
related to the electronic parts of a piece. Regarding
instrumental notation, it can also be used for mix-
ing rhythmic notation (voices in OpenMusic) with
proportional time notation (like chord-seq objects)
in a single score, which is a longstanding problem
in music-editing tools. Moreover, the possible func-
tional relations set between the different objects
and the general programming features make score-
representation issues and their implications in terms
of temporal representation interact with computing
issues, which we believe to be of major importance
in computer-music composition systems. The po-
tential of this new type of score can therefore be fully
achieved in a CAC environment such as OpenMusic.
The sheet extends the previous types of docu-
ments, the patch and the maquette, and may be used
for various purposes in complement with them,
particularly when there is a special interest in mu-
sic notation. The many possibilities of embedding
these different documents inside each other is yet
another essential characteristic of the system and
should help composers to freely combine the various
abstractions and structural levels of their musical
works.
Future developments will probably concern the
setting of more advanced time relations between the
objects in the sheet editor. The temporal landmarks
present or defined by the user in the musical objects
indeed allow one to envisage possibilities for their
mutual synchronization (Stroppa and Duthen 1990).
An object could be positioned in the time axis using
one of its internal temporal landmarks, possibly
synchronized to a landmark of another object from
another track of the sheet. In the future, persistent
temporal relations between the objects could hence
be implemented, using, for instance, the interval
logic operators from Allen (1983), or other temporal
Bresson and Agon 45
Figure 17. Building a
model of Mikrophonie I in
OpenMusic. This patch
shows the creation, by
programming and using
their graphical editors, of
some of the components of
the sheet from Figure 16.
Left: a part of the curve of
track 4; right: some filter
settings visible on track 3.
These objects are then
collected in a sheet object
factory.

logic formalisms as described, for example, by
Marsden (2000). Another important related issue
to be addressed concerns the pagination of score
editors to allow for the display and printing of larger
scores and musical works.
Acknowledgments
The authors would like to thank Catherine Durand,
Michael Fingerhut, and Georges Bloch for their help
in proofreading this manuscript.
References
Agon, C. 1998. OpenMusic: un langage de programma-
tion visuelle pour la composition musicale. Ph.D.
Thesis, Universite Pierre et Marie Curie, Paris 6, France.
Agon, C., and G. Assayag. 2002. Programmation visuelle
et editeurs
musicaux pour la composition assistee
`
par ordinateur. 14eme
Conference Francophone
sur lInteraction Homme-Machine IHM02. New
York: Association for Computing Machinery, pp. 205
206.
Agon, C., et al. 1998. Objects, Time and Constraints in
OpenMusic. Proceedings of the 1998 International
Computer Music Conference. San Francisco, California:
International Computer Music Association, pp.
406415.
Agon, C., K. Haddad, and G. Assayag. 2002. Repre-
sentation and Rendering of Rhythmic Structures.
International Conference of Web Delivering of
MusicWedelMusic. Piscataway, New Jersey: IEEE
Computer Society Press, pp. 109113.
Allen, J. F. 1983. Maintaining Knowledge About
Temporal Intervals. Communications of the ACM
26(11):832843.

46 Computer Music Journal

Assayag, G. 1998. Computer Assisted Composition
Today. First Symposium on Computer and Music.
Corfu, Greece: Ionian University, pp. 917.
Assayag, G., et al. 1999. Computer Assisted Composition
at IRCAM: From PatchWork to OpenMusic. Computer
Music Journal 23(3):5972.
Bancquart, A., M. Andreatta, and C. Agon. 2008.
Microtonal Composition. In J. Bresson, C. Agon, and
G. Assayag, eds. The OM Composers BookVolume
2. Paris: IRCAM/Editions Delatour, pp. 279302.
Boulez, P. 1987. Timbre and CompositionTimbre and
Language. Contemporary Music Review 2(1):161171.
Bresson, J., and C. Agon. 2006. Temporal Control over
Sound Synthesis Processes. Proceedings of Sound
and Music Computing Conference, Marseille, France:
Centre National de Creation
Musicale, pp. 6776.
Ebbeke, K. 1990. La vue et loueProblematique
des partitions dans la musique electro-acoustique.
Contrechamps 11:7079.
Ellis, N., J. Bensoam, and R. Causse. 2005. Modalys
Demonstration. Proceedings of the 2005 International
Computer Music Conference. San Francisco, California:
International Computer Music Association, pp.
101102.
Hamel, K. 1998. NoteAbility, A Comprehensive
Music Notation System. Proceedings of the 1998
International Computer Music Conference. San
Francisco, California: International Computer Music
Association, pp. 506509.
Kuuskankare, M., and M. Laurson. 2006. Expressive
Notation Package. Computer Music Journal 30(4):67
79.
Laurson, M. 1996. PatchWork: A Visual Programming
Language and Some Musical Applications. Doctoral
dissertation, Sibelius Academy, Finland.
Laurson, M., and M. Kuuskankare. 2002. PWGL: A Novel
Visual Language based on Common Lisp, CLOS and
OpenGL. Proceedings of International Computer Mu-
sic Conference. San Francisco, California: International
Computer Music Conference, pp. 142145.
Marsden, A. 2000. Representing Musical Time: A
Temporal-Logic Approach. Lisse, Netherlands: Swets &
Zeitlinger.
Nienhuys, H.-W., and J. Nieuwenhuizen. 2003. LilyPond:
A System for Automated Music Engraving. Proceed-
ings of the XIV Colloquium on Musical Informatics
(XIV CIM 2003). Florence, Italy: Tempo Reale/AIMI,
pp. 167172.
Orfali, R., D. Harkey, and J. Edwards. 1996. The Essential
Distributed Objects Survival Guide. New York: Wiley.
Smith, L. 1972. Score: A Musicians Approach to
Computer Music. Journal of the Audio Engineering
Society 20(1):714.
Stroppa, M., and Duthen, J. 1990. Une representation
de structures temporelles par synchronisation de
pivots. Colloque Musique et Assistance Informatique,
Marseille, France: Laboratoire Musique et Informatique
de Marseille (MIM), pp. 305322.
Bresson and Agon 47