document at i on

• Research reports
• Musical works
• Software

AudioSculpt
Cross-Synthesis Handbook

by Mario Mary

Second edition, April 1996

Copyright © 1995 Ircam. All rights reserved.

This manual can not be copied, in whole or part, without the written consent of Ircam.

This manual was written by Mario Marcelo Mary under the supervision of Marie-Hélène Serra,
and was produced under the editorial responsibility of Marc Battier, Département de la Valori-
sation, Ircam. English translation by Richard Dudas.
AudioSculpt
Design Philippe Depalle, Chris Rogers
Program Chris Rogers
Project supervision Gerhard Eckel

SVP
Design Philippe Depalle
Program Philippe Depalle, Gilles Poirot, Chris Rogers, Jean Carrive

Analysis modules
Algorithms Xavier Rodet, Philippe Depalle, Guillermo Garcia, Ernst Terhardt,
Boris Doval

Documentation
Coordination Marie-Hélène Serra
Manuals Marc Battier, Jean Carrive, Peter Hanappe, Mario Mary, Brice
Pauset, Marie-Hélène Serra

This documentation corresponds to version 1.0 or higher of AudioSculpt and to version 1.3A
of SVP.

Apple Macintosh is a trademark of Apple Computer, Inc.

AudioSculpt is a trademark of Ircam.

First Edition, October 1995

Ircam
1 place Igor-Stravinsky
F-75004 Paris
Tel. 01) 44 78 12 33
Fax 01 44 78 15 40
E-mail : ircam-doc@ircam.fr
IRCAM Users group

The use of this software and its documentation is restricted to

members of the Ircam software users group. For any supplemen-
tary information, contact:

Département de la Valorisation
Ircam
Place Stravinsky, F-75004 Paris

Tel. 01 44 78 49 62
Fax 01 44 78 15 40
E-mail: bousac@ircam.fr

Send comments or suggestions to the editor:

E-mail: bam@ircam.fr
Mail: Marc Battier,
Ircam, Département de la Valorisation
Place Stravinsky, F-75004 Paris

http://www.ircam.fr/musinfo
To see the table of contents of this
manual, click on the Bookmark Button
located in the Viewing section of the
Adobe Acrobat Reader toolbar.
Contents
List of examples 6
Résumé 7
General Information 8
Introduction 9
Source-Filter Cross-Synthesis 11
Adjusting the Analysis Parameters 13
Example 1 index 1, 2 and 3 19
Example 2 . 1 index 4, 5 and 6 21
Example 2 . 2 index 7 22
Example 3 . 1 index 8, 9 22
Example 3 . 2 index 10 22
Example 4 . 1 index 11, 12 and 13 23
Example 4 . 2 index 14 26
Creating a Text File for Dynamic Sound Processing 27
Example 4 . 3 index 15 29
Example 4 .4 index 16 29
Generalized cross-synthesis 32
Example 5 index 17, 18 and 19 34
Example 5.1 index 20, 21 and 22 36
Example 5.2 index 23 39
Example 5.3 index 24 39
Example 5.4 index 25 40
Example 5.5 index 26 and 27 41
Example 5.6 index 28 42
Example 5.7 index 29, 30 and 31 43
Example 6.1 index 32, 33 and 34 45
Example 6.2 index 35 47
Example 7 index 36, 37 and 38 48
Example 8.1 index 39, 40 and 41 50
Example 8.2 index 42 53
Example 8.3 index 43 54
Example 9.1 index 44, 45 and 46 56
Example 9.2 index 47 59
Example 9.3 index 48 60
Example 9.4 index 49 60
Example 10 index 50, 51 and 52 61
Cross-Synthesis by Mixing 65
Example 11 index 53, 54 and 55 65
Example 12 index 56, 57 and 58 67
Appendix 68
Example 13. 1 index 59, 60 and 61 69
Example 13.2 index 62, 63 and 64 71
Example 14.1 index 65 and 66 71
Example 14.2 index 67 and 68 74
Example 14.3 index 69 76
Conclusion 77
Annex: List of options 78
Index 79
List of examples

Example 1 index 1, 2 and 3 19

Example 2 . 1 index 4, 5 and 6 21
Example 2 . 2 index 7 22
Example 3 . 1 index 8, 9 22
Example 3 . 2 index 10 22
Example 4 . 1 index 11, 12 and 13 23
Example 4 . 2 index 14 26
Example 4 . 3 index 15 29
Example 4 .4 index 16 29
Example 5 index 17, 18 and 19 34
Example 5.1 index 20, 21 and 22 36
Example 5.2 index 23 39
Example 5.3 index 24 39
Example 5.4 index 25 40
Example 5.5 index 26 and 27 41
Example 5.6 index 28 42
Example 5.7 index 29, 30 and 31 43
Example 6.1 index 32, 33 and 34 45
Example 6.2 index 35 47
Example 7 index 36, 37 and 38 48
Example 8.1 index 39, 40 and 41 50
Example 8.2 index 42 53
Example 8.3 index 43 54
Example 9.1 index 44, 45 and 46 56
Example 9.2 index 47 59
Example 9.3 index 48 60
Example 9.4 index 49 60
Example 10 index 50, 51 and 52 61
Example 11 index 53, 54 and 55 65
Example 12 index 56, 57 and 58 67
Example 13. 1 index 59, 60 and 61 69
Example 13.2 index 62, 63 and 64 71
Example 14.1 index 65 and 66 71
Example 14.2 index 67 and 68 74
Example 14.3 index 69 76
Résumé

Ce manuel est une illustration d’une des possibilités les plus intéressantes offerte par le programme AudioSculpt :
le synthèse croisée. A travers une série d’exemples sont expliqués différents aspects de ce traitement qui lie in-
timement deux sons.
Le manuel s’accompagne d’une série de fichiers de son utilisés par les exemples. Ils sont disponibles votre CD-
Rom.
Comme ce manuel a une vocation pédagogique, il est conseillé au lecteur d’étudier les exemples l’un après l’au-
tre.

7 - AudioSculpt - Cross Synthesis Handbook

1 General Information

AudioSculpt is an Ircam program which permits the visualization and processing of sounds; it runs on the Mac-
intosh (either 68k or Power PC). Sounds can be visualized in the form of either an amplitude/time envelope or a
sonogram. The processing options currently available (time-expansion /compression, transposition, cross-syn-
thesis, etc…) are those of the program SVP, which is based on the principles of a two-channel phase vocoder.
SVP was originally developed on the Unix platforms Dec (Mips and Alpha), SGI and NeXT. To run the SVP
program in a Unix environment it is necessary to type a command line such as:
svp -v -t -T -A -Z -Scymbale -D2 -M1024 -N1024 cymbale-D2
The various options in the command line specify which treatment to apply to a sound and with which parameters.
In contrast, AudioSculpt was conceived from the beginning as a graphic interface to SVP in order to facilitate the
use of the program.
In place of a command line, the user is able to select a processing command from a menu, and fill in the param-
eters of the various options within a standard Macintosh dialog box. It is always possible, however, to revert to
the older method of controlling SVP by selecting Use Command Line from the menu. For each type of process-
ing presented in this manual, both forms (dialog box and command line) will be given.

8 - AudioSculpt - Cross Synthesis Handbook

2 Introduction

Cross-synthesis requires the use of two sound files. If the two files are of different lengths, the resulting sound
file will have the length of the longer file (before version 1.3 of SVP the shorter file’s length was used). Be careful
not to confuse the length of the sound file with the duration of the sound actually heard, which may be shorter
than the actual size of the file due to silence which could be either at the beginning or at the end of the file.
There are three types of cross-synthesis available within SVP:
• cross-synthesis by mixing (add mode)
• source-filter cross-synthesis (mul mode)
• generalized cross synthesis (cross mode)
Currently, AudioSculpt only gives access to source-filter and generalized cross-synthesis from within the Pro-
cessing menu. Cross-synthesis by mixing must be requested by using a command line.
Explained in the simplest and most general manner, cross-synthesis by mixing (add mode) is based on the mixing
of two sounds, source-filter cross-synthesis (mul mode) on the transformation of one sound by another, and gen-
eralized cross-synthesis (cross mode) on the exchange of characteristics of and interpolation between two sounds.
However, the user may go much deeper into the use of each of these three cross-synthesis modes in order to re-
alize original and creative cross-synthesis by calculating different types of hybrid sounds whose sources may no
longer be identifiable.
In order to realize a desired cross-synthesis, there are two basic considerations to take into account:
1. the choice of sounds to cross, and
2. the cross-synthesis mode, and the manner in which to use it.
In this tutorial, the user will find different examples geared to provide a guide for his or her personal work.
Note from the editor: because this handbook is really a tutorial which takes you gradually through all the steps
involved in doing cross synthesis, you are advised to read the sections in the order they appear. For instance, ex-
planation on how to adjust analysis parameters in introduced in the 3rd chapter devoted to Source-filter cross-
synthesis. We hope that reading the handbook in that manner will help you fully understand the processes in-

9 - AudioSculpt - Cross Synthesis Handbook

volved in achieving successful cross-synthesis. Do refer to the sound examples that come with this handbook1.
The recorded examples have been compiled as examples of effects made possible by cross-synthesis with
AudioSculpt.

1. The Sound examples are available on a DAT tape or on the Forum CD-ROM.

10 - AudioSculpt - Cross Synthesis Handbook

3 Source-Filter Cross-Synthesis

This type of cross-synthesis involves multiplying the short-term spectrum of one sound by the short-term spectral
envelope of another sound, which causes a filtering of one sound by the other’s spectral envelope. A sound’s
spectral envelope is calculated by using a linear prediction algorithm (LPC).
To begin processing, first open the sound files which you would like to cross by selecting Open from the File
menu (or by typing command-O). Once the sound files are open, you may select the type of cross-synthesis in the
Processing menu:

The following dialog box will appear:

11 - AudioSculpt - Cross Synthesis Handbook

It is in this dialog box that you must specify the file to be filtered and the filtering file. The sound to be filtered
is by default the sound file which is in the active window (the window whose borders are greyed). If you would
like to select another file to be filtered, you must close the dialog box (Cancel), click on the desired file’s window
(to make it active) and re-select the type of cross synthesis desired.
non-active window active window

In order to select the filtering sound, you may choose between any of the currently open sound files.

12 - AudioSculpt - Cross Synthesis Handbook

Click on Analysis Parameters to open the window which allows you to set the spectral analysis parameters: win-
dow size, hop size, analysis window type, number of poles determined by LPC, size of the FFT, and synthesis
window type.

Adjusting the Analysis Parameters

In theory, the analysis window size used on a sound should be equal to at least four times the period of the sound
(for periodic sounds), or at least four times the period of the lowest partial contained in the sound (for non-periodic
sounds). AudioSculpt automatically calculates the window size according to this rule when you type a frequency
value in the Fund. Frequency box (see below). As such, the frequency you give should correspond to either the
fundamental frequency of the sound (for periodic sounds) or the lowest frequency present in the sound (for non-
periodic sounds). Generally, since the sounds to be crossed will more than likely not have the same characteris-
tics, a different window size will be automatically choosen by the program for each sound file. The FFT size must
be the same for both sounds since the two spectra calculated by the Fourier transform must have the same number
of points. By default, AudioSculpt sets the FFT size to the first power of 2 greater than the larger of the two win-
dows. The hop sizes of the two analysis windows are equal to 1/8 of their respective window sizes by default.

13 - AudioSculpt - Cross Synthesis Handbook

Assuming that the two analysis windows are of different sizes, AudioSculpt will, by default, create two different
hop sizes. This implies that the short-term spectra of the two sounds will not be calculated at the same rate in each
of the two sound files. When the resulting sound is synthesized, AudioSculpt will calculate the sound according
to the temporal base of the spectra in the primary sound file, thereby changing the speed of advancement of the
secondary sound in relation to the primary.
For example, if the hop size of the primary sound is less than that of the secondary (analysis rate is faster in the
primary sound), the rhythm of spectral changes which the output sound inherits from the secondary sound will
seem faster than in the original. Conversely, if the hop size of the primary sound is greater than that of the sec-
ondary (analysis rate is slower in the primary sound), the rhythm of spectral changes inherited from the secondary
sound will seem slower than in the original. There is, therefore, a temporal compression or expansion inherent
in the second sound when a cross synthesis is calculated using two different hop sizes. One should be aware of
this phenomenon, and know that one simple way to avoid it is by using identical analysis window sizes. In general
it will suffice to increase, within reasonable limits, the size of the smaller analysis window to be equal to the size
of the larger, since the larger window gives a better frequential resolution. This phenomenon of temporal scale
distortion may often yield interesting results, in which case two different hop sizes may be kept.
Occasionally, the theoretical window sizes may not be the best to realize a cross-synthesis. In certain cases, by
reducing the size of the windows in respect to the theoretical minimum, the cross synthesis may gain an additional
sonic quality. Indeed, by reducing the frequential resolution of the spectra, one could also obtain a spectral mix
which might be particularly disturbing.
The button labeled Factory Settings will re-initialize all analysis parameters to their default values.

14 - AudioSculpt - Cross Synthesis Handbook

 filtered sound filtering sound

size samples

hertz

step samples

poles

samples

Once the desired values have been chosen and set within the Analysis Parameters dialog box, click on OK, and
then on Save in the Cross Synthesis Source Filter dialog box. You will then be presented with yet another dialog
box in which you can name the resulting sound file and select the folder in which it will be saved. You can click
on the Normalize checkbox to normalize the output sound file, i.e. scale the amplitude of the sound to its maxi-
mum.

15 - AudioSculpt - Cross Synthesis Handbook

 Folder in which
the sound file will
be saved

Normalization of the
Name of the resulting sound output sound’s
amplitude
The same cross-synthesis may also be requested by using a command line:

This is the command line equivalent toa source-filter cross-synthesis which is performed on the files caixixi and
vl-jeté:

16 - AudioSculpt - Cross Synthesis Handbook

If you need to brush up your SVP options, please refer to the list of options given at the end of this handbook, or,
better yet, see you AudioSculpt or your SVP manual.
CAUTION: you may change the order of any of the arguments to SVP except the last, which is always the name
of the resulting sound file. If you use a command line you should indicate the folder which contains the sound
files which you want to cross, and the folder(s) where you would like to keep the parameter files and comment
(console) files. Select Set Default Folder>Sound Folder from the File menu.

to open the following dialog box:

17 - AudioSculpt - Cross Synthesis Handbook

where you may search for the folder which contains the sound files.

Once you have indicated the folder where the input sound files are to be found, AudioSculpt will always look
there for them by default; you may, however, change the default directory at any time. To set the default folders
where AudioSculpt will search for parameter files and save console files, select Set Default Folder>Parameters
Folder and Set Default Folder>Console Folder (both in the File menu), respectively.

18 - AudioSculpt - Cross Synthesis Handbook

Example 1 index 1, 2 and 3
The graphic representation of sounds in AudioSculpt (amplitude/time envelope or sonogram) allow inspection of
different aspects of the sound such as those calculated in this example of source-filter cross-synthesis. The spec-
tral content of the instrument caixixi is quite close to white noise; its sonogram shows a very compact spectral
disposition, contrary to that of the violin where the progression of harmonic partials (horizontal lines) is very
clear.
The resulting crossed sound contains the spectral organization of the filtered sound, but with the dynamic profile
of the filtering sound. Otherwise stated, one can recognize the sequence of attacks of the violin with jeté bowing,
but with the sonority of the instrument caixixi. One could call this a caixixi played “jeté.”
Please note that with this type of cross-synthesis, there can be a definite loss of high frequencies in the resulting
sound. This is due to the spectral multiplication which reinforces the common amplitude zones, leading to an
attenuation of zones containing weak amplitude.

19 - AudioSculpt - Cross Synthesis Handbook

20 - AudioSculpt - Cross Synthesis Handbook
Example 2 . 1 index 4, 5 and 6
A harp arpeggio is filtered (in source-filter mode) by a trumpet using a wah-wah mute which opens and closes
rapidly. The resulting sound has a hybrid sonority in which the filtered sound predominates.
Here are the analysis parameters used:

and the equivalent command line:

21 - AudioSculpt - Cross Synthesis Handbook

Example 2 . 2 index 7
This is the inverse of the previous example; the trumpet is now filtered by the harp. All of the cross synthesis
parameters remain the same, but the order of the sound files and the name of the resulting sound have been
changed.

Example 3 . 1 index 8, 9
These two examples complement the preceding ones. Using identical cross synthesis parameters, the trumpet will
be crossed with a small sequence played on the berimbao. The resulting sound contains a counterpoint between
the timbral fluctuations and rhythmic action contained in the two sounds.
The command line to filter the berimbao by the trumpet with wah-wah mute is as follows:

Example 3 . 2 index 10
This is the command line to filter the trumpet with wah-wah mute by the berimbao:

22 - AudioSculpt - Cross Synthesis Handbook

At this point we will begin presenting sound examples constructed from several processing operations in order to
go deeper into the workings of AudioSculpt.

Example 4 . 1 index 11, 12 and 13

We will begin with a source-filter cross-synthesis between a clarinet with a sforzato attack followed by subito
piano, crescendo and diminuendo:

amplitude

time

and a cymbal with a strong attack followed by a diminuendo:

amplitude

time

The cymbal will be used to filter the clarinet:

23 - AudioSculpt - Cross Synthesis Handbook

Since the cymbal’s sound contains many partials, we will increase the definition of the spectral envelope of the
analysis by setting the order (i.e. the number of poles) of the linear prediction algorithm (LPC) to 50:

24 - AudioSculpt - Cross Synthesis Handbook

The resulting sound has the following time/amplitude envelope:

25 - AudioSculpt - Cross Synthesis Handbook

Remember that the filtering sound allows only those frequencies to pass which are contained in its spectrum. The
source sound passes through this filter.
Note that the two source sounds have amplitude envelopes which are rather different, and that the envelope of the
crossed sound itself has a profile quite unlike those of the sound sources. Its envelope more closely resembles that
of a plucked string, and in it one can distinguish four zones as marked in the figure.
• The amplitude of the attack is very strong because it is an important part of each of the source sounds,
containing many frequencies common to both.
• Just after the attack, the envelope falls sharply because, even though the “cymbal filter” remains open
with strong amplitude levels, the clarinet partials have rather weak amplitudes.
• The amplitude of the crossed sound then remains stable, because, even though the clarinet’s amplitude
increases, the “cymbal filter” begins to close.
• After 3.2 seconds, the clarinet is filtered by an empty signal. The spectra of the clarinet are multiplied by
empty spectral envelopes, since there is no sound in the cymbal file after this time. The resulting ampli-
tude is therefore nil.

Example 4 . 2 index 14

This is the inverse of the preceding example. Here, the cymbal is filtered by the clarinet; all of the cross-synthesis
parameter values remain the same.

The resulting amplitude envelope resembles that of the preceding example, but the predominant timbre is that of
the cymbal:

26 - AudioSculpt - Cross Synthesis Handbook

Many successive treatments may be applied to a sound in order to arrive at a particular type of sound desired. The
user is encouraged to try different types of treatments (time-stretching, transposition, etc…) using either fixed or
dynamic parameter values. Dynamic parameter values are those which change over time, and can be supplied to
AudioSculpt in the form of a text file.
The following section gives an example of applying a dynamic time-stretch to the sound we have just created.

Creating a Text File for Dynamic Sound Processing

By selecting New from the File menu (or by typing command-N), you can open an empty text-editing window to
create a dynamic parameter file:

The commands results in a new window:

27 - AudioSculpt - Cross Synthesis Handbook

In the figure below, we have created a simple file which contains points in time with corresponding coefficients
for temporal compression/expansion.
Caution: If the coefficient values are less than 1 there will be a temporal compression, if they are greater than 1
there will be a temporal expansion or dilation (time-stretching); a value of 1 will not change the duration of the
original sound.
Between any two points in time there will be a linear interpolation between the coefficients. In other words, each
coefficient will change continuously in value until it reaches the next coefficient value indicated.
When you save the file, you will be prompted to name it and should be sure to remember the folder where you
have saved it. In this example the parameter file has been named dilat1.
time temporal expansion (dilation) or compression coefficient

The resulting sound will begin with a time-compression which becomes stronger until 0.3 seconds, at which point
the compression slowly becomes weaker until 0.5 seconds where the compression coefficient is equal to 1.0 (no
compression or expansion). Between 0.5 and 1.0 seconds, the coefficient becomes greater, thereby time-stretch-
ing the sound, and continuing until 2.0 seconds, before returning to 1.0 over the remaining part of the sound. All
in all, the sound will be time-compressed in a variable manner from 0.0 to 0.5 seconds and time-stretched from
0.5 to 3.5 seconds.

28 - AudioSculpt - Cross Synthesis Handbook

Example 4 . 3 index 15
In order to use a text file to control dynamically the temporal expansion and compression of a sound, begin the
processing by selecting Time Scale Modification from the Processing menu with the window of the dynamic
parameter file active, or, if using a command line, indicate the name of the file just after the option -D.

The time scale modification gives the file a total duration of about 7.5 seconds. The attack stays very short, but
the resonance was stretched by more than twice in an irregular fashion.

Example 4 .4 index 16
The following example further modifies the previously created sound by applying a transposition to it. In order
to make a dynamic transposition, you should use a text file such as:

29 - AudioSculpt - Cross Synthesis Handbook

 time transposition values in cents

Transposition values are expressed in cents, for example:

• 100 cents = one half step higher
• -200 cents = a whole step lower (or two half steps lower)
• 0 cents = no transposition
The maximum transposition allowed is two octaves in either direction. (+2400 or -2400 cents).
Begin the processing by selecting Transposition from the Processing menu, with the dynamic transposition file
in the active window…

or, when using a command line, indicate the name of the transposition file after the option -trans:

30 - AudioSculpt - Cross Synthesis Handbook

 transposing option name of the transposition file

Transposition without time correction will modify the duration of the sound; if you would like to preserve the
sound’s original duration, click on the time correction checkbox in the Constant Transposition dialog box (or
add the option -D in the command line). If you wish to make a constant transposition, type the transposition in-
terval in cents directly into the space provided in the dialog box.
In our example, the transposition has given the sound a deep drone with little fluctuations in pitch — it is evident
that the sound has evolved into something far from the cymbal and clarinet of its origins.

31 - AudioSculpt - Cross Synthesis Handbook

4 Generalized cross-synthesis

This type of cross-synthesis allows the separate combination of the amplitude and instantaneous frequency spec-
tra of two input sounds. For the amplitude spectra of the primary and secondary sounds, A1 and A2, and their
instantaneous phase spectra, ƒ1 and ƒ2,(related to instantaneous frequency) on a given window, the spectrum of
the output sound for that window is defined as:
A = X A1 + x A2 + q A1 A2
ƒ = Y ƒ1 + y ƒ2
The Amplitude Scale Factors correspond to the coefficients X and x of SVP, the Phase Scale Factors to coeffi-
cients Y and y and the Amplitude Cofactor to coefficient q.
To begin processing, open the files that you would like to cross in AudioSculpt using the command by selecting
Open from the File menu (or command-O). As you now know, you have the option to use either fixed or dynamic
(time-varying) parameter values. To use fixed parameter values just choose the type of cross synthesis desired
from the Processing menu:

32 - AudioSculpt - Cross Synthesis Handbook

In the dialog box which opens, select the sound files and adjust the cross-synthesis parameter values. Then, click
on Analysis Parameters to enter the analysis parameter values, as in the case of the source-filter cross-synthesis.

Using dynamic parameters with a cross-synthesis is much like using them with time-stretching or transposition.
Begin by creating a text file with the crossing parameters. Next, make the window containing the text the active
window and select the type of cross-synthesis from the Processing menu. The following dialog box will open:

Select the sound files you wish to use, click on Analysis Parameters to modify the analysis parameter values, as
before, then begin the processing by clicking on Process and Save.

33 - AudioSculpt - Cross Synthesis Handbook

Example 5 index 17, 18 and 19
This is an example of spectral interpolation from one sound to another, a typical application of the generalized
cross-synthesis mode. To begin with we will choose two sounds that resemble each other — an English horn and
a cello — and proceed by constructing a transition from one to the other using a dynamic parameter file. The text
file used (crossfile-1) indicates the temporal changes of the crossing parameters X, x, Y, y, and q. Do not forget
that this file must be in the currently active window when the Generalized Cross Synthesis command is selected,
otherwise AudioSculpt will assume the use of fixed parameter values.
Here is the parameter file used for the crossing:

sound 1 sound 2

time X x Y y q

only the primary sound

the secondary sound begins to appear
transition
only the secondary sound

coefficients for the coefficients for co-multiplication of the amplitude spectra

amplitude spectra the instantaneous
frequency
spectra

The following is a graphic representation of the parameter file crossfile-1:

X Y
1.0

0.7 y

0.3 x
0.2
q
0.0
0.0 0.5 0.8 1.6 2.7 4.8 sec.

(The X and Y curves are superimposed)

34 - AudioSculpt - Cross Synthesis Handbook

The English horn is the only sound at the beginning of the output file since both the X and Y values are equal to
1, and the x and y values (for the cello) are set to 0. These values are retained until 0.5 seconds, so during the first
half second only the English horn will be heard. The transition to the cello sound begins after this point, when,
from 0.5 to 0.8 seconds, the amplitude of the secondary sound begins to increase1. By 1.6 seconds the transition
is already evident, and, from 2.7 seconds until the end (at 4.8 seconds) only the secondary (cello) sound is heard.
We will use the following analysis parameters:

This is the equivalent command line:

1. One should always remember to initiate such a cross at a very subtle rate, since our hearing
perception is very sensitive to changes made to a stable signal.

35 - AudioSculpt - Cross Synthesis Handbook

 cross synthesis mode name of the crossing parameter file

This timbral interpolation was greatly facilitated because the both English horn and the cello have a large quantity
of similar timbral characteristics and a similar acoustic ambiance at many pitch levels. As we can see, the choice
of sounds to interpolate is a very important decision in this kind of work. When two sounds are similar, an inter-
polation between them may be made solely by the interpolation of their timbres.
By using a parameter file for cross-synthesis you can obtain a very smooth and homogenous interpolation; any
potential imperfections in this gradual timbral transformation are masked by the other parameters of the sound.
However, in a cross between two sounds which have different acoustic qualities, each change in the sound, how-
ever slight, will bring these differences immediately to our attention.

Example 5.1 index 20, 21 and 22

The following example offers different types of interpolation between two very different sounds. Here, the sound
of a tam-tam is crossed with a succession of short vocal attacks.
The interpolation will take place in “stages”. To this end, we will use a dynamic crossing file which will change
the parameter values just before each vocal attack, and remain stable until the next attack.
time X x Y y q

stages

1 tam-tam
2

6 voice

36 - AudioSculpt - Cross Synthesis Handbook

With the window of the crossing file active, select Generalized Cross Synthesis from the Processing menu, to
open the dialog box where you can select the files to be crossed.

Click on Analysis Parameters to open the dialog box to adjust the analysis parameters.
We will use the following analysis parameters:

Click on OK, then Process and Save to begin the cross-synthesis.

The following equivalent command line may also be used:

37 - AudioSculpt - Cross Synthesis Handbook

Here is the amplitude/time envelope representation of the two source sounds followed by their resulting cross-
synthesis:

The interpolation has been made discreetly at the attack of each vocalization.
We will now create a series of examples using the same two sound files with the same analysis parameters as
above. Our variations will be made only to the dynamic parameter file controlling the cross-synthesis. In this
manner you can begin to better understand the many diverse possibilities inherent in this type of cross synthesis.

38 - AudioSculpt - Cross Synthesis Handbook

Example 5.2 index 23
In this file the “stages” progress by “jumps”.
time X x Y y q

In order to process this cross-synthesis by using a command line, it will suffice to change only the name of the
crossing file and the name of the output file.

The resulting sound:

The “jumps” give a different color to each vocalization.

Example 5.3 index 24

Here there are no more “stages”;

39 - AudioSculpt - Cross Synthesis Handbook

 time X x Y y q

The changes are continuous, but made in a zigzag fashion.

Example 5.4 index 25

The primary sound file is heard alone at the beginning (between 0 and 0.35 seconds), and the secondary sound
file is heard alone at the end (from 1.82 to 2.3 seconds). The transition takes place between 0.35 and 1.82 seconds.

time X x Y y q
1.0 X Y
x y
sound 1 0.7

sound 2

0.0 q
0.0 0.35 1.82 2.3 sec.

40 - AudioSculpt - Cross Synthesis Handbook

Example 5.5 index 26 and 27
In developing the same idea, the voice will now be crossed with “tongue-rams” on the flute; the analysis param-
eters are the same as before. Below are both the dynamic crossing file and its graphic representation.
teime X x Y y q

1.0 X
Y x
0.5

0.0 y q
0.0 0.36 0.8 1.1 1.46 1.9 2.3 sec.

As we can see, the crossing between the two sound files will not be made in a linear fashion, in order to make a
more convincing interpolation.

Here is the command line equivalent:

The file cross-voix-tongue-28 is the resulting interpolation between the vocal attacks and the “tongue-rams” on
the flute.

41 - AudioSculpt - Cross Synthesis Handbook

Example 5.6 index 28
Here is an example of cross-synthesis in sequence.
By using a digital sound editing program (such as Sound Designer by Digidesign, for instance), you can paste
example 5.4 after example 5.5 to obtain an interpolation in two steps.

tam-tam voice tongue-ram

One can further develop this idea by editing a larger series of cross-syntheses together. A return to the point of
departure may even be made by performing a cross-synthesis from the last to the first sound in the sequence.

42 - AudioSculpt - Cross Synthesis Handbook

Example 5.7 index 29, 30 and 31
Our last example in this series will not use a parameter file, since our crossing parameters will remain constant.
Our source sound files will be almost the same: the voice will contain only four attacks, and the tam-tam’s res-
onance will be longer. Select Generalized Cross Synthesis from the Processing menu to open the dialog box
to select the sound files and enter the parameter values for the cross-synthesis.
Constant values of cross-synthesis parameters

We will use the following analysis parameters:

43 - AudioSculpt - Cross Synthesis Handbook

The equivalent command line is as follows:
Constant values of cross-synthesis parameters

The values entered in the Analysis Parameters boxes will create a distortion in the time scale of the secondary
sound tam-tam2, due to the use of different hop sizes for the two sounds’ analysis windows. The resulting sound
file will have a duration of 50 seconds, in spite of the fact that the two source sounds tam-tam2 and voix-Eh-4
have durations of 1.474 and 6.402 seconds, respectively. If the two hop sizes are set to equal values, the duration
of the resulting sound file will be equal to that of the larger input file (tam-tam2), whose duration is 6.402 sec-
onds. Although AudioSculpt will synthesize sound in the output sound file as long as it finds a frequency spec-
trum in one of the input sounds, it will, nonetheless, continue the cross-synthesis using silence if it does not.

44 - AudioSculpt - Cross Synthesis Handbook

In this example the two hop sizes are respectively equal to 500 and 64. As such, the spectra of the sound voix-
Eh-4 are spaced much farther apart than those of tam-tam2. The ratio between the two rates is therefore 500/64,
or 7.81. In performing the cross-synthesis, AudioSculpt will calculate the sound Cross-voix-tam2 using the tem-
poral scale of the primary sound file voix-Eh-4, which will thereby distort the time scale of the file tam-tam2 by
a factor of 7.81. Because tam-tam2 contains more spectra than does voix-Eh-4, the cross will continue until the
end of the file tam-tam2. The resulting duration can be calculated by multiplying the size of the sound file tam-
tam2 (6.402 seconds) by this dilation factor, which gives us a total of 50 seconds.
The next two examples show how large modifications can be made to a resulting cross-synthesis by slightly vary-
ing the parameters used for the cross. A very low bass clarinet sound will be crossed with a cymbal. The analysis
parameters for both examples 7.1 and 7.2 are identical, but different dynamic crossing files will be used.

Example 6.1 index 32, 33 and 34

The following parameter file will be used to control the cross:

time X x Y y q

This file controls a very simple cross. The output file begins with the primary sound alone, followed by a rapid
transition to the secondary sound beginning at 0.5 seconds, and, from 1.6 seconds, the secondary sound is heard
alone. With the crossing file’s window active, select Generalized Cross Synthesis from the Processing menu
and select the sound files in the dialog box.

Here are the analysis parameters we will use…

45 - AudioSculpt - Cross Synthesis Handbook

…and the equivalent command line for the whole cross-synthesis:

The resulting sound has the following amplitude envelope:

46 - AudioSculpt - Cross Synthesis Handbook

Example 6.2 index 35

The modifications made to the dynamic crossing file are few, but they nonetheless produce noticeable differences.
Here is the modified crossing file and its graphic representation:
time X x Y y q

The command line equivalent is as follows:

This is the amplitude/time envelope of the resulting sound:

47 - AudioSculpt - Cross Synthesis Handbook

The resulting sound is a hybrid which moves and gets transformed over time. In particular, you can follow the
glissando produced by the parameter y from the beginning of the sound to 0.8 seconds. Around 2.5 seconds the
resonance of the cymbal becomes more apparent.

Example 7 index 36, 37 and 38

Here, a short double-attack on the tabla is crossed with a tam-tam with long resonance. As the attack is generally
the most characteristic part of an instrumental sound, its modification will render the sound difficult to recognize.
During the attack, a sound is unstable since the spectrum is made up of transients which stabilize only after the
attack. This example modifies the transients in the tam-tam’s attack. This type of cross may be affected on any
combination of instruments and can be applied to the idea of micro orchestration of complex sounds.
The parameter file used for the cross is carefully calculated to modify the resonance of the tam-tam.
time X x Y y q

X
1.0

Y x
y

q
0.0
0.0 2.2 0.7 1.0 1.5 2.5 2.6 sec.

As before, two sound files are selected,

48 - AudioSculpt - Cross Synthesis Handbook

and the following analysis parameters are used:

The equivalent command line:

49 - AudioSculpt - Cross Synthesis Handbook

and the resulting sound:

The following example describes one possible method for a better realization of a cross-synthesis.

Example 8.1 index 39, 40 and 41

A violin attack is crossed with a cymbal.

50 - AudioSculpt - Cross Synthesis Handbook

We will first try a cross using the following parameter file:

time X x Y y q

violin
attack
transition
resonance
of the
cymbal

The following files will be selected in the same manner as before…

… using the following analysis parameters:

51 - AudioSculpt - Cross Synthesis Handbook

The equivalent command line:

and the resulting sound:

52 - AudioSculpt - Cross Synthesis Handbook

 Maximum amplitude Maximum amplitude
peak of the violon peak of the cymbal

Example 8.2 index 42

The maximum amplitudes of the two instruments are too distant from each other in the resultant sound, provoking
a discontinuity (see above similar figure in example 9.1). In order to better realize this cross, it is necessary to
bring these peaks closer together by modifying the point in time where one finds the cymbal’s peak in the enve-
lope of the resultant sound.
The third step changes from .8 to .4 seconds

time X x Y y q

The following command line equivalent may also be used:

The resulting sound:

53 - AudioSculpt - Cross Synthesis Handbook

The continuity between the two sounds is evident in the resulting sound both upon listening to it and in its graphic
representation (in the form of an amplitude envelope) as well.

Example 8.3 index 43

Here, two lines have been added to the crossing file (1) to further improve the continuity between the two sounds.
The violin is held at maximum until 1.5 seconds (2) while the amplitude of the cymbal continues to increase (3).
To make the cymbal’s resonance less evident,envelope a slight oscillation is added to the parameters which con-
trol the sounds’ phases (4).
time X x Y y q

2 3 4

The command line equivalent:

54 - AudioSculpt - Cross Synthesis Handbook

Here is the resulting sound represented by both an amplitude/time envelope and a sonogram, with a graphic rep-
resentation of the crossing parameter file crossfile-52, below.

1.0
X
Y x
0.7

0.5
y

q
0.0
0.0 0.2 0.4 1.0 1.5 2.5 3.2 sec.

55 - AudioSculpt - Cross Synthesis Handbook

Example 9.1 index 44, 45 and 46
In this example a violin is crossed with a saxophone. The crossing parameter file used produces a rather strange
result; to better understand the reason behind this, one may compare this with examples 9.2, 9.3 and 9.4. The
following dynamic parameter file is used:
time X x Y y q

The files are selected in the dialog box, as shown,

using the following analysis parameters:

56 - AudioSculpt - Cross Synthesis Handbook

The command line equivalent:

The resulting sound:

57 - AudioSculpt - Cross Synthesis Handbook

 1 .0

0 .0

By carefully observing the sonogram, one notices that the glissando heard in the sound does not correspond to its
spectral representation — in fact, it seems to move in contrary motion to it. This is a typical example of a para-
doxical sound.

58 - AudioSculpt - Cross Synthesis Handbook

Example 9.2 index 47
Here the coefficients relating to the primary sound have been kept, but those relating to the secondary sound are
constant and equal to 1 for x and y and 0 for q.

time X x Y y q

x y
1.0
X Y

q
0.0
0.0 0.88 2.64 4.1sec.

This is the equivalent command line…

… and the resulting sound:

59 - AudioSculpt - Cross Synthesis Handbook

Example 9.3 index 48
Here, the amplitude coefficients (X and x) for the two sounds have been preserved (with respect to Example 10.1),
but coefficients Y and y have a constant value of 1.
time X x Y y q

1.0
Y y
X

x
q
0.0

The resulting sound:

Example 9.4 index 49

For the last example in this series, the amplitude coefficients (X and x) are conserved as before, but the three re-
maining coefficients Y, y and q are constant and equal to 0.

60 - AudioSculpt - Cross Synthesis Handbook

 time X x Y y q

1 .0
X

Y y q
0 .0

This combination creates a very metallic sounding output.

Example 10 index 50, 51 and 52

A cymbal is crossed with a flute using a rather complex parameter file:

61 - AudioSculpt - Cross Synthesis Handbook

 time X Y x y q

Here is the selection of files in the generalized cross-synthesis dialog box:

The analysis parameters:

62 - AudioSculpt - Cross Synthesis Handbook

The command line equivalent:

The resulting sound:53

63 - AudioSculpt - Cross Synthesis Handbook

 1.0
X

y
q
x
0.0

The parameters Y and y, as they are defined in the file, impose rather large variations in pitch. The most evident
to the ear is probably the glissando produced by parameter Y at the beginning of the sound.

64 - AudioSculpt - Cross Synthesis Handbook

5 Cross-Synthesis by Mixing

This type of cross-synthesis is analogous to the mixing of two sounds. The coefficients X and x play the role of
faders and allow control of the weight of each sound in the mix. This type of cross-synthesis is currently only
available by using a command line, using the option -Gadd.
To begin processing, open the command line dialog box by selecting Use Command Line from the Processing
menu.
If you are using constant parameter values, you must type these values just after the options -X and -x (respec-
tively, the two options are relative to the primary and secondary input sounds).

Constant
amplitude
Cross synthesis values
by mixing sound1 sound2

Example 11 index 53, 54 and 55

If you use dynamic crossing parameters (those which vary over time), you should create and save a text file con-
taining these parameter values.

65 - AudioSculpt - Cross Synthesis Handbook

 Amplitude coefficients
sound 1 sound 2

time X x

The name of this file must be given just after the option -Gadd in the command line.

name of the crossing file

Do not forget to correctly specify the folder where AudioSculpt will look for both the parameter files and sound
files you will be using (by selecting Set Default Folders from the File menu) before initiating the processing.
Here, our resulting sound has the attack of a cello with the resonance of a drum. The crossing file creates a very
rapid transition.

Graphical representation of file addfile-1

1.0
X

0.3
0.2 x
0.0
0.0 0.5 1.0 3.0 sec.

66 - AudioSculpt - Cross Synthesis Handbook

Example 12 index 56, 57 and 58
In the last example of this series, a timpano is crossed with a contrabass. The parameter file used for the cross
creates a transition which is much slower than in the previous example.
The following is the dynamic parameter file used for the cross:

time X x
1.0
X

x
0.0
0.0 0.3 1.0 2.0 3.0 3.8 sec.

This is the command line which will be used:

Cross-synthesis by mixing (add mode) is a useful tool not only for the mixing of two sounds, but, more impor-
tantly, for the creation of micro-mixes. The two preceding examples illustrate the precise control of amplitude
one has over the mixing of two sounds when using this type of cross-synthesis.

67 - AudioSculpt - Cross Synthesis Handbook

6 Appendix

This appendix includes those examples which “cross the boundaries” of the diverse types of cross-syntheses so
far explained in this tutorial. These examples are presented to provide a springboard for ideas for those who are
searching for new fields of audio processing in which to explore.
The examples were realized with the help of the program PatchWork1 (also developed at Ircam), which served to
create complex dynamic parameter files for unconventional control of cross-synthesis, transposition, and time ex-
pansion/compression.

1. In PatchWork, the selected time resolution (shown in the time column of the parameter file)
is not necessarily in accordance with the step size used in AudioSculpt. When it is smaller,
AudioSculpt reproduces only the variations which match the step size. the finer ones are skip-
ped.

68 - AudioSculpt - Cross Synthesis Handbook

Example 13. 1 index 59, 60 and 61
This example uses cross-synthesis by mixing (add mode) in which the parameter file contains variables for X and
x whose sum is always equal to 1. This is equivalent to using two faders to control the output level of the two
sound files. The amplitude values in the parameter file oscillate very rapidly in an irregular fashion, which gives
the global impression of a progressive transition between a trumpet and a flute accompanied by a rapid granula-
tion effect.
Here is the crossing parameter file:
time X x

To create this long file, a patch was constructed in PatchWork, using a stochastic algorithm to generate the oscil-
lations while controlling the percentage of predominance of one sound or the other, as well as the interval of os-
cillations.

69 - AudioSculpt - Cross Synthesis Handbook

Here is the graphic representation of the file addgrains-4:

Sound 1

Sound 2
0.0 0.37 2.25 2.6 sec.

The command line used:

The amplitude/time envelope of the resulting sound:

The oscillation between the two sounds produces different types of “grains” during the transition. This type of
granulation has a masking effect during the transition to the flute sound. The idea to introduce elements which
capture the auditory attention just at the moment of transition is just one of the many possible means leading to
inexhaustible creative possibilities.

70 - AudioSculpt - Cross Synthesis Handbook

Example 13.2 index 62, 63 and 64
With the same type of cross synthesis, and using a crossing parameter file similar to that in the previous example,
a saxophone melody is crossed with the sound of a caixixi. The granulation produced by this parameter file soft-
ens the rugged character of the caixixi.
The parameter file addgrains-6 resembles that of the preceding example, and will not be displayed here due to
its length. Here is the command line:

Example 14.1 index 65 and 66

This example uses the option -trans to generate a vibrato for the sound of the cello. Again, the dynamic parameter
file which controls the transposition was created within PatchWork by using a patch to determine the frequency,
amplitude envelope, waveform and delay leading to any kind of vibrato.

Original pitch Maximum depth + - 7 cents

+7 cents. . . Delay
. . . . . . . . . . . . . . . . . . . . . . . . . . . .
. .
End of the vibrato
. .
. .
. .
. .
. .
- 7 cents. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ..
0.0 0.8 2.56 4.33 7.0 sec.

The dynamic transposition file used to produce the vibrato:

71 - AudioSculpt - Cross Synthesis Handbook

72 - AudioSculpt - Cross Synthesis Handbook
time cents

73 - AudioSculpt - Cross Synthesis Handbook

The command line used:

Example 14.2 index 67 and 68

One can further extend the idea of using the option -trans to impose a vibrato upon a sound by generating a fre-
quency modulation to be imposed upon a sound file. It is sufficient to create a vibrato faster than 20 Hz for the
first lateral frequency bands to appear.

74 - AudioSculpt - Cross Synthesis Handbook

 time cents

In this example, the sound of a flute is colored with frequency modulation induced from a dynamic transposition
file.
The beginning of the file fmfile-9 is shown in the figure. This file defines a vibrato whose frequency is 200 Hz,
and whose amplitude varies between, +100 and -100 Hz.
To achieve good spectral and temporal definition, the analysis is made with a large FFT (flag -N4096) and a very
small hop size (flag -I20).
The following command line is used:

75 - AudioSculpt - Cross Synthesis Handbook

 1 2 3

This example of frequency modulation imposed on a sound file is given, once again, to provide a point of depar-
ture for the advanced AudioSculpt user. Once this principle is understood, it can be applied to other types of mod-
ulation, such as amplitude modulation and wave shaping.

Example 14.3 index 69

The same transposition can be used to simulate a flutter-tongue by using a large window size and the default hop
size value.
The following command line is used:

The transformation from flutter-tongue to ordinario is obtained by using a version of the above transposition file
whose vibrato begins at the maximum amplitude and gradually decreases to nothing.

76 - AudioSculpt - Cross Synthesis Handbook

7 Conclusion

The use of text files containing parameter values allowing the realization of sound processing which var-
ies over time opens inexhaustible possibilities when one takes into consideration that these files may be
created in a program other than AudioSculpt itself (for example, PatchWork).
At the beginning of this work, I constructed different systems to create text files based on parameter en-
velopes, as well as for the graphic representation of existing files. I named these systems “editors”, and,
with their help, I extended the possibilities of cross-synthesis within AudioSculpt. Some of these exper-
iments, described in the appendix of the AudioSculpt manual show that, with a little imagination, one
can discover new creative applications of the program.
In the appendix to this tutorial, the reader will also find several examples of unconventional sound pro-
cessing which, I hope, can be musically very powerful.

Conclusion 77
8 Annex: List of options

Here is a summary of available SVP options:

-v display errors in the console window.

-t display the analysis window’s average position in the console window.
-A analysis of the primary input sound.
-a analysis of the secondary input sound.
-a analysis by linear prediction (LPC). Default order = 30 poles.
-Z perform a re-synthesis.
-G cross synthesis between two files followed by one of the three modes: add,
mul or cross.
-S name of the primary sound file.
-s name of the secondary sound file.
-M size of the analysis window (in samples) for the primary sound file.
-m size of the analysis window (in samples) for the secondary sound file.
-N size of the FFT (in samples) for the primary sound file (the option -n for the
secondary sound file is not used since the FFT size must be the same for both
sound files.)
-I hop size (in samples) for the primary sound file.
-i hop size (in samples) for the secondary sound file.
-J type of synthesis window.
-W type of analysis window for the primary sound file.
-w type of analysis window for the secondary sound file.

For a detailed explanation of SVP options, refer to your AudioSculpt or your SVP manual.

Annex: List of options 78

Index

Generalized cross cynthesis 45

A Generalized cross synthesis 43
Amplitude Cofactor 32 Generalized cross-synthesis 9
Amplitude Scale Factors 32
Analysis Parameters 13
Analysis window 13, 78 H
Apple 2 Hop size 13, 14, 44, 78

B I
Battier M. 2 Interpolation 41, 42

C L
Carrive J. 2 Linear prediction algorithm 11
Cents 30 Linear prediction coding 78
Command line 8 LPC 11, 13, 24, 78
Console Folder 18 number of poles 13, 24
Constant Transposition 31
Cross-synthesis
add mode 9, 67, 69, 78 M
by mixing 9, 67, 69 Mary M. 2
cross mode 9, 78
generalized 9, 32, 34, 37, 43, 45, 62
mul mode 9, 78 N
source-filter 9, 11
NeXT 8
Normalization 15
D Normalize 15
Dec 8
Alpha 8 O
Mips 8
Depalle Ph. 2 Options 78
list 78
Dialog box 8
Digidesign 42
Doval B. 2
Dudas R. 2
P
Dynamic transposition 29 Parameters Folder 18
PatchWork 68, 69, 71, 77
Pauset B. 2
E Phase Scale Factors 32
Poirot G. 2
Eckel G. 2
Poles 13, 24, 78

F R
Factory Settings 14
FFT 13, 75 Rodet X. 2
size 13, 78 Rogers C. 2
Fund. Frequency 13
S
G Serra M.-H. 2
Set Default Folder 17, 18
Garcia G. 2

79 - AudioSculpt - Cross Synthesis Handbook

Short-term spectral envelope 11
Short-term spectrum 11
Sonogram 19
Sound Designer 42
Sound Folder 17
Source-filter synthesis 9
Spectral envelope 11, 24
SVP 8, 9, 17
Synthesis window 13, 78

T
Terhardt E. 2
time correction 31
Time scale modification 29
Transposition 30

U
Unix 8
Use Command Line 8

W
Window size 13

80 - AudioSculpt - Cross Synthesis Handbook