«Harmonics» project
Alex, Bo, Fred, Malik
2024, july 8th
Abstract
Tonnetz is
software developed by Frédéric Faure, Malik Mezzadri,
Alexandre Ratchov and Bo VanderWerf. Tonnetz is aimed to
play music with just intonation, with
MIDI instruments and also with
acoustic instruments, via microphone. It also controls a
slider-flute. It contains an
automate that can play, according to a
Markov graph.
This document is a tutorial. It explains different aspects of the program tonnetz, from installation in
1, to the abstract concepts and practical use. There is an other document called “reference” that gives all mathematical details and software code explanations. The objective of this program “Tonnetz” are multiple: from research in musicology, pedagogical use, to compose music, play and improvise on stage.
Documents and references:
1
Installation of plugin Tonnetz and softwares on the computer Mac-Os
- If not already done, download Synth that contains the sounds for the synthetizer 'midisyn' of Alex.
- Download the latest version of the plugin Tonnetz.
- If MacOs refuse to install, go to directory Downloads, use the key Ctrl and left clic on the file and Open it in the menu
2 How to use Tonnetz
2.1 Programme Tonnetz (StandAlone)
La version StandAlone signifie que ce programme Tonnetz est indépendant sur l'ordinateur.
2.1.1 Lancement du programme
- Start:
- In Applications/Fred/Tonnetz
- Settings:
- in the upper left corner: Options/Audio_midi_settings/Audio_buffer_size: choose 128 samples and in Active_MIDI_Inputs: connect your device (EWI or Keyboard).
2.1.2 Comment enregistrer des sessions audio et vidéo
L'idée expliquée ci-dessous est de connecter:
- TonnetzBlackHole_2ch QuickTimePlayer ou Audacity pour enregistrer un fichier audio (et video)
- et TonnetzBlackHole_2ch casque ou haut_parleurs pour entendre en même temps pendant l'enregistrement.
Installations:
Configurations:
- Dans le programme Tonnetz, menu/Options/Audio_midi_settings/Output_Audio: choisir Blackhole_2ch.
- Dans le logiciel LadioCast, mettre sur une ligne: entrée: BlackHole_2ch, sortie: carte son ou casque ou haut-parleurs. Cela permettra d'entendre le son produit sans latence.
- Enregistrement audio:
- Enregistrement audio avec QuickTimePlayer: lancer QuickTimePlayer, menu/Fichier/Nouvel_enregistrement_audio et avant de lancer l'enregistrement, près du bouton rouge, choisir en entrée: blackhole 2ch.
- Enregistrement audio avec Audacity: menu/preferences/Reglage_Audio/Enregistrement: choisir blackHole_2ch.
- Enregistrement audio + video:
- avec quicktime player: touches: Cmd + shift + 5 ou menu/Fichier/Nouvel_enregistrement_de_l'ecran et avant de lancer, dans option microphone, choisir blackhole_2ch.
- (Optionnellement exporter le fichier video .mov en H.264 .mp4 pour ensuite le lire sur des pages web html.)
3 Versions and news
21/01/2024, version 1.1.1
Version simplifiée du plugin Tonnetz qui permet de:
- Jouer avec l'ewi ou clavier. Choisir le son dans 'audio out'. Choisir le tempérament.
- Avec l'ewi, passer/sortir du mode 'hold' en mordant un peu et effacer les notes en mordant fort. Il y a une jauge 'crunch' qui montre le seuil. Ainsi, en mode hold, on peut poser des accords.
- Si on rejoue une note existante, cela l'éteind.
- On peut jouer sur differents canaux de l'EWI, par exemple
- canal 1: en mode hold pour construire des accords,
- canal 2: en mode hold off, pour jouer un solo.
- On peut modifier les accords avec la souris, en cliquant sur les notes (on/off) et modifier la force des notes avec la molette.
- Save/Load des accords et faire evoluer des accords en choisissant différents paramètres.
- Jouer en temperament adaptatif ou fixe et décaler les tempéraments fixes selon les directions .
19/02/2024, version 1.1.9
- Il y a un seul package appelé Tonnetz qui contient:
- Tonnetz_Synth qui est un synthétizer for LogicPro or GarageBand, etc. This is an AU Plugin.
The suffix Synth means “synthetizer” and that on output it gives some sounds from an internal synthetizer (by Alex or Fred).
- Tonnetz_MidiFX est un plugin for LogicPro, this is a AU Plugin.
The suffix MidiFX means “midi effect” and that on output it gives midi events. Beware that it is not accepted by GarageBand. On LogicPro, to have sound, you need to put after this plugin, any external synthetizer which accepts pitch-bend and calibrate the pitch bend to the range (e.g. Diva).
- Tonnetz_Analyzer is for LogicPro or GarageBand, etc. This is an AU Plugin.
The suffix Analyzer means “audio effect” and that on input and output it takes some audio. on peut jouer avec un instrument acoustique et en particulier la flute à coulisse.
19/02/2024, version 1.1.10
- Dans la section, “1 Temperaments/ 3 Display Settings”, un mot de passe est requis pour accéder à l'onglet “8 For developpers”.
- Représentation en portée musicale “Staff” dans “1 Temperaments/ 3 Display Settings”.
- On peut éditer un tempérament fixe dans la section “1 Temperaments/5 Edit”
- Il faut au préalable sélectionner un tempérament fixe.
- On modifie les notes, soit à la souris sur la fenetre tunplot ou la fenetre netplot, soit avec les widgets.
- Penser à utiliser le bouton droit de la souris pour basculer en mode freeze.
- On peut sauvegarder et rejouer des accords dans la section “1 Temperaments/2 Chords/ 1 Save/Load” et faire évoluer les accords dans la section “1 Temperaments/2 Chords/ 2 Change Chord”
- Si on utilise un synthe extérieur avec le plugin Tonnetz_MidiFx, on peut tester si le synthe (par Diva) est bien configuré pour les messages de microtonalité, avec le bouton de la section “2 Midi / Midi out / External Synth/ Test Pitch bend”.
20/04/2024, version 1.1.12
- Une version StandAlone du programme que l'on peut utiliser directement dans Applications/Fred/Tonnetz
23/05/2024, version 1.1.13
- On peut programmer et faire jouer un automate dans la section “4 Markov Automate”. Il composera en direct selon des suites d'instructions situées sur les sommets d'un graphe de Markov.
21/07/2024, version 1.1.16
- Il y a une interface graphique (comme des pads) pour jouer la suite d'accords: dans “1_Tempéraments/2_Chords/Show_sequence”.
30/07/2024, version 1.1.17
- Pour chaque canal midi entrant (ch_in sélectionné en haut à gauche) on peut associer un instrument et des effets spécifiques (voir “2 Audio/Audio_out”).
4 Sessions passées
5 Objectives of the project
5.1 First ojectives
5.1.1 For Alex & Fred
- Developp the plugin «tonnetz», midi->midi:
- that works as a AU plugin on MacOs, in LogicPro or Ardour or Reaper and explain how to use it.
- We could use the plugin tonnetz with an EWI (or MIDI keyboard) to record some multiple Midi tracks with chords and get sound from any synthetizer.
- Connect «tonnetz» with Alex's plugin synthetizer «midisyn» (midi->audio) to get the special sounds and effects of Alex.
- Connect «tonnetz» with a plugin «pitch-detection» (audio->midi) so that Malik can use its «flute à coulisse» or flute or voice or «flute à nez»,
- Developp the plugin «midisyn» (synthetizer midi->audio).
- Developp the plugin «pitch-detection» (audio->midi).
- Developp the plugin «markov» (midi->midi) that interacts with «tonnetz» and allow to generates some midi patterns followings rules from Markov graphs.
- In order to simplify the use for musicians, propose a unified plugin (audio,midi-> audio, midi) that contains all the previous plugins.
5.1.2 For Bo & Malik
- Use the plugins to compose and record music.
5.1.3 Together (Alex, Bo, Fred, Malik)
- Also with Paul Albenge and Aline, we meet together, in order to prepare a concert and some support: audio record documentary, ...
5.2 Some conferences?
Une conférence sur les tempéraments anciens et actuels?
5.3 Objectives
- Un disque
- Documentaire de 15' + 30' de morceaux
6 Pedagogical tutorial: «how to start with the plugin tonnetz»
Before starting to use the plugin tonnetz let us give a short introduction that gives an idea of “what tonnetz is” and a few important “key words” for after.
6.1 Summary of the tonnetz plugin with few key-words
6.1.1 MIDI,Audio, DAW and plugins
- Audio signal means a sound that has been transformed to a sequence of numbers (i.e. digitized).
- the time is cut in small intervals , e.g. Hz intervals in a second called the rate frequency.
- The amplitude is also approximated by integers in the interval with integer called the bit sampling, ex: in the picture below but usually we take for good precision.
- MIDI signal is a sequence of numbers that represent some instructions between musical devices (keyboards, computer, etc) in electronic music.
- The list of all instructions is defined by the MIDI association.
- For example the note with maximal volume is represented by the sequence of numbers (here means 'note on', means 'C' and is the maximal value).
- A DAW means Digital Audio Workstation and is a software in a computer that acts as a “recorder”. It allows to record some audio signals and MIDI signals on tracks and play these tracks on output speakers.
- We will mainly use Ardour, that is a efficient and free software. However most of professionals use Protools on windows PC or Logic-Pro on MacOs (these are proprietary software).
- An audio plugin is a computer software that can be used in a DAW for transforming Audio signals (and/or) MIDI signals. In our case, “tonnetz” is a plugin that takes audio (stereo) and MIDI on input and output:
6.1.2 The tonnetz
- By definition, the tonnetz is the space of all notes with rational frequency ratios, i.e. fraction, called just intervals in music, starting from an arbitrary original note which is the "diapason" (A at 440 Hz). In principle, this space of "just notes" is accessible to all musicians simultaneously. But this note space is infinite - there are even infinitely many notes between each note, just like the rational numbers Q. See this short video on numbers. Because of this infinity, in practice, we have to use a temperament to access some of these "just notes".
- For a musician, a temperament is a limited number of tonnetz notes to which he has access, via the keys of his instrument (C,C#,C etc). In other words, a temperament is a choice of just notes. There are two types of temperament:
- fixed temperament, where the choice is fixed during the play. You can use a predefined fixed temperament, such as the (historical) fixed temperaments of Zarlino, Pythagoras, Harmonic, etc. (and their transpositions), or use a personal customized fixed temperament.
- an adaptive temperament, where this choice depends on the leader notes present (played by all leader musicians). This guaranties a better coherence in the harmony.
- A musician is in leader mode if the notes he plays and holds influence the adaptive temperaments of the other musicians and himself. Otherwise, the musician is said to be in follower mode.
7 Theoretical description
7.1 Harmonics and just intervals
7.1.1 Human voice and importance of harmonics and just intervals
Human voice is created by the periodic vibration of
vocal cords. From
Fourier analysis, there are harmonics in every periodic signal like the human voice, a sound of a string instrument, a flute or a trumpet, etc.
Harmonics are therefore present in the human voice and other musical sounds.
Just intervals are the intervals between different harmonics. Our brain is well trained to analyze and recognize human voice via identification of these just intervals, so we can understand that just intervals play an important role in music. This is what we will be concerned with this program.
Auditive illusion:
This
video (read the given description) illustrates how the brain creates virtual fundamentals from just intervals, as it would like to give the information to consciousness that a human voice is present. See
wikipedia about Missing fundamental.
7.1.2
What are the harmonics
The harmonics of the fundamental note for example (subscript here means octave ), is the following series of notes (and correction in half-tone)
where the index is the harmonic number and the subscript is the correction with respect to the equal temperament. For example, harmonic 3 is with a correction of halftone. Harmonic is with a correction of halftone, that is very perceptible. What is important to observe and enough to remember for after is:
- Harmonic is called the fundamental.
- Harmonic is an octave from the fundamental.
- Harmonic is a fifth (plus an octave) from the fundamental with of halftone (not perceptible).
- Harmonic is a major third (plus two octaves) from the fundamental with of halftone.
- Harmonic is a minor seventh (plus two octaves) from the fundamental with of halftone.
- Harmonic is a augmented fourth (plus three octaves) from the fundamental with of halftone.
- Harmonic is a major sixth (plus three octaves) from the fundamental with of halftone.
Numbers
are
prime numbers and are enough to construct every other numbers, e.g. harmonic
is one octave above harmonic
, hence
octaves and a fifth above the fundamental. Harmonic
is a fifth and an octave above harmonic
, hence two octaves and two fifth above the fundamental, etc.
Frequency of the harmonics:
if
is the frequency of the fundamental, then the frequency of harmonic
is
, and frequency of harmonic
is
, etc. These frequencies are slightly different from the frequency obtained in equal temperament (more on this below). More informations about harmonics are in the appendix
sec:Harmonics.
7.1.3 What are just intervals
Between the first harmonics, there are the following intervals:
- a fourth between harmonics ,
- a minor third () between harmonics ,
- another different minor third () between harmonics ,
- a tone () between harmonics ,
- another different tone () between harmonics ,
- a triton () between harmonics ,
- another different triton () between harmonics , etc
These intervals between the harmonics of a given fundamental are called just intervals and will have important roles after.
Ratio of frequencies of just intervals:
In the example above, if is the frequency of the fundamental, then the frequency of harmonic is , and frequency of harmonic is . Consequently we get that the ratio of frequencies of the major third just interval isSimilarly the ratio of frequencies of the fifth just interval isMore generally, there is a just interval between two frequencies if the ratio iswith two small integers (say ).
Dissonance of a just interval:
We define the
dissonance of the just interval
as
for an
irreducible fraction , and with “
” being the
natural logarithm function. (There will be a simpler expression of
later).
7.1.4 Just intonation
Just intonation in musics means that the we play music only with just intervals.
We will use two different and complementary representations for the notes constructed with just intervals: the pitch space and the tonnetz space. We will explain after the concept of temperament that explains how we choose 12 (or less) notes per octave.
7.1.5 Equal temperament
In every culture of every time people played with just intonation, i.e. just intervals. The only exception comes from Europe in the mid of XIX-th century, where they introduce the
equal temperament, where the octave is divided in logarithmic scale in 12 equal intervals. The consequence is that in the equal temperament, only the octave remains a just interval. The other intervals are not. But the fifth and fourth are very close to just intervals, so that the difference is unperceptive.
Later, we will not consider the equal temperament (except in the pitch space representation, giving some dashed lines of references).
7.2 Tunplot: the pitch space
The pitch is very well known by musicians, it represents the position of a note on a piano keyboard. Precisely the
pitch of a note of frequency
is defined by
where
Hz is the frequency of the
tuning fork and
is its (MIDI) key number. In other terms
represents the position of a note on a MIDI keyboard, where integer values are exactly on the keys (ex:
is for
) but non integer values of
are also allowed, between the keys.
For example the pitch of
is
and the pitch of
is
, hence
is a pitch at “half-distance” between
and
, sometimes called
quarter-tone.
In the program, the representation of the pitch space is called: “tunplot”.
Example 7.1.
On the following example, three notes have been played with the keys
. They are represented by the green disks. Their horizontal position gives their pitch
, and the
dashed lines are is the equal temperament, i.e. for integer value of
. (In black are other accessible notes by the temperament, but not yet played).
In this example, we observe that is a little lower than the equal temperament line. The number on the left of each line is the octave range number.
7.3 Netplot: the tonnetz space
In German, ton = “note”, netz = “lattice”, so “tonnetz” is the lattice of notes.
Euler in 1750, proposed a
tonnetz to represent just intervals, that is a two-dimensional lattice. He used it to represent just intervals with fractions of the form
with integers
. This is not sufficient for our purpose, we will need to use all the
prime numbers in our fractions, so we define the tonnetz space as follows: a fraction is written uniquely as
with some integers
. We represent
as a point in a lattice. In practice we will represent the axis of
, the axis of
are written in small fonts. The value of
(the octave) is not represented, instead we represent the octave position of the note on a circle.
In the program, the representation of the tonnetz space is called: “netplot”.
Example 7.2.
On the following example (same as
7.1), three notes have been played with the keys
. They are represented by the green domains. (In grey are other accessible notes, but not yet played). The horizontal axis is for
value, the vertical is for
and the octave range value is the position on each circle. From this picture, up to a factor
, we can read that the ratio of frequencies are
and
. We also read that
are in the same octave range
and
is in the octave range
. We can deduce actually that
is a just fifth and
is a just major third. This was not visible from the pitch representation.
Summary 7.3.
We not need to be a mathematician to read this picture. We only need to observe that the notes are neighborhoods on the horizontal axis which is the axis of just fifths and that are neighborhoods on the vertical axis which is the axis of just major third.
Example 7.4.
Here is an example with the three axis . This is a chord with . The note is along the axis which is the axis of just minor seventh.
Example 7.5.
In this more complicated example, we have played the keys . On the pitch axis, we have added the fractions, where has the fraction , this is the reference note. For example has the fraction with respect to . On the tonnetz it is represented on the axis , that starts from on the upper-left. Similarly the axis is on the upper-right, the axis is on the right, the axis is above.
7.4 Formula for pitch values
We have seen of the tunplot figures above that the pitch of notes differs slightly from the equal temperament shown by the vertical dashed lines. Here we explain how to compute the pitch corrections from the position of the note on the tonnetz.
Here is a table and we explain after how to use it with an example.
Axis
|
interval
|
Number of half-tone
|
pitch correction
|
|
|
in equal temperament
|
with respect to equal temperament
|
|
fifth
|
|
|
|
major third
|
|
|
|
minor seventh
|
|
|
|
augmented fourth
|
|
|
|
minor sixth
|
|
|
|
half-tone
|
|
|
|
minor third
|
|
|
Consider the following example, called harmonic temperament later:
In this example the reference note is and his pitch is given by the equal temperament, i.e. . We see on netplot that is a major third from , so as can been observed on tunplot. Similarly we deduce the following table of pitchs for this “Harmonics scale”
Note
|
pitch (i.e. half-tones) w.r.t.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
What is important to remember is that:
- Intervals along the axis, i.e. fifths, do not differ from the equal temperament intervals by a perceptible quantity. And similarly for axis.
- Intervals along the axis of major third get a strong pitch correction (of half-tone) for each positive step.
- Intervals along the axis of minor seventh get a strong pitch correction (of half-tone) for each positive step.
7.5 Temperaments
There are an infinite number of rational numbers, so in practice if we use an instrument with 12 keys per octaves, we have to choose which frequencies to play. Such a choice of frequency for each key is called a temperament.
7.5.1 Fixed temperament
If this choice is fixed, we say that we play with a
fixed temperament. The program will allow to play with different fixed temperament, like the famous “
Pythagorean temperament”, the “
Zarlino temperament”, the “Harmonic temperament” etc.. but also on some fixed temperament build by the user. The advantage of fixed temperament is that some fixed “musical mode color” can be explored. The dis-advantage of fixed temperament is that not all the just intervals are available, and some intervals sometimes sound bad.
Zarlino temperament in C:
Here is the picture for available notes
We observe on the tonnetz space that it is made with just fifths and just major thirds. On the pitch space, we observe that the temperament is the same on each octave range. One advantage of the Zarlino temperament is that we have just major triads chords like , . But beware that some other triads are not the usual just like: or .
Harmonics temperament in C:
Here is the picture for available notes
We observe on the tonnetz space and pitch space that it is made from the first few harmonics of (up to harmonics 25).
7.5.2 Adaptative temperament
Adaptative temperament has been defined and proposed in the paper [
1].
Instead of using a fixed temperament, it is also possible to adapt the frequency of some key to the present situation, depending on the notes that are played currently (or recently) and depending on some (deterministic) rules. We say that we play with adaptative temperament. The advantage of adaptative temperament is that all the just intervals may be available. The disadvantage is that the frequency associated to a key may change, and this can be a difficulty to the player. To overcome this difficulty this is very useful for the player to look at the available notes on the tonnetz space and on the pitch space.
Definition of the adaptative temperament:
Suppose that there is already a set of notes, i.e. a chord that sounds. The adaptative temperament depends on this chord . We have to choose how many keys per octave we want to be able to use. This is . Then the octave range is divided in equal intervals and in each of these intervals, we choose the note that is the most consonant with the notes already present in the chord (among the infinity of rational numbers in this interval).
More precisely we use the dissonance function of a note with respect to the chord made of notes.where is the dissonance of the just interval made by the pair of notes . (On the tonnetz is just the distance). The resonance function is just the opposite .
Example 7.6.
In this example, the chord is made of the only note . We choose notes per octave. For this we are allowed to use only the keys in each octave. On the first pitch space we represent a vertical bar at (almost) every rational number, i.e. possible new note , and the hight of the bar is the resonance . Each octave is decomposed in equal intervals and the black lines is the selection in each interval that gives the highest bar, i.e. the most resonance note, with respect to . This is the adaptative temperament with notes per octave, for the chord . The other pictures are as above.
Observe that the temperament is not the same at each octave. We observe interestingly that the scale in octave is the “blues scale”.
Example 7.7.
In this example, the chord is made of four notes . We choose notes per octave.
We observe that the adaptative temperament depends indeed on the chord .
8 Some examples of just intervals, just chords and chord progressions
By definition of just chord is a set of notes such that each interval is a just interval. Hence a just chord is a set of notes displayed on the tonnetz.
For example a major triad is displayed as a triangle on the axis with a right angle at the bottom left. Here is :
For example a minor triad is displayed as a triangle on the axis with a right angle at the top right. Here is :
For example a
dominant seventh chord is displayed as a
tetrahedron on the
axis. Here is
:
In Section
7.1.2, we have seen there are different just minor thirds between harmonics
and
. Correspondingly there are different
minor seventh chords. For example here is a first
:
Here is a second non-equivalent :
Observations: the first is composed of fifths and thirds. The second is composed of fifths and minor sevenths. They sound very different (see the video).
8.1 Great cadence
9 Advanced use of the plugin
9.1 Automation
Automation means that some parameter of the plugin can be recorded on a MIDI track and play after. For example a parameter can be the choice of 'temperament'.
This allows that a parameter evolves during the reading of a MIDI track.
9.2 Adaptative temperament with different N_keys and levels
- Choose adaptative temperament with 4 keys/octave
- Choose adaptative temperament with 6 keys/octave
- Choose adaptative temperament with 3/6/12
- In octave 4, put hold notes in leader mode: C,Ab,E, this play a minor triad. Then play in follower mode, not hold, in higher octaves.
9.3 Many musicians, how to use in session
We propose here a setup how to use the plugin(s) in a session with many musicians (here Bo and Malik).
We propose as seen on the picture below,
- On input there are audio (in green) and midi (in red) signals. Each input is connected to an individual midi track with a tonnetz plugin. In fact all the plugins communicate in the computer and share the same list of notes and the same tonnetz space. Beware that a plugin VST3 can not communicate with a AU plugin, so all the plugins should be of the same type VST3 or AU.
- On output of the plugin we can choose to use the internal synthetizer (IS) of Alex that gives an audio output. This audio output is then sent to a mixer table.
- On output of the plugin we can choose not to use the internal synthetizer of Alex but send the midi signal to an external synthe, and after the audio signal to the mixer table.
- The musicians have a direct access to the plugin parameters through a computer console. They can also send automatic commands to the plugin using a Foot-Pad midi controller. The commands can be assigned from the computer console.
- In practice the developpers can have also an access to the computer from an external computer, using “AnyDesk”.
Details on midi connections:
- if the controleur uses a DIN connection, one has to use either a convertissor or the midibox.
A EWI
A.1 EWI USB
A.2 EWI 5000
Ewi 5000 is a MIDI saxophone.
A.2.1 Configuration with the software EWI5000
- set Key delay = 2. (not more not less)
- Fingering: EWI
A.2.2 Configuration (once for ever)
The program tonnetz uses
bite events from EWI5000. You have to configure the EWI5000 as follows, (see page 17,18 of the
manual for details)
- Configure the bite sensor:
- No Filter, No Pitch bend, no Breath sensor
- But CC enabled: En with a dot, Cn is 20 (= x14), nb is 0 and bt is 27.
As a result, the bite event will send a midi message: where is the value.
A.2.3 Midi messages
EWI5000 sends the following messages (given here in hexa):
- Note on
- Note off:
- Breath control fine, coarse:
- Controller Potamento, on/off and coarse:
- PitchWheel:
- Bite event: where is setup as explained above.
References
1F Faure, M. Mezzadri, and F. Ratchov, "Analyse et jeu musical en tempérament juste adaptatif", https://hal.archives-ouvertes.fr/hal-01119499link (2015).