Augmenting the Cello

Adrian Freed: adrian [at] cnmat [dot] berkeley [dot] edu Frances Marie Uitti: francesmarieuitti [at] yahoo [dot] com David Wessel Michael Zbyszynski {wessel,mzed}@cnmat.berkeley.edu

 

Abstract:

Software and hardware enhancements to an electric 6-string cello are described with a focus on a new mechanical tuning device, a novel rotary sensor for bow interaction and control strategies to leverage a suite of polyphonic sound processing effects.

Keywords

Cello, chordophone, FSR, Rotary Absolute Position Encoder, Double Bowing, triple stops, double stops, convolution.

1.     INTRODUCTION

This paper describes the fruits of the collaboration between the renowned cellist Frances-Marie Uitti and CNMAT researchers in the fall of 2005 sponsored by a UC Regents lectureship program. The augmented cello completed during the collaboration was used in performance at the end of her residence on November 4th 2005.

The starting point for the project was a 6-string cello built by Eric Jensen [4]. The main, unusual feature of this electric cello is a deep notch in front of the bridge co-designed by Ms. Uitti and Mr. Jensen. This allows Ms. Uitti to play using two bows simultaneously-one above and one below the strings-for chordal and other polyphonic textures [16] [12]. We were curious how much of our previous work on polyphonic signal processing for guitars could be leveraged for a bowed instrument in the hands of player who has already vigorously pursued the polyphonic potentiality of the instrument.

We will describe a new solution to the problem of changing tunings of the open strings, a matrix of switches and pressure sensors installed on the instrument, a novel bowed rotary encoder and the software used in the debut performance of the instrument.

1.1     Tuning Augmentation

Ms. Uitti uses a variety of non-traditional tunings to take advantage of the possibilities afforded by multiple stops and two bows.

The combinatorial elaboration of sounding strings for multiple stops described in Table 1 takes into account the limited access of the underbow to two strings. Other practical considerations make a couple of the stops difficult but the table clearly shows the advantage of 6 strings over 5 or 4 - especially for triple stops.

Table 1. Available stops for double bowing

	Strings
Stops	4	5	6
4	1	2	3
3	4	9	12
2	6	9	13

 

The conflicting constraints of string displacement; stopping-hand reach, spread and strength; and the bridge arch result in a practical limit of six strings. Lap steel guitar players, freed of the reach and stopping pressure constraints, play 6-9 stringed instruments[2]. The additional constraint of the curved bridge to allow bowing of separate strings precludes adding as many strings. Chordal fingerings become more limited as the neck becomes broader, especially those chords where the little finger or ring finger needs to depress the lower strings while other fingers need a maximum curve to access the upper ones. These considerations explain why bowed chordophones such as the cello and viola d'amore have not explored the extremes of stopped string count achieved for the lute and theorbo.

The analysis so far only addresses the bowability of triple and quadruple stops. What pitches are actually available depends on additional, more complex constraints from the interaction of the stopping-hand reach and the chosen tuning. Ms. Uitti has already approached the limits of what is humanly possible with her stopping hand so the free design parameter is the tuning of the open strings, e.g. Scelsi's 4th string quartet [8].

The interesting question of which families of tunings to use will be the subject of a future paper. We choose here to focus on the ergonomics of quickly changing tunings: during a piece and even during a note, a technique used occasionally by banjo players and guitarists and developed to its extreme by Adrian Legg [3].

One approach to supporting different tunings is to use independent pitch shifting DSP algorithms on the signals captured by piezoelectric pickups under each string at the bridge. This method is used commercially for guitars and used notably by musicians who adopt many unusual tunings, Joni Mitchell, for example, who composes using scordatura tunings as a starting point [13].

During a previous project on hex guitar signal processing we identified several important challenges with electronic pitch shifting:

1)      Numerous noticeable artifacts in the shifted sound.

2)      Conflict between the acoustic sound and electronic sound in live performance

3)      Unacceptably long latencies especially for low-pitched strings.

Our solution to these problems was to augment the cello by adding a mechanical tension-modulating device at the heel of the instrument.

Figure 1. Cello heel with string tuning device

This device was originally developed for guitars by Hipshot Inc [10]. We adapted it to the cello - primarily accommodating the larger cello string end. The device is normally floated from the heel of the instrument but we instead added it to an extension of the heel to maintain the existing short string length. This affords bowing below the bridge on the short strings.

Three possible pitches are available for each string  adjusted by set-screws allowing for microtonal, 1/4 tone, 1/2 and whole tone tunings.

This arrangement works well avoiding problems with previous methods and we suggest it is a good example of the benefits of exploring non-electronic solutions to instrument augmentation challenges.

2.     Gesture Sensing Augmentations

Foot control is commonly used in live performance especially with computer-based scores. We experimented with many foot pedal options and confirmed our early suspicions that these are hard to use in practice. Cellists use their legs to counteract the considerable torque generated by bowing. Their feet have to be firmly planted on the floor to comfortably do this for long periods with the necessary stability to support solid performances. Alternatives have been explored to this seated playing position including stands and harnesses[9] but these are not widely accepted on ergonomic and practical grounds. We therefore decided to focus our efforts on new interaction opportunities for the fretting and stopping hands - the core of the cellist's technique.

2.1     The stopping hand

For the stopping hand we provided a row of FSR's (Force Sensing Resistors) on the edge of the neck closest to the low-pitched strings. These were centered at the semitone positions of the string. This provides both a natural location (already thoroughly part of the cellist's technique) and no part of the hand can inadvertently touch this part of the instrument. The semitone positioning also suggests a convenient labeling of each control in a score.

FSR's have the advantage over switches of having a low profile and providing an extra control dimension (pressure). They also cost no more because the installation cost dominates the parts cost.

On the other edge of the neck we installed a continuous pressure-sensing strip accessed typically with the thumb.

FSR strips are cheap and convenient but unlike knobs and sliders they don't provide any tactile memory of a parameter setting. We addressed this by adding a slider. This most commonly was used to adjust the sound balance between processed and direct cello sound.

 

Figure 2. Cello Body showing neck and body FSR

We also installed a switch array directly below the bridge and an arrow of circular FSR's at the top of the body of the instrument. The switch array is used to make major "preset" changes during performance where the tactile feedback of the switches was important to confirm the change. Installing a small touch screen here would have allowed us to label the presets but we note that some performers prefer instrument interfaces where there is no dependence on visual feedback.

Figure 3. Cello Heel with switch array, hex pickups and slider

We attempted to sense string stop position using a resistive strip designed as a "ribbon" controller but found it too wide and short for this application. We also to measure the electrical resistance of the string from a conductive fingerboard to the nut but found that the distance/resistance function was highly non-linear and varied from string to string, presumably because of the exotic alloys and solid wound and stranded construction techniques used in cello strings. These difficulties were a turning point for the project: where we decided not to try to measure and track traditional cello-playing gestures but instead augment the instrument with new possibilities.

2.2     The Bowing Hand

For the bowing hand we introduced a novel application of a rotary absolute position encoder, a device that outputs a voltage corresponding to the angle of rotation of a shift from a reference position. We attached a wheel to the shaft of a commercially available encoder with a surface preparation that the bow could easily grip. We installed the wheel behind the heel of the instrument where it can be thought of as an extension of the "short string" bowing technique.

Figure 4. Sensor Wheel

3.     Sensor and Sound Data Capture

All the resistive and switched inputs for gesture sensors were translated into voltages between 0 and 5v using simple resistor divider networks. These signals were carried on a multiwire cable to a DB25 connector plugged into one of the two Sensor ports of CNMAT's Connectivity processor [1].

Figure 5 CNMAT Connectiviy Processor

The piezo sensors for each string and two additional piezo pickups near the tail of the short strings were converted by custom-built charge amplifiers built into a special daughter card for the Connectivity processor.

Figure 6. Short String Piezo Pickups

These analog signals are conditioned, converted into digital signals, serialized and aggregated into an Ethernet stream that was processed by custom software in Max/MSP. Sound output was also routed through Ethernet packets to the connectivity processor and demultiplexed into 8 balanced analog audio outputs.

4.     Performance Software

We elaborated and augmented ideas originally developed for an earlier polyphonic guitar project [15] to reflect Ms. Uitti's aesthetic needs.

Each idea was implemented as a separate Max/MSP patch and each patch was controlled by a main supervisory patch that managed all the signal and gesture routing and also switched active patches according to selections by the performer.

One programming challenge is to give the performer as much meaningful control as possible without overwhelming them with parameters that they will find useless or, worse yet, distracting. It is important to work in a style that allows the programmer to quickly remap controllers and values to any location in the patch, and empowers the performer to feel that the software is actually responding to her actions.

To that end, overall control of the performance subpatches was managed using a combination of OSC (Open Sound Control)[14] and the pattr family of objects. Each of the hardware sensors was given a unique address in an OSC namespace, allowing individual subpatches to tap into the appropriate control data.  Configurations that activated one or more subpatches were stored as presets in the pattrstorage object and triggered via the switch array (below the bridge). Smooth crossfades between successive configurations were achieved with pattr's built-in interpolation features.

 

These features allowed the cellist to dynamically remap the meaning of her performance gestures according to the needs of the musical situation, quickly and smoothly moving between one set of patches and the next.

No matter what patches are in effect, the cellist always has control of her throughput gain, and the overall gain of the effects. Single controllers are mapped to each these gains, and remain fixed throughout the performance. This was important to allow the performer to react instantly to the musical situation, especially if the processing does not fit the character of the musical moment.

Figure 7. Performance patch

We briefly describe in the next sections some of the more compelling subpatches available.

4.1     Vocal Effect

For this effect we used a separate bank of five resonant formant filters for each string. These were tuned dynamically by interpolating between vowel pairs stored from a data set that included a,e,i,o,u for soprano, alto, bass, contrabass, and tenor voices. The appropriate vocal data set  was matched to the tessitura of each string. Vibrato was created artificially by interpolated delay line modulation and modulated by pressure of the fingerboard FSR strip. This was used as a micro-rhythmic contrast against Uitti's normally fluctuating vibrato, creating changing beating patterns and synchronizations. Vowel pairs were chosen using the fingerboard FSR's and interpolations were driven by the patch.

4.2     Double-stop Convolution

The key idea of this patch is to use a separate convolution for all the double stop combinations and to process and spatialize the output of the convolved pairs independently. Since the convolution was performed by FFT's we were able to save computation by sharing the forward transform of each string signal.

Convolution works well in this situation because sound is only output if there is a signal in both inputs of the convolution. This is a fruitful area of exploration because double stops are a reliable musical gesture and the performer has immediate access to many independent streams of processing without having to choose them ahead of time with other gestures.

Figure 8. Double Stop Convolution

4.3     Quad Granular and Circular Panning

Two patches were combined in this effect with the intent of surrounding the direct sound of the cello with a diffused aura of related fragments. The fragmentation was achieved with a pair of stereo granulators, specifically munger~ (from the PerColate [11] collection). These were set to create relatively long (2000ms ±200ms), widely spaced (500 ms ±250ms), irregular grains. Grains were generated from a 3000ms buffer, and could play back either forwards or backwards at the speed of the original performance. Each granulator is independent, and their outputs were interlaced and sent to the circular panner.

The panning patch diffused the sound in a circular array, maintaining a 180 degree separation between each channel of each granulator. That is, if left and right for the first granulator appeared at 45 and 225 degrees from the listener, the second granulator would appear at 135 and 315 degrees. Each granulator generated grains at random locations in their stereo field, so the result was a complex constellation of sounds. The entire sound field was rotated by the performer using the rotary encoder behind the heel of the cello. This gave the performer sensitive and expressive control of the direction and rate of the perceived motion. The angular displacement of the sounds was generated by Ville Pullki's VBAP objects[5], allowing the angle to be specified independently of the specific number and location of loudspeakers.

5.     Future Work and Conclusion

We will explore the use of touch panel displays for labeled buttons and the use of two-dimensional pressure sensing panels on the side of the body.

The position encoding wheel/bow sensor interaction shows a lot of promise in the augmented instrument context. We are exploring use of detents and weights to see how much tactile feedback can be exploited by the musician. We are also exploring new instrument interfaces built around this sensor. We will explore the addition of a servo motor to the drive of the encoder, a strategy that has been explored to research violin bowing [7].

We used surface wiring and temporary adhesives to provide the most flexibility in the development of the augmented instrument. Now that the design issues are settled we will mechanically integrate the sensors and bury the wiring within the instrument. We note that current construction techniques in solid-bodied musical instruments do not provide the channels and cavities in the neck of the instrument to facilitate this and suggest that simply routing cavities in the body of instruments for transducer electronics is insufficient to embrace the potential of modern sensing technology and the ambitions of future musicians.

The solutions developed in this collaboration can be further enhanced with a newly designed instrument and we can accommodate some of the ideas we were forced to discard. In particular we will be able to integrate stop position sensing and we will significantly augment the control possibilities of the new instrument by marrying it with a sensor-laden bow [6], a project already in the initial phases of design and construction by F.M.Uitti in her Sonic Lens Project. This sound/vision project supported by Stichting Steim and the Biennale of the Amsterdam Film Museum involved the triggering and manipulation of film using bowing gestures.

6.     Acknowledgements

We gratefully acknowledge the prior work of Matt Wright and John Schott and the support the Chambers fund, UC Regents, and Waves Audio Ltd.

7.     References

 

1.         Avizienis, R., Freed, A., Suzuki, T. and Wessel, D., Scalable Connectivity Processor for Computer Music Performance Systems. in International Computer Music Conference, (Berlin, Germany, 2000), ICMA, 523-526, http://cnmat.CNMAT.Berkeley.EDU/ICMC2000/pdf/connectivity-processor.pdf.

2.         Bechtel, B. Lap Steel Tunings, 2006, http://www.well.com/user/wellvis/tuning.html#8string.

3.         Ellison, T. Against the Grain: A Chat with Adrian Legg, 2006, http://www.solidairrecords.com/AMR_interviews/legg.html.

4.         Jensen, E. Jensen Electric Cellos, 2006, http://www.halcyon.com/jensmus/cello.htm.

5.         Pulkki, V. Virtual sound source positioning using vector base amplitude panning. Journal of the Audio Engineering Society, 45 (6). 456-466.

6.         Rasamimanana, N.H. Gesture Analysis of Bow Strokes Using an Augmented Violin IRCAM, Université Pierre et Marie Curie, Paris VI, 2004.

7.         S. Serafin, M.B., S. O'Modhrain, C. Nichols. , Expressive controllers for bowed string physical models. in DAFX, (Limerick, Ireland, 2001).

8.         Sciannameo, F. A personal memoir: Remembering Scelsi. The Musical Times, 142:1875. 22-26.

9.         Steinberger, n. Cello Stand Instructions, 2006, http://www.nedsteinberger.com/instruments/cello/cr/celloinstr.html.

10.       Trilogy. Hipshot Trilogy Multiple Tuning Bridge, 2006, http://www.hipshotproducts.com/trilogy.htm.

11.       Trueman, D. and DuBois, R.L. PeRColate, 2001, http://music.columbia.edu/PeRColate/

12.       UItti, F.-M. Two Bows, 2006, http://uitti.org/twobows.html.

13.       Whitesell, l. Harmonic palette in early Joni Mitchell. Popular Music, 21 (2). 179-193, http://www.journals.cambridge.org/action/displayAbstract?fromPage=online....

14.       Wright, M., Freed, A. and Momeni, A., Open Sound Control: State of the Art 2003. in International Conference on New Interfaces for Musical Expression, (Montreal, 2003), 153-159, http://www.music.mcgill.ca/musictech/nime/onlineproceedings/Papers/NIME0....

15.       Yeung, B. Guitar Dreams SF Weekly, San Francisco, 2004, http://www.sfweekly.com/Issues/2004-01-07/news/feature.html.

16.       Zorn, J.E. Arcana. Granary Books, 2002.