Interdisciplinary Research
Thesis abstract
In his ‘Uncanny Valley’, Mori posits that, when a non-human entity appears too ‘realistic’ or human-like, users respond with unease. To date, most synthetic speech research has ignored the phenomenon of the ‘Uncanny Valley’, striving for greater ‘realism’ and verisimilitude to human speech. Distinct from ‘realism’, however, is the concept of ‘liveness’, which is more akin to ‘spontaneity’ or ‘truthfulness’. This thesis documents the development of the Place, Authenticity, Time (PAT) framework, a framework that seeks to support the design of synthetic voices by making them appear more ‘live', rather than more ‘realistic’. The framework is derived both from the performing arts and from computer science, and casts synthetic voices as ‘synthetic voice actors’. Over the course of this thesis, a software system is built and a number of studies are undertaken to reveal whether simple, performing-arts-derived, paralinguistic/prosodic modifiers can improve the user perception of a synthetic voice’s ‘acting’. These studies show a preference for a modified synthetic voice over a non-modified synthetic voice in straightforward head-to-head comparisons, but no clear preference in more complex interactive implementations. The PAT framework emerges from these studies as a model of the ‘problem space’ confronting the designer of synthetic voice actors. It suggests ways in which acting, scripting and setting can be used to evoke ‘liveness’, and to sidestep issues of ‘realism’ and the accompanying user unease.
Keywords: Speech synthesis, computer speech, TTS, liveness, synthetic actors, lifelike computer characters, synthetic agents, virtual actors, virtual characters, anthropomorphic agents, emotional agents, believable agents,, avatars, ventriloquism, acousmêtre
RESEARCH
Recent Journal Articles
Liveness' in human-machine interaction. Performance Arts and Digital Media. 7(2): December 2011
'Place, Authenticity Time: a framework for synthetic voice acting.' International Journal of Performance Arts and Digital Media. January 2009.
Conference presentations and performances
My Voice and Me: A melodrama for synthetic voice. Creative Speech technology Network. University of York
18/10/2011
'‘Whose Voice is it Anyway?’ demonstration and performance. (re)Performing the Posthuman a conference on performance arts and posthumanism. Centre for Research in Opera and Music Theatre University of Sussex, Brighton, 21-22 May 2010
A telephonic Installation. Sonic Interaction Symposium. York University UK. 2nd April 2009
P.A.T. (a telephonic computer voice actor). 'Theatre Noise' Central School of Speech and Drama. London 22nd April - 24 April 2009
'Frank' an artifact for public engagement in speech technology. Interspeech 2009. 7th - 10th September 2009
Other Activities 2009
Convener of the inaugral meeting of the Creative Speech Technology Network. Scarborough March 18th
Earlier research activities
2008 'P.A.T. Testing' (Newell, C.; S. Andrews and I. Gibson) a one hour performance piece shown at the University of Hull Scarborough Campus.
2008 'Please wait with me' (Newell, C. and S. Andrews). A telephone based installation shown at Sight Sonic in York.
2007 'Call Centre' (Newell, C.; S. Andrews; A. Edwards et al.). A telephone based installation shown at the Digital Music Research Network in Leeds.
2006 Tide' (Newell, C.; S. Andrews; J. MacDermott et al., 2006). A digital art exhibition exhibited at the British HCI Conference 2006.
2005 Edwards, A and Newell C Achieving a lively voice, International Journal for Language Data Processing 28.2/2004, Pages 133-151, ISSN: 0343-5202.
2005 Newell,C and Edwards A. Unnatural but lively voice synthesis for empathic, synthetic performers, Pages 137-143 AISB 2005 Social Intelligence and Interaction in Animals, Robots and Agents, Joint symposium on Virtual Social Agents, The University of Hertfordshire, ISSN:1 902956492.
2004 Newell, C and Edwards, A. To be or to seem to be: that is the question. Unnatural but lively, speech synthesis for synthetic performers: models from acting, Pages 124-131 AISB 2004: Symposium on Language, Speech and Gesture for Synthetic Characters AISB (Artificial Intelligence and the Simulation of Behaviour: The University of Leeds, ISSN:1 902956390.
PRACTICE
2001 - Digital Artist. Sepia Sands. University of Huddersfield research project. Software prototype only. This was an experimental attempt to present the e-shopping experience in a narrative, multimedia environment. Preliminary exploration of the
2001 - Digital Artist. Norman A digital video taster for live web show. Produced using web-cam technology and crude audio sampling.
2000 - Digital Artist. University of Huddersfield research project. Experimental composition tool merging poetry, morse code and chaotically-generated musical bytes.
1997 - Digital Artist. AHRB research and development grant. An online opera. Adapted from the pamphlet by Jonathan Swift.
1996 - Digital Artist. Birmingham Conservatoire. One year funded research project. Web-based music experiment merged Yamaha MIDI XG technology with streaming spoken audio.
1994 - Digital Artist. An early attempt to merge live and digital media. The only technology available was an Atari games console, and a voice modulator/pitch shifter. The show was presented in an historic greenhouse within a London Park in about 110 degrees of summer sun. The computers crashed and the screen images were invisible against the sunlight.
Opera and Theatre
2002 - Director: Carmen, Bizet. Diva Opera. European Tour
2002 - Director: La Cenerentola, Rossini. Mid Wales Opera.
Published as a Newtonian Production Video: david_pugh@lycos.co.uk
2000 - Director. Curlew River, Britten. University of York
1999 - Director: Pierrot Lunaire, Schoenberg. Black Hair, University of York
1999 - Director. A Whitehouse Cantata, Bernstein, Nagano, London Symphony Orchestra
1998 - Director. The Magic Flute. Mozart. Mid Wales Opera
1998 - Digital Artist. Newell, Barker, Chance.
A major Arts Council commission.
1997 - Director. Salome, Strauss. Bournemouth Orchestras Tour,
1996 - Director. The Tempest, Shakespeare. The Green Room, Manchester
1996 - Director. Noises, Sounds and Sweet Airs, Nyman. Festival di Fano, Italy
1996 - Director. The Turn of the Screw, Britten. Aldeburgh Festival
1996 - Director and Devisor.On the Road with Ron and Ros.
Arts Council commission.
1995 - Director. Tosca, Puccini. Pavilion Opera International Tour
1995 - Director. Rigoletto, Verdi. Mid Wales Opera.
Published as a Newtonian Production Video: david_pugh@lycos.co.uk
1995 - Director. The Coronation of Poppea, Monteverdi. Birmingham Conservatoire
1995 - Director. A Midsummer Night Sir Colin Davis. LSO
1994 - Stage Director. Barbican Theatre London
1994 - Director. The Merry Widow, Lehar. Pavilion Opera International Tour
1994 - Director. Tales of Hoffman, Offenbach. Mid Wales Opera.
Published as a Newtonian Production Video: david_pugh@lycos.co.uk
1993 - Director. Blood Wedding, Lorca. Birmingham Conservatoire
1993 - Director. Alice in Wonderland. Gardner Arts Centre
1993 - Director. Barber of Seville, Rossini. Mid Wales Opera.
Published as a Newtonian Production Video: david_pugh@lycos.co.uk
1993 - Director. Carmen, Bizet. Pavilion Opera International Tour
1992 - Director. Die Fledermaus, Strauss. Mid Wales Opera.
Published as a Newtonian Production Video: david_pugh@lycos.co.uk
1992 - Producer. Fungus the Bogeyman, Briggs/Carter. Belgrade Theatre, Coventry
1992 - Director. The Turn of the Screw, Britten. Aldeburgh Festival
1991 - Director. La Pazzia Senile, Banchieri. MMTT at the Purcell Room, South Bank, London
1991 - Director. Into The Woods, Sondheim. Birmingham Conservatoire
1991 - Director. Higglety Pigglety Pop, Knussen. Barbican London
1989 - Director: The Golem workshops. Almeida Festival, London
1990 - Director. Die Entfuhrung aus dem Serail. Mozart. Pavilion Opera International Tour
1990 - Director. The Bartered Bride, Smetana. The Birmingham Conservatoire
1989 - Verse coach. The Merchant of Venice, Shakespeare. The Peter Hall Company, London
1989 - Associate Director. A Midsummer NightGlyndebourne Festival and Touring Opera
1988 - Director. La Traviata, Verdi. Macau International Festival
1988 - Director. Die Fledermaus, Strauss. Pavilion Opera International Tour
1988 - Assistant Director. The Late Shakespeares Shakespeare. The National Theatre, London
1987 - Adaptation Director. La Traviata, Verdi. Glyndebourne Festival Opera, The Royal Albert Hall, London
1987 - Director. Christopher Columbus, Offenbach. Birmingham Conservatoire
1987 - Associate Director. The Electrification of the Soviet Union, Osborne. Glyndebourne Festival Opera and Tour
1987 - Associate Director. La Traviata, Verdi. Glyndebourne Festival Opera and Tour
1987 - Associate Director. Albert Herring, Britten. Royal Opera House, Covent Garden, London
1987 - Director. Die Zauberflote, Mozart. Pavilion Opera International Tour
1987 - Director. The Siege of Rochelle, Balfe. John Lewis Opera, London
1987 - Director. The Devils Wall, Smetana. University College Opera, The Bloomsbury Theatre, London
1987 - Director. Fantastic Mr Fox, Dahl/Carter. Gardener Arts Centre, Brighton
1986 - Director. Phantastes, Barker. The Place Theatre, London
1986 - Director. Cosi Fan Tutte, Mozart. Pavilion Opera Tour
1986 - Associate Director. Porgy and Bess, Gershwin. Glyndebourne Festival Opera
1986 - Associate Director. Higglety Pigglety Pop and Where the Wild Things Are, Knussen. Glyndebourne Festival Opera and Tour
1986 -Associate Director. Albert Herring, Britten. Accademia Philharmonica, Roma
1986 - Director. La Boheme, Puccini. Ashcroft Theatre, Croydon
1986 - Associate Director. Albert Herring, Britten. Summerscope, Queen Elizabeth Hall
1986 - Director. Nabucco, Verdi. Epsom Playhouse
1985 - Director. The Canterville Ghost, Wilde/Barker. Hoxton Hall, London
1985 - Associate Director. Albert Herring, Britten. Glyndebourne Festival Opera and Tour
1985 - Associate Director. The Coronation of Poppea, Monteverdi. Glyndebourne Festival Opera
1985 - Associate Director. A Midsummer Nights Dream, Britten. Glyndebourne Festival Opera
1985 - Director. Hands Across the Sky, Hopkins. London Chamber Opera
1985 - Director. The Marriage Between Zones 3, 4 and 5, Barker. Opera Viva, The Place Theatre, London
1984 - Director. A Midsummer Nights Dream, Britten. Guildhall School of Music and Drama, London
1984 - Director. Ariadne auf Naxos, Strauss. Opera Viva, London
1983 - Assistant Director. Intermezzo, Strauss. Glyndebourne Festival Opera
1983 - Assistant Director. Idomeneo, Mozart. Glyndebourne Festival Opera
1983 - Assistant Director. The Love of Three Oranges, Prokofiev. Glyndebourne Festival Opera
1983 - Director. Opera Scenes. The Wexford Festival, Ireland
1982 - Director. Il Trittico, Puccini. Abbey Opera, London
1982 - Director. The Threepenny Opera, Brecht/Weil. Opera Viva, London
1982 - Director. Don Pasquale, Donizetti. Pavilion Opera Tour
1981 - Director. Cosi Fan Tutte, Mozart. Pavilion Opera Tour
1981 - Director. The Triumph of Virtue, Scarlatti. Guildhall School of Music and Drama, London
1980 - Director. Ariadne auf Naxos, Strauss. Opera Orfeo, London
1979 - Director. L Opera Orfeo, London
1979 - Director. La Serva Padrona, Pergolesi. Guildhall School of Music and Drama, London
Television and video production
1987 - Assistant Stage Director. The Electrification of the Soviet Union, Osborne. BBC broadcast
1986 - Assistant Stage Director. Where the Wild Things Are, Knussen. TVS broadcast and video release. UPC#: 032031445192
1986 - Assistant Stage Director. Higglety Pigglety Pop, Knussen. TVS broadcast and video release. UPC#: 032031445192
1985 - Assistant Stage Director. Albert Herring, Britten. BBC broadcast and video release. ASIN: B0009PQWYY
1986 - Assistant Stage Director. La Traviata, Verdi. BBC broadcast and video release. ASIN: B0000541Y0
1985 - Assistant Stage Director. The Coronation of Poppea, Monteverdi. TVS broadcast and video release. ASIN: B00019HP1M
1986 - Stage Director. Arts Review of the Year. LWT broadcast.
1982 - Stage Director. Songs over the Garden Wall. BBC broadcast
Audio recording
1995 - Stage Director and Dialogue Coach. A Midsummer Night London Symphony Orchestra conducted by Sir Colin Davis. ASIN: B0000041ET