The Complexities of What We Hear: Robot Mouths and Speaking Pianos

Gemini robot in conversation

Gemini robot in conversation
© Peter Čuhalev with CCLicense

The drive to understand the coordination between the mind and body that produces language is a powerful source of innovation. The following videos represent attempts to replicate the human voice by means of a physical apparatus or robot rather than by digital sampling and recording techniques alone. This requires and stimulates a greater understanding of both how the human voice is produced and how it is heard.

Wien Modern was one of ten organizations asked to make an artistic contribution to the European Environmental Criminal Court Forum 2009 event in the Palazzo Ducale, Venice, Italy. Their project: to perform the text of a proclamation released earlier by the EECC as a musical composition, without setting it in a traditional sense. To that end, Austrian composer Peter Ablinger transferred the frequency spectrum of elementary school student Miro Markus’ voice to his computerized piano. Breaking down the voice into snatches of frequency, much as a computer might break down an image into pixels, Ablinger was able to reconstruct Markus’ recitation into pitch and rhythm on the keyboard. The result is astonishing, although having subtitles really helps clarify the text. The relationship of our speaking voice to pitch and rhythm is quite complex and the “music” of a voice is unique and integral to how that voice is understood.

Video via TheMcphearson on YouTube.

Scientists at Kagawa University in Takamatsu, Japan are trying to achieve a better understanding of how the mind and body work together to produce speech. The talking robot below consists of a silicone mouth, vocal cords and tongue; a plaster nasal chamber for increased resonance; an air pump that acts as lungs and a rudimentary computerized brain which has the capacity to associate movement with sound and learn through auditory feedback. What the human mind and body does subconsciously can only be reproduced with a great deal of trial and error, but the robot can learn and execute an extremely simple song, make vowel sounds, and produce some words. Next step? This mouth is currently being fitted with teeth so that it can produce fricative consonants. Look out, world!

Video via Diginfonews on YouTube.

Be a Part of our Third Web Project

© Dayna Bateman with CCLicense

© Dayna Bateman with CCLicense

Synkroniciti is proud to announce our third web based project. We invite you, our readers, to create new art based upon the themes Synkroniciti has explored in the past three weeks: city, shadow and mud. We encourage submissions in any discipline, including the realms of music, theatre, film, dance, visual art, and literature. We will edit the submissions to create a video to be featured here on the Synkroniciti site, on Youtube, Vimeo and Facebook. All artists will be credited in the video. You can view our previous videos here.

Please submit one of the following by 11:59 PM CST on Sunday, July 21st:

a video of your artwork

This may consist of  a video journal detailing the process of creating the artwork or a performance of the artwork or a combination of both. Any performance of the artwork should take no more than three minutes. You may send as much video journal material as you like.

an audio track of your artwork

Artists who work with sound may want to explore this option and should realize that audio tracks will be used to accompany the images of other artists. The audio track should be no more than three minutes long.

a file of images

Visual artists may want to explore this option and should realize that their images will be synced with an audio track of another artist. We will accept up to twelve images from each artist.

We regret that we do not accept written materials, but encourage artists such as authors and composers to submit their works in another format more suited to video. We encourage you to read, illustrate or animate your text.

Submissions can be shared with us via dropbox here.

There is no submission fee. Once the finished project is out please evaluate your experience. If the experience was beneficial for you we ask that you acknowledge that with a donation to Synkroniciti. You are free to set the amount of that donation and we are happy to accept any amount.

I will be writing a poem to be performed and included in the video as well. I look forward to taking this journey with you!


Made for Flight: Alouette, the Femmebot

Eastern Meadowlark (Sturnella magna), from The Burgess Bird Book for Children by Louis Agassiz Fuertes

Eastern Meadowlark (Sturnella magna), from The Burgess Bird Book for Children by Louis Agassiz Fuertes

Writers are encouraged to write about what they know. This has merit, but does it shortchange our imagination or compassion? When we take the time to tell the story of others, which requires empathy, interpretation, care and research, this can broaden our understanding and cross borders. The stories that inspired the following video poetry are not mine, but they are not unfamiliar. They float between us and around the corner from us, becoming part of the great cloud of our subconscious mind. When the unspeakable happens it makes ripples that resound through all of us.

Alouette is the French word for a family of birds English speakers know as larks. It is also the title of a French nursery song known all over the globe. The song is intended as a way to teach children the parts of the body, but, as with a great deal of children’s songs, there is a sadistic streak in it that cuts deep.

Lark, nice lark,
Lark, I will pluck you…

I will pluck your back. I will pluck your back.

And your tail!  
And your feet!  
And your wings!  
And your neck!  
And your eyes!
And your beak!  
And your head!  

Alouette is also the name of the Femmebot, who identifies this song with unmentionable abuse that has rendered her damaged and changed her nature. But who is she and what is that nature?

Cool Noise: Daft Punk’s Random Access Memories and Giorgio Moroder

© CarlosMatallanaDiaz with CCLicense

© CarlosMatallanaDiaz with CCLicense

Daft Punk became popular in the 1990s and is credited with some of the biggest hits of the French house music scene, including Phoenix and Rollin’ and Scratchin’. Audiences in the United States probably know them best for One More Time. The electronic music duo of Guy-Manuel de Homem-Christo and Thomas Bangalter is known for its theatrical showmanship as well as its musical virtuosity, performing in iconic android costumes featuring helmets that are the epitome of cool. Their shows also feature impressive lighting displays and strong visual and story-telling elements.

It has been a few years since Daft Punk produced an album, but their newest offering, Random Access Memories, becomes available May 21st. This album is creating some exciting buzz as Daft Punk, in partnership with The Creator’s Project, has collaborated with famous electronic musicians to create something new, often hearkening back to previous decades and to more acoustic music. This is a great example of crossing genres to find new styles. I can’t wait to hear what they have cooked up! Here is a little taste, featuring robots in the desert, something that Synkroniciti wholeheartedly embraces.

Video via PoufyGB on YouTube.

The next video, the first of a series produced by The Creator’s Project about the new album, is an interview with award winning producer and electronic musician Giorgio Moroder, famous for his work with many recording artists, including disco hits with Donna Summer, and for his film scores, which include Midnight Express, Flashdance, Scarface and Top Gun. Listen to him tell you about how he first met the synthesizer and how he and Donna Summer created hits like Love to Love You, Baby. He is among the collaborators that Daft Punk selected for Random Access Memories and he has a great deal of praise for the work and its creators. So much coolness in one place!

Video via CreatorsProject on YouTube.

Playful Depths: I’m a Cyborg, but That’s OK

Are humor and honesty effective in dealing with death and mental illness? Are we held back by propriety and clichés?

When the white coats take away Young-Goon’s grandmother, who believes she is a mouse and has been eating only radishes, Young-Goon retaliates with a suicide attempt. Young-Goon has become convinced that she is a robot and cannot eat. Her attempts to recharge herself using batteries and power cords are dangerous and less than successful. Placed in a mental institution, she makes friends with Il-Soon, a kleptomaniac schizophrenic who fears he is shrinking and will vanish into a dot one day. He knows she needs to eat, so he fabricates a device that turns rice into energy.


Park Chan-wook’s I’m a Cyborg, But That’s OK wades into the issues of aging and mental illness with an impish grin which never fades. You can watch the trailer here. Young-Goon has had such a difficult time dealing with her grandmother’s dissolution that she tries to cut out her feelings, asking Il-Soon to steal her sympathy so that she may kill the doctors in white coats to avenge her granny. Being a cyborg gives her life purpose; as a robot she is designed to destroy. The scenes in which she ridiculously imagines herself a weapon, shooting bullets out her fingers and dropping shotgun shells from her mouth, are some of the most entertaining of the film. I think many people, pressed too far by life or society, have fantasized such power. It isn’t real, and it doesn’t solve anything, as Young-Goon eventually accepts. The film’s combination of violence and humor seems to be a way to talk about the rage that humans feel when they question the meaning of life. This suggests that the way to rehabilitate those who fantasize about violence is not to suppress it, but to talk through it.


Similarly, Il-Soon is afraid of not existing, of fading into a dot. At  some level we all have this fear or doubt, and, for many, religion addresses this. Il-Soon finds the thing that helps him the best is to care for Young-Goon, who has even less going for her than he does. When he serves her, his life has meaning. He learns to love Young-Goon as if she were his own self.

There are some moments in the movie which are shocking and difficult, even amidst the madcap humor, but it is an honest attempt to work through existential issues without the normal cliches we use. What seems to be irreverence may actually be some sort of homage. Can we respect another’s journey even if it doesn’t resemble our own?

A Winsome Robot: The Animated Robot Band from Machinarium

It seems that video games are becoming more and more creative and artistic. Machinarium is an adventure game developed by Amanita Design. A robot called Josef must reassemble himself after being tossed out on a scrapheap and set off for the city, running into puzzles and brain teasers on the way. Josef is named after Josef Čapek, who coined the term robot.

Here is an artistic touch, the song Robot Band, featuring Josef dancing with a small band of characters: one playing didgeridoo; one playing something that looks rather like a saxophone, but sounds like a clarinet; and one playing the drums. It’s a catchy little tune that is played well and the graphic is absolutely enchanting. Who wouldn’t want to play?

Video via MikescCZ on Youtube.

Don’t Quit Your Day Job: Robot Musicians

© Mr Wabu under CCLicense

© Mr Wabu under CCLicense

A whimsical and eye-opening tour of robots performing. Your printer won’t be going on tour soon, but technology is improving!

First up, a cover of Maroon 5’s Moves Like Jagger by the EOL Robot Band.  If you find the lead singer is curiously reminiscent of Steven Hawking, you’ll be interested to know that the Vocals Digital DECtalk Express used here is the same type of unit Hawking used back in the 1980’s, controlled here by a Genesi Linux box. He’s a real scream, “uuuuuhhh”. The rest of the band is fairly impressive: Robot Snare, Robot Bass Drum, Robot Tambourine, Robot Keyboard and HP Scanner, who pretty much steals the show.

Video via bd594 on YouTube.

Perhaps the original song has enough electronic components that the band, excluding the singer, of course, doesn’t sound very far removed. How well do they do without the singer? Try this version of Marilyn Manson’s The Beautiful People. HP Scanner is in fine form again and we now have Robot Bass.

A bit mechanical? Well, what about something more acoustic… say brass and percussion? Meet Toyota’s Concero Quartet: Harry (trumpet), Dave (trumpet), Chuck (tuba), and Ritchie (drums). Not too shabby, and no one is miscounting their rests!

Video via gustavokenichi on YouTube.

I think they managed to put the appropriate ego into the lead trumpet player. If that doesn’t scare you, the next robot band from the University of Pennsylvania is comprised of quadrorotors, the same technology that pilots many automated military drones. The execution is much weaker than that of robots specifically designed to play music, but the visual is entertaining.

Video via UnivPennsylvania on YouTube.

All of these robots perform pre-programmed or learned routines. What about a robot that improvises? This is Shimon, created by the Georgia Institute of Technology. He learns to play in the style of jazz greats while improvising with a human partner.

Finally, what about replacing a vocalist with an android? Okay, I can hear you reciting singer jokes, but this is actually the most difficult sound to reproduce. Feast your eyes on HRP-4C, a female android that sings, complete with voice imitation software.

Perhaps she will do well at karaoke, but I can’t imagine her capturing a resonant female voice. My guess is that pop singers who rely on autotune should be slightly worried.

What do you think? Do robots have a chance as musicians?

Quote for Today: Erich Fromm

Sculpture by Bryan Alexander Davis © bitterherbs via CCLicense

Sculpture by Bryan Alexander Davis
© bitterherbs via CCLicense

The danger of the future is that men may become robots. True enough, robots do not rebel. But given man’s nature, robots cannot live and remain sane, they become “Golems,” they will destroy their world and themselves because they cannot stand any longer the boredom of a meaningless life.

Erich FrommThe Sane Society

Quote for Today: Steven Pinker

Today’s ubiquitous, networked computers have an unprecedented ability to do mischief should they ever go to the bad. But the only mayhem comes from unpredictable chaos or from human malice in the form of viruses. We no longer worry about electronic serial killers or subversive silicon cabals because we are beginning to appreciate that malevolence—like vision, motor coordination, and common sense—does not come free with computation but has to be programmed in. …Aggression, like every other part of human behavior we take for granted, is a challenging engineering problem!
― Steven PinkerHow the Mind Works