By Matt Kelly
Small Times Correspondent

CAMBRIDGE, Mass., Jan. 2, 2002 — Be careful when you ask Joseph Paradiso how microelectronics can influence the arts. His answer is liable to leave your feet spinning.

While many MEMS applications focus on the nuts-and-bolts worlds of aerospace or defense, Paradiso has tinkered with the technology for a more aesthetic pursuit: dance. The research scientist at the Massachusetts Institute of Technology

null

Yuying Chen makes music by dancing with MEMS-equipped shoes at a Wearables Fashion Show.

(MIT) bolted a few microsize gyroscopes and accelerometers to footwear and tracked dancers’ movements as they took to the floor. He programmed a computer to translate those motions into musical tones.

Now, dancers and choreographers can lace up a pair of Paradiso’s shoes, turn on the computer and make music just by moving.

“I threw everything I could think of at the shoe,” he said. “It was kind of a technical exercise. I put in every sensor I could fit.”

The MEMS dance shoes — formally known as “expressive footwear” — measure how a dancer moves his foot across a predetermined floor space. As the dancer taps his foot, slowly spins or runs across the floor, the sensors feed that information to a nearby computer. The computer then interprets how that data should be played as music through a system of algorithms and prerecorded musical sounds.

The result depends on the dancer, since his personal movements steer how the music sounds. Someone making slow, graceful turns might create broad, soothing chords. A tap dancer’s rapid-fire steps might get the audience bopping with a series of sharp bursts.

“The shoes are responsible for generating the music, so as you dance, you’re also composing at the same time,” said Mark Haim, a New York choreographer who has used Paradiso’s shoes several times. “Your compositional know-how is dependent on your familiarity with the system. That is, the placement of specific sounds and samples on the floor grid and the sensors on the shoes that set off specific sounds and events.”

Paradiso got the idea for the dance shoes while visiting a Yamaha computer lab in Japan in 1997. He saw a bodysuit with sensors to detect human motion, “and I thought, ‘Wow, you can do so much more with the foot.’ ”

As director of MIT’s Responsive Environments Group, Paradiso had already dabbled with motion and location sensors and the necessary MEMS technology; he had created a baton with motion sensors at each end. For the dance shoe, he decided to measure every type of motion possible.

Paradiso and his students squeezed 16 sensors into their prototype, a black Capezio Dansneaker. It consisted of an insole with sensors to measure pressure at the heel, toe and ball of the foot, and a circuit board attached to the outside ankle to measure speed, radial motion, height from the floor and location in the room.

The “MEMS potpourri,” as Paradiso calls it, is housed in the circuit board. It includes two accelerometers to measure tilt, speed and acceleration as dancers kick and spin. A sonar receiver processes “pings” from various points in the room to determine location; a microcomputer embedded on the circuit board compiles all the sensor information into digital data transmitted through a small antenna.

Paradiso and his students created their prototype in 1998. They spent the next two years mounting the equipment onto other shoes and field-testing them at various dance festivals and electronics exhibitions. Paradiso’s group is now working on a next-generation model that can quickly clip onto the back of a shoe and occupy less space.

It costs several hundred dollars right now to make each shoe, Paradiso said. He believes that cost could fall significantly in mass-manufacture.

All of the MEMS devices are commercially available from companies such as Analog Devices Inc. or Measurement Specialties Inc. Paradiso and others say that the shoes do not pioneer new technology, but rather compel new ideas about existing MEMS know-how.

“Well, it’s certainly innovative,” said Daniel Siewiorek, a professor at Carnegie Mellon University in Pittsburgh and an expert in wearable computers. “The components are all there [already] … it’s coming up with a new way of thinking.”

Siewiorek said he has seen demonstrations of Paradiso’s shoes. “There’s a niche that Joe is in,” he said. “I don’t see a lot of other people in there yet, but entertainers will do it as a novelty.”

Haim, the New York choreographer, agreed. “I imagine that for the time being, it would be a novelty item because of the complexity of setting up the system and the amount of technological expertise involved in operating it,” he said.

Paradiso agreed that while dancing is the shoes’ sexiest use — “that’s the Cadillac of it all” — the underlying principles can be applied to studying and managing a variety of motions.

For example, Paradiso said, rehabilitation hospitals may find the shoes useful in helping disabled patients learn to walk after an accident. Similar equipment already exists to measure a patient’s motions, “but you have to wire them up. This take a little fewer measurements, but allows more freedom,” he said.