Hyperacoustic Instruments ("Scratch Input")

1. Invention title: Hyperacoustic Instruments ("Scratch Input")
2. Summary explanation -- how it works: With this invention, we can turn any object into a musical instrument! We can also outfit any smartphone, computer, car dashboard, or aircraft cockpit, to make it sonically responsive, for hands-free operation so that instead of looking at a button, you can hear the button exactly in the way you touch, press or twist it. This allows head-up operation of complex controls in mission-critical operations.

   More generally, we can turn any surface into a user-interface for multimedia computing and data input!

   For musical applications, hyperacoustic instruments are a class of truly acoustic, yet computational musical instruments. These interfaces and instruments are based on physiphones (where the initial sound-production is physical rather than virtual), which have been outfitted with computation and tactuation, such that the final sound delivery is also physical. The result is a highly-expressive musical instrument that gives a "sweet sound" and "sweet feel" for customers.

3. Example commercial applications:
   •Low-cost manufacturing of expressive musical instruments for sale in mass markets. This technology allows us, for example, to turn blocks of wood or plastic which normally produce a dull "thud" when hit, into a beautiful-sounding, well-tuned musical instrument. Each block sounds with one note of a scale, when it's hit. This can be a DIY kit, or can be manufactured all-complete, for sale to childrens' parks, musical instrument stores, etc..
   The manufacturing cost is greatly reduced, by using ordinary materials as a playing surface. Our technology turns those cheap, ordinary materials into a beautiful-sounding musical instrument, that can also be made extremely durable and vandal resistant.
   •High-end musician market: A musician's control surface that the studio or musician can project onto any surface (positioning a projector-camera device that would be sold), and that surface can be played directly, and interacted with due to tactile feedback. We have created features such as feedback vibrotactility, and sensory fusion with computer-vision and acoustic pickups. This device can be used in live concerts, and in the studio recording industry.
   • In another example, a single plank of wood, which normally makes a dull "thud" sound when hit, is turned into a continuous-pitch xylophone in which the initial sound production originates xylophonically (i.e. as vibrations in wood), as input to a computational user-interface. But rather than using a loudspeaker to reproduce the computer-processed sound, the final sound delivery is also xylophonic (i.e. the same wood itself is set into mechanical vibration, driven by the computer output). This xylophone, which we call the ``Xyolin'', produces continuously variable pitch like a violin. It also covers more than 10 octaves, and includes the entire range of human hearing, over its 122 centimeter length, logarithmically (1 semitone per centimeter).
   •Other examples include pagophones in which initial sound generation occurs in ice, and final sound output also occurs in the ice.
   •Smartphones, computers, car dashboards, and aircraft cockpits, outfitted with this technology to make it sonically responsive, for hands-free operation so that instead of looking at a button, you can hear the button exactly in the way you touch, press or twist it. This allows head-up operation of complex controls in critical operations.
4. Links to technical papers associated with the invention:
   •Hyperacoustic Instruments,
   •The Xyolin,
   •Physiphones,
   • Making a badly tuned or unpitched instrument play in perfect harmony.
   •This invention is also called "scratch input", a highly successful and vibrant field of research and practice that we founded through the above papers.
5. Patent applications filed or issued: US Pat. 8017858. Additional patents in-progress.
6. Prototypes constructed: Several working prototypes have been constructed by S. Mann, R. Janzen, and research team.
7. This invention is jointly developed by S. Mann and Ryan Janzen, who would be equal partners in commercialization efforts. The hyperacoustics invention can be extended and deepened, if desired, by including a few other developers who are already skilled in the art of hyperacoustics.

Return to a list of some of Steve Mann inventions