Dirty Tangible Interfaces

are a new concept in interaction design that forgoes the dogma of repeatability in favor of a richer and more complex experience, constantly evolving, irreversible & infinitely modifiable.

watch the demo on Vimeo »

We have built a prototype based on granular or liquid interaction material contained in a semi-transparent glass dish that is analyzed by video tracking for its 3D relief. This relief, and the dynamic changes applied to it by one or more players, are interpreted to drive concatenative sound synthesis, allowing them to mold landscapes and to plow through them in an inherently collaborative, expressive, and dynamic experience.

The prototype instrument consists of a dark box containing a video camera, a semi-transparent dish, optionally surrounded by LED lights to perform in darker conditions, and containing the interaction material.

Several kinds of interaction materials are possible: dry grains (plastic granulate, tapioca grains, peas, marbles), plastic balls, water with ink(s), ice cream, soft chocolate… depending on the desired expressivity, precision/randomness ratio, and inertia of movement wanted. The dish is captured by the camera from below in order to obtain a gray-scale image of the interaction material, that is then converted by the analysis software into a 3D depth image that activates the sound synthesis.

The grayscale camera image is the source for detection of various parameters:

• • density of interaction material
• • motion, quantity of movement applied to the interaction material
• • colors
• • shapes

For dynamic gestural control, our approach is to detect where there is movement in the material. Therefore, a moving blob detection is implemented by first creating the difference image from the blurred grayscale image camera, and then detecting the hierarchical blobs on this (see CCGL Dirty Tangible Interfaces screen print →).
This means that fast movements will automatically result in a deeper blob being detected, because the difference is greater.

In CCGL Dirty Tangible Interfaces, the user input (ie. typically the finger tips) is drawn as several paint brushes with their respective strengths being translated on a red-to-purple scale.

The applications that can use — and derive from the use — of DIRTI are very diverse. Here's an example: a simple terrain editor where holes can be dug, in order to reach water.

In the near future, we are planning to build several visualizations for performers as well as spectators. Here is a more focused view of the depth blobs, outlined and displayed in 3D according to their "grayscale value":