Explauralisation

Let's say a building contains a musical symphony that unfolds in the act of exploring architecture, how can we make this experience audible, i.e. auralise it, and make the active role of the beholder explicit?

E.A.R. Evaluation of Acoustics using Ray-tracing

E.A.R. is a computer program developed for this project to visualise and auralise the acoustics of a space using Ray-tracing. Ray-tracing is a way of following the sonic energy that is emitted through space. Rays are traced to generate an impulse response which represents the decay of sonic energy in time at the place of the listener. The impulse responses are generated for three separate frequency bands, because material and air absorption parameters are different in respect to different wavelengths.

Consider the following room in which a loudspeaker is separated from a listener by a wall that does not run up to the ceiling:

This evaluates to the following impulse response. Higher frequencies feature more specular reflections which results in some prominent peaks in the early reflections. Whereas for the lower frequencies, which are more diffusely reflected, shorter non-specular paths can be found, which results in more evenly distributed impulse response. Note that in this case transmittance is not considered, even though E.A.R is capable of modelling that.

The impulse response is calculated independently of the actual sound that is being emitted from the loudspeaker. Therefore, in order to make the acoustical implications of the architectural space audible, a sound file needs to be convoluted by the impulse response. For that matter that sound file has to be split into the three frequency bands using an equalizer algorithm. Ideally, the sound file would not contain any spatiality, so it should either be recorded in an anechoic chamber or be computer generated. A short click makes the impulse response itself distinctly audible, but the response can just as easily be added to, for example, the first measure of B.W.V. 999.



S.I.N.E. Storyboard of an Individual's Navigational Events

But architecture is not a mere collection of static volumes, its aural experience is enriched by its visitors engaging architecture, by the sounds that a building provokes from its beholders: footsteps, slamming doors and the occasional conversational mutter, making the building's inherent vocabulary of sounds explicit. Therefore, to conceive a musical representation of architectural space, it is just as important to consider the visitor's interactions and freedoms in experiencing architecture. To model this we take a path that is followed by a visitor and map the sonic events that the visitor invokes to a sound file, which in turn can be fed to E.A.R. to incorporate the acoustical and reverberant qualities of its surroundings. As an example consider this scene which leads to the following sound file.

Combining E.A.R. and S.I.N.E.

By navigating in space, the visitor functions as his own personal mixing device, mixing the configuration of sound sources around into a personal experience. In E.A.R. the active role of the visitor is approximated by generating impulse responses, as described above, for several distinct keyframes along the path of the visitor. Generating an impulse response for every frame of the sound file (usually 44100 hz) would simply be too time consuming. Therefore, these successive impulse responses need to be interpolated as the sound file progresses. This procedure is illustrated by the following scene, in which the visitor moves along a hallway towards music playing on another floor. This scheme yields a total impulse responses that needs to be generated of No Sound Sources × No Keyframes × No Frequency Bands.

The first row of impulse responses illustrates the sonic energy of the rightmost sound source at the location of the listener, as he moves to the right. As the listener approaches the sound source, the time, required for the first rays to hit the listener, diminishes, whereas the intensity of the impulse response increases. The second row illustrates the impulse responses that are generated for the footsteps of the listener. When the listener passes the curved roof, some distinct echoes are present, whereas under the lower ceiling the energy decreases more gradually. The audible result of this scene follows below.

Some notes on Stereo

By default E.A.R. models the listener as an omni-directional microphone. One could model a stereo listener by placing two omni-directional listeners in a head model, but E.A.R. also comes with the possibility to use a combined stereo listener. This listener models the inter-aural intensity differences (Δi) and inter-aural temporal differences (Δt) of incoming sound rays (vin) according to the following scheme. This model could be extended by incorporating Head Related Transfer Functions to model the actual pinna of the listener, which provides additional directionality cues for incoming sounds.

Screenshots

Because it is important for a widespread use of the tools in the field of architectural design and other disciplines, designing a user-friendly interface for E.A.R. and S.I.N.E. has been an important part of this project from the very start. E.A.R. and S.I.N.E. operate as add-ons for the free open source modelling package Blender. A schematic overview of the 3d viewport is listed on the right. The settings panel for render settings, material properties and object properties are listed below.

Visualising impulse responses

Instead of focussing on a moving configuration of sound sources and observers, one can also focus on the spatial variations amongst impulse responses. When aligning a collection of listeners in a grid, the fluctuations in perceived sonic energy can be mapped to a perpendicular offset of a wire mesh, to visualise the sounds waves as they move through space. This grid of listeners can be rendered simultaneously to re-use the calculations for the successive reflections of the sound sources.