secondary research
IBM
Time: April 19 – April 26
Brief: Design a way to counter some of the alienating effects of remote working online.
My group: Melanie, Nayla, Lea
After the feedback we decided to narrow down our focus to a specific topic. We thought about creating a 3D matrix that would record the movements of one person and translate them into a 3D matrix of another, allowing the person to remotely feel another. We also considered various avatars, virtual environments and so on, but the more we thought about it, the more we realised that we do not want to replace the human contact and that we should find something else to focus on.
The idea about the sound came to me as I was talking to my cousin who is one of the managers in a contemporary theatre in Ljubljana. She was talking about the struggle they were facing while holding international conferences with different artists online. She explained that during the usual in-person conferences attendees always managed to have lively debates, but ever since being transferred online, conversation has been much duller and more forced, due to an unnatural way of hearing and inability to interrupt the conversation or jump in.
Sounds presentation.
As I mentioned the idea of focusing on the sound, the whole group found the idea very intriguing. We decided to look at different aspects of sounds to see what our possibilities for this project were. Since the study of sound is a multidisciplinary field that examines the relationship between the physical world of acoustics and the perceptual world of hearing, Melanie delved into the theories of Social Presence, Mediated Communication and Cognitive Dissonance. The first focuses on the ability of communication media to transmit social cues and the differences in physical proximity inherent in using various media. Mediated Communication relates to the use of any technical medium for communication purposes and Cognitive Dissonance occurs when a person holds two contradicting beliefs and the need one has to change them until they become consistent.
Nayla dug into psychoacoustics and how sound induces emotions in people. Studies show that sound may provoke different emotional responses in different listeners due to individual differences. The sound can also be stimulating for the learning process and imagination. Lea researched how our hearing functions and how our brains process different sounds. She learned that since we are constantly picking up sounds from our environments, our brains will determine the important sounds and focus on them in order to save our bodies’ energy, as well as that hearing is the most sensitive sense to humans. She also came across the phenomenon called the Cocktail Party Effect, where the brain can tune in to a single sound amid background sounds. Then she also looked into 2D sound (our current teleconferencing system), where the sound is essentially flattened. When we hear a 2D sound, it takes us out of an environment and makes the environment feel unreal, or movie-like. This happens because humans localize sound naturally and use it for the orientation in 3D space.
Example of 3D sound.
I focused on the 3D sound which consists of audio effects that manipulate the sound produced by speakers and headphones in a way to recreate sound in a much more natural way, so that it is closer to what you might hear in real life. Although 3D sound is not a new technology, as it was introduced some time ago, it only came to life with the development of virtual reality and games. As I looked more into 3D sounds in teleworking, I came across Voxeet, an API that enables 3D sound in conference calls by the position of people’s icons on the screen. It is, regrettably, available only in some apps and is charged by the minute. I also discovered various articles on imagination and learning in correlation with sound. One of the studies conducted an experiment on blind and blindfolded children where they described the environment to children and then children drew a picture of it. First, they did it with a normal 2D sound and afterwards, they used a 3D sound. The conclusion was that the pictures done after the 3D sound showed a better use of imagination and memory. A similar study also showed better results in understanding and learning when 3D sound was used in comparison to 2D sound.
References
Bronkhorst, A. W. (2015) The cocktail-party problem revisited: early processing and selection of multi-talker speech. Attention, Perception, & Psychophysics, 77(5), 1465-1487.
Cobb, S. C. (2009) Social presence and online learning: A current view from a research perspective. Journal of Interactive Online Learning, 8(3).
Hobbs, D. and Armstrong, J. (1998) An experimental study of social and psychological aspects of teleworking. Facilities, 16(12/13), pp.366-371.
Hyder, M., Haun, M. and Hoene, C. (2010) Placing the participants of a spatial audio conference call. In 2010 7th IEEE Consumer Communications and Networking Conference (pp. 1-7). IEEE.
Lombard, M. and Ditton, T. (1997) At the heart of it all: The concept of presence. Journal of computer-mediated communication, 3(2), JCMC321.
Lumbreras, M. and Sánchez, J. (1999) Interactive 3D sound hyperstories for blind children. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems (pp. 318-325).
Sánchez, J. and Sáenz, M. (2005) 3D sound interactive environments for problem solving. In Proceedings of the 7th international ACM SIGACCESS conference on Computers and accessibility (pp. 173-179).
Study, B. S. C. and National Institutes of Health. (2007). Information about Hearing, Communication, and Understanding. In NIH Curriculum Supplement Series. National Institutes of Health (US).
