Tsukuba University of Technology is a unique national university corporation in Japan where only hearing or visually handicapped students learn.
We aim to find and evaluate a tool for training listening capabilities among hearing-impaired persons through exploring long-term effects and benefits of training through playful interaction with music.
With six years of experience with teaching computer music to hearing-impaired students at Tsukuba College of Technology (now National University Corporation, Tsukuba University of Technology), we believe that hearing-impaired people have an interest in music and anxiously hope to enjoy music. Thus, we set our goal to propose an assistant system for the hearing-impaired people to play music in an ensemble; an ensemble with both hearing-impaired people and people with hearing abilities.
So far we have conducted a series of experiments to understand how hearing-impaired people recognize emotion in music. We used drum performances that are played in an improvised style with an intention of an emotion, drawings intending to represent an emotion, moving pictures that are generated from drawings, and combinations of music and pictures as stimuli and asked people with and without hearing impairment to recognize emotions in the stimuli. Consistent results were: 1. There was no significant difference (5%) in recognizing emotion between hearing the two subject groups. 2. Fear was the least recognized emotion in any type of stimuli.
Music players generate an expressive performance by adding several types of deviations, a slightly quicker performance for example, to each note in a score. Since the process of adding deviations has not been explicitly described because it is difficult even for a player to specify them and the degree to which they are applied to each performed note, a musical performance has been explained in an abstract way and with subjective terms. Thus we have developed a system that helps users better understand and objectively assess a musical performance through simultaneous performance visualization and listening.
A single media can convey emotional intention--a musical player plays an instrument expressing one's emotional intention and listeners recognize the player's intention, for example. In this research, we try to understand 1. the relationships between human recognition in single media data and their physical features and 2. whether the combination of the multimedia data that intend the same emotion might enhance the intention compared to the single medium data.
Presentation slides at TMH Department of Speech, Music and Hearing in KTH Royal Institute of Technology and Centre of Excellence in Music Reseach in University of Jyväskylä and University of Helsinki.
4-3-15, Amakubo, Tsukuba, 305-8520 Japan