Accessible web for deaf and hard of hearing with transparent multimodal sign language interpreter module : doctor of philosophy thesis
Dostopnejši splet za gluhe in naglušne s pomočjo transparentnega večmodalnega predvajalnika znakovnega jezika s podnapisi : doktorska disertacija

english

doktorska dela

2011

Univerza v Mariboru, Fakulteta za elektrotehniko računalništvo in informatiko

World Wide Web is becoming increasingly necessary for everybody regardless of age, gender, culture, health and individual disabilities. Unfortunately, the information on the Web is still not accessible to deaf and hard of hearing Webusers since these people require translations of written forms into their first language: sign language, which is based on facial expressions, hands andbody movements and has its own linguistic structure. This thesis introducesa possible solution (method) for providing accessible information tothe deaf and hard of hearing on the Web. The Sign Language Interpreter Module (SLI Module) method combines three different types of modalities: visual, auditory and kinesthetic. The visual modality is represented by the video of a sign language interpreter with the addition of subtitles. The auditory modality is reflected as speech and the kinesthetic modality is defined as user interaction; activating sign language videos on demand. In comparison to previous technological Web solutions, the innovation of the proposed SLI Module method is that it envelops various modalities for delivering written information. The presentation of this information as sign language videos with subtitles can be easily integrated into original Web sites, while fully preserving the layout's structure. This is possible due to the use of Web specifications such as HTML for layout presentation and the ubiquitous use of JavaScript scripting language for interaction. In addition, the method uses external W3C Timed Text format for subtitling. The analysis ofWeb Content Accessibility Guidelines 2.0 recommendations showed that these recommendations are still too general and inadequate for deaf people who use sign language as their first language. For this reason, the thesis argues thatthe main problem of the Web accessibility guidelines is that they focus mainly on functionalities instead of user experience. In the thesis, the reasons for selecting sign language videos over synthetic signing, such as avatars, are addressed. The process of producing sign language videos is described in depth. The thesis analyses present Web dictionaries on the Web and introduces motivations factors and development of a Web glossary on demand(SLI Glossary) using the SLI Module method. Additionally, this approach can be used for monitoring Web users? activities and can also be used as an indicator for further sign language videos recordings. The SLI Glossary methoddoes not represent a text-to-sign-language automatic translator, but a way to enable single word sign language translations to original Web sites. Based on the experience obtained from the evaluation studies, the SLI Module method is gaining wide acceptance within the Deaf community as it assists themto get the written form of information by using a multimodal information retrieval. Finally, the SLI Module follows worldwide declarations and legislations regarding human rights on equal access to information, since it facilitates sign language as a primary communication channel. The goal of the thesis is also to start an initiative for Slovenian sign language recognition and increase people's awareness about its usage.

07.08.2023