its really difficult to imagine, but cognitive skills and perceptional imagination are both two facts, which are very useful in identifying this phenomena

Your question appears to concern the semiotics of verbal language. But verbal language can also be written, and to believe that writing is but a metalanguage of spoken language results from a misconception. Written language is a semiotic system of its own, which does not necessarily depend on hearing. And, of course, "sign language" is a kind of writing.

[What a strange expression, "sign language"!!! Any language, including the spoken one, in a sense is a sign language!]

If you go to research the topic through Peircean perspective of Semiotics, you have some answers. What you hear is audisign. What you see is vidisign. Neurologically, the brain recruits your auditory cortex to other modes of perception in case the auditory stimuli is not available. This is called brain plasticity. The visual semiotic process has a process of its own. There are also other modes of cognition in which the tactisigns, haptisigns or kinestheticsigns are used. Infact the semiotics outside the auditory mode is very large if you go further on the Peircean model. Saussurean model has only a limited scope without the hearing mode of cognition.

I agree with Kandamkulathy Jijo . I think that  the Peircean model.offers very strong basis for the understanding of the communication between hearing impaired and hearing abe persons. It is a question of  the articulatation of two semiosis systems: different signs and interpretants. 

Thank you guys for your very informative inputs. If I'm  going to view it in semiotics, can I use the phenomenological approach as my research method?