Hello Marc Champagne , interesting question. Let me answer in detail, in a structured but pedestrian manner:

(1) Waveform

First of all, we have to be clear about what we define as waveform: the sound produced is captured by microphones (or equivalently decoded from a received coded signal) and it is analysed as a signal s(t), where t is usually discrete, hence the function s(t) (for continuous time) becomes a sequence also called a time series.

This is called a speech signal. It is analysed in the time domain, in the frequency domain or any transform domain (wavelet, or other time-frequency representation)

Quantization and pattern recognition on di-phones, which are transitions between phonemes, can be performed. Other methods are possible.

This corresponds to an analogy for cooking: you are given a plate, with a cooked dish, and you are trying to get to the recipee of the cook: how the dish can be made...

(2) Analyse further speech production

Humans speak by shaping their mouth and vocal tract, and the airflow in these, etc... This is well described and understood, and medically used by speech therapists ("orthophoneticians") to help people overcome difficulties they may have in generating audible and understandable speech.

An other angle of analysis is the lip reading methodology used by people with hearing disabilities. This assumes a good visibility of the speaker: mouth, lips, eyes and other elements of body language which give cues. With this, you can develop an automated lip face and body-language reading system.

The speech production system of human can be modeled in flexibly deformable connected tubes, and the vocal tract mechanisms of air flow, such as the glottal closure (for which I have developed a detector, years ago).

This proactive modeling of speech production versus the "waveform based methods" correspond to the intelligent automatic piano (or other instrument) which plays the score you input into it versus the recorded playing as a waveform.

(3) Conclusion

Speech coding has made tremendous progress when it has started to combine analysis and synthesis (the method is called analysis by synthesis, and is used in codecs fro the era of CELP, code excited linear prediction) used in the first digital mobile communication system globally standardised: GSM (the codec was "RPE-LTP by Kroon Deprettere).

Now is maybe a time to revisit how combining production modeling with auditory recognition models could bring new systems of practical interest.

Does it help you?

Let me know

Here is an algorithm by Arjo Chakravarty that uses a measure of complexity of a signal to identify phonemes. It makes it possible to do speech recognition on an Arduino.

https://github.com/arjo129/uSpeech/wiki/How-%C2%B5Speech-Detects-Phonemes

https://github.com/arjo129/uSpeech

Regards,

Joachim

I really maybe should not answer this question because its full intent may be beyond what I can address. But, perhaps (indeed) just for humor, wouldn't an auto-telegraph supplemented with smiley faces, ETC. do most of that job? And, beyond that: we do have quite a lot of speech recognition capabilities; once the "wave forms" are so recognized, that could seemingly be turned into any form; what would be needed to supplement THAT 'recognition' is a dictionary (knowledgeable of contexts, and determining which/what context(s)); and a phrase dictionary determining w/r to a phrase, which may have several meanings, its use (meaning) in a particular context (or context of contexts); in addition, perhaps we then just need a way of processing emotional expression (distinctly NOT of a huge many types -- so not a big problem). [ P.S. A context analysis of subsequent sentence(s) may have to be done to see if initial context determinations may have to be altered. ]

My area is cognition and cognitive development (ontogeny) and empiricism which perhaps gives me some insight in addressing this Question : I have developed a system for coming to empirically understand thinking and concepts (all levels, including the most abstract) -- SEEING (literally) their beginnings, OR at their inceptions, as real-time, observables i.e., __AS__ CONCRETE OVERT BEHAVIOR PATTERNS (though the concepts may very much seemingly be inherently ACROSS-CONTEXTS -- because at least that IS the way they become most significantly (and, maybe even quickly) -- YET beginning as singular instances).

Hello Marc Champagne , interesting question. Let me answer in detail, in a structured but pedestrian manner:

(1) Waveform

First of all, we have to be clear about what we define as waveform: the sound produced is captured by microphones (or equivalently decoded from a received coded signal) and it is analysed as a signal s(t), where t is usually discrete, hence the function s(t) (for continuous time) becomes a sequence also called a time series.

This is called a speech signal. It is analysed in the time domain, in the frequency domain or any transform domain (wavelet, or other time-frequency representation)

Quantization and pattern recognition on di-phones, which are transitions between phonemes, can be performed. Other methods are possible.

This corresponds to an analogy for cooking: you are given a plate, with a cooked dish, and you are trying to get to the recipee of the cook: how the dish can be made...

(2) Analyse further speech production

Humans speak by shaping their mouth and vocal tract, and the airflow in these, etc... This is well described and understood, and medically used by speech therapists ("orthophoneticians") to help people overcome difficulties they may have in generating audible and understandable speech.

An other angle of analysis is the lip reading methodology used by people with hearing disabilities. This assumes a good visibility of the speaker: mouth, lips, eyes and other elements of body language which give cues. With this, you can develop an automated lip face and body-language reading system.

The speech production system of human can be modeled in flexibly deformable connected tubes, and the vocal tract mechanisms of air flow, such as the glottal closure (for which I have developed a detector, years ago).

This proactive modeling of speech production versus the "waveform based methods" correspond to the intelligent automatic piano (or other instrument) which plays the score you input into it versus the recorded playing as a waveform.

(3) Conclusion

Speech coding has made tremendous progress when it has started to combine analysis and synthesis (the method is called analysis by synthesis, and is used in codecs fro the era of CELP, code excited linear prediction) used in the first digital mobile communication system globally standardised: GSM (the codec was "RPE-LTP by Kroon Deprettere).

Now is maybe a time to revisit how combining production modeling with auditory recognition models could bring new systems of practical interest.

Does it help you?

Let me know

Hello, Thank you for your question.

You mention semiotics of knowledge. Do you mean tacit knowledge? In other words, the AI platform can interpret facial expressions or body language when a command or response is given without words? Or, for example, an AI platform can imply to somebody to infer how to swim or sing...maybe by demonstration? I am unclear, but your question raises even more questions.

I agree with dr.Renaud Di Francisco completely

Most likely this way: sound from the micro is sampled and subjected to a FFT. The spectrum changes in time, that gives a spectrogram. You can think of a spectrum as of a high-dimensional vector, and the spectrogram is a trajectory through this high-dimensional vector space. Each phoneme has optimal trajectories through this space, and with a Hidden Markov Model the observed spectra can be mapped to a likely state transition, which yields a sequence of phonemes. These phonemes can be parsed with NLP methods, e.g. Cocke-Younger-Kasami algorithm. Parsing reduces the sentence to a nonterminal symbol, which stands for a typical pattern, like "Alexa, buy {me} $number {of} $goods". Such patterns are associated with procedures and perform actions on remote computers.

Regards,

Joachim

You really need to define what you mean by waveform. If you mean any time series (scalar or vector) then it is hard to see how anything you do wouldn't fall into that category.

However, in theory you could build a set of event detectors (phonemes? allophones? automatically learned acoustic phenomena?) that work directly on analog or digital sound with only limited history, and produce a time series of bits (detected or not) and based on these alone recognize what was said.

That's the closest I can think of something that might be classified as not waveform-based.

Y(J)S

Dear All:

I sometimes post follow-ups to my own answers to alert any interested readers that it is not atypical for me to edit an Answer (often to make additions) for up to an hour after the initial posting. THIS is NOT the sole purpose of this present post. Rather, I would like to provide what may well be a useful synopsis.

You will note, in my original Answer, the repeated use of the word "context" AND, indeed, this is a (even THE) major point. Bringing a context(s) (sometimes, a situation(s) or a circumstance(s)) to another's mind (to the reader or listener's mind) is the POINT OF COMMUNICATION. You displace some of whatever is on the hearer's mind with what you are talking about; then further talk is to shift context(s) (within some larger context) more and more as you continue on -- elaborating for some reason, so the listener "sees" more and more to indicate your intention(s), goal(s) or at least your appreciations and/or proclivities. AND: The bottom line is: no other view of communication is consistent with empiricism (other supposed kinds of communication could not develop (Darwin)).

Communication is setting and shifting contexts for some purpose(s) (an organismic thing). This is what ultimately must be represented when translating wave form talk (OR when translating writings) to whatever different form; never losing sight of this is essential for AI (as is coming to a clear understanding of the emerging TRUE cognitive hierarchies during ontogeny -- everything must be essentially (if only partly) OF the organism (the context, in some way, clearly contributed to BY the proactive Subject organism)). The essays associated with my Ethogram Project describe the nature of a TRUE hierarchy (

Thank you for sharing this Question

Sound does not need conversion, it is already generated as waveforms , either as environmental air pressure waves or as electrical signals in microphones or as digital signals in CDs etc. Speech recognition software simply tries to identify the structure of the waves in temporal and frequency domains to find out how which parameters are used to encode information.