It is well known that audio compression (e.g., MP3, AAC) usually processes the audio data frame-by-frame. However, I am curious about the feasibility of single frame based processing.
A commonly accepted notion is that frame based processing has time resolution of audio data while a single frame processing does not have. This is similar to comparing DFT and STFT.
However, why we need time resolution of audio signal during compression? For a given audio clip, its single frame FFT has super frequency resolution (huge points) and no time resolution. However, we can still calculate tonal and non-tonal elements, masking curves, and generate quantization index, etc. In this way, the modifications of any frequency bins will be reflected throughout time domain whenever this frequency appears along the time axis in the compressed time domain audio samples.
I personally do not see any potential problems of performing single frame compression as described above. The only problem I can imagine is in terms of hardware implementation for huge DCT points. But the computational complexity of FFT is O(nlogn) which approaches a linear function of n when n is large. Hence I do not see this as a big problem with the consideration of rapid developed computer capabilities.
Please help to point out my mistakes in the above statements.
Latency and memory requirements both then depend on the number of samples - for communications applications, latency is critical. Compression may be less effective as the transformed signal probably would no longer be sparse in the frequency domain (when averaged over the entire music or speech signal).
As I said, compression will probably be less effective as you are more likely to have energy present right across the frequency domain. Consider a short musical scale played on a piano. With framing, each frame is likely to only contain one significant frequency component, so the transformed frame will be very sparse. The entire signal, however, has energy present at many frequencies and so will not be sparse. This is even more the case for speech which is a time-varying mix of random and periodic components.
Also, using framing, it is trivial to add a checksum to each frame so that you can just drop any frames that are damaged. With a single block transform, you are at much greater risk of losing the entire block (you would probably need to add a lot of forward error correction).
Might also be worth mentioning the transient smearing issue. Energy in a transient portion of the audio signal can result in artefacts during the psychoacoustic compression stages. These can appear both before and after the transient in time, causing the well-known 'pre-echo' problem. MP3 compression alters frame sizes in response to signal content in an attempt to minimse this effect.
DRF's answer pretty much nails the fundamental from an information theory perspective. There's also the psychoacoustic perspective (which, not quite surprisingly says the same thing). The ear is something of a time/frequency analyzer. We don't hear the full spectrum of a whole piece of music at once but on the other extreme neither do we hear each sample individually. The spectral content of a part of the signal that's coming 1 second in the future has no bearing on what the signal sounds like now. So there's no point in taking it into account. If you want to base a compression algorithm on things the ear can and cannot discern its operation should roughly mirror that of the ear.
Can I suggest that the answers above all fit into the category of the statistics of the signal are changing a lot over time. There is no good way to model this as a block, so the work is done in frames that are assumed to have moderately constant statistics over their epoch. As mentioned, when there are rapid changes, then encoders often shorten the block or frame over which analysis is done to deal with these sudden changes.
You want to increase the size of the audio frame to contain the whole audio message. At first i think such question needs investigation to answer it.
But i think the frame size is determined by the latency in the transmission system.
It is required to carry out all the signal processing in such latency time. One other important point is that the transmission medium may be dynamic such that the decoding may be very complicated or even impossible if we increased the frame size.. I think the size of the frames are dictated more by the more complex decoding process that the coding process. Also, the transmission medium imposes restriction on the frame size..
Also the cost of the processing even if it increases only proportional to the size of the frame must be considered as one can not use powerful computing platforms for all communication equipment.
An another important point that audio signal is not a continuous signal in nature but contains interruptions.
I think the optimum frame size varies among applications.
Best wishes
- Badges
- Science topic