In numerical analysis, a sparse matrix is a matrix in which most of the elements are zero. By contrast, if most of the elements are nonzero, then the matrix is considered dense. The fraction of zero elements over the total number of elements in a matrix is called the sparsity (density).

In general, the concept of “sparsity“ is defined and utilized in signals and systems domain and has been shown that many signals including images can be approximated by a linear combination of predefined signals.

Probably the best place to start learning about sparsity in image processing domain is its application in denoising. The main assumption is that the representation of the clean signal is sparse while the noise is dense over the predefined signals.

However, it has other applications in image processing like object/activity recognition, super pixel resolution, image segmentation, etc.

In numerical analysis, a sparse matrix is a matrix in which most of the elements are zero. By contrast, if most of the elements are nonzero, then the matrix is considered dense. The fraction of zero elements over the total number of elements in a matrix is called the sparsity (density).

I recommend you read the paper of John Wright: "Sparse Representation for Computer Vision and Pattern Recognition". 

Sparsity appears in the statistics of natural images in the wavelet domain. In that domain, most part of the energy of the image is concentred in a few coefficientes of  large amplitude. This is a good property for image compression (the information is contained in a small fraction of the image wavelet domain). Sparsity is also good for noise reduction, due to noise has its power equally distributed through the domain, and large amplitude signal coefficients can be easily recognized.