Generally the information theoretic measures are based on probability distributions. These distributions take into account the individual values of the pixels. Is there any literature which tries to extend the probability measures to regions or patterns in the images.
Thanks in advance for your responses
There are two ways for handling the problem of patterns. One is to build a model of the pattern. This may be a probabilistic model like for instance a Gaussian noise model or a Markov random field model, or some signal processing like a wavelet transform. The other approach is to use a universal code like Lempel-Ziv that look for pattern and compress accordingly. Often the best is to combine the two approaches.
Hi, see the linked page on maximum entropy for some pointers regarding your question.
https://cmm.cit.nih.gov/maxent/letsgo.html
There are two ways for handling the problem of patterns. One is to build a model of the pattern. This may be a probabilistic model like for instance a Gaussian noise model or a Markov random field model, or some signal processing like a wavelet transform. The other approach is to use a universal code like Lempel-Ziv that look for pattern and compress accordingly. Often the best is to combine the two approaches.
This is an excellent question with many possible answers. In addition to the very helpful observations that have already been made, there is a bit more to add.
A study of randomness relative to local structures in images is given in
https://www.researchgate.net/publication/256721123_Local_Shanon_Entropy_Measure_with_Statistical_Tests_for_Image_Randomness
See Section 4.2, on page 9, where local randomness and patterns are considered.
Article Local Shanon Entropy Measure with Statistical Tests for Imag...
My first thoughts have to do with hierarchical levels: 1) the pixel level; 2) on the structural level (aggregated pixels); and 3) at the interpretive level (recognizable objects).
Maybe someone has viewed entropy/information at various levels?
@Antonio Lucero: Maybe someone has viewed entropy/information at various levels?
That is a very interesting question for followers of this thread. Entry level entropy is considered in
https://www.researchgate.net/publication/274891219_An_overview_of_the_emerging_HEVC_standard
This is not quite the same as viewing entropy at different levels such as the entropy of pixels vs. the entropy image regions such as those in a Voronoi tessellation of an image.
Image entropy based on thresholding (the focus of the following paper) is closer to what you have asked about.
https://www.researchgate.net/publication/227284783_Image_Segmentation_Using_Excess_Entropy
and
https://www.researchgate.net/publication/222143546_A_new_method_for_gray-level_picture_thresholding_using_the_entropy_of_the_histogram
Conference Paper An overview of the emerging HEVC standard
Article Image Segmentation Using Excess Entropy
Article A new method for gray-level picture thresholding using the e...
I am very thankful for all the answers and the resources above. I have also found a few approaches which I think can be useful.
The publication takes into account the structural representation of images (Laplacian eigenmaps) and the entropy measures to present a registration algorithm. This paper also talks of calculating the entropy over the histograms of gradients rather than intensities.
http://people.csail.mit.edu/wachinger/articles/MedIA-Structural-2012.pdf
Structural entropy can also be crucial:
https://www.researchgate.net/profile/Szilvia_Nagy/publication/248491545_Structural_entropy_in_detecting_background_patterns_of_AFM_images/links/5747685808ae14040e28d250.pdf
Moreover this paper about the applications of Quantum entropy from quantum information theory:
http://download.springer.com.uml.idm.oclc.org/static/pdf/305/art%253A10.1007%252Fs10946-016-9554-z.pdf?originUrl=http%3A%2F%2Flink.springer.com%2Farticle%2F10.1007%2Fs10946-016-9554-z&token2=exp=1483400593~acl=%2Fstatic%2Fpdf%2F305%2Fart%25253A10.1007%25252Fs10946-016-9554-z.pdf%3ForiginUrl%3Dhttp%253A%252F%252Flink.springer.com%252Farticle%252F10.1007%252Fs10946-016-9554-z*~hmac=6636df04a4c13c81b1fa305755d9f65e3af9bdec2b276806eeae0825a097a7d9
This paper is an interesting take on the extension of the Shannon's entropy to the images based on the local structure in the images and patterns.
https://arxiv.org/ftp/arxiv/papers/1609/1609.01117.pdf
Article Structural entropy in detecting background patterns of AFM images
Thank you for sharing the information that answers your question regarding entropy measurements considering local structure and image patterns.
I have nothing more to add to the information contained in the publications obtained from the links offered. Even you propose an interesting article with all the information you need to answer your main question.
regards
Jose Luis
It would seem to me that entropy at the pixel level has to do with the probability distribution of intensities. For example, a uniform intensity or uniform distribution of intensities would have high entropy; whereas discrete clumps of similar intensities would have lower entropy.
At the pixel neighborhood level, spatial inhomogeneities, such as exhibited by textures would have relatively low entropy which could be captured by "joint edge" measures of straightness, curvature, or corners, together with the associated edge gradient values.
At the recognizable object level, the entropy associated with the overlap of the distributions of classes of objects in extracted feature space would be high for features that do not separate the classes well, and would be low for features that do a good job of separating the classes of objects.
These would be three separate ways of characterizing the entropy in imagery. I doubt there would be any value in combining the entropy measures from these three hierarchical levels. You choose the level that pertains to your domain of interest and report your estimates of entropy at that level.
- Badges
- Science topic
- Similar topics
- Mathematics
- Applied Mathematics