t-Distributed Stochastic Neighbor Embedding -- It seems the method reduces dimensions but insights might help. If I run t-SNE hundred times,why should I select the solution with the lowest KL divergence? Is there a theoretical guarantee?
# Machine Learning
# Data Visualization
You can look up these articles to clear your doubt regarding the lowest KL divergence:
https://www.analyticsvidhya.com/blog/2017/01/t-sne-implementation-r-python/
https://towardsdatascience.com/checking-out-dimensionality-reduction-with-t-sne-78309b2ca67d?gi=f02e543af122
See the link below
KL divergence is a metric to match distributions (higher dimensional space to lower dim. space where we find the mapping).
you can think of it as cross correlation for distributions, in a loose sense, and relative entropy/mutual information in information theory.
The Kullback–Leibler divergence can be interpreted as the expected extra message-length per datum that must be communicated if a code that is optimal for a given (wrong) distribution Q is used, compared to using a code based on the true distribution P. If P and Q are close, the difference in the coding lengths is small.
Mutual information is the Kullback–Leibler divergence of the product P(X)P(Y) of the two marginal probability distributions from the joint probability distribution P(X,Y) — i.e. the expected number of extra bits that must be transmitted to identify X and Y if they are coded using only their marginal distributions instead of the joint distribution. Equivalently, if the joint probability P(X,Y) is known, it is the expected number of extra bits that must on average be sent to identify Y if the value of X is not already known to the receiver.
http://jotterbach.github.io/2016/05/23/TSNE/
These video tutorials are very helpful to understand t-SNE insight.
https://www.youtube.com/watch?v=NEaUSP4YerM
https://www.youtube.com/watch?v=ohQXphVSEQM
https://www.youtube.com/watch?v=W-9L6v_rFIE
[1] van der Maaten, Laurens, and Geoffrey Hinton. Visualizing Data using t-SNE. J. Machine Learning Research 9, 2008, pp. 2579–2605.
[2] van der Maaten, Laurens. Barnes-Hut-SNE. arXiv:1301.3342 [cs.LG], 2013.
[3] Jacobs, Robert A. Increased rates of convergence through learning rate adaptation. Neural Networks 1.4, 1988, pp. 295–307.
[4] Wattenberg, Martin, Fernanda Viégas, and Ian Johnson. How to Use t-SNE Effectively. Distill, 2016. Available at How to Use t-SNE Effectively.
You can follow the below links
[1] https://www.analyticsvidhya.com/blog/2017/01/t-sne-implementation-r-python/
[2] https://indico.io/blog/visualizing-with-t-sne/
- Badges
- Science topic
- Similar topics
- Biological Science
- Kinesiology
- Running