SVD can be used to obtain low-rank approximations. The methodology for training DNN architectures is inspired by the fundamental property of SVD.
This may help you:
Conference Paper Singular Value Decomposition and Neural Networks
Sheeja Nirish
The singular value decomposition (SVD) method divides a matrix into singular vectors and singular values. The SVD is commonly utilized in the computation of various matrix operations, such as matrix inverse, as well as in machine learning as a data reduction approach.
Deep neural network (DNN) designs with varying degrees of complexity to approach the SVD. The training process for these DNN designs is motivated by SVD's essential characteristic, which is that it may be utilized to create low-rank approximations.
SVD or PCA, ICA, LDA etc., are good for conventional data preprocessing or linear transformation from complex/nonlinear system to a simple/Euclidean linear system. In deep learning, the alternative is Convolution over Neural Networks = CNN.
Dear Sheeja Nirish :
The singular value decomposition is a matrix factorization technique where a matrix is decomposed into a product of a square matrix, a diagonal (possible rectangular) matrix, and another square matrix. The application of SVD in DNN can be studied in the following articles:
https://link.springer.com/chapter/10.1007%2F978-3-030-30484-3_13
https://link.springer.com/chapter/10.1007/978-3-319-13560-1_65#:~:text=In%20this%20paper%2C%20we%20propose,change%20distributions%20of%20singular%20values.
Regards;
Ehtisham
Yes, it is still using such as 1) SVD-Based Screening for the Graphical Lasso IJCAI 2017 2) Spectral Clustering Using Multilinear SVD: Analysis, Approximations and Applications AAAI 2019 3)Tensor-SVD Based Graph Learning for Multi-View Subspace Clustering AAAI 2020
- Badges
- Science topic