Machine learning methods, particularly Neural Network models, have been increasingly being adapted for forward and inverse problems in science and engineering communities.
When it comes to inverse problems, also known as model discovery, identifying the coefficients in PDEs or ODEs, the advantages of ML over established methods in the field of system identification are almost never discussed. In machine learning, as in system identification, the function space (polynomials, trigonometric functions, or derivatives) is pre-selected by the user. If so, why not solve for the coefficients using established regression techniques? Why use Machine Learning? For example, what is the role of Machine Learning in SINDy? What are the advantages of Machine Learning for inverse problems (or model discovery) if the user needs to select the function space apriori?
Hello, Chennakesava Kadapa Sir
I have used the package PySINDy to your question on the role of ML in SINDy. So what I saw from PySINDy was the functions (e.g., polynomials, trigonometric functions) that it has to represent potential terms in the governing equations.
So, in general, ML algorithms can help extract suitable terms by evaluating their significance and relevance based on the data type being used.
Can you please elaborate?
How does ML help in extracting the terms? What are the criteria for evaluation? Are some of the terms removed from the function space before the algorithm finds the coefficients?
If the neural network is a "black box" with no user visibility of discovery algorithms between input and output, then we cannot know how the extraction of differential coefficients by the neural net happens. Even if the user inputs a priori parameters to help set up the function space, there will be some uncertainty about the output, and it is prudent to check the ML's coefficients using other methods. Possible ML advantages would be speed and time saved in determining useful coefficients if other methods are cumbersome.
Chennakesava Kadapa, Sir, so PySINDY often starts with an overcomplete library of candidate functions that contain more terms than necessary. The algorithm iteratively refines the model by removing insignificant terms.
Regarding your question on criteria for evaluation:
SINDy employs a library of candidate functions representing potential terms in the governing equations. Machine learning algorithms help in selecting the most appropriate terms from this library. They typically involve assessing the significance and relevance of each term in capturing the system's dynamics. You can use the Bayesian Information criterion or any other cross-validation technique.
Shashwat Sourav Thanks for your response!
Could you point me to the papers on SINDy or in the libraries where the filtering of terms is done?
Hi,
As far as i know, original SINDy algorithm uses sparse (L1 constraints) regression to find parsimonous representation. If I am not mistaken, the sparse regression used is not based on the sequential thresholding because it is too unstable. See some studies of Brunton's lab.
Petr
I have been wondering why linear regression models are referred to as ML models. The only thing that comes to mind is how the model is calibrated. In ML, linear models are trained on training data and evaluation metrics are obtained on test data, whereas classical regression models use statistical inference. Incidentally, the evaluation metrics are different for ML and classical statistical models (e.g., cross validation score vs. chi stats. or other parametric inference).
Thanks, Petr Henyš!
Do we really need ML for sparse regression? I guess this is a more pertinent question.
We can use the same metrics for both methods.
Chennakesava Kadapa, Sir, you can look at this page - https://pysindy.readthedocs.io/en/latest/examples/10_PDEFIND_examples/example.html.
Paper - https://arxiv.org/pdf/2004.08424.pdf
- Badges
- Science topic
- Similar topics
- Computer Science and Engineering
- Programming Languages
- Python