Suppose 2 methods are shown to do sliding mode control of a given system. Please tell me if we can compare chattering in both by MATLAB or some other way
I suggest you follow https://books.google.com.au/books?hl=en&lr=&id=_EeES0_bR0wC&oi=fnd&pg=PR3&dq=overview+on+sliding+mode+control+in+quantitative+way+physically&ots=oXvF7vHOiY&sig=OYDBph2YbyBxGmHB2eI5LmaLoC4#v=onepage&q&f=false
besides the suggestions already given chattering can be quantified computing RMS values over a fixed time interval once steady state chattering has developed
This approach is also used quite often for quantifying jitter in high precision mechanical systems with gearboxes, such as in telescopes, surgery robotics, instrumentation
I suggest you use the control total variation and chattering indicator for quantifying the chattering effect and to measure the smoothness of the control signal. you can see also the following papers:
Article Adaptive high‐order sliding mode controller‐observer for MIM...
Article Adaptive High Order Sliding Mode Controller/ Observer based ...