Hi,

Why would you transform data before using a RF: it is counter intuitive to do that and also not clear when you compute variable importance measures

RF are used mainly because they accept any coding, so you should just keep the original scale and coding

It is not really clear to me what your expected outcome is.

• What kind is your outcome?
• What package and software are you using?

Dear Stefano,

We found that some of our variable data is Non Normal Distribution so we did transformed them using asin and also log transformation before running Random Forest (RF) to rank the relative importance of each predictor

Then in the RF, we need the outcomes with Mean of squared residuals and %Variance explained by each response variables in each group of variables. I implemented a series of Random Forest models (Breiman, 2001) to examine the relative importance of the 10 explanatory variables. My Random Forest models’ specifications included 1000 trees (ntree=1000), and the number of explanatory variables per node set to three (i.e. the square-root of the available number of explanatory variables; Diaz-Uriarte and Alvarez de Andres (2006)). I assessed model performance using OOB (Out-of-bag error) and report the mean of squared residual and the percentage variance explained to indicate model precision. Predictors importance was assessed based on IncNodePurity. The Random Forest models were run using the “randomForest” in the statistical programming language R studio Version 1.1.463 (Liaw & Wiener, 2001).

If you back-transform your optimal forecasts obtained for the transformed series, you'll get optimal forecasts in terms of the median, see this paper for more explanations (page 3 of the preprint):

Chapter Forecast Error Measures: Critical Review and Practical Recommendations

If you want to evaluate accuracy after back-transformation, you need to use the out-of-sample MAE/AvgRelMAE, as described in the above paper. More details on the AvgRelMAE can be found here:

Article Measuring Forecasting Accuracy: The Case Of Judgmental Adjus...

Dear Bang Nguyen Tran :

Glad my suggestion helped. if you want to go deep into error metrics and loss functions, please let me recommend my PhD thesis (Davydenko, 2012):

Thesis Integration of judgmental and statistical approaches for dem...

in section 3.2.4. "Finding Optimal Point Forecasts and Estimating Systematic Errors" you'll find discussion of how to find optimal forecasts depending on the loss function. Basically, it has been well-known since Zellner (1971): if you want optimal estimates in terms of MAE, you need the median of forecast density, if you want optimal estimates in terms of MSE, you need the mean.

As we note in (Davydenko and Fildes, 2016), when back-transforming your forecasts, you'll only be able to get the median of forecast density (as it is invariant to such kind of transformations). So MAE becomes the best choice for accuracy evaluation.

If you want to average MAE across series, you'll need the AvgRelMAE proposed on page 63 of (Davydenko, 2012). This metric was later published in this IJF article (Davydenko and Fildes, 2013):

Article Measuring Forecasting Accuracy: The Case Of Judgmental Adjus...

The chapter I recommended (Davydenko and Fildes, 2016) is actually an improved version of the above paper.

So, as you can see, there's been a lot of research in this area, actually - choosing the right metric is not straightforward and different measures may lead to conflicting results, so it's important to have a really good and reliable approach here.

We've recently presented a new framework for forecast evaluation (Davydenko et al, 2020):

Conference Paper Data Formats and Visual Tools for Forecast Evaluation in Cyb...

See page 33 of (Davydenko et al, 2020) for notes on forecast evaluation after back-transformation. Importantly, when you're using MAE, it makes no sense to evaluate bias (see page 33). But on transformed data you can still compare bias across series using the AvgRelAME metric (proposed in Davydenko, 2012, p. 64). Generally, I would recommend the framework proposed in (Davydenko et al., 2020) for your task. It involves steps for visual analysis and formal statistical tests. Our new guide on this topics is about to be published soon.

Would appreciate it if you could cite the PhD thesis (Davydenko, 2012) in your research in the context of the choice of the error metric. Thanks.

Regards,

Andrey Davydenko