I have run a number of time-consuming complex simulations: -approx. 200 samples
-input parameters are not definable as simple numerical values (they are like: low, mid, high, etc.)
So, input parameters and output values are known but the model is so complex as it's like a black box.
Is it possible to sort the input parameters by their importance? If yes, How?
You need to clarify the question a bit more. Are your inputs drawn from a finite set of integers (which would have a 1-1 correspondence with a set such as {low, mid, high})? For inputs drawn from a continuous real interval and with a differentiable (even if unknown in closed form) output function, sensitivity analysis is relatively straightforward. In other cases, it tends to be far more problem specific.
The most important parameters will be the least certain ones and the ones that produce the most variation in output. You can bootstrap for sensitivity analysis of studies with small samples or use a univariate analysis, where you vary one parameter at a time and observe the output response
Quenouille's jackknife may be an alternative to Efron's bootstrap. Both have their merits.
Brief comparisons of bootstrap and jackknife are:
https://en.wikipedia.org/wiki/Resampling_(statistics)#Bootstrap_2
https://www.researchgate.net/post/What_is_the_main_difference_between_Jackknife_and_Bootstrap_method
If you send data to a remote website that runs an analysis and sends you numbers back, then the only thing you can do is contact the website administrators and hope they will be helpful.
If you write the program either in SAS, R, or a lower level language like C or Fortran, then the solution is relatively easy. However, in this case I don't see how the model could be unknown.
Can you rewrite the program? Use the "black box" for verification and use published literature for equations and theory. Then compare your new program to the black box output to see if you get the same answer.
Dear all, many thanks for your comments. After some studies I found that I should probably use a nonparametric test method. The DV values may not be normally distributed. And the IVs are mostly categorical/ordinal. There are more than 2 groups. And I also want to test interactions to highlight moderator variables. Are two factor kruskal wallis or Friedman test good choices? If so, which one's better, and do you know/recommend an excel or python free tool to perform it?
I guess I found the answer to my question here: http://depts.washington.edu/ilab/proj/art/index.html
Paresh Date Leendert Van der Tempel I would appreciate it if you could also reply to my new question at:
https://www.researchgate.net/post/How_do_you_define_a_dataset_for_glm_when_one_level_of_a_factor_is_meaningless_for_one_level_of_another_factor
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size