I'm not responsible for the paper you refer to.

However, I do have some knowledge on how

to combine uncertain knowledge about an

event. If you are interested in a continuation

you may contact me on e-mail [email protected].

Dear Michael,

you can combine all IV in one model if you use the SEM command in STATA.

in the paper you refer to a is the normalisation constant, i.e. sum over all Ck given Dns

So you need to think how many alternative models you have (which is the point of Bayesian statistics).

But the main issue here is: do you want to know how much variance your IV explain? Why are you not using eta2 then? Or any other effect size measurement.

From what you wrote P(C) = .5 sounds like 50% chance (binary variable) to get C, and P(C|D1) does not change this probability, so D1 adds no information whereas P(C|D2) does, D2 raises the chance from 0.5 to 0.8

Translated to your verbal vs non-verbal description: if D1 is verbal then they do not effect the outcome of C but so does the non-verbal = D2 (or vice versa).

For a treatment of a different but related problem, see

"Coherent Combination of Experts' Opinions"

by

A. P. DAWID , M. H. DeGROOT and J. MORTERA.

Test (1995) Vol.4,No.2,pp.263-313.

Dear Michael,

May be you could check the chapter on "coherence variable" in the book titled "Bayesian programming" (http://www.crcpress.com/product/isbn/9781439880326). It explains a formal way to address your problem by adding an extra boolean variable.

Assuming that D1 and D2 are conditionally independent given C, posterior odds for C are prior odds (= 1) times likelihood ratio for C from D1 (=1) times likelihood ratio for C from D2 (= 4), which is equal to 4. Thus, posterior probability for C is 4/5, which is equal to 0.8. So yes, 0.8 makes more sense than 0.65.

You can confirm this by using a naive Bayes classifier, which allows for missing data, so you can have all independent variables in one model. If you don't get the same answer, then you should question the assumption that the two sets of independent variables are conditionally independent given the classifier variable.

@Alexander: seem to address more the distinction between reliability (how trustworthy) and precision (how much of an expert) which we have described in the spatial domain, i.e. Pfuhl et al. (2011) Precision and Reliability in Animal Navigation. Bulletin of Mathematical Biology. volum 73 (5).

Applied to Michaels data, one could state that the comeplte data set has twice the reliability of the incompelte data set (or any other value depending on how many data points are missing), then it is easy to calculate P(C|D1,D2) with our formula.

Thank you all for your responses. I am slowly going through the materials you've given me. Just a clarification, I am using random forest, support vector machines and ada to obtain the probabilities (basically using the same method with all these models). I am testing the efficiency of the models not based R^2 but based on a 10x 10-fold cross validation and obtaining recall, precision, f-measure etc. These machine learning algorithms do handle missing data by either omitting or using kmeans or something similar. But this does not work because I don't want them to exclude from my model cases where verbal data are not present. I just want to have the non-verbal and use the verbal as an additional for the cases that they exist. This way I want to measure if by this addition of verbal data the predictive ability of my model increases.

If I were to make the switch structural equation modeling or bayesian hierarchical modeling, do you know how do this methods fare compared to machine learning methods?

Bayesian statistics is applied probability theory. So you need to ask what is the joint probability of the variables that you will treat as random. It sounds like you have a regression in mind and that you are willing to treat the some variables as fixed, the independent variables, as fixed. given those What can you say about the joint probability distribution of the dependent variables, the Ys. Can you assume that they are independent of one another in your two groups? If so the likelihood is the product of the two likelihoods. But careful with missing data. A good book on missing data is "Statistical Analysis with Missing Data" by Little and Rubin.

All the best with this and have a happy holiday!

Michael,

Most of the answers thus far haven't dealt with the missing data issue. I agree with Charles Lawrence that you need to take missing data into account into your modeling exercise. First, are the data missing (completely) at random, or are there variables that account for whether the data are missing or not? Second, does "missingness" predict your DV?

You could create a dummy binary variable (missing vs not missing) to test this _and_ also to test whether missingness moderates the effects of the IVs for which you have complete data. It isn't clear from your description whether the IVs with missing data all are missing on the same cases, but if so then this dummy-variable approach would be easy to implement. It should then be possible to include the non-missing IVs in your model by fixing their coefficients at 0 conditional on the dummy variable taking the value "missing" and leaving the coefficients free to be estimated conditional on the dummy variable taking the value "not missing".

HTH,

--Mike

Another approach is Bayesian model averaging (BMA). A good web resource is

http://www.stat.washington.edu/raftery/Research/bma.html . Under BMA the predicted outcomes are based on both models weighted by their respective posterior probabilities.There is an R package BMA for doing the analyses.

I think when you apply results in Eq. (8), you have to be very cautious about a very important assumption they made, i.e., the two sets of data are conditionally indepdent.

Now, you are trying to combine the inference from two models but not two sets of data.

One way you can do is to use stochastic averaging:

P(C|D,M1,M2)=P(C|D,M1)*P(M1|D)+P(C|D,M2)*P(M2|D)

where: M1 and M2 are the two models (or model classes); D is the data; C is a variable you want to predict its probability

P(M1|D) is the evidence of model M1 and similar for M2.

Some clarification would help: (1) Are the underlying data from your 2 models independent? (2) Are the probabilities you specified "event probabilities", or probability tails, i.e. P-values?