My only experience in anything remotely like this is a bayesian brain.

In order to answer that, don't you have to answer how they can be applied? don't they have to be tested? By asking that, I am suggesting that there may be things that are represented that don't go according to common thinking. Is it possible that you can look at your results as part of a larger combinatorial space of answers, some known, some unknown, some true, some false? Can you characterize what the subspaces of a larger set of answers would be in addition to what you expect. Also, do any of the results look like they reflect missing data?

If this was bayesian data that was part of a bayesian brain, I would also ask if you could track how the associations between data that occurred during the storage process came together for a given result. Also, how is conflict resolution handled--this could account for some questionable answers--sometimes these systems, when not able to put out a correct answer, will return ANY answer--that is their nature. Perhaps you simply have to throw the bad answers away if they are the equivalent of human mistakes.

I hope this helps.

Karri

You could be overfitting your data - what i mean here is that you are capturing noise within the dataset (microarrays are particularly noisy) -

and your model is partly fitted to noise (and thus gene that don't make biological sense) - you test the model using the training data and it seems good - but the only true test of a model is unseen data.

You need to apply an external independent test set - or a cross validation routine.

Before we test on the unseen data, a verification process could be adopted first.

Move to the verification stage, we may meet the following situations:

1. rules according with the common sense or current biological research outcome->useful rules?

2. rules without any biological meaning so far (looks strange)-->reserved rules?

3. rules against to the common sense or current biological research outcome-> fake rules?

A. Because of many new technologies has been developed in recently years, modern biological research grows rapidly. But so far, there exists no `closed theory' of molecular interactions. Hypotheses that are based only on theoretical arguments have been rare in biology. Most biological research hypotheses have been derived from previous experimental fi ndings.

For this reasons, reserved rules could be considered to kept in the rule bases.

B. If all fake rules came from the data noise, or model/classifier over-fitted (or model ill-trained) . I think these rules should be deleted from the rule bases, because they fit the unreal data (noisy data)? but,

1. if the noise of the training data and unseen testing data are under the same level (from the similar experiments), deleting these fake rules may brings bad diagnosis results (testing results). for classification purpose, keep it?

2. if the training data and the testing data are different, for example, from two different labs, or under different experimental conditions, deleting these fake rules may brings better diagnosis results (testing results).

--------------------------------------------------------------

are these understanding correct?

---------------------------------------------------------------

Reference: Cancer Diagnostics With DNA Microarrays by Steen Knudsen (2007)

http://www.amazon.com/Cancer-Diagnostics-Microarrays-Steen-Knudsen/dp/0471784079

Definately a good start, IMO. I really liked the overfitting remark. I would have to test that to understand its implications, but there is always a possibility of this with rules-based systems.

When developing microarray based predictive models overfitting is a very real danger for the following reasons:

1) High levels of noise (especially from different patients in comparison to cell lines)

2) High dimensionality

3) Limited sample numbers in comparison to numbers of probesets

In my opinion you should not try to interpret the biological role of probesets until you have completed model construction and evaluation. Therefore your hypothesis is that an array corresponds to a particular phenotype/class and gene are selected on the basis of being able to partition the dataset as opposed to putative biological roles- avoid no bias in gene selection

Also when trying to determine functional roles the selected probesets bear in mind that gene expression analysis is only a single part of the story and genes don't have function, proteins do. For example, a gene may be transcribed and detected by your array however it could be targeted by a miRNA and the transcript degraded or translation repressed in which case its function is impaired.

I've recently published some work on using cross model validation to select genes from a microarray to construct a partial least squares model. you can read the paper here and I would also recommend some of the references contained within.

http://www.ncbi.nlm.nih.gov/pubmed/21115077

In brief the routine uses a double cross val loop. The inner loop runs the gene selection - when the stopping criteria is reached the subset of remaining genes is used to construct a model and the held out set is used to test the model. The accuracy and subset are stored, the algorithm iterates and the process continues until all outer splits have been tested against. I randomised the dataset and repeated a number of times to negate partitioning bias. At the end I then determined the frequency at which gene was selected over all algorithm iterations applied a threshold and trained a final model.

A good way of thinking about is that your validation both the gene selection and model itself. We use the best classification (or calibration in our case) as our objective, rather the frequency of selection for our genes when multiple data splits were used so as to avoid overfitting.

Only at that stage was literature mining used to determine potential functional role of any selected genes. There will also be some "sanity check" gene e.g. p53.

Hope this helps.

Colin