I have developed a fuzzy rule based model for diagnosing cancer data (from microarray experiments).
Everyone says rule based models are transparent model, or understandable models, or interpretable models.
But I did some detailed investigation on the generated rules, not all of these rules seem to make sense. Some of the rules (rule sets) looks obvious against to (or strange to) some basic biological sense, but they still can give very good diagnosis results.
Are these rule still useful, even they against to common sense?
Verification and Validation is an essential stage of inferential outcome in any FL Expert System. Now going back from "very good diagnosis results" in reverse engineering manner, you need to either update/modify/redefine the membership functions or remap-the physical/biological phenomena which is not obvious/common sense.
For my person experience so far, microarray data are particularly noisy data, these incorrect rules may mainly came from the noise.
Before we test the models on the unseen data, a verification process could be adopted first.
Move to the verification stage, we may meet the following situations:
1. rules according with the common sense or current biological research outcome->useful rules?
2. rules without any biological meaning so far (looks strange)-->reserved rules?
3. rules against to the common sense or current biological research outcome-> fake rules?
A. Because of many new technologies has been developed in recently years, modern biological research grows rapidly. But so far, there exists no `closed theory' of molecular interactions. Hypotheses that are based only on theoretical arguments have been rare in biology. Most biological research hypotheses have been derived from previous experimental fi ndings.
For this reasons, reserved rules could be considered to kept in the rule bases.
B. If all fake rules came from the data noise, or model/classifier over-fitted (or model ill-trained) . I think these rules should be deleted from the rule bases, because they fit the unreal data (noisy data)? but,
1. if the noise of the training data and unseen testing data are under the same level (from the similar experiments), deleting these fake rules may brings bad diagnosis results (testing results). for classification purpose, keep it?
2. if the training data and the testing data are different, for example, from two different labs, or under different experimental conditions, deleting these fake rules may brings better diagnosis results (testing results).
--------------------------------------------------------------
are these understanding correct?
---------------------------------------------------------------
Reference: Cancer Diagnostics With DNA Microarrays by Steen Knudsen (2007)
http://www.amazon.com/Cancer-Diagnostics-Microarrays-Steen-Knudsen/dp/0471784079
Yes, I agree. I had observed the limitations due to absence of non- existence of `closed theory' of molecular interactions. However, I feel that reserved rules and fake rules could serve as a data set for updating/tuning the membership function representation for underlying fuzzy sets; especially representations using "hedges". In fuzzy systems -the so called fake rules- can be put to use in the context of Rough Set techniques of Lower and upper approximations. (See ROSE Software for data exploration- for quick reference http://idss.cs.put.poznan.pl/site/rose.html)
By defining membership function & representing fuzzy set then ask question(rule based)
- Badges
- Science topic
- Similar topics
- Computer Science