Dear Almo, this is one of the superior papers on case based reasoning in terms of artificial intelligence and is evaluated in terms of percentage matching to the real outcome (spectral pattern or time series or decision etc)
http://www.idi.ntnu.no/emner/tdt4173/papers/Aamodt_1994_Case.pdf
My prediction is that this approach will also become popular in renewable energy systems automation and control approaches.
kind regards
Gerro
Evaluating textual CBR systems is difficult and there is currently no best way. Crude evaluation have used IR-based techniques such as precision, recall and f-measure where proposed textual solutions are broken into words (or n-grams) and seen as equivalent of documents in IR. Machine Translation evaluation measures such as BLEU and NIST can also be applied but give good correlation where multiple versions of the actual textual solution is available; see link below. You might want to look into the areas of natural language understanding and text summarisation for more evaluation ideas.
http://www.springerlink.com/content/12u62128v7663236/?p=d69f5af432d14a9090994c8124c87018&pi=3
Stefanie Br ¨uninghaus and Kevin D. Ashley in (Br¨uninghaus & Ashley
1998) evaluate Textual CBR Approaches and found accuracy as the evaluation measure, since in their domain, the non-assignment of categories is relevant.
For more check this link
http://www.google.com.ng/url?sa=t&rct=j&q=&esrc=s&source=web&cd=3&ved=0CCQQFjAC&url=http%3A%2F%2Fwww.aaai.org%2FPapers%2FWorkshops%2F1998%2FWS-98-12%2FWS98-12-007.pdf&ei=V3ZCVJDXNKre7AbX9IHQDg&usg=AFQjCNGZUqSOg9kIkllWgI5XW0eCCv9SkA
I agree that the evaluation measure might be a straight-forward accuracy calculation for TCBR scenarios with structured solutions. But this is not directly applicable to textual solutions e.g. FAQs! The earlier answer will be more useful for textual solutions.
- Badges
- Science topic
- Similar topics
- Computer Science and Engineering
- Artificial Intelligence