Hi Krzysztof,

You need to represent both, query and documents in the same internal representation. You could use Vector Space Model, a model very used and tested into information retrieval systems. So, I think that a easy solution for your scenery could be the follow:

The documents (articles) could be condensed via a keyword extraction algorithm. In this sense, I propose you KEA [1]. Then, the keywords extracted could be stored for every single article in the database. When an user formule a keyword-based query, you need match the query with keywords stored in the database. This is a simple idea, but should work.

Best regards.

[1] http://www.nzdl.org/Kea/

Hi,

This is an information retrieval task. I suggest you to read the book by Manning (http://nlp.stanford.edu/IR-book/).

One of the simplest (but also efficient and effective) way is, as Yusniel already pointed out, to express both the documents and the query in the vector space model (https://en.wikipedia.org/wiki/Vector_space_model)

1) Pre-process your documents: remove stopwords and do stemming (and other processes you can find in the book)

2) Extract the vocabulary from the documents (all the remaining unique words)

3) Index the documents using the extracted vocabulary to create a term-document matrix, where terms(words) are in rows and document in columns. The most easy is to count the frequency of each word in each document: https://en.wikipedia.org/wiki/Document-term_matrix

4) Process the query with the vocabulary you extracted from the documents to form a query vector, also counting the number of times a word inside the query (usually they appear only once)

5) Compute the cosine similarity between your query vector and each one of the document vectors.  https://en.wikipedia.org/wiki/Cosine_similarity

6) Sort the similarities in descending order and select the top k documents as the more relevant for your query.

You can perform several other steps between the steps described above to improve the retrieval performance, such as extract n-grams instead of individual words, weighting the words in a different way than the frequency (e.g. term-frequency-inverse-document-frequency), vector normalization, Latent Semantic Indexing, etc.

In Python you can use the nltk package to do most of this things. Here is a webpage where they show how to do it:

http://stackoverflow.com/questions/12118720/python-tf-idf-cosine-to-find-document-similarity

Finally, there are other models different from the vector space, such as the probabilistic retrieval.

For searching purposes I use a semantic network.

When I search for protein, also documents are

found that contain amino acid, peptide, enzyme,

oligomer, ribosome... The semantic network does

not only suggest alternative words, it also

provides a semantic distance so that relevance

can be computed. This distance is gotten by

random walk and taking the logarithm of

visit count. There's a catch: all documents

that come into question must be loaded and

be searched through word by word. A serious

restriction for use in networks,

but on a local hard disk no problem.

Regards,

Joachim