What would be the most suitable machine learning approach to classify a web site? Yes, we can apply text mining concepts on web content but the problem is that the content is not only in a single language, it also has some images and video content too. The other thing is that web content is more semi-structured, unlike the simple text content of any document.
Webpages can be categorized using users of a search engine.
- Acquisition of categories
These can be gotten from the search phrases, mostly one or
two keywords. If enough people enter the same search phrase,
it deserves a new category. (Can also be used for keyword
autocompletion like in Google)
- Spy on the users' click haviour.
If a webpage is in a search result list and a user clicks on the
corresponding link, this increases the probability that the
webpage is in the category associated with the search phrase.
It may also increase the rank of the webpage in relation to
the category. Clicks can be detected with JavaScript.
- Category disambiguation.
If people search for "wolf" as animal and "wolf" as surname,
this will lead to distinct clusters of clicked webpages, which
can be detected e.g. with k-means clustering. Then split
the category into wolf_animal and wolf_surname.
- Rank rotation
In order to obtain a categorization as above for lower ranked
webpages, occasionally increase their rank temporarily,
so that the may appear among the first few result pages.
- Delayed click analysis
If a user enters a search phrase P1, which corresponds to
a category, and he doesn't find it at the first try, the user
will probably find the desired webpage by subsequent
searches, maybe a more complicated search phrase P2,
which does not deserve a category. In this case the
webpage should be assigned the category for P1.
Regards,
Joachim
Webpages can be categorized using users of a search engine.
- Acquisition of categories
These can be gotten from the search phrases, mostly one or
two keywords. If enough people enter the same search phrase,
it deserves a new category. (Can also be used for keyword
autocompletion like in Google)
- Spy on the users' click haviour.
If a webpage is in a search result list and a user clicks on the
corresponding link, this increases the probability that the
webpage is in the category associated with the search phrase.
It may also increase the rank of the webpage in relation to
the category. Clicks can be detected with JavaScript.
- Category disambiguation.
If people search for "wolf" as animal and "wolf" as surname,
this will lead to distinct clusters of clicked webpages, which
can be detected e.g. with k-means clustering. Then split
the category into wolf_animal and wolf_surname.
- Rank rotation
In order to obtain a categorization as above for lower ranked
webpages, occasionally increase their rank temporarily,
so that the may appear among the first few result pages.
- Delayed click analysis
If a user enters a search phrase P1, which corresponds to
a category, and he doesn't find it at the first try, the user
will probably find the desired webpage by subsequent
searches, maybe a more complicated search phrase P2,
which does not deserve a category. In this case the
webpage should be assigned the category for P1.
Regards,
Joachim
Bhaumik,
I would say that there is no single machine learning algorithm but a mix of algorithm strategy that might be useful. you will need:
*NLP for semi structured text processing and meaning
* Image processing for image feature detection
*ML for image classification
Each of these can have one or more stages of processing and learning algorithm involved.
Using Human computing as Joachim suggested can also cover some of the shortcomings which you may get in some classification tasks such as in image classification or in generating some labels for supervised learning.
nice!
This information (feedback) can make the bigger artificial inteligence for me. Is it?
- Badges
- Science topic