Hello, everyone
I am interesting the Content Extraction from HTML web pages. Now I use the HTML tags for dividing the block of web page and use the tag-to-text ratio and anchor-text-to-text ratio and title density to extract main content. But all of HTML tags don't appropriate where content extraction. SO I want to know what tags are more accurate and more suitable for web page' cleaning? Thank You all...
Hai, Good Morning.
Your question is very interesting. For Content extration from HTML web pages is having: there are two issues which can be observed.
1. The source base of the content extraction. Typically, a content extraction method uses either the Document Object Model (DOM) representation or the plain HTML source code.
2. The general approach to do content extraction. Gottron divides the content extraction approach in two categories namely single document extraction and multiple document extraction.
Since our operational settings require that our content extraction module runs during data acquisition, we need a relatively lightweight and fast method to do extraction. In general, single document extraction is relatively faster rather than multiple document extraction because it only considers the document at hand during extraction without looking into other documents from the same host.
Kindly see the attachment for more information.
Thanks.
Hi,
Kindly see the following attachment also for your question. I hope, its very helpful to you. ok.
Have you checked out the Boilerpipe open source project? Maybe it can help you in the full text extraction from Html pages.
https://code.google.com/p/boilerpipe/
Thank you all!
I really want to know that I consider the content extraction based on Line-block concept. The line-block concept means that it will take from the start tag to the end tag. For example, ...,
..
and so on. But I am testing and writing the code for it, it is corrected for the correct HTML format file. If the HTML format is wrong such as the start tag includes but don't the end tag, the code shows the wrong answer and error. So how to handle these coding and what tags are only used for the content extraction?
try visual ping
https://visualping.io/
you can visually select what you want to extract
- Badges
- Science topic
- Similar topics
- Analytical Chemistry
- Extraction