Hello Community,
I'm not very familiar with these kind of tasks and my web development skills could also be better. If anyone is familiar with how to get the source code of an external web site using jquery, javascript, ajax or whatever it would be very cool! :-)
The situation is as follows: I have a list of URLs that were built using a CMS. To the CMS, however, I have no access whatsoever. The previous task was to move all the images by Drag&Drop from an old destination to a new one. Since I'm sure that a lot of images were skipped, my idea was to search the source codes for each occurance of "" appears. Then within each of these substrings I would search and test whether the new path is missing, and if so, I would list the image name and the page it appears on. That would facilitate the task of checking for forgotten images a big time. My idea was to obtain the source code of these URLs as a sting-variable and fill it into a text area within an html form. Then, once one site was completed, it would do the same for the next page, and so on. The problem with this method (php) is that it requires that the site is refreshed after loading each source code. Maybe some javascript method would be faster?!
I would also appreciate other, maybe less awkward ideas. Also if anyone could hint me to some code snippets I would be very happy.
thanks in advance,
Martin
There is a C# library for this kind of problems. It's called HTML Agility Pack. Check the references.
http://stackoverflow.com/questions/846994/how-to-use-html-agility-pack
http://www.c-sharpcorner.com/UploadFile/9b86d4/getting-started-with-html-agility-pack/
http://www.mikesdotnetting.com/article/273/using-the-htmlagilitypack-to-parse-html-in-asp-net
one tip, especially if you go to stackoverflow: developers probably won't consider the URLs or tags you are looking for to be 'source code', so that term will confuse them. If I understand you correctly, you want to have a piece of software visit a bunch of web pages, as if you were browsing them, and extract a list of tags for display or further processing. If that's correct, it's usually called 'web scraping'.
https://en.wikipedia.org/wiki/Web_scraping
If you have a bunch of URLs and you have PHP, then a quick 10 line script will give a simple solution.
- Loop through the URLs
- Use file_get_contents to read each webpage into a string (the scrape!)
- Use preg_match_all to seek the tags within the string (pattern matching)
- Display / log matches
- Continue loop
This may need to more "stuff/tweaking" depending on the scale of the problem (how many URLs have you to got to check?)
Here is a simple code snippet for exactly the task you are looking for without the need to loop through individual tags.
- Badges
- Science topic
- Similar topics
- Computer Science and Engineering
- Programming Languages
- Python