Hello,
Does anyone have any idea about the efficient methods of extracting data like age, name, email and so on from unstructured data like texts?
Sometimes there is enough of a pattern that it can be imported into Excel. You may need to look for commas, tabs, or other characters used to delimit the fields. Otherwise, you'll need to resort to programming and write some parsing routines. If the data is in HTML or uses a "markup language", you may be able to use the tags to delimit the data. I've found that most records end with a carriage return character or carriage return/line feed character pair, so you can use this the get the individual records, and then further sub-parse them. The programming language you use will have string manipulation libraries or instructions that allow you to look for sub-strings and copy them to new strings.
Using commandline tools like sed and grep from linux OS will do that. You could install cygwin on windows and then use these same tools from the windows commandline.
Thanks Jacob and jack for the answers. I am wondering if I have a text like this: "He is 24 years old.", will still the proposed methods can recognize that 24 is the age of a person?
Excel VBA has a function IsNumeric(), and I'm sure other languages have similar functions. It looks at a string and returns a value (True, for this example) if the string could be converted to a number. Worst case, you need to write a function that looks for characters that are 0 through 9
Thanks Jack for the answer. I was thinking if there is a specific pattern recognition or data mining for doing that. I will go through excel VBA to check if it will work for me or not.
So does each file have more than one person in it? If so does name and email also occur in the same file.
grep '[0-9][0-9] years' testfile.txt >> resultfile.txt would get you all lines in the file with (2 digits, a space and the word year)
You can pipe | stuff from one command to another
grep '[0-9][0-9] years' testfile.txt | grep '[0-9][0-9]' >> resultfile.txt
If not, is each person's details in a single paragraph. You could break the paragraph up with awk or sed then grep the details out. The commandline on linux is so incredibly powerful
Thank you very much Jacob for the great feedback. I will go through your recommendation.
- Badges
- Science topic
- Similar topics
- Phytochemistry
- Extracts