I want to build a corpus to test a language identification system. Can you suggest links to collect textual data for these languages transcribed in Arabic characters :
Arabic
Pashto
Balochi
Kashmiri
Kurdish
Punjabi
Persian
Uighur
Urdu
Sindhi
Malay
Hello there,
I can't really help you with the others, but Arabic is very rich in corpora.
Just a quick SourceForge search will throw up dozens of results. https://sourceforge.net/directory/os:windows/?q=arabic+corpus
If you make a bit clearer what genres you want in your corpus, I can be of more help, but there should be a few there to get you started.
Dear Sadik,
Sketch Engine contains corpora for Arabic, Punjabi, Persian, Urdu and Malay. You can prepare corpora for the remaining languages yourself using the WebBootCAT functionality in Sketch Engine. In case of any questions, just ask the support team of Sketch Engine at [email protected] (they could also do that job for you, but there would be some costs involved).
Best regards,
Milos Jakubicek
http://www.sketchengine.co.uk
https://www.sketchengine.co.uk/quick-start-guide/create-your-corpus-lesson-4/
The languages listed are the few languages that still use the Arabic script. Formerly, however, the Arabic script was used for writing many other languages, including Turkish, Albanian, Bosnian, Spanish, Swahili and Chinese. It is likely that including such diverse languages may add to the value of the project.
- Badges
- Science topic