Hello William, 

may I draw your attention to the Leipzig Corpora Collection http://corpora.informatik.uni-leipzig.de/ ?

There are corpora in more than 200 languages, incuding Persian/Farsi.

The corpora are sentence separated and sentence scrambled. Each sentence comes with a source URL. Some more data like word frequencies and word cooccurrences are included.

Smaller corpora (up to 1 million sentences) can be downloaded directly as plain text or MySQL database files at http://corpora.informatik.uni-leipzig.de/download.html

Larger coropra are available on request for non academic purposes without any charge.

Our largest Persian corpus consists of nearly 8 million sentences with about 200 million running words.

Please have a look at our data and feel free to contact me.

Uwe Quasthoff

Hi william

what words or register are  you specifically seeking for ?

Hello William, 

may I draw your attention to the Leipzig Corpora Collection http://corpora.informatik.uni-leipzig.de/ ?

There are corpora in more than 200 languages, incuding Persian/Farsi.

The corpora are sentence separated and sentence scrambled. Each sentence comes with a source URL. Some more data like word frequencies and word cooccurrences are included.

Smaller corpora (up to 1 million sentences) can be downloaded directly as plain text or MySQL database files at http://corpora.informatik.uni-leipzig.de/download.html

Larger coropra are available on request for non academic purposes without any charge.

Our largest Persian corpus consists of nearly 8 million sentences with about 200 million running words.

Please have a look at our data and feel free to contact me.

Uwe Quasthoff

Hi Rahimi,

  I'm not looking for a particular word--I'm looking for a general reference corpus, comparable to the FROWN of English, the British English FLOB, the Russian National Corpus, etc.  A general representation of language can serve at least as a point of comparison for a specifically constructed corpus.  So for example I might collect English language speech about "climate change," and then conduct keyness testing against the FROWN corpus.  Then I can make an empirical claim about the words that appear in the climate change corpus in conspicuous numbers--either conspicuously over-present or under-present.  I want to be able to conduct similar keyness testing on Farsi text collections.  I hope that makes sense.

Hello Uwe,

  thank you so much for responding.  Generally I have used representative national corpora such as the Lancaster Mandarin Chinese Corpus, FROWN English, oSAC Arabic, etc.  As I understand it, the Persian corpus I found on the download page (pes-ir "2011-web-100k") is somewhat different--it is a news/webcrawl of sentences.  But since the data I most often analyze is social media data, I think it would work fairly well.  Am I correct in thinking that the "pes-ir_web_2011_100K-sentences.txt" file is 100k Persian sentences randomly crawled from the web in 2011?

Sketch Engine also has monolingual Persian. Not downloadable but usable online.

Hello William,

the naming convention for corpora at http://corpora.informatik.uni-leipzig.de/download.html is as follows (explained with the example "pes-ir_web_2011_100K"):

pes: language code according to ISO 639-3

-ir: optional country code

_web: genre. news texts (_news, _newscrawl), Wikipedia (_wikipedia) or randomly crawled web pages (_web)

_2011: year of crawling

_100K: size in sentences, here: 100.000 sentences

Due to a change in the language codes you find older Persian/Farsi corpora under fas instead of pes.

Hi William,

Another Persian corpus that may be of interest to you is the Uppsala Persian Corpus

http://stp.lingfil.uu.se/~mojgan/UPC.html

William,

You can compile a web corpus (for almost any language) yourself by a tool called SpiderLing:

http://corpus.tools/

(There are also other useful corpus-related tools to be found at the site.)  I do not know for Farrsi, but for "large" langages you can expect to download as much as  billion tokens a day :-)

Best,

Vlado B, 10:25

Vladimír, I was not aware of that resource--thank you so much!

Thank you Lars for responding--that's very h helpful!

http://ece.ut.ac.ir/dbrg/bijankhan/

https://en.wikipedia.org/wiki/Hamshahri_Corpus

https://www.sketchengine.co.uk/documentation/wiki/Corpora/TalkBankPersian

https://en.wikipedia.org/wiki/Tehran_Monolingual_Corpus

http://link.springer.com/chapter/10.1007%2F978-3-642-19437-5_6#page-1