Hello Michael,

the question you ask raises some interesting questions. I'm not sure if I fully grasped the aims of your project (so apologies if my answer is a bit on the side), but I'll venture my thoughts (based on my own background in corpus linguistics).

First, as Kevin pointed out above, genres could probably play a role here, alongside factors such as register (e.g. degree of formality), publication channel (writers in various magazines or journals might be using in-house style guides that could potentially influence the ratio of words and sentences), as well as plain old individual author preferences. Personally, I would either make sure that the samples I compared were balanced with respect to these factors, or else simply include them explicitly as factors in my statistical model.

Second, I think it's important to define the level at which you're carrying out comparisons. I got the impression from one of your subsequent comments that you're aiming at comparing individual texts. Since you mention adjusting for language differences, I think this could well be handled differently depending on what the level of granularity of your study is (language vs. individual texts).

Third, it might be worthwhile exploring whether any differences in text length/density have a *practical* importance (as opposed to mere statistical significance). The easiest way of assessing this probably comparing effect sizes or correlation patterns for different factors.

Finally, at the end of the day I think your specific research questions will inform you about what makes sense. Here's an example from my own research, based on a paper I co-authored: we looked at sentence alignment in a parallel corpus of texts translated from Norwegian into Spanish (not exactly what you're doing, but close enough). We found that various length measures (sentences, number of words, ratios) could (for our purposes) be reliably reduced to a simplified binary opposition of short vs. long texts. Also, and perhaps somewhat surprising, we found that there were no signficant differences in text length between authors, genres, and translators. In the end, we decided that, although Spanish translations on average tended to be longer than the Norwegian originals, length was simply not an explanatory factor (for our purposes, it should be stressed). If you would like all the details you can access the full paper from my profile, or download it here: https://bells.uib.no/index.php/bells/issue/view/56/showToc (Jenset & Hareide, "A multidimensional approach to aligned sentences in translated text").

Good luck!

Christian thank you for the article.

My approach is simple if I strip it down to the basics. I have the same texts in Spanish and in English. I need to compare these specific texts and not any other corpora. I want to measure the statistical difference in words and sentences. These two metrics are indicators of the content. Should the Spanish text have more sentences then it is likely that they contain more information than the English texts.

However, I have the suspicion that languages have a natural variation. You may need more sentences to describe the same thing in Spanish. So, I suspect that I need to take into account this variance. To do that I can obtain any text in two languages that is translated faithfully and get the number of words and sentences. These are going to be two numbers for each language (words and sentences).

The article that you gave me seems to suggest that i don't need to control for the natural variance between languages however that doesn't sound right. Maybe, I misinterpreted something.

There exists translation transformations. How do you translate realia - culture-specific things, for example?.. Sometimes you need to use 2 sentences in order to translate 1 word, otherwise your reader won't get you. Are you a translator?

Hello, Michael. I have no experience with this type of analysis (i.e. the punkt tokenizer). However, I can agree with your observation that in some instances it may take more modifiers and/or a different word order to communicate the same message in Spanish as opposed to English. So, I don't know if you can use a word count and this type of parsing to answer questions about variation between Spanish and English. Dr. Henriette Langdon has done some backward translations for me from English to Spanish and back to English. I would definitely suggest you google and pose this question to someone who is bi or multilingual, with a linguistics background and background with the software you are using. You raise an interesting question and I will monitor updates to this discussion to learn more.

Dear Tsikerdekis,

Regarding natural variance, it can be overlooked since the sample is located out of control limit. Translators may tanslate the same texts differently

As far as sentence count concerns, i suggest you count clauses as a unit of translation.

I think,you need to add some semantic meaning to the sentences or know the context of the sentence ,rather than to exact translate the words in sentences, this similar work when we doing it for sentences in Arabic and English Language..

Good luck

Have you tried using CLAWS Tagset (free trial is available online) or AntConc software? I used it for my analysis of English language used by engineers. However, I am not sure whether these software supports Spanish language or not? A trial run will be good.

Thank you all for your responses.

The texts that I am testing are not translations (e.g., the English text was not translated to Spanish). These are texts that were developed by different people interdependently about the same topics. Some ideas are likely to have been borrowed between the two while others omitted.

The sentences and words will allow me to see if the text have an equal length of "information" or not. My control text can be anything that truly has been translated faithfully, I used Harry Potter books but the Bible or Quran could have worked just as effectively.

I could add semantic meaning to each sentence or do a coding analysis although that would take a looong time, so I am trying to find indicators (no. words of sentences) that may hinder what's happening. Then, I can find representative examples to demonstrate when information has been omitted from one text.

I have no idea about a metric that would pass peer-review however.

For example, I could use that metric to transform the second column:

New metric = no. of words in the text of interest for the English language / no. words in the control text for the English language

Then do the same for Spanish.

Would that classify as an adjustment for the variation?

PS: I though about using concepts such as name entities but the issue is that you need good corpora that can do that and between languages reliability drops. Especially for other languages with smaller number of native speakers.

Use the variationist method, created by William Labov (1966)

Hiram thank you very much! Looks like there are some ideas there. I am going to have a look and report here the results.

For anyone else interested I found one article at Labov's webpage describing variation and quantitative analysis:

http://www.ling.upenn.edu/~wlabov/Papers/QRL.pdf

Michael, i would argue that just looking at the number of sentences and words is insufficient to detect, if the one side has more information than the other side in you pair of texts. I have observed substantial differences in the ratio of words L1 to words L2 in parallel corpora, where we assume that the content is equivalent. E.g. in one corpus the ratio was 0.95, in another 0.98, and so on all the way to 1.08. And these were all fairly large corpora. I would expect that when you have 10 Spanish books and their English translations, you will see a similarly large variation. Of course, you can always calculate a mean and a standard deviation. But for a new pair of documents you can not decide, if the two sides have the same or different amounts of information.

Actually, are you interested in comparing the amount of information, or if two texts have the same information? For the second one, my suggestion is to look at what people have done on comparable corpora: identifying and then aligning comparable corpora. There, the idea is to detect the parallel parts in a bilingual corpus, which may have both parallel and non-parallel segments. One way to approach this is to learn an alignment model (essentially a dictionary) on an available parallel corpus and then do, for example, cross-lingual information retrieval. An alternative approach is to translate one side into the other and then use machine translation evaluation metrics to identify the closed matching sentence, again to detect the (mostly) parallel parts.

Stephan, I agree and see your point. Looking at just sentences it is shallow. Two sentences can convey the same information as one at times and so that won't give me an objective measurement. Translating the text and then comparing them would be an options however, it would take me a long time to do so. I prefer to use quantitative established methods for identifying text features and then pick parts from the text which represent what the quantiative result suggest.

Christian's idea sounds great on that regard. I've searched already for information density but the consensus on what it is and how it is been calculated varies. One text suggest Lexical Density (lexical words per clause) (http://livingknowledge.europarchive.org/images/publications/teich-fankhauser-final09.pdf) while another suggests Sentence Length (http://www.corpora.uni-hamburg.de/gscl2011/downloads/Program_CL-TS-MT_with_Abstracts.pdf#page=9). If there is a better formula could you point to the exact article please? :-)

The issue is that I am not doing research in linguistics, I just want to use techniques so that I can assert that there is a difference between the text and identify why that is (connect it statistically to different factors).

Lexical density is hard to calculate without a tagged corpus and it would take me forever to tag each word in the texts so that I can obtain just the lexical words. On the other hand, sentence length is a much easier metric to calculate but as an information density measure I don't think it works. To be honest, what I seek is more like "information volume" than density. So what I want is to know how much text is there and how dense it is. In this regard, I can see two potential metrics that would satisfy this condition:

- number of sentences * avg sentence length

So if I have two texts with 100 sentences but one has a lesser sentence length it would mean that probably there is less volume of information there.

- number of unique words may be another alternative. It's simple but it is being incorporated in metrics such as lexical diversity.

For both of these metrics I would have to use already established texts (Bible, Quran, Little Prince, etc.) to produce my control variables and then I can adjust the final result by division.

For example:

Lets say that the information volume = number of sentences * avg sentence length, for my spanish and english texts is 100 and 100 respectively. I calculate the same variable on my control texts that are in theory carbon copies and just translated and I get a score of 50 for Spanish and 40 for English. In other words, the spanish language has a tendency to use more sentences*sentence length. By dividing, I will get a score of 2 for spanish and a score of 2.5 for English. Thus, while the initial scores were 100 for both languages, I control for the variation and the English text got a slightly higher score accounting for that variation.

I know that this way the final numbers cannot be interpreted directly but statistically I can then compare the values over multiple texts and see what happens between the two languages and their means.

What do you think of this method?

information density is a better measurement than counting words when dealing with more than one language. For example, in some languages, particularly the agglutinating languages, we add prefixes, suffixes, infixes, etc. to words as part of the grammar. The result is a single word with high information density. A single word in one language would translate into several words in another language, but the information density would be the same. It will be easier to compare information content across languages by counting words if the study were confined to the same language group, e.g. Spanish and Catalan, than it would be for English vs. Navajo, two languages in very different groups, each with very different grammar rules and information densities per word.

Concerning information density and word counting, you need to compare genres of text by subject area and understand what each culture considers verbose or appropriate/inappropriate in which settings. Without this background, you will not be able to do much accept guess.

Hello Kevin,

Great point on text genres. For example, poetry may be verbose just to get the meter and rhyme to match - poetic license, which we would not have in scientific prose, which aims to be as concise as possible with no concern to be poetic. The best way is to compare the translation of exactly the same text across different languages - prose or poetry, noting that this comparison will not be perfect because what is poetry in one language will be prose in another. If the text is the same, it will provide the best basis for comparison. This should be done with a regard to the strong domain of each language, if any. For example, a translation into English of a text in Italian about music is likely to have more words than the original text because music is a strong domain of Italian.

Hello Michael,

the question you ask raises some interesting questions. I'm not sure if I fully grasped the aims of your project (so apologies if my answer is a bit on the side), but I'll venture my thoughts (based on my own background in corpus linguistics).

First, as Kevin pointed out above, genres could probably play a role here, alongside factors such as register (e.g. degree of formality), publication channel (writers in various magazines or journals might be using in-house style guides that could potentially influence the ratio of words and sentences), as well as plain old individual author preferences. Personally, I would either make sure that the samples I compared were balanced with respect to these factors, or else simply include them explicitly as factors in my statistical model.

Second, I think it's important to define the level at which you're carrying out comparisons. I got the impression from one of your subsequent comments that you're aiming at comparing individual texts. Since you mention adjusting for language differences, I think this could well be handled differently depending on what the level of granularity of your study is (language vs. individual texts).

Third, it might be worthwhile exploring whether any differences in text length/density have a *practical* importance (as opposed to mere statistical significance). The easiest way of assessing this probably comparing effect sizes or correlation patterns for different factors.

Finally, at the end of the day I think your specific research questions will inform you about what makes sense. Here's an example from my own research, based on a paper I co-authored: we looked at sentence alignment in a parallel corpus of texts translated from Norwegian into Spanish (not exactly what you're doing, but close enough). We found that various length measures (sentences, number of words, ratios) could (for our purposes) be reliably reduced to a simplified binary opposition of short vs. long texts. Also, and perhaps somewhat surprising, we found that there were no signficant differences in text length between authors, genres, and translators. In the end, we decided that, although Spanish translations on average tended to be longer than the Norwegian originals, length was simply not an explanatory factor (for our purposes, it should be stressed). If you would like all the details you can access the full paper from my profile, or download it here: https://bells.uib.no/index.php/bells/issue/view/56/showToc (Jenset & Hareide, "A multidimensional approach to aligned sentences in translated text").

Good luck!

More specifically, the variationist comparative method. See Shana Poplack's work 1993, 2000, 2001, etc. Also see Miriam Meyerhoff 2009

Hello,

In simple way i'd count the variance in the alignment after translating from source to target, use algorithms that you might find in Koehn book, statistical translation.

http://www.statmt.org/book/