Hi,

just thought I'd add a couple more points to this very interesting discussion. I think that for a quantitative analysis some thought should be given to the degrees of freedom. If the quantitative analysis will look at, say, 8 features per sentence, then you'll need more data than if you'll simply be counting observations of the sentence type per time period. For my PhD thesis on English diachronic syntax (available from my profile), I used all the data I could suck out of the YCOE & Penn corpora, since I didn't know exactly how many features would be included in the final analysis

Also, since the topic deals with corpora and Latin diachronic grammar: you might want to check out Barbara McGillivray's 2013 book Methods in Latin Computational Linguistics (http://www.brill.com/methods-latin-computational-linguistics), which has a chapter discussing a corpus-based approached to diachronic changes in Latin argument structure.

I don't think there's a straightforward answer to this. Generally, I would say, bigger is better as long as the corpus is still a corpus (with accurate meta-information, and consistent makeup across periods), and not merely a collection of texts. Second, there's a huge difference between looking into high frequency items such as function words and doing lexical research. For research on, say, auxiliaries or determiners, a corpus of a couple of hundred thousand words per period will do, as long as each period contains a decent number of texts from different authors. For word order, an even smaller corpus may be sufficient. For research on lexical items or moderately frequent grammatical constructions, several millions of words per period are generally necessary to get a balanced picture. You could also use a big corpus and then take a random sample if the number of hits is too high. In that case though, it's good practice to first go through your full list of results and filter out noise before you take the random sample, as it's hard to know in advance what the noise will be like, and it's unlikely it will have a distribution similar to what you are looking for. So sometimes a corpus can also be too big to be practical, if it would require you to go through 100,000 hits before randomizing.

Generally, you would like to have at least a couple of hundreds of good hits – evenly distributed, so excluding idiosyncratic outliers – for each period in your corpus to make quantitative analysis reliable. But especially when dealing with older data this target is often hard to get.

Hello,

I would like to add a generic hint on a good introduction to the development of linguistic corpora (edited by Martin Wynne). The book is freely available at:

http://www.ahds.ac.uk/creating/guides/linguistic-corpora/

For your question, ch. 1 (by John Sinclair) might be particularly useful.

Thank you for suggestions! I am looking at word order distribution across Latin, however, my focus is narrowed to only infinitival transitive clauses, and it is true that this type is less frequent than declarative clauses.

To have a balanced chronological data, the next question is the chronological division - if I go with a century, ex, 1st, 2nd, 3rd etc - some documents are listed as, for example, at the end of the 1st and at the beginning of the 2nd. There is also a recent work by Pr.Gries and Pr. Hilpert about the clustering classification of chronological stages (The identification of stages in diachronic data: variability-based neighbor clustering). What are your thoughts?

Chronological ordering is another tricky matter. Ideally you need to know date of manuscript and date of original (unless you're working on autographs), and then decide on whether you consider the language as representative of the time of the original, or rather as representative of the time of the copyist. This decision too may depend on the kind of phenomenon you're looking into. If the word orders you are looking at remain grammatical across the time examined, and there's evidence that the copies are reliable (e.g. few obvious transcription errors such as word substitutions that don't make sense or skipped bits, haplology etc.), then it seems safe to take the date of the original as your point of departure (if this date is known for the majority of the texts). There will always be cases that are hard to classify. Either you make a blind decision, e.g. by attributing all texts to the middle of the range in which they are dated (systematically assigning century turns to either one period), or you decide case by case taking into account all of the available philological information. While the second method ideally results in better chronological ordering, there's the obvious risk of imposing your idea of chronology on the texts.

Also, you don't need to divide periods into centuries. Another sensible way of dividing periods would be along significant historical events, which probably had an impact on language use (e.g. in the case of increased social mobility) - provided, of course, that the periods are still of comparable length.

As regards the paper by Gries & Hilpert: the idea of bottom-up classification of stages is  a great idea, but its execution is far less obvious. In one of their case studies, they discuss the fate of the prefix ge-. Yet it appears that they haven't looked into all spellings of ge-, which has probably resulted in overestimating the steepness of its decline around 1150. While this is a problem that can in principle be avoided, others cannot. The corpora they use are the Penn Parsed Corpus of Middle English & the York Corpus of Old English. However, these corpora pose certain problems when comparing Old to Middle English, one of them being the fact that certain texts that 'look like Middle English' have been included in the Middle English corpus, which are generally dated though to the Old English period (for a detailed discussion see chapter 3 of my book Constructions and environments). Which leads to the main problem: the dating of the texts themselves needs to be fairly accurate to make bottom-up clustering more reliable or informative than top-down periodization. I doubt that this will be the case with your research.

Dear Peter,

Thank you for a great and helpful explanation!

Hi,

just thought I'd add a couple more points to this very interesting discussion. I think that for a quantitative analysis some thought should be given to the degrees of freedom. If the quantitative analysis will look at, say, 8 features per sentence, then you'll need more data than if you'll simply be counting observations of the sentence type per time period. For my PhD thesis on English diachronic syntax (available from my profile), I used all the data I could suck out of the YCOE & Penn corpora, since I didn't know exactly how many features would be included in the final analysis

Also, since the topic deals with corpora and Latin diachronic grammar: you might want to check out Barbara McGillivray's 2013 book Methods in Latin Computational Linguistics (http://www.brill.com/methods-latin-computational-linguistics), which has a chapter discussing a corpus-based approached to diachronic changes in Latin argument structure.

Hi,

I'd just like to add that my book also describes a corpus-driven valency lexicon for Latin verbs, which can be used to study word order patterns and morpho- syntactic patterns for verbs.

Hi.  -  Just a statistician who stumbled across this interesting discussion in an area unknown to me.  I note that the chapter that Manuel refers you to does have a section on size. I suppose in your field you may rely more on descriptive statistics than inferential statistics, but I'd think there would be room for both. There seem to be substantial practical problems to overcome that would make measurement error a problem. But still, some use of regression might be very interesting. Does the rise of one phenomenon 'predict' (a misleading term in statistics, not forecasting) a rise in another?  If so, what standard error do you have on that prediction, and what sample size do you need for a small enough standard error to be confident of your conclusion?  That sort of size problem does not seem to have been considered yet here.  - Thanks - Jim 

Thank you for very helpful comments! I was not aware of some of the references.

James, indeed, I am using logistical regression. In fact, there is another statistical question whether to treat chronological periods (centuries in this case) as a categorical or continuous predictor. It looks like (but I may be wrong) previous diachronic studies treated them as  categorical.

Thank you!

I'll give a shot at your last question, Olga (although I'm sure James also has something to say on that).

Coding the chronological periods as categorical or continuous depends partially on what kind of assumptions you can reasonably make about your chronological variable (e.g. assumptions about normality). I think it also depends on what makes sense in the context of your analysis: if coding the chronological variable as categorical, would you end up with hundreds of levels in that variable? You then need to ask whether that would be a problem or an advantage.

Your question also evokes another thought: should the chronological variable be treated as a predictor (i.e. a fixed effect) at all, or should it rather be a random effect? For my PhD thesis I experimented with the chronological variable as a fixed or random effect, and obtained a much better model fit by treating it as a random effect in a mixed effects/multilevel model.

Actually, I'd just say it sounds like Gard's PhD thesis is the place for you to look.  - Jim 

PS - Oops. BTW, I should have said estimated standard error of the prediction error above, not standard error of the prediction.

Dear Gard,

Great point. I have only recently started learning about a mixed model. All my past experience was  a "traditional" sociolinguistic fixed effect model. Do you think that placing a chronological variable into a random will not hinder the interpretation of results - as I am tracing how word order has changed over time? (I am going to read your thesis this weekend!)

Thank you!

I just came across this discussion and might have just two cents to add to the $100 of useful things that have been said …

Regarding amounts of data for logistic regression models, Baayen cites a "rule of thumb" that "there should be at least fifteen times more observations than coefficients" (2008: 195); though this says nothing yet about how evenly the observations should be distributed across the levels of the dependent variable (I don't know if there's a "rule of thumb" for that, too, but it should certainly be considered).

As for how to treat the time variable, I'd say that in principle continuous is better that categorial (on the rationale that one should not needlessly lower the level of measurement). If it seems more reasonable to use time periods (i.e. categories), I would suggest to treat them as ordered categories, i.e. an ordinal variable (such that, e.g., "Old" < "Middle" < "Late").

And a third cent, in response to Gard's answer: If I understand mixed-effects models correctly, by using the time variable as a random effect, you factor out the different baselines in the different time periods, so what you get in the fixed effects is the effects of the other variables irrespective of the time period. If you're interested in change, that would be not what you want. Another option could be to treat the time variable as a moderator variable, that is to include its interaction with each of the other factors in the model. Interaction with time = change. (But I realize we're digressing from your original question..)

[Baayen, R.H. 2008. Analyzing Linguistic Data. Cambridge University Press.]

David,

Thanks! Good points. I also like your idea of Old > Middle > Late, since some centuries have sparse data, and this method will yield a better chronological balance.

Hi Olga & David,

sorry for leaving this very interesting discussion hanging. I think the point you're both making about change and time as a random effect is interesting. My reply would be: it depends at least partially on what is being investigated. If the aim is to investigate word order change, say "VO" or "OV", as a function of a diachronic variable to obtain changing rates of OV and VO then it might be worthwhile to include the diachronic variable among the fixed effects. In my project I was more concerned about measuring the relative importance of the fixed effects (by period, with separate models, and with a single model) than with the change in the dependent variable per se.