Hi David,

I understand your problem and I have faced similar ones.

How about testing your repeatability with including and not-including the shy fish, to see if the repeatability is comparable? They are after all part of your population, therefore I would be inclined to keep them in the data set, but as you mention the distribution of your data becomes heavily skewed,and that matters for measuring  repeatability...

I have found an article which deals with measuring repeatability of qualitative data, and data that has a beta distribution (you might be able to find your answer in there).

Good luck with finding the ultimate solution, and if you have please post it.

Elske

Thanks for your responses!

Elske: yes my intention is to keep those data. I can try with and without those fish, but the the problem is in the distribution I have when I include them, which is right censored. Thanks for the paper, I'll come across it!

Kate: thanks for your detailed response.

1- In my case, the fish that never left the refuge (in any of the two replicates) are around 10-12%. I feel that the trials are long enough for the species, so these non-reacting fish are probably showing true behavior. However, it's not as "easy" as a binomial distribution, since data are quite spread out in time (but of course most fish leave in the first 30 seconds).

2- Interesting...and yes. For variables related to behavior in the arena, those fish should/could be removed. I found similar cases in the literature. But as I said to Elske, I'd prefer not to lose information in the specific case of the "latency to emerge" variable. Otherwise I'd be dropping the really shy fish.

3- I agree, and have to think about it after making some trials.

4- The same

I'm planning to estimate 1) repeatability for each variable and 2) correlation between variables. But using a multivariate mixed model approach (following the paper by Dingemanse and Dochtermann, JAE, 2013 that you probably know) in order to have a measurement of the "between-" and "within-individual" correlation. 

So to construct the models I need to have "recognisable" distributions for the function MCMCglmm. You mention families for count data...should/can latency times be considered count data? This a key issue here. If so, then I can go ahead with these models and estimate repeatabilities as in Nakagawa and Schielzeth, 2010 (Biological reviews).

So in summary, hopefully, I'll manage to include these fish and work with a easy-to handle distribution, like (quasi) Poisson or negbin...let's see.

Further comments/advise will be appreciated :)

David

Hi David,

when measuring time until leaving and you have individuals not leaving at all, you might want to try "survival analysis". This method comes from medicine where they often measure time until death, depending on treatment. Usually, not all patients die in he end, so this method was developed to deal correctly with those values. They are called"censored data" because you do not know when the animal would have left if you had not stopped the experiment. Including the data as if they had left might be wrong and leaving the data points out could also give a wrong impression. 

But it might be that with your goal of measuring repeatability, survival analysis is not the optimal method, as it compares effects of treatments. Anyway, just wanted to mention it.

Best

Andra

Thanks Andra, I read about about survival methods but I'm not sure how to apply them in my case. But thanks for the suggestions, it probably deserves some more reading.

David

Dear David

I know the problem quite well, as we did similar experiments with some fish species. You may find some papers concerning this issue in my list. In our experiments the "latency to do something" was in every case only one of the different behaviours we observed and our first step was always do run a PCA including all individuals and all measured factors and values (indeed including the 1200 sec. for individuals that did'nt leave the shelter). Regularly, recorded data and calculated PCA were always as robust that 10-20% of non-leaving individuals were no problem in the analysis. And indeed, as mentioned before, these individuals are also part of the population that should, to my opinion, be included in the analysis!

Greetings

Jost

Thanks Jost, 

So let me see if I got it right. Your first run a PCA analysis with all the variables measured in a particular test (latency to exit, time close to an object, time active, ...) and then estimate repeatabilities (and then correlations or whatever you are interested in) with the results of the PCA analysis instead of the original variables. Is this what you mean? In this case I understand you never look at the repeatability of the original variables...

Greetings, 

David

Dear David,

yes your got it, except concerning the repeatability. This is just to increase the number of replicates that are done, however with the same individuals and, thus, have to be evaluated as repeated measurements (pseudoreplicates). Sometimes one can see some changes from the first run to the next and (if there are three) to the last that may give some hints of learning, but this is regularly not in the focus in our work (cf. the paper Goldenberg et al. 2014). Despite if you are using only replicates or as we did it in many studies pseudoreplicates (of course using appropriate statistics to deal with), in every case we use all factors and all values of all experiments to run the PCA and then use the first or sometimes the first and the second axis of the PCA as response variable in our further analyses. Hope it is clear what I meant.

Greetings

Jost

Some updates of what I found:

It seems that survival analysis are one good way two go, and since I'm not an expert in this field I'd recommend the reading of "The Statistical analysis of behavioral latency measures" by Sergey V. Budaev (1997) to see all the potential analysis that can be done with censored data. Not very recent paper, but still helpful.

I found that other people working in behavioral syndromes and personality have circumvented this issue using structural equation models (SEM) under a Bayesian approach. Specifically, I found quite a good explanation in Edeelar et al. (2012; attached). They state that:

Because we obtained a number of censored data, we applied a Bayesian approach to SEM. Censored data is truncated data that has been limited in their values because of constraints to the temporal or spatial range of the study. Censored data are frequent in both experimental and observational studies, but their inclusion in traditional GLM or SEM is problematic. Omitting all cases with censored values leads to a bias in the data set and the results since the most extreme individuals are deleted. In addition, statistical power is lost. Especially in behavioral studies of animals, where sample sizes are often below 25 individuals (Taborsky 2010), this is highly undesirable. Alternatively, simply using the minimum or maximum values for the censored values underestimates the variance among individuals, also leading to biased estimates, inflated repeatability estimates, and violation of distributional (e.g., normality) assumptions of the data. Some methodology has been developed that allows for the proper use of censored data, such as data imputation or survival analysis (see Quinn and Keough 2002). The option  chosen here is a Bayesian analysis that implements an estimation of the posterior probability distribution of each censored data point as part of the fitting procedure, using the information contained in other correlated variables. Hence, likely values for censored data were estimated based on the fitted model (and its credibility), and variability in the censored data was iteratively incorporated during the model fitting (Arbuckle).

I will try different options with my data and see how it goes...

David

I just noticed that MCMCglmm function in R accepts censored distributions, for example "cengaussian" or "cenpoisson", which is pretty much what I was looking for. Haven't tried it though. But worth exploring. Does anyone have experience with this? It looks promising!

Hi David,

This has become a very interesting discussion. I am collaborating on a project where we have five repeatability measures with as response variable the time spent near a novel object and the latency, with a maximum time limit. This sounds similar to what you have and seems to come closest to being described by a beta distribution: a continuous distribution limited between 0 and 1 (using proportion of time). However, I am struggling as well to analyse the repeatability using this distribution. I have been reading about MCMCglmm but have not been able to run the right code on my dataset. Have you been able to do this succesfully yet? It would be nice to see your code.

Best,

Jolle

Hello Jolle.

Which family distribution are you using in MCMCglmm? I think beta is not an option. 

I'm still doing some tests with different families and models, so I don't have anything definitive. Estimating repeatability is quite easy when you have gaussian, poisson or binomial distributions, you can use the package rptR in R for that purpose (or do it manually with the MCMCglmm output). But I'm not sure how you can estimate it when you use other less common family distributions, like "cengaussian", or of course "beta".

This paper is a good one, but repeatabilities are estimated only for gaussian data (not normally our case I guess):

Dingemanse, N. J., & Dochtermann, N. A. (2013). Quantifying individual variation in behaviour: mixed‐effect modelling approaches. Journal of Animal Ecology, 82(1), 39-54.

Thanks for your interest, 

David

Hi David and all, 

I was very excited to see this discussion! 

I have been attempting to run repeatability analyses with the disturbance responses from identified individual turtles (from snorkelers). Repeatability using Multiplicative overdispersion model with binary variables was simple (using function rpt.binomGLMM.multi in the rptr package (see link and associated paper)). However, having issues with latency measures as well. 

For example the time it took for an individual turtle to resume foraging post disturbance (TTRF) was set at a cap of 120 seconds. If the TTRF for an observation took beyond 120 seconds then the turtle was given "120 s" for its observation (~ 20 % observations are 120 seconds). Please see histogram attached.  

Treating this as count data, we could potentially use either an overdispersed poisson distribution (using rpt.poisGLMM.multi function) or we would believe it is negative binomial. Ideally using the rptr package to get repeatabilities however negative binomial does not exist as a choice in this package.

Is using the Poisson models with square-root link (rather than log link) acceptable? 

David (or anyone else!) find any more solutions to this problem or thoughts?

Many many thanks!

Luke

http://rptr.r-forge.r-project.org/

Hi Lucas.

My histograms are exactly the same! And I didn't find a proper solution. However, in my case, latency times are just one variable over a total of 5 for the same experiment (and the other variables are not censored). So I run PCAs with the 5 variables to get a principal component that summarizes the target behavior for that experiment. That component is no longer censored (or at least very little) so I could potentially solve the issue, although is not probably the most elegant way to do it. In my case non-responders were just 10%.

MCMCglmm in R can handle censored distributions, so you will have no problem to model your data in that framework, but the problem again is that it does not exist an estimator for repeatability when you use a censored family distribution (they only exist for gaussian, poisson and binomial).

What about using icc in R? I'm not sure if it can be used somehow with non-normal data...anyone?

Good luck!

In some sense, the censored data will always be unknown (somewhere above the time limit). If few individuals reach the limit, there are probably not much of a problem (but this small problem still needs to be acknowledged in the manuscript). Data can be analyzed with survival analysis and for repeatability you would get a rough idea about the effect by using Spearman Rank correlation (given that you have only two measurements). But if a lot of individuals reach the limit, I guess the only real solution for calculating repeatability is to do the exeriment again and increase observation time - censored data is unknown and there is really no way to make it known. Lots of censored data is not very informative, and I think nothing will really improve the situation in that case.

Hi all, 

I was apart of this chat a long time ago and thought I might chime in again. I simplified the censored problem and decided to run PCA across my behavioral response variables (thanks David) and subsequently ran repeatability analysis using the rptr package (see Nakagawa and Schielzeth 2010). This was effective and clearly showed how some individuals displayed "repeatable" behaviors (as seen through PC scores). If still struggling to find a solution and in need for an effective alternative, see the recent pub I've uploaded on my profile. Cheers!

Hi all,

I agree with David that this is a case of censored data. A survival analysis would be perfect to use, but this does not generate residual variance for the random effect, which makes it not usable for estimating repeatability. MCMCglmm with censored data is, so far as I know, a good alternative (I used cengaussian, see plot) and allows you to calculate the repeatability. The results from MCMC were similar as what I got from lmer, so I think this is a good alternative to use.

Anyway, this is a really interesting topic!

Kind regards,

Bram Vanden Broecke

I was going to give a response to this question suggesting Bayesian estimation, but saw that David Villegas-Ríos already cited and quoted our paper, so no need to do that anymore!

Hey David! Such a rich discussion here! I am going through a similar problem in two tests (but with mice). To make it more complicated (or not), one of the dataset is a longitudinal data with 4 times points and two measures (both are "time until event") and a latency in a box later on. Imma give a try to the Bayesian thought, but if you have other suggestions too!