Yes logarithmic transformations compress the scale. Best, David

Don't they (logged and not means) refer to different experiments?

Please provide more details:

- what was the analysis, what test are you referring to?

- what was the result (estimate, p-value)?

- what excatly did you enter in G*Power and what method did you use?

- what was the actual sample size (total and per group, if applicable) and what did G*Power say?

@Jochen,

Thank you for your response. In writing this, I found a mistake that I made.

Here are the details that you asked for.

Experiment: I am looking for sex differences in probing behavior of the Asian Citrus Psyllid. The insect also has different color forms (call them green and brown). There are four “treatments:” Female green (FG), female brown (FB), male green (MG), and male brown (MB). Green females are the largest, brown males the smallest, however there is considerable overlap in the distribution of the sizes. I have 84 variables, and I get 4 significant outcomes using a Tukey test (alpha=0.05). I get no significant differences if I use a Bonferroni correction for 84 variables. My sample size is 20, and an experiment of this size is about 4 months of work. Sample size distribution in published papers has three modes, n=10, n=15, and n=20. I have three choices:

1) Abandon all hope.

2) Declare that there is no sex effect: the null hypothesis is true.

3) Declare that the sex effect might be present but with sample sizes below (this threshold) you cannot detect it. If I use the existing data and find that my next experiment would require a sample size of 194, then this is the easy option. As long as I am stuck with sample sizes close to 20, the sex effect will always be non-significant. I should then design experiments that use a mix of males and females, but not include this information in the design or analysis. The alternative is to do experiments only using females and assert that the differences between males and females is so small that my results apply to the entire population.

- what was the analysis, what test are you referring to?

NumPrbs: p-values F-test 0.1173

Mean (SE)

FB 4.0782 (0.3472)

FG 4.7834 (0.3395)

MB 4.5638 (0.3324)

MG 5.2942 (0.3641)

MnPrbs: p-value F-test 0.2356

Mean (SE)

FB 8.2221 (0.1956)

FG 7.8681 (0.1913)

MB 7.9788 (0.1873)

MG 7.6446 (0.2052)

NumPrbs: This is the number of probes each insect made (a count). It has been square root transformed. It is the average of 20 insects for each treatment.

MnPrbs: This is the average duration of each probe for each insect. The value is log transformed. It is the average of 20 insects for each treatment.

Tukey test shows no significant difference. First number for NumPrbs is the least square means estimate of the square root transformed mean number of probes per insect. The second number is the standard error of the LSMeans.

- what was the result (estimate, p-value)?

- what exactly did you enter in G*Power and what method did you use?

G*Power 3.1:

1) Test Family: t-test

2) Statistical Test: Means: Difference between two independent means (two groups)

3) Type of power analysis: a priori: Compute required sample size – given a, power, and effect size.

4) Tails 1:

5) Effect Size (Determine, n1=n2)

a. Mean group 1: 4.0782

b. Mean group 2: 4.7834

c. SD group 1: 0.3472

d. SD group 2: 0.3395

In writing this out I found an error.

I entered SE from a regression for SD from a sample.

That is bound to turn out bad!

f. Effect size is 2.05

6) Alpha error: 0.05

7) Power: 0.95

8) Total sample size 12

9) Actual power 0.951

- what was the actual sample size (total and per group, if applicable) and what did G*Power say?

So I need to go back to the raw data and get a real standard deviation. Given that I will analyze the transformed values, do I want to use the mean and standard deviation from the raw data, or from the transformed data? Will anyone care which choice I make?

Raw:

Mean 18.045, SD 10.531

Mean 25.304, SD 14.395

Total sample size 134 (67/treatment)

Transformed (since I will use transformed data, I am guessing that this is the right choice):

Mean 4.078, SD 1.2169

Mean 4.783, SD 1.5916

Total sample size 178 (89/treatment)

Thank you for the detailed description. It already helped you to spot errors ;)

a) These 84 variables, are these variables you need (or want) to adjust the probing for? Are the sex and the color the two factors, for which you will find the differences in the probing counts (after adjusting for the 84 covariables)? Why do you test these covariables? Is that of interest?

b) Have you considered that the sex difference might be different between the two colors (that is: if there is a sex:color interaction)? If yes, you should model the sex and the color main effects and their interaction. If not, why do you want to distinguish the 4 sex/color combinations?

c) The sample size of 20, does it refer to the number of insects of a sex/color combination (so the total sample size is 80 then)? I wonder if that sample is too small to reliable estimate at least 84 coefficients...

d) That's difficult considering c, but: have you considered interactions of sex and/or color with other covariables?

There will be more to follow.

Regarding your last question: you should use the transformed values for the power analysis.

@Jochen,

a) Probing is the general term in contrast to non-probing. Probing is broken down to pathway, xylem ingestion, phloem contact, phloem salivation, and phloem ingestion. There are then a convoluted set of standard variables calculated from these behaviors. The exact number depends on who you ask. My list of 84 comes from European scientists studying (mostly) aphids. A simpler list is often used in the USA.

The 84 variables are not independent, but the degree of correlation varies.

b) Sex and color can be two variables with an interaction term. They can also be a single variable with four treatments. There is no "value" associated with these. There is some biological interest, but mostly the question is: do I need to include sex as part of experimental designs? It is a factor that is always present. Sex makes a big difference in other insect systems. Sometimes males and sometimes females are better vectors of plant disease. It is an issue sometimes raised by reviewers. With the behavior data, the best I can hope for is answering the question "is there a difference?" The why will have to be on the next grant.

c) Test are mostly univariate. Discriminant analysis or factor analysis is done sometimes after variables are removed. "Important" variables are identified by p-value from univariate model F-test, or sometimes by a stepwise method (proc stepdisc in SAS).

d) Total sample size was 80. Without a technological innovation, there will be few studies that increase sample size above this. A poster at a scientific meeting claimed two treatments and a total sample size of 60.

Usually this is 80+. The experiment is run until the poorest treatment has 20 insects. The best treatment might have 30 insects at this point.

e) I can tell the difference in behavior for a psyllid on an old versus a young leaf of the same plant. I can tell the difference between the upper and lower side of a leaf. I can tell the difference in behavior for a psyllid on two different cultivars of sweet orange (both cultivars generally considered good hosts, ideal for maintaining laboratory colonies). I can easily find a sublethal effect of insecticide.

f) Other covariables: I have to hope that there isn't a sex by color by temperature interaction. One could replace temperature by other things, like endosymbionts. I do qualify the issue by stating that these results apply only to healthy psyllids. It is possible that there will be a sex by color by pathogen interaction. So someone (probably not me) will have to do a similar experiment with diseased psyllids.

g) I looked for behavioral effects of old versus young leaves and top versus bottom surface. If there is a sex effect, then the size of my experiment has just doubled. I can choose to only do half the experiment, but then my conclusion only applies to half the population in the field. I can reduce the number of treatments from four to two, but then I cannot ask questions like "is the effect of leaf age greater than or less than the effect of leaf side?"

I hope that answered all the questions.

I am not an etomologist, and it seems that I'm not expert enough here to help you substatially. I Still don't really understand how probing is evaluated and how what role all these variables play and how they are analyzed/modelled. One would need to understand all these details to effectively help you (I think), and I won't be able to learn all this through RG posts. So I think it would be the best for you to contact a local statistician (who should ideally be interested in ecological/etomological problems) to cooperate.

b) Then you should use a model with that interaction. Modelling 4 groups limits the probability to identify sex differences.

c) Ok, as I understood, you take one by one of these 84 variables (or a "significant" subset) as the predictor, besides sex and color, to predict probing? You should not select "important" variables by significance. Statistical significance is not identical to importance! The selection should be done based on theoretical considerations. And even if some variables are excluded, they remain part of the multiple testing problem! If you argue that experience (data) is (or should be) used to select the variables, then is is really cruical that this selection is not based on the same data you use for your research question! You should then screen the literature or have own pilot data (but again: theoretical considerations are better!).

Then, it might not be wise to fit separate models for each (remaining) predictor, because the different predictors may have combined impacts and they could even interact with sex or color. But the small sample size may make this difficult. For each coefficient in the model you should have at least 5-10 insects, so 40 insects would be just enough to investigate sex and color, and adding another single binary or metric predictor that interacts with both (sex and color) would require another 40 insects.

Right now I find the setup (considering or having to consider) so many covariables too complicated. I would suggest to focus on a better controlled subpopulation (aphids on the bottom side of young leafs of the same aged untreated and pthogen-free plant species/cultivars grown under similar climatic conditions etc.). Then show what the sex and color effect under exactly these conditions is. You don't know then if this might be the case under different conditions as well, but proofs (it may proof) the principle, and if you have shown this one can agrue that it is therefore worth to consider sex and color in similar experimens even under different conditions (different cultivars, old leafs, etc). I don't think that you need to show a sex/color effect under all thinkable combinations of cofactors, and I think that is is far too demanding to identify those cofactors under which sex/color plays a role. Select conditions of which you think the sex/color effects are most likely and do this focused experiment -- if this is possible! If this is not possible and you do have a set of data of which you now have to get the best out, I stay with my recommendation to involve and sit together with a local statistician.

@Jochen,

Yes, probably better to abandon this. Yet I hate to leave someone more confused than when they started.

Insect behavior: I hook up an insect as part of an electric circuit. I get out a recording that looks like an electrocardiogram or electroencephalogram. It has repetitive patterns that correlate with where the insect's mouthparts are in the plant. If the mouthparts are not touching the plant this is non-probing. A probe starts when the mouthparts touch the plant and penetrate the cuticle. The probe ends when the mouthparts are withdrawn. However, somehow in this period the insect must acquire food. Within the probing period, there are other behaviors that can be identified.

Probing has (in my case) five behaviors: pathway (mouthparts searching for food and water), xylem ingestion (drinking), contact with phloem, phloem salivation, and phloem ingestion (eating). There are limits, such as an insect cannot do two behaviors at the same time, and an insect cannot go from xylem ingestion to phloem ingestion without going back to pathway. If you add up all the time spent in each activity you get the recording time.

Each insect is a linear series of behaviors, and one recording might have twenty or thirty probes. I can use this linearity to calculate a diverse set of variables. The insect is the replicate.

I can calculate counts, means, and standard deviations. I can calculate proportions like the proportion of time spent ingesting phloem relative to the total time spent in contact with phloem (contact + salivation + ingestion). I calculate time to first phloem ingestion event, or time to first sustained ingestion event (an event longer than 10 minutes), or number of short probes after the first phloem ingestion event. With luck, you are less confused.

Equating importance with p-value is a fight that I have been trying to win for years and failing. It is like a shiny fishing lure promising great things if you bite. Think of a case where your boss wants you to identify the important variables. You set up a meeting to discuss the issue and the first thing they tell you is "I hate math." The second thing is they helpfully show you published papers with this approach. See, it works. It is accepted practice, why are you being so difficult?

A part of the problem is that there is no theory, at least not in the sense that would help with analysis. The data are not normally distributed, so the Europeans use non-parametric methods and the Americans use transformations. The Americans use MLE methods on small sample sizes. In some cases they have confused MLE with nonparametric. However, the method is published in peer reviewed journals so it must be correct. I am being difficult again.

Timothy, thank you so much! That is extremely interesting. I haven't had a clue that such things are possible :) I am definitely less confused and more impressed than I was before, but I have to admit that I still don't really understand what these 84 variables are and how they are considered in your model(s). But that's ok. I see that one really has to dive deeper into that topic, what needs a considerable commitment of time and brains. I thus still think that it is a really good idea to involve a statistician - not only to solve your present problem but also to learn ways to tackle similar or related future problems.

Your paragraph starting with "Equating importance with p-value..." reads exactly like my very own life!

I can possibly make some comments on the last paragraph:

Nonparametric methods have three disadvantages: i) they have low power on small sample sizes, ii) they don't give you even a hint of the size of effects (at least not in any practically interpretable units) and don't allow to model relationsships quantitatively, and iii) they don't allow you to analyze interactions (in a meaningful way).

Data transformations can be useful, but there are often better tools that don't require data transformations. This is especially the case when analyzing counts (e.g. as in your case, as I understood). Here, the variability could (should) be modelled with the Poisson distribution or with the negative binomial distribution. This had the advantage that the model coefficients are interpretable (in contrast to a square-root transformation of the response, where an interpretation of an isolated coefficient is usually impossible (there is a sum of coefficients under the square-root, so they can't be put into independent terms).

Thank you so much again for all the explanations and please let me know if you think I could help.

@Jochen,

You help just by giving answers on RG and being part of the community. However, I'll keep the offer in mind, thank you. I sometimes have questions that are too narrow/specialized for RG.

As scary as it might be, I am the leading statistician at my research station. I have a minor in statistics, but that is more experience than most biologists. I took most of the applied statistics classes that the department offered at the time. I then realized that if I took two theory classes I could get a minor. I loved the class in probability. The other one not so much. I got through because the other students were equally terrible rather than any great skill on my part.

The main campus of the University of Florida is 3 hours away. They do have a good consulting service, and I kick some questions that way.