I think, my question is even more conceptually weird than it appears:)

Treatments are 100% defined by experimenter, let's say, controls (A) and exposed to some toxic substance (B). Then, many potentially interdependent parameters (biochemical markers, physiological performance, you name it....) were measured. The question is - what measured parameters are associated with the exposure?

Why logistic regression would not be an appropriate method to address this question?

Oh, yes, and thank you for your reply!

This is an interesting question. If I am understanding the actual experimental situation correctly, you have a single binary independent variable (treatment vs control), and a number of continuous dependent variables (biochemical markers, physiological performance, etc.)

And it seems like you are drawn toward logistic regression because, if you flip around the IV and DVs, you would have a nice simple single analysis to run. Is this all correct so far?

The problem I see is that the coefficients you get from the logistic regression will not quite tell you what you want to know. The resulting slope for, say, "biomarker 1", will not tell you about the effect of the treatment on that biomarker, which is presumably what you're interested in. Rather it will tell you about the relationship between treatment and that biomarker, while controlling for the relationships between treatment and all the other "predictor" variables. But there is no sensible reason to apply those controls in this case: The other variables are also dependent variables, not potential confounds.

So I don't think logistic regression is the way to go. I also don't think classification tree analysis is relevant, because the dependent variables you are trying to predict are not categorical.

The options I would suggest looking at are either:

Hope that helps!

Paul & Matt - thank you (!!!), I really appreciate this discussion.

The whole idea is unconventional, I agree. However, the differences between "treatment by experimenter" and "state" that differ between the subjects could be just a matter of semantics. If I randomly pick study organisms from the field and infect them with a pathogen - it would be treatment (=independent variable), followed by a collection of dependent variables (=biomarkers). However, if I pick already infected individuals and continue with the same collection of markers as above, their infection state would qualify for a dependent variable...

Multivariate methods are usable, indeed. My first choice was to do PLS with treatment as a single dependent variable (categorical), which works reasonably well in the sense that I can identify a few (2-3, all continuous) markers that define a significant component. In principle, MANOVA or PERMANOVA can do the same, although MANOVA assumptions would be violated with the data I have.

However, odds ratio (OR) is something that I find really attractive and useful in the context of the story... which brought up this question: why logistic regression should not be applied for estimating (OR) of a response (=marker) being higher or lower as a result of treatment exerted by experimenter...The objections that Paul has mentioned, would not be valid if dependent variables are few and none of them are categorical as far as I can see.

Paul: prompted by your suggestion to use CTA, I tried it, but I don't really see that "levels" used for tree branching could be used in a similar way as OR when interpreting their importance. But I gave it a really quick try... need to dig more into the references that you gave and, perhaps, re-think the OR necessity. Also, need to read some more about the linear models with categorical response variable...

Excellent! I will discuss this with a colleague of mine (who actually owns the data) and try to convince her that this is exactly what we need.

I really appreciate your offer. More later/E

Hi Paul,

The categorical variable "treatment/control" is the independent variable in this study. I understand that the OP initially talked about treating this as the response/dependent variable in a statistical analysis. But the reality of the actual experimental design is that this is a manipulated variable, so therefore it makes little sense to regard it as a response/dependent variable.

There appear to be multiple response variables ("biochemical markers, physiological performance, you name it..."), and my tentative reading is that these variables are continuous.

You say that CTA can handle a continuous response variable, and mention your citation 9 (http://optimalprediction.com/files/pdf/V3A18.pdf), which is interesting. This said, I find that description somewhat confusing for two reasons:

1) In the cited article, the relevant analysis is one where the class/response variable is geopotential height anomaly (GHA), and the predictor variables are ordinal symptom ratings. I find this hard to understand, because GHA is the putative cause in that study. Treating it as the response seems to be a workaround to deal with the problem that there were actually multiple response variables (ratings of different symptoms). But the result of this transposing of IV and DVs is that the model is unable to answer even simple questions about causal effects like "What is the effect of a unit increase in GHA on sleep problem ratings?" This problem would also crop up in OP's case, where there are multiple response variables, not just one. Can CTA work with multiple response/class variables in a single model?

2) To me it's ambiguous whether one can really say that the CTA model is able to deal with a continuous response variable. The final model in the cited article 9 ultimately just splits GHA level into three categories and attempts to predict into which category an observation falls. So the model can in a very general sense work with a response variable that's continuous, but is unable to produce continuous predictions. That seems a significant limitation, no? If one wanted to use a decision tree approach with continuous response variable, why would one be using CTA and not a regression tree?

I don't mean to denigrate CTA; I'm no expert in the area, and it seems like an interesting technique. But it is not the right tool for every data analysis problem.

Hi Paul,

>The objective is to discriminate the class variable on the basis of the attributes. 

That isn't my reading of the OP's research question. The question, as I take it, is: What causal effect does the treatment have on the measured attributes? Perhaps some clarification from Elena about this would save further confusion - we may be disagreeing in part because we're interpreting the research question differently!

Morning (here in Stockholm:))

I simply enjoy your conversation. I think that for many experiments in biology, medicine, toxicology, etc., with many measured variable involved, classification and causative regression-based methods are interchangeable... I realize how horrible this sounds... The question is not about just flipping X and Y but about re-formulating what you are after. 

Our experiment is essentially a classical toxicological experiment called bioassay (now I am practically forced to talk details). Sediment-living animals were picked from the field, split in 2 groups and one group was exposed to a sediment from contaminated site (treatment), whereas the other spent the same time in clean sediment (control). Following the exposure, we measured a bunch of biomarkers presumably indicative of detoxification, metabolic disorders, etc. This is a very common type of bioassays in environmental toxicology, nothing is species about that.

Now, we want to find out which of the markers are indicative of the expose (causative relationship) to be able to go out in the field and use them for diagnostics and identification of contaminated areas (classification). And now comes the question: this ultimate goal (=classification) is essentially the same as use of biomarkers in clinical studies, where

- logistic regression is used to investigate the relationship between, for example, a disease status and a panel of biomarkers as well as other risk factors (e.g. age, sex, smoking, etc.).

- and then the same regression is used as a classifier to predict probability of the disease state based on markers.

The only difference between this and our appoach is that disease state was not induced by experimenter, whereas toxic exposure in our case was. Is this a really important difference? All in all, the attributes and continuous response variables are essentially the same thing here and this is where I am kind of conceptually stuck. For each specific marker, t-test (or Mann-Whitney) would be totally applicable, but not interesting at all. Any multivariate method would essentially serve as classification. PLSR would be also used as a classification...

In any case, I am very grateful for your input, guys.

Hi Elena,

In my opinion, the way you define the disease status is not important and you can use logistic regression or other suggested methods to make a comparison between the "disease" and control groups in terms of biomarkers (or the other way around, regression is symmetrical). I agree that the t test would work fine, too.

However, to answer your initial question, I don't think you can draw a significant conclusion about the effect of your "treatment" without a true randomized study. I would gather a bunch of sediment, divide it into a prespecified sample of smaller amounts of sediment, measure biomarkers at baseline, randomly subject half the sample size to contamination, measure the biomarkers again, and then analyze the data.

Hi Cosmina, thank you!

Yes, we did have a proper replication for both polluted and unpolluted sediments, this was OK. The initial baseline would not be very informative, because animals grew during the incubation and the markers that we used are induced during ontogeny... But we have controls (unpolluted incubations) for comparison at the termination of the experiment.

The t-test or a non-parametric analogy, while appropriate, would not be particularly helpful for the questions we are after. As transpired during this discussion (at least for me:), it is rather identification of a biomarkers, both individual and as a panel, that are most efficient in discriminating between polluted and unpolluted environments.

It is supposed to be a trivial thing...at least in the beginning:)

I see. I would still measure the baseline for A and B. Then wait for a period of time (duration of ontogeny maybe?) in both groups. Measure again (I suppose this would be a better baseline). Then apply contamination randomly. Then measure the markers again. 

Discovery of biomarkers is not an an area I'm familiar with. I used my general epidemiological knowledge to answer your question, but I think you're looking for more specialized guidance.

Thanks, Cosmina. This particular experiment is already finished and the baseline would not be possible to get:) But, we were discussing necessity of having it at some point and I wonder if you could point to a reason (or a reference) why is this crucial? There will be more experiments like this and this issue is clearly coming up again and again.

As long as we have animals coming from the same population (cohort, environment) as our experimental group, would not it be enough to compare the endpoints post-exposure between the groups after confirming that handling stress does not cause a measurable response (just once and for all for each specific endpoint)?

I'm not sure if I would call it crucial, but it is rigorous. The control and the experimental groups need to be as similar as possible at baseline so that the differences you notice in the end can be safely attributed to the "intervention" (the contamination in your case). If there are differences between the groups at baseline, these need to go into any regression models you have planned for your data analysis. The measurement of baseline data is even more important in non-randomized experiments (like your's). One important purpose of randomization is to balance the measured and unmeasured factors between the study groups, factors which could influence the measured outcome. 

I'll try to be more specific. I collect soil samples which contain bacteria. I divide them in two groups, control and experimental.  I want to measure whatever markers I want to look at in the end (perhaps after an initial incubation time, if this information is known by your lab or the literature), but also others that I think may influence the results . So, let's say I measure M1, M2, M3, M4, M5. I find that the control group has two times the level of M2 and M5. I have no idea what this means and whether it will influence the level of the markers at the end of the experiment. I run the experiment (contamination and no contamination) and measure the level of markers again. In order to compare the level of M5 between the two groups, I need to control for their level at baseline. The control group had already started with a two-fold greater level of M5. And in the end the control group had a three-fold greater level of M5. How would I have interpreted this result if I did not have the initial information that my control group had in fact started out with a greater amount of M5? I would have seen a larger effect than it really was (a confounded effect). 

As for reference, I would recommend Kenneth Rothman's Introduction to Epidemiology for a nice, easy-to-read description of confounding, biases, and other good stuff.

ok, I see, thank you. In ecotoxicological bioassays, the animals are usually coming from a homogenous population (ideally a lab culture) and are synchronized in terms of their development and growth status. Therefore, it is generally assumed that they are "equal" and it would be relatively uncommon to take a baseline measurements within a bioassay exposure. Essentially, if a control group and a treatment would differ in their initial status, the experiment should not start... But these are standard tests and we are doing a lot of "less standard", which means that we should think about that.

Elena,

What's the verdict? Can logistic regression (indeed multivariate ANOVA)  be used as an appropriate tool to build a predictive model for a binary outcome? All medical statistical books say YES (ref: Kirkwood, Stern). In clinical decision science in respect to diagnosis (i.e which signs/symptoms predict diagnosis) it is a mainstream analysis.

On a level of reasoning I don't see a contradiction that it cannot be technically used to for biomarker-based model to predict the binary outcome "responder" vs "non-responder".   Reasons "why not" , please?

I have not been able to find any strong argument for not using it...

MANOVA is not as functional in this sense as logistic regression because of not very comfortable (at least for my data) assumptions. PLS-DA is a good alternative though that is fully acceptable.