Dear friends, hello! I have been interested in one question for a long time and I really want to discuss it with you. I will be glad to any of your responses. Thanks a lot in advance. Regards, Sergey.
We often read: a research was performed and it was established, based of statistical data processing, that output parameter A is significantly affected by factor B. A question arises - is it possible at all (in principle!) to reliably establish the influence of a particular factor based on the processing of statistics obtained as a result of a passive experiment. Suppose that the output parameter A is a person's longevity, and for simplicity we establish that A can take only two values - zero (less than 80 years) and one (more than 80 years). As an analyzed factor B, we take the drink of red wine over the age of 60 years. For simplicity we will consider this factor also as a binary, taking only two possible values - zero for "non-drinkers" and one for "drinkers".
Suppose cohort consists of 1000 people, and after processing the collected statistics we have got results:
a) 600 of them did not drink red wine over the age of 60, and 400 people drank it.
b) Of the 600 who did not drink, 360 did not live to be 80 years old, and 240 survived 80 years.
c) Of the 400 people who drank,160 did not live to be 80 years old, and 240 survived this age.
It is clear, that the factor “drinking of red wine over the age of 60” strongly influences for longevity - when switching from its value zero to one, the chance to survive 80 years increases from 40% (=240/(360+240)) up to 60% (=240/(240+160)). The using of standard statistical methods (Fisher's exact test) confirms this conclusion: p-value
Sergey Porotsky, first of all, please note that the singular of the word "phenomenon" is exactly that. And that's what you needed up in your question. The plural of that word is "phenomena" Many people make this mistake. I'd like to save you from joining the club. :-)
Now, having read your situation, I think you need to use multiple regression, probably with dummy variables.
I hope that helps. And if anyone disagrees with me, that's fine. Hopefully they would have a better suggestion.
This is a very important question. The answer is "When you can reliably predict a future outcome." Statistical significance testing is a matter of data quality testing. A statistical test is a minimum data quality. The data must meet this minimum for further consideration.
The further consideration is what is needed to make a reliable prediction of a future outcome. A reliable outcome is a controllable outcome. An effect is significant when you can show that the uncertainty in the outcome is much smaller than the effect. Here, significant equals controllable. All the uncertainties must be included in the control (amount of alcohol, type of alcohol, income, education, ...)
I am attempting to address this same question. See my project "Scientific Method."
Dear Robert Trevethan, hello!
I have changed word "phenomena" on the title - thanks for remark!
About your suggestion. First, in my opinion we should not use Regression, we rather should use Classification - due to binary type of output parameter. Second, it may be performed only for linear classification - e.g., for Linear Kernel of SVM. Usually optimal kernel isn't linear, it is rather RBF and how can we evaluate significance (influence for output parameter) for RBF Kernel using? Regards, Sergey.
Dear Joseph L Alvarez, thanks for your quick answer. I will carefully study your project ASAP. Regards, Sergey.
Sergey Porotsky, Hello! And please forgive me, but I'll try to help you with English again: The past tense of drink is "drank", not drinked! I know, I know: English is a horrible language.
Now to your problem. I don't fully understand your situation, but I think you are trying to predict a binary (dependent) outcome from a number of independent (predictor) variables. If that's the case, I think you could well aim for logistic regression. That is the kind of regression that deals with a binary outcome.
But I could be wrong. I'm sorry, but I don't understand terms such as "Linear Kernel of SVM", and that might be limiting the advice I can give.
Feel free to get back.
Dear Robert Trevethan, hello!
I have changed word drinked for drank on the title - thanks for remark!
About my question. My aim isn't predict outcome, I rather want to estimate influence (significance or not) of single factor for outcome. Variables are not independent, they rather strong correlated.
Regards, Sergey.
Sergey Porotsky, hello again! I have to confess that I no longer (definitely) understand what you are trying to investigate. I wish I did because I'd like to help if that was possible. Maybe some others can help.
Actually, I still think that some form of regression is what you seek. Are you familiar with regression? If not, I'd encourage you to check it out. It need not be difficult (there are probably some good YouTube presentations).
But I might be barking up the wrong tree . . .
Robt.
Dear Joseph L Alvarez, thanks a lot for your detailed answer. Let’s discuss it.
About your answer "When you can reliably predict a future outcome. Statistical significance testing is a matter of data quality testing. A statistical test is a minimum data quality…”. Unfortunately, I don’t agree with you. My aim isn't to predict outcome, I rather want to estimate influence (significance or not) of single factor for outcome. Sometimes it is impossible reliably predict output parameter (i.e. values of sensitivity, specificity and other classification metrics are too low), but it is possible to evaluate significance of the single factor. Consider above example with 5 binary factors: drink of red wine over the age of 60 years, severe illnesses by the age of 60, gender, financial status and education. If we have large cohort (e.g., 10,000 people), we can separate initial cohort for 16 groups such that in each single group values of all factors besides “drink of red wine over the age of 60 years” are fixed. Now for each single group it is easy to evaluate correct influence of the factor “drink of red wine over the age of 60 years”, although it is impossible to correct predict output parameter for single record. But unfortunately this approach isn’t applicable for not-binary factors – e.g., for 4 factors with 5 values for each factor we should consider 625 groups! What is your opinion on this? Regards, Sergey.
Sergey Porotsky What do you not agree? Prediction? Statistical significance? Minimum quality? Do you disagree with all quoted sentences?
You say, "My aim isn't to predict outcome, I rather want to estimate influence." Why estimate the influence if you do not intend to say why it is important? If you say why it is important, you are making a prediction.
If you can make 16 blocks you should have some binary tests to estimate influences. With these estimates you can estimate total uncertainty. It is the uncertainty in the estimates that is necessary to assert reliability.
What do you mean by significance? A statistical test?
Sergey Porotsky , I think that Joseph L Alvarez is asking you some important questions. Those questions make me wonder whether, all along, you have not understood what the meaning of the word predict is, in a statistical context. Your objections to the help that we have attempted to provide you with make me think that you might not have understood other aspects of statistics.
Or are those of us who are trying to help you simply not able to see the nature of your question(s)? Maybe that's the case.
Joseph L Alvarez and Robert Trevethan, once more about my question. My question is NOT theoretical or methodological. This is a simple and concrete practical question - it is useful or useless to do something. For example, from the point of view of human longevity, is it useful to drink red wine? Suppose I have statistics (for example, 10,000 people), where for each person there are his values for a number of factors, and there is the value of the output indicator (binary) - he lived to 80 years or not. Of course, you can solve a prediction task for new person: based on statistics, build a classifier (Logistic Regression, SVM, NN, NB, etc.), perform cross-validation, select control meta-parameters, estimate the impact of sample size, …. This is a difficult and interesting task, but my task is much simpler. I do not need to predict anything, I just want to conclude, based on the some processing of the available statistics - is a certain factor (for example, drinking red wine) significant for the certain output parameter (for example, the longevity of a living life)? Is it useful to drink wine or not?
When we have only one factor taken into account, this is easy to do - an example was higher, and we get that drinking red wine increases the chance to live to 80 years from 40% to 60%. When we have several factors (5 in the example above), but they are all binary, it is also easy to do such an analysis: to separate the initial cohort into groups with fixed values of all factors, except for "drinking red wine". But what to do when we have a lot of factors, and besides, they are not binary? How in this case to make a conclusion about the usefulness of drinking red wine? You give important, but too general advice (“When you can reliably predict a future outcome. Statistical significance testing is a matter of data quality testing. A statistical test is a minimum data quality. The data must meet this minimum for further consideration...”).
Once again - I am NOT going to predict anything, I just want to process the statistics and get an answer to a simple and specific question - is it good or not to drink red wine.
Robert Trevethan, about your answer "lActually, I still think that some form of regression is what you seek. Are you familiar with regression? "
Yes, I know what is regression and have used it a lot of times (e.g., SVR - Support Vector Regression). But... regression is used for Numerical (Continious) output parameter, for binary outcome (Positive Vers. Negative, Yes Vers. No,...), we should not use regression, we should use Classification.
Sergey Porotsky "This is a simple and concrete practical question - it is useful or useless to do something. ... I just want to process the statistics and get an answer to a simple and specific question - is it good or not to drink red wine."
Let us dispense with the semantics prediction.
Let us dispense with the repetition of your question.
Say you conclude it is good to drink red wine. What is your criterion for concluding that red wine is good? Assume there are no other factors involved.
Joseph L Alvarez, I only can repeat part of my first post :
"It is clear, that the factor “drinking of red wine over the age of 60” strongly influences for longevity - when switching from its value zero to one, the chance to survive 80 years increases from 40% (=240/(360+240)) up to 60% (=240/(240+160))". But this assumption ( Assume there are no other factors involved) isn't correct, really we should to take into account a lot of factors!
Sergey Porotsky
What is your criterion for concluding red wine is good?
Joseph L Alvarez, you seem to "know your stuff". Might I ask you, therefore, if you think that Sergey's question could be addressed appropriately with logistic (multiple) regression? He seems to have a categorical variable (Y) that he wants to predict from a number of predictor variables (Xs), some of which are categorical (e.g., male/female, wealthy/poor, educated/uneducated) and others of which (in Sergey's words "etc. and etc.") might be equal interval.
In my view, using logistic regression would take into account that he has a dichotomous Y variable, and he could create dummy variables for his categorical predictor variables.
What do you think? Am I barking up the wrong tree?
RT.
Joseph L Alvarez , if we take into account only one factor (drinking or not drinking of red wine), we can use any criterion , result will be same (drinking is very useful) :
ODDS Ratio (OR) = 2.25
Risk Ratio, RR (Increasing of Chance) = 60%/40% = 1.5
p-value for Fisher Test = 7e-10
Sergey Porotsky
You list several results. Let's take one, RR.
An RR of 1.5 is small. If you set your criterion to require the usual RR > 2, then drinking red wine is a non-factor.
Say you are ready to accept an RR of 1.3. Now examine the uncertainty of RR = 1.5. A simple count data uncertainty analysis indicates acceptable bounds. A total uncertainty analysis considering all assumptions and data collection uncertainties will likely find the RR of 1.5 is not sufficient.
Say you have an RR = 5. Drinking red wine is a strong factor. Now is the time to consider the uncertainty caused by confounding and known factors. What is the possible influence of education? Can it be quantified? What is the influence of financial status? Can it be quantified? ...
Robert Trevethan
There is not enough information to suggest logistic (multiple) regression. The simple example provided is not enough for dose response nor any repressible effects.
Joseph L Alvarez, hello. First, in my opinion, RR = 1.5 is very-very good result. To increase chance live up 80 years from 40% to 60% is excvellent result.
OddsRatio (B=1 Vers. B=0)=2.25 with Сonfidence Interval [1.7…2.9]. confirm this conclusion - it is useful to drink red wine.
Suppose, that other factors are also strong. If we perform independent analysis of RR, OR or Fisher test for these factors, we can get similar results - they are significant. So, may be situation, that really "drinking of red wine" isn't strong factor - rather women live more, than man. And simultaneously women prefer to drink red wine and man prefer to drink vodka. How can we estimate factor "drinking of red wine" in this case? To separate initial cohort for group of man and for group of women and perform analysis for factor "drinking red wine" two times - for women and for man. But what to do, if there are a lot of factors and they are not binary?
Sergey Porotsky If you consider RR=1.5 a very-very good result, what is a weak result? Confirmation by other tests does not make it any stronger. I keep asking you, "What is your criterion?" You refuse to give one. I offered the criterion of
RR > 2.
How do you decide the data are good enough? Can you give an answer instead of more questions?
Joseph L Alvarez OK, Criterion is ODDS Ratio > 2. Once more : if we take into account ONLY one binary factor, it isn't problem to evaluate it's significance - by means of OR, RR or Fisher test.
The data set shows longer lives for persons over the age of sixty who drink red wine. The odds ratio is 2.25, which is above the criterion of 2.0. The immediate conclusion is drinking red wine prolongs life. This conclusion is reached assuming all other factors equal or unimportant. This conclusion must be examined and interpreted.
Can you quantify the prolonging of life? The data show number of drinkers and nondrinkers alive at age 60 and age 80. Is there data for other ages? If so, can a distribution be established to estimate average difference in life expectancy or some other quantity? The difference in the groups might be unimportant if prolonged life is small.
Quantifying prolonged life will determine if further investigation is necessary.
The original question indicated that only binary comparisons were possible. The original question indicated other factors were available for binary comparison.
Compare the drinkers and nondrinkers by available factors. Are factors distributed equally in drinkers and nondrinkers? If so, no further consideration is necessary as the assumption of all other factors are equal applies.
If the two groups are equal in other factors, consider the degree of inequality and test in a binary fashion if confounding is indicated. Assess the degree of confounding.
Are there factors not included in the data that might confound the result? Estimate how much effect an unknown factor must have to confound the result.
Dear Joseph L Alvarez, thanks a lot for your detailed reply. Regards, Sergey.
When conducting experiments there is a condition for its success is that all factors are fixed except the factors under study, and this can not be applied in your case, there are many factors that can affect the life of human except red wine, including chronic diseases, epidemic diseases and other behavioral issues, and therefore
Dear Ali Alhayany, you are fully right, if we we perform Active Experiment. In this case we can build groups such, that all other factors will be fixed. But I have asked about Passive experiment - in this case we should not fix values of other (not analysed) parameters.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size