If you score a sentence based on the presence or absence of an error, the score is dependent on the error hence the predictor (error) is binary and the score is non-binary. Is it possible to apply some regression model in such a case. As logit and probit regression models apply to cases that are reverse of what I need. Any other method for solving the issue?
You can use a linear regression model, where the score is the dependent variable and the error is used like a dummy independent variable, togheter with other predictors, both continue variables and qualitative ones. For the qualitative ones you should transformer them before in n-1 dummy variables, if n is the number of modalities.
Thanks a lot for the response. I shall go ahead with the standard simple linear regression model then.
Yes, of course, you can go ahead as Casper suggests. You may also keep in mind the potential for assigning subtler values to the now-binary predictor; such as, "critical error" and "small error."
Hi, what I am stuck up at is that I have 6 errors . I got some evaluation done based on these errors in a sentence and got the sentences scored and know which is more critical and which one is the least one but I wish to assign weights to these errors.... but how I am confused. I can think of two ways:
1. I take error as one predictor and assign 6 values (1-6) for each error and scores obtained for each sentence is the dependent variable.
2. I take 6 errors as 6 different predictors, but the values for each of these predictors is 0 and 1. 0---for absence and 1 for presence of an error and the score obtained is the dependent variable
In case of 1 I can use only a univariate linear reg. and hence would not get weights for each of these errors but only one value, for slope and the intercept.
In case of 2 I will get multivariate linear reg. and feel will get weights of each of these errors independently. Different slopes for each error.
What do you suggest?
OK, I will try. To begin, please help me a bit:
How many individuals provided an evaluation?
How many evaluations were performed by each individual?
Did you design the test so that there is just one error in each evaluation sentence-case and, if not, how did you design it?
What is your real goal, and how do you expect to use the results?
Hi, I will explain the things further:
1. 45 individuals have evaluated a set of 100 sentences.
2. Each eng sentence was translated to the target lang and the target sent has only one of the six errors. Thus each eng sentence has 6 translations each translation has one of these errors. Thus each individual evaluated in total 600(100*6) sentences.
However, I further understand that there will be situations where a translated sent will have more than one error, for that I feel we will need to go in for Interaction effects between the errors as well . (But for that probably I need to get sentences evaluated again with target sent having more than one error)
3. The goal is to find a suitable weight (or penalty) for each of the errors which can help determine the score for a sentence provided you know which all errors occur in the target text.
Ok, it is clearer now.
It looks like an interesting experiment. I see that you may have enough data for potential findings.
I assume that you do not care to find the bias of each sentence as you may already have integrated this in your study. Therefore, i propose that you keep it simple and begin by running 6 main experiments, each including the dataset that is relevant to one error. Please include only sentences with one error.
FOR EACH EXPERIMENT: Run in 2 steps. Step one runs 45 models (one for each individual) and each dataset includes 100 data for each individual. Step two combines all individuals in one model.
What do you think?
I think its a good idea, I was so far thinking of taking up only 6 diff experiments for each error and had not thought of going in for different models for each individual.
If I have understood it right, I go in for 6 experiments. each for 1 error. Also for each experiment. we go in for a model for each individual with the set of evaluated sentences i.e 100 (for 1 error).
I will try the experiments. and see how the things come out. I will check out for the methods to be used for models combining or can we go in for model selection also...?
From what you are saying, you have got the right idea. Essentially, in Step 2 we do what you had in mind initially. Step 1 is supportive for examining individual bias.
The modelling of this experiment could easily get very complex. However, at this stage, a simple model will do. Just make sure you use the same model specification throughout.
- Badges
- Science topic
- Similar topics
- Methodology
- Research Methodology
- Research Papers