How to normalize numeric ordinal data from [1, Infinity) to [1,5]? I want to normalize relative scores which is ordinal by nature. But range of score can be from [1, Infinity). So, I need to bring it on scale of [1,5]. Can anybody help me figure it out?
Data values are double (float) type values.
This is a classical case for transformation. Several transformation techniques can be used here. IRT is one such example as mentioned by Matthew Kerry. For simplicity, I can illustrate the transformation to the following structure: 1-100 becomes : 1.001 to 1.100 as so on. Choose your values wisely to accommodate your range of values. Remember that ordinal rankings have no direct statistical context with their absolute values, but to their relative scale and variance. Statistical analysis is build on this core principle. Handling infinity is challenging. Hard to answer without the proper context. Good luck with your analysis.
Dr. Arumugam Thiagarajan Matthew Kerry
My data is more biased towards small values. I mean data contains large amount of values b/w [0.0, 8] and very less frequency of bigger values. But, after normalization to [1,5] data should be uniformly distributed b/w [1, 10].
If we do linear transformation, due to a single large value only, scores of small ones will be too diminished.
Can you suggest me a better approaches?
Once the data is normalized, the distribution of values are of no concern. You should be good with analysis. But make sure you test the normality of the data (eg. Shapiro Wilkin test) and your p value is more than 0.1. You have to test various transformations based on the variability of your data to obtain normality. This will be an iterative process. Generally, statistical packages come with a suite of transformation options. It is unwise to recommend a solution with knowing the nature and context of the data.
We had a similar problem in the context of educational assessment where very often people work with "judgement scales" of both types, i.e. two-sided bounded (like your [1,5]) as well as one-sided bounded (like your [1,inf] or [0, inf]), and where they have to merge those distinct scales in some way.
Fortunately, there are not so many ways to do such mergings, if you want to keep things simple (Ockam's razor) and choose the right mathematical framework which suggests the appropriate transformation to you. Just picking the one or another transformation to fit your data e.g. to a normal (or some other) distribution is IMHO an unsuitable statistical approach which doesn't do justice - plausibly - to the inherent structure of your data.
For instance, in our case, it was plausible to assume a sub-additive merging of scores, something of the form a + b - a * b, and derive a structure which turned out to be equal to a semi-module. Knowing that, everything else could be derived - without guesswork.
The 1-1 transformation from a bounded to a unbounded interval we used had the relatively simple form: 2-log ( score / (1 - score) ), where 2-log is the binary log function, and scores are numbers between 0 and 1.
The resulting numbers are between -inf and +inf, but of course we could adapt it such that the resulting numbers start from 0 onwards (your case). Because the transformation is one-to-one, it has an inverse which goes the other way back. Works great! [ for the aficionados: connection with IRT obvious but not intended!)
Probably there are statistical techniques for doing (significance) tests with your data, but that is none of our concern in the given non-research context. But it appeared (quite) simple to define correct means and standard deviations on data from a bounded range (obviously different from the usual aritmetic or geometric means!).
- Badges
- Science topic
- Similar topics
- Computer Communications (Networks)
- Simulators