I have a data set for a railway wagon:
It consists of 9 measurement parameters per wheel (hollow wear, flange thickness, etc) and there are 8 wheels on a wagon. I need to work out a health index per wheel taking all 9 parameters into account. Y = mX + C , where Y is the health index (unknown, needs to be calculated). This leads to the use of unsupervised machine learning techniques (clustering in particular). I have used the k-means algorithm and ran it for each individual wheel. The result with k=5, was a 5 row, 9 column matrix. I don't know how to interpret the result and how I can relate it to the health of the wheel. I would really appreciate it if someone can assist me in this regard.
I am little bit confused after reading your question. It seems you are trying to do regression but you mentioned that you applied K-Means where k=5. Why k=5? I believe you can calculate y value using a regression model. Are the values of 9 measurement parameters are numerical? If possible please share the data.
For result interpretation, you need to use evaluators such confusion table in case of classification simulation.
Good day Aparna Sathya Murthy and Md Anisur Rahman . Thank you for your reply. I was afraid that I was unclear around my description of the problem. I was really trying to say that I do not have values for the output, hence I cant apply linear regression. I understand that I need to take the clustering route for this reason.
Maybe I must start off with a bit of a background. My aim is to predict the time to next maintenance action for a wagon. I want to determine the health index for an individual wagon wheel based on the 9 measurement parameters. Monitoring the health index of each wheel continuously, you will be able to call in a wagon for service at an appropriate time.
Regarding the way I came to using k = 5; I have used the elbow method to determine the optimum k value. To be honest, I do not fully understand the results given by the k-means algorithm. If someone can explain that to me I will greatly appreciate it. I know that k-means can handle big datasets and it is easy to use but is it the best algorithm to use or can you suggest another algorithm maybe more accurate than k-means.
I am attaching some sample data. It is random values and in millimetres.
Y=mX+c is a linear regression problem whereas k means clustering is an unsupervised classification problem. If you want to apply k-means clustering for k=5, you do not need the linear expression. Start with random seeding and apply k means algorithm for every new seeding in the iteration.
5 rows indicate means of 5 clusters, 9 columns because it is a 9-dimensional data (9 parameters).
You must perform a regression. You can try many models and use some process to define the best model.
Unfortunately you can use any cluster algorithm to perform this task
From your description, it seems that you want to divide the wheels first (into 5 clusters), then try to calculate the health index of each wheel from different clusters by a linear regressor. The resulting matrix you got from k-means may represent the centroids of the 5 clusters, shown by rows, with 9 parameters, which are shown by columns.
Btw, doing a prediction is a regression problem that requires labeled data.
Thank you very much for your great response Aparna Sathya Murthy I really appreciate it. Like you said Zhaojun Jia , I need to perform a regression action to do a prediction. However, I need to go from unlabeled(9 measurement parameters/wheel) to labeled(health index/wheel) data. The 9 parameters must at the end tell me something about the current condition(health) of the wheel as a collective. As soon as you have this information, you should be able to make a prediction.
Will a possibility be to set k=8 to represent the 8 wheels and the 9 values per row to represent the mean value per parameter?
From the sounds of it, you are using a semI-supervised approach given that you have decided the k value for your clustering.
This clustering from the sounds of it should be done to group the wheels in question in the data, I would personally do a likelihood to require repairs
In the data you do not have any value for the outcome variable (Y). Therefore, it is not possible to apply regression and any other classification algorithms on the data. However, if you want to predict the value of Y for future records then the following steps (semi-supervised) may help you:
After clustering, you will know which record belongs to which cluster. Therefore, for each record you can assign the value for outcome variable Y. For example, if record 1 belongs to cluster 1 then you can assign value 1 to the output variable Y for record 1. Similarly, you can define the output value Y for each record. Then you can apply a classification algorithm such as J48 decision tree on the labelled data. The model on the labelled data can be used for future prediction.
However, if you want to apply a regression algorithm on the data then you need to add a correct output value (i.e. not the cluster index) for each record and then you can build a regression model on the data. I believe it will be difficult to assign correct output value for each record.
Therefore, I would suggest you to use J48 decision tree instead of a regression algorithm as you do not have any historical output value.
Thank you Anisur, this will be of great help. I will look into your suggestions.
Regards
- Badges
- Science topic
- Similar topics
- Public Health
- Health