I'm trying to investigate the variations in the preferences of the 142 users in a specific interface which give users ability to choose they own interface design. also, I'm trying to know the consistency of these preferences for each user at each login to this interface.
only 4 users / 142 participant preferences were matched (0.0281).
94.5% was the average of similarity in preferences for each user(3 times of selected sessions).
Are this results significant?what is suitable test can be applied in this kind of experiments?
Dear Nader,
could you describe your setup in more detail please? More specifically:
- 142 users logged on to your system for several times, right?
- What is it exactly that you are referring to as a "session". Does that refer to each occasion a user logged on to your system? Was there a fixed number of sessions?
- What exactly do you mean by a "match"?
- How did you compute the "similarity in preference" and what are you hoping to see with this value?
- Could you outline your research question a little more? Also: what is your hypothesis? In order to assess statistical significance you need a null and an alternative hypothesis.
One more request: please, please, next time ask about the data analysis BEFORE you collect your data. I experienced situations in the past when the only advice I was able to give was: throw the data away (because it cannot serve its purpose) and start again. Well, in your case I am still optimistic.
Best regards,
Jan
First,Sorry for ambiguity.
The users were logged on the online exam system three times within two weeks, every login they choose their online exam interface preferences.
H1: There is a difference in the online- exam interface preferences based on user identity.
H01: There is no difference in the online exam interface preferences based on user identity.
H2: There is a difference in the interface preferences for the same user in different logins on the online exam.
H02: There is no difference in the interface preferences for the same user in different logins on the online exam.
When I compare the 142 user's preferences templates, only four users preferences templates were the same(100 % match).
94.5% was the average of similarity in preferences for each user(3 logins).
Dear Nader
I think that Chi-Square test ( Goodness of fit ) will be suitable for your data, using test for equal categories to test Null hypothesis.
Good Luck
Dear All,
something similar to a Chi-Squared test was my first thought, too. Then we would have a design: Occasions (3 levels) X Interface (unknown levels). But it is hypothesis 2 that makes me ponder. The Chi-Squared test does not take repeated measures into account. If we add 'users' as a variable, the expected frequencies in many cells will be zero, which is definitely too low to get a useful Chi-Squared (because it requires expected frequencies of 5 and above to provide a reliable estimate). I have to give it some more thought.
Regards, Jan
Thank you for interesting Dr. Khalid and Dr. Jan.The chi-squared was my first choice,but unfortunately, I don't have expected frequencies. Chi-squared test is used to determine whether there is a significant difference between the expected frequencies and the observed frequencies in one or more categories .
Dear Nader
The expected frequency can be calculated, by supposing equal categories, so in this case you have observed frequency and expected ( total observation divided by the number of categories ) according to Null hypothesis which supposes no differences between groups. My advice to test two groups in each test.
You can perform your test by SPSS software and the program will calculate the expected frequency, your option when use SPSS is equal categories.
Good Luck
- Badges
- Science topic