Hello Serban,

As rated quality of wifi is likely to be an ordinal score, then, strictly speaking, I don't believe you'll be able to talk about x% improvement in that variable. As well, I'm not sure I understand how it is that you'll require control and test groups for an existing data set. If attention is reserved for the explanatory power of rated wifi service, after all other relevant variables/features have been accounted for, then I think you'd likely want to compare: (a) log likelihoods; or (b) change in satisfaction status; or (c) adjusted odds ratios of satisfaction between two models:

1. An explanatory model that has all relevant variables _except_ wifi rating;

2. Model 1 with wifi rating added.

That will come closest to giving you the "marginal" or value-added contribution of wifi rating as regards satisfaction status.

Apologies if I've misunderstood your query.

Good luck with your work.

Hello David,

Thank you for your inputs!

Regarding your doubts, I can explain a little bit better the rationale behind the control & test groups.

After applying the decision tree and extracting the features’ importance ( = which service separates the best the satisfied passengers from the non-satisfied ones) I received some feedback from some of the professors in the committee. They were asking about the level of confidence that we can have in these results. Investigating further (statistics are not one of my strong points as I’m an aerospace structural engineer by studies) I understood that this level of confidence arises from hypothesis testing.

That’s why I am trying to find a way to generate these control & test groups from the dataset that I already have - in order to test the hypothesis that indeed, improving the percieved quality of the wifi service results in a larger number of satisfied passengers.

I hope it’s clearer now. If you think that there are other methods from which I can obtain the confidence level in my results, please let me know!

Again, many thanks!

If I got your question correctly, there is no need to determine the control and test groups. It is enough to separate cases into different disjoint groups.

Hello again Serban,

OK, so the usual nomenclature is "training" (or model building) and "test" (or validation) groups. The training data set is used to develop the model, then the model's accuracy/efficacy is ascertained by applying it to the test/validation sample.

With a large enough initial sample, you could randomly select some fraction of that to serve as the training/model building sample, and hold the remainder out to use as the test/validation sample. There are fancier schemes (so-called k-fold models) that data mining folks like to use, but this is the basic framework.

Choose the initial sample size to be sufficiently large to allow a high degree of precision (and, stability: all other things equal, sample results are less volatile across large samples than across moderate or small samples). It's up to you to declare what degree of precision you'd like for any parameter estimates deriving from model building, and to select the training sample size accordingly.

Good luck with your work.