I am trying to understand patterns in a very large institutional database. Unfortunately I can only get the data from, “dashboards.” I can construct elaborate cross-tabs, though never everything I would like at once. For all the these dashboards / cross-tabs it’s (logically) possible to reconstruct a datafile for statistical analysis. That is, rather than having cross-tab after cross-tab where cells are the number of participants, I can (in principle) make a file where each row is a participants and each column is one of the values from a cross-tab. But since we’re talking about tens of thousands of participants, this would take years to do by hand. Is there any software where I could upload the cross-tabs and have a computer produce a datafile for me? When I tried searching, I found many options for creating dashboards from a datafile, but none that do the reverse. Thank you for your help! ~ Kevin
The dashboard likely provides only aggregate information and does not allow individual students' data because it would allow individual students to be identified and that would be against FERPA. However, your institutional research department should allow you access. You'll have to sign some confidentiality forms. There are functions in R for recreating rectangular data from contingency tables, but you would need an array of dimension equal to all the variables. If this is for CSU Monterey, they should allow access without problems for you, and other places will likely allow it with a justification and a MoU (Memorandum of Understanding).
If you have a large cross tabulation there is no point in turning into an individual level file - both the aggregated and the individual file will have the same information content . And you can do the same analysis say using a Binomial logit model ( with a varying denominator) instead of the equivalent Bernoulli model (with a denominator of 1) - you will get exactly the same results.
See Article Multilevel perspectives on modeling census data
So you could do a few variables at a time to see the major patterns, but you always have to be aware of potential confounding
and the problem of Simpson's paradox
https://en.wikipedia.org/wiki/Simpson%27s_paradox
Even if you have access to the full individual table you may want for efficiency to turn it into a large table for analysis
Uncovering interactions in multivariate contingency tables: a multi-level modelling exploratory approach
Kelvyn Jones, Ron Johnston, David Manley
Methodological Innovations Online, 9. DOI: 10.1177/2059799116672874
Article Uncovering interactions in multivariate contingency tables: ...
Thank you all for your feedback. Unfortunately, I didn’t learn R in graduate school (only SPSS), so hearing you can use it to convert tables to a datafile is yet another reason for me to learn. Yes, getting the data directly from the creator of the dashboards would be ideal, but unfortunately priorities of different offices do not always line up. Thank for the article , it’s a very similar issue and I’d wondered if I could do analysis of interactions directly from the cross-tabs. Concerning Simpson’s paradox, or other statistical demonstrations, is there any evidence that researchers would be less likely to miss important interactions, or more likely to see false positive distinctions with small N aggregating across many cross tabs, than if data were analyzed where each row is a participant and each column a variable (that would appear along the sides of a cross tab)? This could be helpful for explaining why we need access to a datafile rather than just dashboards. If anyone does know of software, free, online, or requiring purchase, that I could use to go from cross tabs / dashboards to a “raw” data file, please let me know.
- Badges
- Science topic
- Similar topics
- Medicine
- Physiology
- Biosignals
- Biomedical Signal Processing
- Statistical Analysis