R programming language
I am considering if is it appropriate to use two different randomly chosen samples coming from one huge database to proceed two logistic regressions separately on the same subject?. The main cause is a low power of my computer and no possibility to use own written multimatching function that binarizes whole data into 0 and 1 (follow / not follow).
The database consists of 1 500 000 obs. and 54 variables (data.frame). The DV reflects the act of following one of two presidential candidates (1 and 0) and IVs reflect the act of following particular media outlets appearing on Twitter (also 1 and 0). The aim is to present association between media and political agenda and predictive power of particular media.
Unfortunately, I am forced to sample the data because of the computing time. Hence, I am going to randomize two samples (2 x 100k records), proceed the regression, and then, confirm the first one using the second one. Is it consistent with methodological / statistical art ? Thank you in advance.
Yes, you can report experiment 1 and then experiment 2. You can write general methods for both experiments or method 1 and two. The same thing goes for the discussion.
Yes, you can report experiment 1 and then experiment 2. You can write general methods for both experiments or method 1 and two. The same thing goes for the discussion.
Generally, what you are attempting aligns with methodological standards. You are using two (sub-)samples to conduct the same analysis and expect very similar results. However, I wonder why you are not using 200k random records in the first place, as this should result in the same computing time without the need of two separate models.
Hi, Jacek,
It is the duty of the researcher (study developer) to justify the methodology based on all relevant theoretical grounds--a function of the experimental hypothesis under investigation. There is no "yes" or "no" answer to such a phenomenon--the answer is situation (theory, method, and application)-specific.
Have you looked into "K of N-unfolding" and bootstrap (50% with replacement) methods? These methods would help you address and understand well the reliability and validity aspects of your model.
PS: If and when you are interested in more powerful modern methods than logistic regression, think of the following paper as being a "red pill" which begins to expose the legacy statistics matrix: https://odajournal.com/2020/06/28/what-is-novometric-data-analysis/
- Badges
- Science topic
- Similar topics
- Social Science