Answering as a non-expert,  I don't think that unequal sample sizes are particularly problematic for a t-test.  There are other assumptions of t-test that you should verify, though.  For example, the equality of variance for the groups.  If the variances differ, be sure you are using a Welch's t-test or other appropriate test.

There are of course other tests you can use, depending on the hypothesis you are trying to test, and depending on the nature of your data.  For example, two-sample permutation test, two-sample Mann–Whitney test, and others.

But to the larger discussion: How do you know that the difference in means is "spurious"?  If you already know the answer, why are you bothering to do the t-test?

I imagine what you really mean is that even though the means are _statistically_ different, that the difference is _practically_ meaningless.  But those are two different questions.  The p-value does not tell you if the difference between two groups is meaningful or interesting in any practical or scientific sense.

And ,yes, as sample size increases, the power of a statistical test to detect small differences in means can increase.  It may be useful to calculate the effect size. [Although Cohen's d may assume a similar number of observations.]  It also may be useful to plot means with some measure of variation, like standard error of the mean or a 95% confidence interval.

Answering as a non-expert,  I don't think that unequal sample sizes are particularly problematic for a t-test.  There are other assumptions of t-test that you should verify, though.  For example, the equality of variance for the groups.  If the variances differ, be sure you are using a Welch's t-test or other appropriate test.

There are of course other tests you can use, depending on the hypothesis you are trying to test, and depending on the nature of your data.  For example, two-sample permutation test, two-sample Mann–Whitney test, and others.

But to the larger discussion: How do you know that the difference in means is "spurious"?  If you already know the answer, why are you bothering to do the t-test?

I imagine what you really mean is that even though the means are _statistically_ different, that the difference is _practically_ meaningless.  But those are two different questions.  The p-value does not tell you if the difference between two groups is meaningful or interesting in any practical or scientific sense.

And ,yes, as sample size increases, the power of a statistical test to detect small differences in means can increase.  It may be useful to calculate the effect size. [Although Cohen's d may assume a similar number of observations.]  It also may be useful to plot means with some measure of variation, like standard error of the mean or a 95% confidence interval.

Angel,

You have to check if both distributions are nearly Gaussian. This is easy to do a quick check e.g.. is mean and median close enough ?. If yes, you can use t-Test but with the Satterthwaite Approximation which can take into account both unequal variances and unequal sample sizes at the same time,

see example here

http://www.unm.edu/~marcusj/2Sampletex2.pdf

Now, if your data are non-normal, you cannot use t-Test, please refer to this instead

https://www.isixsigma.com/tools-templates/normality/dealing-non-normal-data-strategies-and-tools/

Good luck

Thank you both for your answers, my main concern was not actually the difference of sample sizes between groups, but rather, the increase in the likelyhood of obtaining small p-values due to sample size alone.  I suspect this is the case because, even though T-test is highly significant (p

Angel,

Your interpretation is probably correct.  But this is the nature of p-values.  Tests like t-test get better at detecting differences as the sample size gets larger. 

I don't know of any simple way to address this, except to report the difference in means, and dismiss it as practically unimportant.

One complication is that, if you are able to dismiss the significance difference in means  from one variable, how do you justify paying attention to the difference in means from another variable?  

Angel: "Furthermore, I wanted to ask if selecting a random subset of data from the large group and running the comparison on it was considered acceptable."

Given that the ideal sample is the entire population (ie measure everybody/everything and you will no longer need hypothesis tests!), I would leave "throwing out data" as an option of last resort. I think the main Effect Size metrics, Cohen's d and Hedges g have ways of dealing with unequal sample sizes.

A sample size of 30 is regarded as a large sample. Therefore having selected a sample of  (n = 986) and realizing (men: 156, women: 830), should not pose a concern. You should go ahead to perform your test of differences of means. That your sample sizes are extremely unbalanced between men and women has nothing to do with your expected results because

i.              Each of the samples are large enough

ii.             The extremely unbalanced sample sizes will be zeroed out by the arithmetic mean (provided that the arithmetic mean is the most suitable average for the variable you are testing)

iii.            The SD that will accompany your mean values will further help your judgment of the results

Yes, you can select a subset of your data and perform a test on that. However, you need to let the computer do the selecting and you need to be careful on how this is done. Also, one selection of a smaller sample size from your data will not be suitable. In this approach you are asking "given a sample size of 10, what is the distribution of possible outcomes in resampling my existing data with that sample size of 10?"

That said, this will not really solve your problem. A better strategy would be to take the statistical results as given, and then add additional details that present a clear case for why the statistical outcome should be ignored. Just stating that you have a statistically significant outcome that is practically meaningless is not enough. So treatment A is different than treatment B with a mean separation of 2 units. I now need to know why the difference of 2 units is meaningless. Can you go beyond the 'I collected too many samples" answer?

Another thing to consider is to now advise your past self on what sample size you should have collected. So based on "this, this, and this factor" I know now that any difference less than 70 is meaningless. Given that I only care about differences in means greater than 70, I should have collected a sample size of 5 males and 5 females. One way to do this is to randomly resample (with replacement or without replacement, though I prefer without replacement) your data. There are three ways that this can be done, and I would contrast the outcome from all three approaches. Assuming that the best sample size was n males and n females, you could:

1) Randomly withdraw 2n observations. Here you are assuming that you have no control over the gender of respondents and that the sex ratio in your sample is similar to that of the population that you studied.

2) Randomly withdraw n males and 5.3n females. Here you are assuming that the sex ratio is more planned, or nearly invariant. It implies that there is something special or unique about the sex ratio that you observed.

3) Randomly withdraw n males and n females. Here you assume that the next study will used targeted sample sizes. Once you get n females you stop recording results from females and continue surveying until you get n males.

Inferential statistics, unlike descriptive statistics, is a study to apply the conclusions that have been obtained from one experimental study to more general populations. This means inferential statistics tries to answer questions about populations and samples that have never been tested in the given experiment.

Inferential statistics infer from the sample to the population, they determine the probability of characteristics of a population based on the characteristics of your sample, they also help assess the strength of the relationship between your independent variables, and your dependent (effect) variables. With inferential statistics, you can take the data from any samples and make generalizations about a population. 

There are two main areas in case of inferential statistics, estimating parameters. This means taking a statistic from your sample data and using it to say something about a population parameter and hypothesis tests, this is where you can use sample data to answer research questions. 

Regards