There are a variety of recommendations for sample size in SEM, but one common one is 10 observations per parameter in your model. If you are testing a "moderator" model then you will need to have that many observations in each treatment category, but if you are only testing mediation, then you can include both subgroups in the same model with a variable representing treating condition.

The standard error of the mean (SEM) is a measure of the dispersion of sample means around the population mean. It is the estimate of the standard deviation of the sampling distribution of a parameter (mean). A random sample is a prerequisite. Only if you have one, the estimation of population parameters (inference statistic: obtaining confidence intervals) makes sense. Even if you have a random sample remember that small samples tend to systematically underestimate the population parameters. Multivariate combinations of characteristics lead to small or no numbers of observations for a combination. A 2x2x2x2 table has 16 combinations. 100 cases evenly distributed lead to approximately 6 or 7 cases per cell. More likely ist a distribution with empty cells and doubt about the existence of these combinations of characteristics in the population.

Are you asking about statistical power or about model convergence?

David's advice pertains to having enough information in the data to enable the estimator to converge on a solution (parameter estimates).

The other important issue is power. For that, Monte Carlo simulation will probably be the best approach, because it can accommodate your exact model.