Point biserial is just a special case of the Pearson product-moment correlation. In fact, the same data may be plugged into any software or calculator that performs a Pearson correlation and the answer will be the same as that obtained from textbook formulas for pt-biseral. Sample size is important for two reasons: precision of the estimate, and power of a statistical test that population correlation = 0. Bigger N is better on both counts. Minimum N to compute and test a correlation is 3 cases, but nobody would want to rely on such a small N. How much bigger can be ascertained if you're able to quantify the desired level of precision (for a parameter estimate) or the desired level of statistical power (against some specific effect size alternative).
It is clear that to calculate a point biserial correlation you need 3 or more data points. To do a point biserial analysis you need more than that. how many? I think this is your question. This is an estimation problem and the solution depends on the variance of the estimator.
The link I of my first answer gives the variance of the sample point biserial correlation. You can isolate $n$ from this formula:
where ro is the population point biserial correlation, P is the proportion of group coded X=1 and Sg is the standard deviation in the formula.
If we denote the (1 - alpha/2) quantile by Za and r the sample point biserial correlation, the confidence interval (CI) is given by (r - Za * Sg, r + Za * Sg). Thus Sg is the key to calculate the CI. The length of the interval is 2 * Za * Sg.
Give some arbitrary but reasonable values to ro and P. For example, if you find that 40% of the subjects in the population have X = 1, set P = 0.4. More important are the values you assign to ro. Values near zero require very large n. Values near -1 or +1 require moderate or small n. You can work this in Excel.
Example, Assume P = 0.4 and suppose you want a CI length of 0.08 and a 0.95 confidence coefficient.
2 * Za * Sg = 0.08 Then Sg = 0.08 / (2 * Za) = 0.02040854
Give some values to ro and apply in the formula above to calculate n. Next I write the code in R language: