Is there any formula/technique or guideline to calculate sample size for pilot randomized trial which has primary outcome as continuous data (to detect mean difference between two groups) ?
The sample size calculation is the same for a pilot study as for a main study. If you start off with no idea about the variance or means then you cannot do a sample size calculation. In this case you run the pilot study with as many replicates as you feel like taking. If you take three replicates then your estimates for sample size calculations for the main study will be subject to the error inherent in a sample size of three. If your pilot study has 20 replicates, the estimates will be appropriately better.
If you want a fast way of understanding the problem better try taking random samples from a random number generator. In this problem you know the underlying distribution so you know what the correct answer must be. Now run a pilot study and estimate sample size. Try other distributions: Gaussian, Uniform, Beta, Gamma, and so forth. Trying other distributions is essential unless you know for certain that your data follow a specific distribution (in which case use that distribution). This activity will not provide the answer you were hoping for, but it will give a better understanding of the problem.
Need to know a lot more information to even try to answer:
Do you have background data on what the means and variances are in these groups? What kind of difference or effect size do you want to detect? What kind of analysis do you plan to use to test for differences? What kind of statistical power do you want? 80%? 90%? I would recommend meeting with a statistician because they will need to know the context of the experiment in order to help.
Is it an actual pilot study? i.e. no background information, just collecting data to use to design an adequately powered study in the future. If that's the case, they are often not statistically justified sample sizes that will test formal hypotheses. But, in general, somewhere between 12 and 20 per group is often used as a rule of thumb cited from various articles, but again, even this could vary depending on the field. Lab experiment animal pilot study and a psychological qualitative pilot study may greatly differ in what is a good sample size.