I'm not all together clear about your research topic, but I do understand what it sounds like you were doing at the very start of your validation work. And it sounded to me like your initial grouping mechanism did not succeed in constructing two truly random case groupings, since the validation group had significant differences with the modeling group on distributions of some key variables. If the initial grouping had been closer to being based on a randomizing criteria, those differences would have been much less likely to have occurred, right? If this initial theory building and testing step is important, then there may be alternative ways to better construct the test and validation groupings. Have you tried sorting the total case data set on the second letter of their last names? Or their third and fifth numeral s in their zip codes? Or some similar fairly random contruct? And then take a median split... These randomizing mechanisms should get your two key groupings closer to having no differences, and allow you the initial staging it seems your analytic work requires. Best of luck..

Are you looking for how to do this via a specific software package, or a general algorithm?

How many variables, and what statistic are you thinking to use to separate the groups?

Sorry, the amount of information you give is not sufficient to answer this question in detail.

In general I would say you might want to cluster your data according to the variables your have. Maybe this clustering gives reasonable groups. Whether you will be able to find two equal groups is another question. Make sure your splitting is justified from a scientific point of view.

E.g. in R with ggplot2 you can facet the data on the variable (colour=~variable)). Binning the variable beforehand might be advisable depending on its type. Other approaches might involve clustering, possibly restricted to this variable.

You might also want to elaborate on the type of variable and what you consider equal and homogenous.

To help you best possible, you might need to elaborate on your data, as the others noted.

You can split your data by random, clustering, rank, etc., but why do you need two homogenous sub-groups?

Let's see if I'm understanding here... Are you looking for the parameter that is the best for splitting the entire set in two equal subset? no matter which is the final composition of the two sets? Have you tried with the RandomForest R package?

I am not sure if this is what you are looking for, but you might take a look at MacthIt (http://gking.harvard.edu/matchit), a R package that allows for matching, giving a set of parameters (variables). I've been using this approach to subset a large sample into two comparable groups, giving a group variable and the parameters I want to be equal between the groups. For example, I have a large heterogenous population, genotyped for a particular SNP (CC, CT and TT genotypes). If wanna match CC and CT carriers based on age, sex, and BMI, I can use this package to extract a subset of CC individuals and CT individuals that doesn't have significant differences in age, sex and BMI.

Take a look at the attached publication that shows an example of application (particularly Table 2).

Article Adenosine Deaminase Polymorphism Affects Sleep EEG Spectral ...

Are you talking about a funciton similar to filter feature of MS excel!!!

If you want to subset the data into two groups that are more homgeneous, that suggests that the most troublesome variable, as you've put it, is a two-level category. Is that correct? If so, then there's no need to literally split the data set. Use the two-level 'troublesome' variable to stratify the data in analysis. Any good statistical package can do this.

Is your study case only or case-control study?

If it's a case-control study, you can try a size-2 clustering to balance the population structure in case and control. Force each size-2 cluster to have one case and one control and dump those unmatched samples.

Why throw away data? There are other options.

You can wite an algorithm for this

Dear all - many thanks for your input, which is gratefully received.

To be more specific. I am interested in comparing blood-borne lipids between groups of patients in a case-control manner to identify disease-associated markers. I have performed an initial screen on one set of patients/controls - and then conducted a validation analysis on a separate group of patients/controls to verify initial findings. Validation analyses look different to initial screening results which after investigation appears due to underlying differences in general blood parameters (e.g. total cholesterol, triglycerides etc) which we were not aware of at the time.

To assess the results more stringently, I would like to:

1) Combine all patients from both screening and validation cohorts to initially assess the overall distribution of blood parameters in the whole population assessed (total n=200).

2) Split the population into 2 equal(ish) cohorts of ~100, which are matched according to total cholesterol. Matching evidenced by p>0.05 (preferably higher) by univariate analyses. Please note, I am not trying to remove data here, merely spread the most awkward variable (cholesterol) evenly between the groups.

3). Repeat steps 1 and 2 for the controls and then compare the metabolomics data in a case/control manner in the 2 new patient/control populations to assess the association of lipid species in a more representative population.

This is, of course an over simplification as the patients/controls will also need to be matched for age, gender, CV risk factors etc, but once the cholesterol issue has been solved the rest should be relatively easy.

I think you are reinventing the wheel. It seems that you are just needing to control for Cholesterol (and other covariates) , which is easily taken care of using an analysis of covariance (ANCOVA). Prior to the ANCOVA you will need to consider the potential for correlations among the X variables (age Cholesterol etc.) and if strongly correlated you will need to conduct preliminary analyses to combine related variables, form propensity scores etc. I would start with a standard reference on multiple regression and analysis of variance such as Neter et al....

http://www.amazon.com/Applied-Linear-Statistical-Models-Neter/dp/0256117365

I'm with John Kern on this one, but I'd add the following. Since you are investigating disease-associated markers, you stratify your populations on total cholesterol, using a published cutpoint of, say, below 200 vs. 200+. People with 200+ will be at higher risk of stroke, heart disease, et al. People below will have less risk. There's really no need to match.

Any comments are reasonable

I'm not all together clear about your research topic, but I do understand what it sounds like you were doing at the very start of your validation work. And it sounded to me like your initial grouping mechanism did not succeed in constructing two truly random case groupings, since the validation group had significant differences with the modeling group on distributions of some key variables. If the initial grouping had been closer to being based on a randomizing criteria, those differences would have been much less likely to have occurred, right? If this initial theory building and testing step is important, then there may be alternative ways to better construct the test and validation groupings. Have you tried sorting the total case data set on the second letter of their last names? Or their third and fifth numeral s in their zip codes? Or some similar fairly random contruct? And then take a median split... These randomizing mechanisms should get your two key groupings closer to having no differences, and allow you the initial staging it seems your analytic work requires. Best of luck..

The parameters will be traced or planned in the collection of data itself. so only the collected data from patient population will show some heterogeneity. there are two basic options. You got the answer already, Split them by clustering. now you have the parameters already. there are many basic clustering techniques where you can set the parameters by yourself Joseph.

more over splitting could be done by N numbers and in N parameters too.

I guess you talk about hundred or thousend or more samples as we have often in vitality screenings of pants in field trials and high heterogeneity in the values of our experimental parameter:

1) Rank the values from lowest to highest. You get for each sample its rank in the group versus parameter value.

First you see if the distribution is symmetric or not. If not you always can transform your Parameter values Y into logY or 1/Y etc and do the same rank-plotting.

2) If you want to split your data set in 2, 3, 4.....etc groups, then select every second or every third or fourth sample from the rank-list and make this way 2 or 3 or 4.....groups of samples with a highly simiar heterogeneity in respect to the parameter you used for the rank pots.

(most tabulator programs can rank your samples upon a click) Good lluck

Reto J. Strasser