Nice one indeed, since there are other followers to the question, it could be very helpful if you would want to share any kind suggestion. Thanks

Good question. The solution comes to my mind is a preprocessing step : applying a dimensionality reduction like PCA with K components on two unequal length sets and then perform the classical regression.

well Hadi, yes in principle could be a solution. Unfortunately my experience with PCA is that in many real (and complex) problems the principal components result not so principal...if a dim. reduction should be applied I suggest backward elimination wrapping method. Probably much more expensive, but better performing.

However, my question is more dealing with cases where dim. reduction cannot be applied for two main reasons: the initial parameter space is already small; all of them are (or it is strongly expected to be) correlated but confused.

For instance, think to cases when sometimes groups of features are occasionally blended one with each other, mixing their information contribution in a way extremely difficult to disentangle...

Therefore in such cases I cannot apply dimensional reduction. Much better would be to have a machine learning or statistical model directly able to manage patterns of variable lenght...

Hi Massimo, this is a very interesting question. To answer it properly, we must first establish why your feature vectors are of variable length. Why are the short vectors short? Is it because some features is missing, incomputable, immeasurable, or simply inapplicable?

There are numerous ways of translating a variable-length vector into a fixed length vector -- previous answers have suggested dimensionality reduction; to this you can add downsampling, truncation, feature space subselection, and application of aggregate measures. The appropriate technique really depends on why your vectors are of variable length to begin with.

Hi Massimo, here's one more way to map your data into fixed-length feature vectors. First, try to define a distance function between two instances of variable number of features. Then, use the distance function and training data to map each instance into a feature vector of fixed length (kth feature in a target instance, x, is the distance between x and kth training example). 

Thanks Guys.

The specific problem arises often in astronomy. For example when close objects are masked within an image. You could calculate the total light flux of any region, but such flux is due to the unknown contribution of more than one source. If you want to disentangle single flux contributions, by representing objects through a vector of fluxes in different bands, the resulting dataset table is like a sparse matrix. But note that here it is not a problem of missing data. Of course one way would be to apply any deblending method directly on the images. But if not, a ML model able to deal with variable structure is required...

I recommend Neural Netwoks (Self-Organisinz Maps: Kohonen or Deep Belief Networks Hinton). But I advise you first to perform a feature selection for features reduction, perhaps some of your features are strongly reliables (correlated).

This sounds like an interesting problem, I'd like to better understand it.  This is what I gather from your sparse matrix comparison:

Each sample can have, at most, d features.  Only a few features will be present for each sample, so if a sample has n features present there will be d-n zero entries (or whatever value you use to indicate missing data).  Is this correct?

I have a few follow-up questions:

1. Do you think the specific features missing (and the number of missing features) is informative?

2. When the features are present, are they continuous-valued? Binary? Categorical?

3. Are any features always missing?  Or always present?

4. Do you know which features should be present/absent for each sample?  In other words, before you calculate the features, do you know which ones will be present?  Or is this unknown until you actually calculate the features.

Hi Matthew, the problem here is that for  some patterns, some of their features have as entries a non linear combination of information coming from other features. And such non linear combinations do not follow an a priori known law. Then a ML machine should be able to deblend the information in the right way. 

It is more difficult of the relatively simple missing data problem...

I'm trying the SVM one-class algorithm, which should be indicated (at least in theory) to deal with such kind of problems, but probably the parameter space should be enriched by means of more information features...

One possible approach is to use e.g. K-Means to cluster all features in all examples (whether its possible/natural depends on your precise setup) and then for each training/test example make a histogram of the feature types appearing in that example. And use the histograms for further training (I guess that this is sometimes called Bag of visual words).

Is it possible to resample the data by some interpolation technique?