Well, you can technically do PCA on anything.  :P

As for what information you'd get, if you performed PCA on population data, you should expect your components to represent the population and not the preferences of individuals. 

Each item was rated by 100 people, so I'd expect the bias caused by different people using different ranking systems to be quite small.  I don't work with survey data, though, so couldn't put an exact number on it.

Please ignore the previous answer.  You could definitely perform PCA/ICA over indivials' data simply by splitting your dataset into the 16 items; one PCA per item.

Can you expand on what "one PCA per item" means? Since PCA is related to finding the eigenvalues of the cor/cov matrix, is the matrix you are suggesting still of all 16 items.

HI Chris and Daniel, 

I don't really see why I would perform a PCA on a single variable...But just to clarify: The goal of the study was to detect components of aesthetic experiences when seeing certain images. The 'survey items' were 16 adjectives on which subjects were supposed to rate the images. To make the task easier for the subjects, each subject only got assigned two adjectives at the beginning of the study. Then they rated several hundred images on these two adjectives. Other subjects got a different combination of adjectives.  

This means that from each subject, I only have ratings for 2 out of 16 possible adjectives. The goal is still to detect components(covariation) between all 16 though. Intuitively, I would say this analysis would be flawed, because I know that the ratings have been obtain from different populations. But I had a hard time finding anything about it. 

But today I found this: http://support.sas.com/publishing/pubcat/chaps/55129.pdf On page 13 it states rather explicitly that you can only perform PCA on data which is measured within the same population including all measures. As soon as 1 variable rating is missing for a subject, the entire dataset from this person must be dropped. 

What do you think? 

Different packages will have different options for dealing with missing values and how to either estimate the entire data matrix or just the covariance/correlation matrix. If you are happy with the estimated matrix, then you can use the procedure (e.g., eigen with correlation matrix, svd with data). I find with imputed matrices you often get oddities like negative eigenvalues. I think having only 2 per person may yield an odd pattern. The reliability of the estimated matrix might be low so you might want to run the PCA on several estimates. It depends though on high you want to do a PCA. (and also the mode of the PCA, Cattell talked about this). There are likely alternative things that you can do depending on your goal.

I also don't understand what it means to do a PCA on a vector (i.e., on one variable). Hopefully Chris will tell us soon and we can learn.

Ohhhh, my apologies.  I thought that the subjects received a subset of images and rated them based on the same criteria.

Still, if you're trying to find out information about the images rather than the subject, I suspect that the large n (100 ratings per feature) would still make subject-to-subject bias small and you could assume it all comes from the same population.  Alternatively, you could "normalise the scales" of each individual participant by converting their responses to some sort of "average response" and "deviation from response metric".  That way, if John rated "average beautiful" as 7/10, and Sylvia rated it as 5/10, you wouldn't be assuming that something is more beautiful simply because John rated it.  I'm sure there's a way to figure out analytically how much variation you expect due to subject-to-subject differences.

Again, though, I don't work with survey data and I'm not a statistician.  Take everything I say as the advice of a layman.  ;)

May I ask why PCA and not EFA? Either way, Mplus can apply full-information maximum likelihood (FIML) using the raw data. 

I have no idea how the sparseness of the data will affect FIML, however. The design reminds me of planned missing data designs, though without a shared item across all participants (often called the X set). Also in the planned missingness literature, the amount of incomplete data per participant never exceeds the amount of complete data per participant. Given these two points, FIML may or may not work well. I suppose that's what future directions (and future methodological research) are for. 

The EFA v PCA discussion is large (and I am sure there are discussion on RG, stackexchange, etc), but relevant to this discussion EFA is more a description of what you want done and it is implemented in different ways (if you like Marr/Poggio's levels, the computational and algorithmic ... these are roughly the why/what and how levels of explanation). Also, the descriptions can vary, so the different ``modes'' I mentioned above and this may relate to doing EFA on a single variable (for example, that would work for a single item with multiple subjects in a time series), but PCA's computational level (the why) is simpler: dimension reduction. Of course in a typical data set both use covariates and can produce results that look similar despite having algorithms that look almost opposite.

@Chris

In the dataset I have, John rated the images on 'beautiful', while Sylvia rated them on 'makes me feel comfortable'. While these concepts are arguably related in terms of 'How do you find this room?', I try to understand whether it is conceptually meaningful to look for underlying components (dimension reduction). My worry is that correlation might be overestimated because of the conceptually relatedness of the measures, while the subjects were not able to give a more differentiated rating; e.g. to make differences clearer. 

Does that really matter if there were 100 Johns and 100 Sylvias?

You'll still have 100 people saying how beautiful the object was, and another 100 saying how comfortable it made them feel.  From that, you could surely make a reasonable estimate of the population response.  

I'm not sure exactly how conceptual (un)relatedness of variables given to a single subject would affect population response, but the hypothesis makes sense.  It's easy to imagine someone given only 'beautiful' and 'makes me feel uncomfortable' to see it as 'good' vs 'bad'.  You've got the dataset to verify this, though.

Chris, Dan, and Jayden - thank you for all of your helpful comments. I'm working with Franziska on this project and just wanted to clarify a few things about the design and analysis. First, we had subjects rate 200 images on 4/16 dependent variables (not 2/16), and these 4 dependent variables were assigned quasi-randomly from the original 16 so that different subjects were asked to rate different combinations of dependent measures. Regarding the issue of "missing data," we attempted to circumvent this problem by performing an item-level (rather than subject-level) PCA. In other words, we calculated the average rating for each image on all 16 dependent variables, and then created a matrix of 200 images (rows) x 16 dependent measures (columns) and performed a PCA on that matrix. The resulting PCA was therefore intended to show the relationships between dependent variables based on their covariance across the 200 images (not based on their covariance across the 800 subjects). Is this a valid approach? 

I wouldn't say it seems invalid, at a glance.

I'm not sure why you're using PCA, though.  (I've only ever used it for dimension reduction).

Would a simple table r values show their covariance better?