Are you talking about multi-target regression?

Regarding error metrics, you may find this work of interest:

Chapter Forecast Error Measures: Critical Review and Practical Recommendations

If you provide some examples, it may be easier to answer your questions.

Yes you CAN do R-squared, though in this setting it's called the the Intraclass Correlation Coefficient (ICC).

https://en.wikipedia.org/wiki/Intraclass_correlation

This might be as simple as looking at he the sums of squares from an ANOVA model and calculating R-squared (coefficient of determination) for the percentage of variance explained by those 5 categories. You could also set up 4 dummy variables for the groups and run a regular linearregression (the 5th group will be the intercept), and this will give you R-squared directly.

Adjusted R-squared might be appropriate, but it won't be very different.

There are other forms of ICC used for different designs, which can get quite complicated, but I think you want the simplest one.

The task you are intending to perform is classification. If your data is numerical, please select some regression models, however., if you have images of videos, please use Bayesian methods, that may be parametric or non parametric. No parametric is much straightforward (here you do not need to consider shape of the distribution, i.e., you do not need to calculate normalizing constants) method to adopt. You may use Kernel SVM etc., as a best choice or some other clustering method

Visweshwar Arulmozhi Nambi

Some questions:

1) As I can understand: on your plot x is category, but what is y?

2) Do you want to evaluate a regression model?

P.S. Please have a look at the error measures we proposed here:

Chapter Forecast Error Measures: Critical Review and Practical Recommendations

If you replace time series id with category id and replace naive method with the mean, you can have the goodness-of-fit formula based on the out-of-sample AvgRelMAE.

Is this paired data (ex. each subject has five different health dimensions A1c, BP, etc.) or is this unpaired (ex. ICC)?

Visweshwar Arulmozhi Nambi

ICC quantifies if within each region, there is low variance. Chi-Square quantifies if within each region the yield is near its goal/ideal value. Kolmogorov-Smirnov measures if a given regions distributions are similar to a reference distribution. Does each region have a goal or ideal amount of yield (or a reference distribution ex. last harvest seasons crop data)? Please add a lot more information about the goal of this project as this is a very technical question and the devil is in the details.

I wonder if the five categories have any strong meaning for what you are trying to do. A related approach (but avoiding putting sampling locations into categories) would be to make use of (semi-)variograms as summaries of the data within different areas, perhaps using variogram values at short and long distances as complementary measures of variation within a region, and then comparing these across regions.