I am not sure to correctly understand your question. I can see at least two different meanings of what you are suggesting. The first one, the trivial one, is about "parameter space exploration": in this case you are looking for (hopefully) best parameters to model a real system. In this perspective, any model (i.e, any combination of different parameters for the same model implementation) could be interpreted as "a different" model. But of course, I suspect that this is not what are you looking for. The second meaning is about "different types of models" (meaning this that the modelling approach itself could be different, in principle; e.g., the level of agency - individual, group, population,...); this is a very interesting question, indeed. You are asking, if I am not wrong, what algorithms have been suggested to evaluate different modelling approaches. I have to admit that I never thought about this; but, the very first thing that pops into my head is "genetic algorithms". You could imagine to have a set of models (potentially, very different ones) and using a sort of fitness function to evaluate them. Of course, you need to properly define a "good solution" from the real system (as a benchmark for your models evaluation). Then, you can run different models, and accordingly with the fitness function you defined, assign them a "score", potentially make them reproduce (eventually with a mutation, e.g., changing some parameters, rules or agents' properties) and finally see what happens. Resulting model(s) could be the most appropriate.

other kind of possible approaches are related to different kind of evolutionary algorithms (e.g., PSO-like) or more statistical-oriented approaches (e.g., likelihood functions).

In any case, I see at least a couple of different "parts" of models where one could try to change things: at the individual level (i.e., at the level of agents' properties, behaviours and interactions), or at the model "structural level" (i.e., the architecture of the model itself). I suspect that your suggestion could be a very interesting topic for a whole research project... I hope this may help (even if it doesn't really answer to your question....)

I've never come across such a thing (though it would be useful). Does a negative count as an answer? Of course people search parameter spaces more or less systematically and have used "clever" optimisation techniques for this:

http://forrest.stonedahl.com/thesis/forrest_stonedahl_thesis.pdf

I think people have also come up with "agents" in restricted languages (like genetic programmes, http://www.sciencedirect.com/science/article/pii/S0165188900000300, so maybe in these simplified agents you could define a "space" clearly) but I guess the wider problems are:

a) What is the search space over possible decision processes for an agent? (As opposed to parameters within a given decision process?)

b) Given how slow many ABM run just once, would it be possible to search such a space (if you could define it) in a remotely feasible time frame?

Ooops. Sorry, I was composing my post between interruptions. I wasn't ignoring Marco's post. I didn't see it till I submitted.

Thinking more though, it gets trickier. Let's suppose we can do this, and we can agree on a set of data that the model needs to generate to "pass" as a candidate model. We still need to figure out what to do with parameter values in each run. We can't tune those as well or we have far too much freedom and can probably prove anything. But then we need to have at least parameter ranges for each possible model style (if people are rational, then they have this discount rate, if they are habitual then they have this satisficing level) and that actually gets a bit weird because it isn't clear how you can be agnostic about how people make decisions while still collecting effective data about the parameters for each possible model. It may be that this is almost a thought experiment. Suppose we could really do this, how might it work and what would we really know as a result?

To Marco and Edmund: Yes, you are quite correct – NOT search through a model's parameter space (some standard techniques -- but in heuristic search through a massive parameter space is surely far from easy!) but search of a hierarchy of AB-models (e.g. next level up by abstraction/aggregation).

NB I am looking for useful algorithms with computational thinking behind them.

Arguably the key steps re crawling over hierarchy of AB-models in computer are: implicit(?) definition of hierarchy of models including operational algorithm moving from one model to another, operational heuristic search algorithm, operational algorithm to assess performance/suitability of a model (does this have to run the model i.e. experimental trials? Edmund gave interesting  thoughts re this)

My little (all too terse!) paper “Algorithmic Analysis of Production Systems…”  -- readable on RG and downloadable – does report some relevant  algorithms (implemented and to a degree tested – using a particular type of programming language confusing called “production systems”) but arguably needs  much further work to develop the ideas. I believe this may be moving in a potentially productive direction (but am I wrong!?). So I am very interested in anyone else’s thoughts about these matters especially if coming from a different background (NB mine is originally in classical AI).

Which is why i posted this question!

Thanks. I will have a look at this. It has been at the back of my mind for some time. For what it is worth, I think I would count production systems as another way of "closing" the search space in the sense that they define a grammar which agents cannot "go outside". For example, we know what is in the set of legal PS possibilities. However, this is only a "half way house" as regards to your original question in the sense that we have no closure (for example) on the set of sensors or effectuators that the agent might have.

Re previous answers, anyone unsure about the meaning of "Production System" in the context of Computer Science and AI might like to glance at the corresponding Wikipedia entry which is a good quick read.

Jim