There is an extremely good publication where the objective was to optimize a semi-conductor factory. You might find it useful to check which metrics they were using: "Generating dispatching rules for semiconductor manufacturing to minimize weighted tardiness"
My latest industrial PhD student, Karin Thörnblad, worked on a flexible job shop problem at GKN Aerospace. She did test several objectives, and found that makespan in particular was not suitable. Here is a link to her thesis, especially the introduction. She also has some published papers that you will find links to:
The FMS performance is measured by its operational availability index, which is determined using the production output rate of the FMS under a variety of time between failure distributions and different operational conditions.
Here could be used many criteria,mainly process cost, productivity, energy comsuption. All of them commonly reflects the performances and depends on the manufacture politic dedided.
FMS should be able to minimise handling and machining. It should maximize the utilization of each macine. During failure the system should be able to run