Good evening

From your question it is not clear from what viewpoint you want to evaluate the design process. For example, one simple method of evaluation is to check if the product was produced within +/- 5 % of the estimated cost and within +/- 5 % of the estimated design time and fees. On other hand you may want to familiarize with the work of Dr Sebastian Macmillan of the University of Cambridge Department of Architecture, and in particularly with his work on the improvement of the design process in architecture (https://www.arct.cam.ac.uk/people/[email protected]).

Regards

I agree with this point made by Vladimir Ladinski.  When evaluating a design process, Azam, one may also consider what the product or service was intended for. A task analysis can give some insight as to whether the design meets intended objectives related to expected task performance. For additional information, you may visit the ResearchGate site for our colleague Neville Stanton who has done very good work in this area.  

I intentionally did not mention to any field and context, in order to gather diverse viewpoints of designer/researchers! So, I’m aware of generality of this question!

Thanks a lot for response !

Dear Azam.

The answer can be summed up in the word 'satisficing'. A word coined in 1947 by Herbert Simon which ask the question; has the design passed the minimal acceptable threshold for doing the job it was intended to do?

For example the Airbus A380 might have come to market within cost and time (it failed on both counts), but the real failure is in not addressing real-world airport facilities or the needs of the travelling public. It fails to satisfice the conditions for being a successful airliner.

By the same token Kinneir + Calvert's UK road signage (1958 and onwards) satisfices (it does the minimal acceptable job of directing road users) without achieving a notional state of best signage possible to design.

This is not necessarily a happy state of affairs for designers, who may hit deadline and budget, and still produce a failing design.

I may suggest reading Zeisel´s "Inquiry by Design" and Peter Rowe´s "Design Thinking" for a comprehensive classification of design processes

I’d suggest that it’s essential to differentiate the forms and criteria which constitute evaluation for the particular design process.

It’s also worth stating from the outset that the criteria suggested here aren’t definitive or exhaustive, and all criteria should be considered multi-layered and multidirectional, creating complexity.

Any design process should have two main forms of evaluation: formative and summative.

Formative evaluation informs each step of the process, from beginning to end, and can be checked against identifiable criteria such as cost, timeframe, unique selling points, functionality, viability/use of materials, fitness for end purpose, or even client response, etc. These criteria, of course, can be combinatory or discrete and are applied at every step, significant or minor in development. Formative evaluation examines, compares and analyses as the process develops.It’s application can change the direction of the process and help overcome or avoid problems. It helps shape the outcome.

Summative evaluation is an analysis of the outcome against the limiting parameters for the overall goal: functionality, cost, time, effort, quality, and context, etc. Summative evaluation analyses the outcome and clarifies any shortcomings in the current phase of the design process and flags them, ideally for resolution in the next application of the cycle.

While formative evaluation guides the quality of the design process by largely, but not exclusively practical means, summative evaluation is more critical and offers an oversight of the strengths and weaknesses of the outcome, offering information that can reach outside the limits of any specific project.

How to evaluate a design process?

I think my answer needs to be as clear as possible. But, a simple answer is not possible when the process so heavily involves the creative and artistic process of making jewelry.

As a jeweler, evaluating a design process varies. It depends on if I am working for another person, if it is a paid job, or if I am creating something for my own satisfaction, or as a gift.

Or, as an added layer of consideration for the above, am I designing something to sell in quantity?

Now if I am going to make an item to duplicate and market, there are plenty of theoretical models that will work. The answer above by Gerald Gibson really hits a lot of very good points.

But many times the evaluation of the creative process is hampered by how insubstantial the artistic process is. Many times design comes from many hours of sitting at a bench sketching over and over until it "feels" right.

Sometimes the process seems to flow out in a torrent of wildly original design. More often than not, it is somewhere in between.

So evaluating the design process of what I would call the "Artistic" process is much more of a personal evaluation.

And that enters the realm of personal opinion, and little or no standard scales to measure with.

So how do I evaluate a design process? With difficulty most of the time.

I fully agree with Dr. Gibson. For every evaluation the (intellectual) platform, the perspective, the (overall) approach, the criteria, and the measures are to be defined.

We also have to take into consideration that "the design process" starts with a fuzzy front end, and ends with a fuzzy back end. There is a contradiction in that these may have larger effect on the whole of the process and results as the more systematically structured and conducted product generation procedures, but their evaluation is much more challenging due to the inherent fuzziness and the intangible boundaries.

Dear all ! I do appreciate your help !

(Industrial designers do not have much familiarity with fuzzy logic! )

In fact, without knowing the formal side of fuzzy logic, industrial designers actually use this approach. It is enough only to pay attention to the interior design of the car, its controls, the cabin of the aircraft and many other technical objects. What is this, if not the embodiment of a real fuzzy approach!

If we accept fuzzy logic equals to systematic thinking; yes Industrial designers know it well and have used it a lot!

But It seems that this term has wider dimensions !

It's not just about system thinking. My answer was related to your question about the criteria that can be fuzzy (beautiful, easy, comfortable and so on). I want to say that even if an industrial designer does not use formal methods of fuzzy logic, he involuntarily adds the list of quantitative criteria to qualitative ones. Thus, we can gradually approach the pro-criteria design (multicriteria optimization), the list of criteria which includes both quantitative and qualitative values. Or only quality. For example, what criteria is guided by a person when choosing cutlery (spoons, forks, knives) from several options. I think that not only at the price, but mostly it is guided by emotional evaluation, i.e. unclear criteria, which he can not even formulate explicitly.

If I may, I would like to say a few more words, since this question interests me. You may find my answer about the use of fuzzy logic and fuzzy criteria in designing far from practical problems. At first glance this is the case, that is, if we are talking about the development of a mathematical model in which the inputs are fuzzy design variables, and the output is a fuzzy criterion, then often for a really working model it takes quite a long time. But what is important here is that when they start thinking in this direction, it turns out to be very important. This not only disciplines thinking, but there are always tools that allow you to see something that until then has passed by. There is not enough space to explain this idea, but I think you understand what I want to say.

Very interesting question and educating answers. Thank you for the recommended books I'll read them.

My attempt to answer the question will be based on my experience as an architect and a researcher. Satisfaction is key in design evaluation. Thresholds of satisfaction exist hence we never have one examiner for design projects. A panel examines and the average is taken. Things that influence the design examination are trends, economy and personal taste. It's always a conflict between the designers choice and the examiner's standards. There are many design principles and the best designs are those that take the principles and apply them uniquely to any situation.

Not a bad discussion! It is interesting that banal criteria are not discussed. As for emotional evaluation criteria, I would like to offer a book Jonah Lehrer "How We Decide". Of course, it is popular, but for people whose design work is related to qualitative evaluations of results (development of clothing, architecture, etc.), it can be useful.

Hi Azam,

It all depends upon on your research topic. You can evaluate the design process by Reverse Engineering very easily.

Now the term "reverse engineering" is rather ambiguous. What kind of meaning do you put in this term and how can this method be used to evaluate the design process?

Bittee