In the mixed-variable heuristic optimization domain, what is done when a categorical variable determines the existence of continuous or ordered discrete variables in each possible solution?
To illustrate, imagine an optimization problem to determine the best tool to cut paper.
In this problem, a variable tool can have the values "knife" or "scissors".
- If its value is "scissors", there's the continuous-valued blade_size variable.
- If it's "knife", there is the same blade_size continuous variable and also a num_of_teeth discrete variable
How can I deal with this problems using some metaheuristic designed to hadle categorical, continuous and discrete ordered variables?
My first tought was to set the problem to the max possible dimensionality and, after choosing the value of the categorical variable, select (if commands) which other variables are going to be optimized and used to evaluate the solution.
This probably will work, but it seems naive to me. Do other more sophisticated methods to deal with this kind of problem exists? If yes, what are these methods?
Dear Victor,
I suggest you to see links and attached files in topics.
-Population-based Heuristic Algorithms for Continuous and ... - ULB
iridia.ulb.ac.be/~mdorigo/HomePageDorigo/thesis/phd/LiaoPhDThesis.pdf
-Latent Class Analysis Frequently Asked Questions (FAQ)
www.john-uebersax.com/stat/faq.htm
-Linear Optimization
home.ubalt.edu/ntsbarsh/opre640a/partviii.htm
Best regards
Dear Victor,
How many values can take your categorical variables?
If they can take only two values it is way easier to solve it as a bi-level optimization problem where you:
(a) Choose one value of the categorical variable
(b) Solve the resulting optimization problem
(c) Choose another value of the categorical value and solve again the resulting optimization problem (nested optimization)
(d)... Repeat the same procedure for all possible values of the categorical variable
At the end, your optimal solution is the one that is minimized further (as you phrase it, this is the "naive approach").
Bi-level optimization is used in many problems (i.e., the toll setting problem in transport where for any possible tax structure an optimization problem is solved). If your categorical variable can take thousand different values though, bi-level optimization cannot be applied anymore (too computational complex).
Best,
Costas
Have you tried a type of analysis called the minute analysis. This analytical framework is especially suited for treatment of categorical variables.
Thank you all for the answers. I am really sorry about the delay in responding back.
That is a great list of readings, dear Mohamed-Mourad, and they helped me on having a broader view about the optimization field. I already use some of them as references (e.g. Liao's work), but unfortunately I couldn't find help with the specific problem in these readings. Maybe this was my mistake, so please give me further indications if you have them.
I'm currently using four possible values, dear Konstantinos. Maybe in the future there will be more, but it wont reach dozens. Bi(or multi)-level optimization is actually a really interesting way of dealing with this problem. I didn't mentioned this fact in my question, but I'm using metaheuristic optimization in model selection of classifiers. The proposed solution is likely to be problematic in this field because of the large number of function evaluations needed. When this happens, it's very likely that the phenomena of overfitting in the second level of inference appears. The solution I found was modifying the metaheuristic I'm using so the categorical variable is optimized first. Then, depending on its values, the optimization of other variables is performed (I'll omit specific details because the proposed solution is yet premature and being tested).
I'm not aware of this analytical framework, dear Narahari, and couldn't find more details about it on search engines. If you could attach some material about this subject or indicate me the title of a work that describes it, I would be pleased.
- Badges
- Science topic