Hi Roman,

Perhaps include a message that says that the input data does not fall into the range of the program, therefore the program cannot take responsibilities in the outcome.

For Reinforcement agent, I guess you can parameter the range?

Yohann

In a simple two-dimensional case, this would certainly be possible.

In a high dimensional case it might not be possible to overlook this.

I am looking for a method to generate this information.

The Bayesian optimization could be an approach. However, I would like to continue using the advantages of deep learning. The topic Baysian Deep Learnig is comparatively young. There is little experience and libraries compared to classical deep learning.

In Deep Reinforcement Learning with continuous actions it is difficult to allow only certain areas. The agent usually chooses a continuous value between a minimum and maximum value.

what deep learning method and software are you using ?

Yohann

I create regression models from measured data. I use them in Deep Reinforcement Learning.

Some of the physical models have more than 10 input values. All in all I have very large data sets.

For both I use Tensorflow.

Hi Roman, I guess you program your Reinforcement Learning agent yourself? In that case you can limit the actions (including exploratory actions) the agent takes, right? So, let it choose any (x,y) combination to investigate in the shaded region? Or do I mistake your question?

Chiel

Hello Chiel,

yes I program my agent myself and can influence the action choice and exploration.

Here is a minimal example:

The agent can set y in a range between 0 and 1. In the range of 0

OK, Roman, I see.

So depending on state values the allowable region for (x,y) is different, but you don't know the exact relationship between the two (state values and region)?

How then do you know that some (x,y) values are not feasible?

On the one hand I can use my measured data for modelling.

On the other hand I can have a look at the distribution of the data. Thereby the gaps in the space becomes visible.

So you don't know the relationship between state values and allowable (x,y) region, but you can have a look at allowable (x,y) data points? What about letting your agent sample from this data (instead of modeling it), so sample from the 'empirical distribution' given by the data points themselves?

That's a very good idea. This would directly maximize the model certainty.

So as a result of this, I choose discrete and no longer continuous actions?

For example, in certain states I would have 3 valid actions, in others 300?

By using Python library......its possible if you are using python for programming.

Keras is an easy to use and powerful Python library for deep learning.

There are a lot of decisions to make when designing and configuring your deep learning models. Most of these decisions must be resolved empirically through trial and error and evaluating them on real data.

As such, it is critically important to have a robust way to evaluate the performance of your neural networks and deep learning models.

# MLP with automatic validation set

from keras.models import Sequential

from keras.layers import Dense

import numpy

# fix random seed for reproducibility

numpy.random.seed(7)

# load pima indians dataset

dataset = numpy.loadtxt("pima-indians-diabetes.csv", delimiter=",")

# split into input (X) and output (Y) variables

X = dataset[:,0:8]

Y = dataset[:,8]

# create model

model = Sequential()

model.add(Dense(12, input_dim=8, activation='relu'))

model.add(Dense(8, activation='relu'))

model.add(Dense(1, activation='sigmoid'))

# Compile model

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

# Fit the model

model.fit(X, Y, validation_split=0.33, epochs=150, batch_size=10)

https://machinelearningmastery.com/evaluate-performance-deep-learning-models-keras/

Yes, Roman, that's the idea. It does (indeed) also mean that your agent only takes actions that are in the dataset, while there may be many other allowable actions. That's a drawback of this method. So, if you are able to come up with a model for the 'feasible region' for (x,y) that would be better...

TensorFlow Probability could be an approach: https://medium.com/tensorflow/regression-with-probabilistic-layers-in-tensorflow-probability-e46ff5d37baf

Thanks Chiel!