Hi, I am using CNN for flood susceptibility mapping and would like to find the feature contribution using Shap, I used the test data set to calculate shap_values but I am getting the following error:
"ValueError: shape mismatch: objects cannot be broadcast to a single shape"
and here is my code. e = shap.DeepExplainer(model, X_test) shap_values = e.shap_values(X_test) shap.summary_plot(shap_values[0], X_test, plot_type="bar")
X_test is an array with dimensions (787, 23, 23, 11) and shap_values [0] is also an array with the same dimensions 787 image, image size is 23 X 23 and has 11 bands.
I would be grateful if someone can help.
Dear Omar Seleem
These links might be useful, have a look:
1. https://stackoverflow.com/questions/16950074/valueerror-shape-mismatch-objects-cannot-be-broadcast-to-a-single-shape
2. https://datascience.stackexchange.com/questions/47287/stacked-barchart-bottom-parameter-triggers-error-shape-mismatch-objects-canno
Kind Regards
Qamar Ul Islam
Dear Omar Seleem :
The reason of this error is being that when plotting you always expect to have the x coordinate and y coordinate to come in pairs. If they don’t have the same length, matplotlib can’t figure out how to plot it.
So when you get this error, double check your input to the plot functions and make sure they are what you meant them to be.
And also have a look on these links too.
1. https://github.com/slundberg/shap/issues
2. https://github.com/slundberg/shap
Regards;
Ehtisham
Dear Ehtisham Lodhi
Thank you for your response.
The input dimensions are the same. However, I am wondering if I can generate the plot in my case as in the documentation they are showing only shap.image_plot() as shown in the below link
https://github.com/slundberg/shap/blob/master/notebooks/image_examples/image_classification/Explain%20an%20Intermediate%20Layer%20of%20VGG16%20on%20ImageNet.ipynb
Dear Omar Seleem :
I get your point. Pls have a look on these links. It will be helpful...
1. https://shap-lrjball.readthedocs.io/en/latest/generated/shap.image_plot.html
2. https://shap.readthedocs.io/en/latest/example_notebooks/api_examples/plots/image.html
Regards;
Ehtisham
Dear Ehtisham Lodhi ,
Thank you for your response.
Yes, I read the documentation. However, I would like to produce a figure showing the contribution of each feature on the prediction ( figure as the figure from shap.summary_plot),
I am thinking about reshape all the shape values that I got from my model and the test data to two dimensional array and see if it works, what do you think? i couldn't find a function that generates these figures in case of using the deep_explainer
Dear Omar Seleem :
Yes, pls reshape all the shape values that you got from model and the test data to two dimensional array and check the results. I am hopeful that it will work in this case.
Regards;
Ehtisham
- Badges
- Science topic
- Similar topics
- Computer Science and Engineering
- Artificial Intelligence