Dear Kamran Shaukat Dar, I'll assume your data is going to the CNN for feature extraction, then through an SVM for your final prediction and you perform backprop on the output of the SVM.

So I see two main ways to improve your model:

1/ Optimize the CNN part by using SotA training methods. I had quite good results by (i) replacing standard batchnorm layers by switchnorm layers. Also, (ii) stochastic weight averaging helps most of the time by smoothing the loss function, and (iii) for most my projects I use an AdamW with a Cosine Annealing LR scheduler, which fasten the convergence of the CNN. All of those steps are really easy to implement with PyTorch Lightning

2/ Once you optimized the training scheme of your CNN, you can tune the 2 remaining hyperparameters of your SVM, C and Gamma. To do so you can use a random search.

Don't forget to keep a separate test-set, to ensure your hyperparameters aren't overfitted on your training & validation sets

I am agree with excellent suggestion of Aparna Sathya Murthy and Clément Poiret.

Try after reducing the dimension of image size.

This depends on the nature of your problem, the complexity of the problem determines how to improve feature extraction based on CNN.

Which is the best way to extract feature!

Hello, I am sending a paper, hoping it will be useful to you, it is a study of the influence of preprocessing, segmentation and optimization in the CNN.

Chapter Accurate Identification of Tomograms of Lung Nodules Using C...

Design a effective CNN model to extract the feature.

Can check out our recent paper https://link.springer.com/article/10.1007/s00521-021-06629-9

You can introduce weights to features for SVM as described in this paper. Also in this paper you can see how the formation of hyperplanes affect with weighted samples in SVM. Have a look