Maybe this is not the answer you are looking for...anyway, I usually perform a grid search on the SVM parameters to find the best set considering the cross-validation error...

You can try to use tuning packages like Optunity or HyperOpt to automate the tuning process. Such packages use general-purpose black-box function optimizers to efficiently find a good set of hyperparameters (much more efficient than grid search, manual search or random search).

Optunity has an example of tuning an SVM in Python using scikit-learn in the link I provided (lines 30-36, a C-SVC is used; nu-SVM would be a straightforward modification). 

http://www.optunity.net

https://github.com/claesenm/optunity/blob/master/bin/examples_scikit-learn_svm.py

I recommande this paper :

Model selection in one-class v-SVMs using RBF Kernels of Unnthorsson, R., Runarsson, T.P. and Jonsson, M.T. (2003).

https://github.com/benmack/oneClass/blob/master/notebooks/oneClassIntro.ipynb

here b. mack explained an occ classifier based on manual model selection.

@Matteo De Felice I believe what you have mentionned is not appliucable when it comes to unsupervised learning in general and outlier detetcion in particular which is the first aim of OneClassSVM

Setting up the OCSVM

For supervised OCSVM classification, the cross-validation technique cannot be applied due to the training frame containing only elements of the positive class (target class). Indeed, it is not relevant to see a positive classifier producing a zero error on subsets of the training base for a couple of parameters (nu and sigma).

The OCSVM supervised classifier is the model chosen for our thesis. The empirical hyperparameter selection model in OCSVM which is a special case of SVMs, and employed by ghafoori, (2016), unnthorsson, (2003) and then taken up by Li, (2010a) in debris detection from a high spatial resolution satellite image is adopted. In their methodological approach, the authors first set the nu parameter based on the fraction of misclassified target training data [1% or 5%], i.e. nu= 0.01 or 0.05 for a classification error rate. The value of nu chosen is 0.01. Next, they proceed to the appropriate kernel size setting (sigma) where the fraction of well classified training data giving a classification accuracy score that tends to (1- nu) is sigma=0.1. The resulting combination of nu and sigma is used in the classification of OCSVM. In this study, we will follow the method of Tax,D. (2001) by incorporating a small amount of negative (non-damage) data during training to increase the performance of OCSVM. The inclusion of a small amount of outliers, other than the target data, helps to improve the good boundary area around the data.

Ghafoori, Z., Rajasegarar, S., Erfani, S. M., Karunasekera, S. & Leckie, C. A. (2016). Unsupervised

parameter estimation for one-class support vector machines. Springer, Pacific-Asia

Conference on Knowledge Discovery and Data Mining, pp. 183-195.

Unnthorsson, R. (2003). Model selection in one class v-svms using rbf kernels. Proc. of 16th

Int. Cong. CAMADEM, 2003, 1-11.

Li, J., Qin, Q., Ma, H. & Yuan, W. (2010a). Study on road damage assessment based on RS

and GIS. IEEE International Geoscience and Remote Sensing Symposium (IGARSS),

pp. 4534-4537.

Tax, D. (2001). One-class classification; Concept-learning in the absence of counterexamples.

Ph. D thesis , ASCI Dissertation Series, 2001. 146 p. (Doctoral dissertation, Delft University

of Technology).