Given that you have a good and representative set of features and/or metrics, then a clustering algorithm (e.g. expectation maximisation) can be useful for initial exploration of anomalies. This is especially useful if you do not have a representative labeled test set.
If you have a labeled test set, then you can apply supervised learning methods, for example support vector machines (SVM). A challenge is however the lack of good test sets. The KDD cup data set is a synthetic test set that is well known for being biased and severely outdated.