I am still new with Apache Spark. I trained a LogisticRegression model using crossValidation. For instance:
> val cv = new CrossValidator().setEstimator(pipeline).setEvaluator(new BinaryClassificationEvaluator).setEstimatorParamMaps(paramGrid) .setNumFolds(5)
> val cvModel = cv.fit(data)
I was able to train and test my model without any error. Then I saved the model and the pipeline using:
> cvModel.save("/path-to-my-model/spark-log-reg-transfer-model")
> pipeline.save("/path-to-my-pipeline/spark-log-reg-transfer-pipeline")
Up till this stage, the operations worked perfect. Then later on, I tried to load my model back for prediction on new data points, then the following error occured:
> val sameModel = PipelineModel.load("/path-to-my-model/spark-log-reg-transfer-model")
java.lang.IllegalArgumentException: requirement failed: Error loading metadata: Expected class name org.apache.spark.ml.PipelineModel but found class name org.apache.spark.ml.tuning.CrossValidatorModel
Any idea what I may have done wrong? Thanks.