On the basis of literature survey you have to first decide solvent system or if there is no much information is available in your plant system you must screen range of solvents from non polar to polar solvents. This will optimize your solvent system and you can analyze and quantify metabolites by using different analytical methods like HPLC.
Extraction of phytochemicals depend upon polarity of the solvent, its concentration, and pH level., But all these depends that what you want to extract.