I think in this case, you can look for any reported ligand in literature for your protein.

Else you can check if your protein is produced by different organism and if its structurally similar, then you can check the action site and the ligand of this molecule and compare it with your protein as a start.

If you can find mutagenesis studies in the literature, you can see which residues are important for ligand binding. Then you can also evaluate the docking results based on these interactions.

You can do an alignment of predocked pose of ligand to post docked pose, and calculated the rmsd value between the poses. Lower rmsd value is preferred.

Or you can retrieved reported inhibitor of the protein with their bioactivities. Dock these retrieved inhibitors and make a plot of Pic50 and obtained docked score. The r2 from the plot can be used for validation. R square greater than 0.6 is desired in these case.

You can validate the docking by using inhibitors of the protein to generate decoys at DUD;E. You then dock the decoys and the inhibitors against the protein. After that, you use the binding energy to generate an ROC curve. If the AUC is greater than or equal to 0.7, it means the docking software can decipher the inhibitors and the decoys

As Michael Afiadenyo suggested you can do ligand enrichment with a DUD-E dataset, and calculate the enrichment factor