Neither method of measuring protein concentration is perfect.

The Bradford assay relies on a comparison to a standard protein. The amino acid composition of the protein affects the signal it generates in the Bradford assay. If the amino acid composition of the analyte protein is not similar to that of the standard, the result will be incorrect. Another problem that can happen with the Bradford assay is that the analyte protein is not soluble in the Bradford reagent and precipitates.

The UV method can go wrong for several reasons. It relies on calculating the UV extinction coefficient based on the amino acid composition. To be correct, the measurement must be made in a strongly denaturing solution, namely 6M guanidine-HCl. If you don't follow this procedure, the result is likely to be incorrect. The protein has to be essentially pure in order to get an accurate result. If the sample is contaminated with other proteins, nucleic acids, cofactors, or any other UV-absorbing substance, the result will be too high. Also, if there is any aggregated material in the sample, which scatters UV light, the result will be too high.

Comparing the direct UV absorption and dye-binding based methods is not reasonable. As Adam B Shapiro indicated, the protein solution should be pure enough to measure the absorption which really belongs to your protein. I do not recommend to use this approach and prefer to use Bradford instead of it. Since the UV absorption takes place by the Coomassie blue dye as Bradford reagent, the absorption rate will be different. Also, the wavelength should be different when applying direct and Bradford assay therefore comparing the direct absorption is not reasonable too. Using BSA (typically) calibration at Bradford measurements will give the results without any interference which you probably encountering at direct measurement.