If - hypothetically - the HR diagram consisted of only Main Sequence stars, whose luminosity follows a monotonic relationship with temperature, you could proceed as:

• (B-V) of a star -> Temperature (Ballesteros' formula) -> Spectral Type -> known Absolute magnitude for that type of MS star (e.g. https://sites.uni.edu/morgans/astro/course/Notes/section2/spectraltemps.html)-> Distance Modulus for the individual star.

The limitations with this are:

• There are of course non-MS stars as well, with the same (B-V) values but vastly different luminosities. A colour-colour diagram would help to distinguish MS stars from giants, but you'd need a 3rd filter (U), and it's a much cruder discriminator of luminosity class than taking spectra.
• Doesn't take account of extinction/reddening, as you flagged.
• Doesn't take account of different metallicities (population effects...Pop II subdwarfs being the main example).
• Temperature resolution decreases in the (B-V) index for very hot or (to a lesser extent) very cool stars, as their spectra shift out of the filter bandpasses. A wider-base index like (B-I) or (U-I) would be better. I appreciate that you may not have had control over the filters used, e.g. if it's archival data.

So while you could proceed with the (B-V) recipe above to get a distance, the result would have to be asterisked: "assuming a Population I MS star, and assuming no extinction".