By lysis buffer, if you mean Laemmli lysis buffer, then it’s not very advisable to proceed for ELISA without removal of SDS and BME/DDT.
In ELISA experiments, if your initial lysate contains detergents and denaturants, they will interfere with binding of capture antibody with protein.
Also, in presence of BME & SDS, bound capture Ab may lose its adherence from surface that will end in lesser binding of protein of your interest. This may rise to uneven binding of capture AB, then uneven binding of lysate to capture AB. So even if it works in presence of denaturants, there could be a possible variation in results for same sample.
You can try precipitating protein by various methods like; chloroform methanol, methanol, TCA or acetone and then reconstitute it into buffer of your interest (PBS+BSA, I guess).
What I do is obtain the cell lysate by the simple free-thaw procedure. It gives beautiful results for detection of intracellular cytokines, as in my case.