I can think of three possible sources of your trouble.
1. The dialysis bag has a small tear that caused your protein to leak out into the dialysis buffer
2. The protein was smaller than the pores in the dialysis bag
3. The protein was insoluble in the dialysis buffer
You can test the first two by concentrating the dialysis buffer and seeing if the protein reappears, and you can check the third point by taking the protein, pre-dialysis, and adding the dialysis buffer and checking for precipitation.