A partial disruption of lacZ by the insert is one of the reasons. They may contain the insert. My recommendation is to avoid them if you can for downstream applications.
It's possible that your small insert is there but the lacZ still has some residual activity even with the added sequence. The X-gal is then only slowly converted to blue such that the older cells in the colony (in the center) have had enough time to convert some X-gal while the newer cells on the periphery have not.
If you could sequence through the whole cloning site including your insert, the clones might be OK. Otherwise, there may be only a subfragment of your insert going in (such as the bacteria got rid of a region that they didn't like) or a distinct low abundance PCR insert that they can tolerate went in preferentially. I wouldn't give up if you have access to a cheap Sanger sequencing service to verify that what you need is there.
Have you done a control with no insert at all? What did you see? You may also need to incubate the plates for longer time (generally about 16-20 hours is recommended) if you haven't already tried that.
Erome, yes, that control was used as well. Incubation was long enough, maybe just a little to long. Unfortunately, sequencing is not possible Christoffer. It's a school project, but it can be used in the discussion ofcourse