or is it a chance that these intermediates are just chucked and by chance these are active against particular organisms. Can someone enlight me in this case with some publications. Thank you
Antibiotics are typically secondary metabolites, not simply a byproduct of primary metabolism. It can be energetically costly for an organism to produce antibiotics, so an organism only wants to do it when it needs to. Organisms have evolved to combat their competitors for a resource. "The evolutionary arms race" is worth looking up. Most microbiology textbooks will have information on this.
In marine and terrestrial environments it is an interesting aspect about how this competition evolves and is regulated for the fitness of the producer: http://mmbr.asm.org/content/71/2/295.short https://femsre.oxfordjournals.org/content/35/5/957 http://science.sciencemag.org/content/337/6098/1107 http://www.nature.com/ismej/journal/v9/n1/abs/ismej2014106a.html