Unfortunately, the attached mechanism did not open. But generally, following reaction mechanism request that you monitor a reagent that must be a key element in the final product formation. The possible activated complexes it can form (pivotal, stable, and pseudo stable) and the stability time of the activated complex. Tac must be lower than the contact time but measurable for you to be able to establish the mechanism.

Due copyright issue, I cant upload complete article, but i have uploaded an image file of mechanism which help.

This seems like any mild base or acid could be used.  It's possible that the sulfonamide (after removal of the N-Br bond) could be providing a trace of acid, but really the TBBDA is an overly complex reagent to complete this reaction.  If you use an acid, then you would be generating water rather than the BrOH that is in the mechanism.  Have a go at something like TFA or Anhydrous HCl in MeOH (acetyl chloride in methanol will give this after a few hours).

I have tried proline, Zinc-proline , NBS. The reaction didnt proceeded as expected.  

It seems like you need a source of electrophillic bromine. If NBS won't work (have you considered different solvents, additives etc? Perhaps a few drops of pyridine or TFA or a protic/aprotic solvent?) An alternative seems to be n-bromo saccharin. Or molecular bromine, but i fear this may destroy or over-brominate your compounds. It is indeed remarkable they would choose to use such an odd lewis acid. Perhaps you can substitute a more common lewis acid? Something like bf3 etherate... Iodine may be worth a shot, as well.

I havent tried any adiditves yet, i have used ethanol as a medium, and the LC-MS analysis of the reaction suggests there is no trace of desired product. I will try using TFA/pyridine and update then.

 I agree with Luke Henderson.  The reaction looks like a condensation plus Michael addition that could be catalyzed by acid or base.  In the mechanism shown, Br+ is acting as a Lewis acid catalyst.  Be careful with that.  There's very little evidence of FREE Br+ is solution.  A Bronsted acid should work as well, in the right solvent.  More usually, these reaction are base-catalyzed.  The driving force is dehydration - eliminating 2 molecules of water between the 3 reactants.  Reminds me of the reactions of "Dimedone", e.g. link:

http://pubs.acs.org/doi/abs/10.1021/ed400457c

I have tried using an acid and the reaction didnt accomplished as expected, it was halted at an intermediate step (Structure attached),

You can try the reaction  with molecular Iodine I2 or even with metal triflate such as In(OTf)3, Cu(OTf)2 etc with suitable sovent

Since your evidence for acid catalysis is lacking, it may be that the mech is indeed driven differently that shown, and may be dependent upon the oxidation potential of the N-Br reagent(s). So you might look up similar items like Tos-NHBr, or di Br cpds etc., NBS lack of reaction may also fit into this line of thought at the nonreactive end.

I strongly appreciate appreciate all the contributions suggesting Bronsted acid/Lewis acids. You also need to understand that since the reaction involves a dehydration stage based on O=H stretch Br may hinder the mechanism which suggests your present observation. I strongly think you need to substitute those Bromides with a stronger HAlCl4/HAlF4 in a substantial low ratio. One of the Crown Complex advantage is that serves as  -O- pivotal stretch than dehydration. If you can suppress dehydration to maintain the  O=H bond , either -O- or the NH- dislocation may be used to monitor your mechanism. Good luck as you search further

I searched this type reaction, but there is no report (by Scifinder). Maybe the common lewis acid cannot promote its formation. On the other hand, the synthesis of 5-member rings was achieved via diazotransformation followed by TM-catalyzed [3+2] cycloaddition. I suggest you use some TM catalysts. Maybe it will work. Good luck!

I would try Titanium isopropoxide or the dicholoro derivative. It has a major advantage in being a good dehydrating agent producing insoluble TiO2, it makes a good lewis acid but is also a bronsted base. All of which are potentially useful in effecting your desired transformation.

The brominating agent is quite peculiar given the lack of any change in oxidation state.

If you are using the indole shown previously then you may need to protect the indole N-H with something moderately electron withdrawing. Otherwise the electronics of the knoevenagel adduct will prevent the Michael addition step as indoles still have significant enamine like character.

Poly(N-bromobenzene-1,3-disulfonylamide) has comparable brominating power, and would be easier to separate (and regenerate):

R. Ghorbani-Vaghei, H. Jalili, Synthesis, 2005, 1099-1102.

R. Ghorbani-Vaghei, S. Hajinazari, J. Chem. Sci., 2013, 125(2):353–358.

NBS is also a source of bromonium, just like TBBDA.