I think you require to add potassium hydroxide (catalyst).

you need to add a base like NaOH or KOH and monitor the progress of the reaction

by TLC.

You need a base as a catalyst like 40% NaOH, or you can do the reaction in DMF in the presence of sodium methoxide. 

I agree with my respected RG colleagues that the reaction needs NaOH or KOH. The attached reference supports a need for a base. My regards.

http://www.ijrpc.com/files/01-277.pdf

Your amine (guanidine)  is protonated.  Free it with a suitable base. Not sure how nucleophilic guanidine is.  Did you check literature about its reactivity?

your  amine in Salt form  that's why bring free amine then put reaction.

Dal

You have the answer from the colleagues but I enclosed two useful references

Good luck

Change the solvent if possible. to DMSO or DMF with ammonium acetate or TBAOH ( Tetrabutyl ammonium Hydroxide in MeOH or in MeOH+ Tolune as catalyst

Guanidinium cation is very stable (guanidine is strong base such as calcium hydroxide) and for deprotonation you need strong base. The best is solution 1 eq KOH in MeOH, addition of 1 eq of guanidinium chloride, 15 min stirring under argon (with CO2 and water the very bad soluble carbonate is formed), fitration of KCl by suction. Methanol solution of free guanidine is ontained.

Thank you for the responses fellow colleagues. They really helped.

So in order for the guanidine hydrochloride to react with the chalcone, I first have to deprotonate it first since guanidium ion is so stable and it will be hard to react with the alpha-beta unsaturated group of the chalcone. What i did wrong at first was not deprotonating the guanidine hydrochlride and that's why i obtained my starting material again after refluxing. This is what I have understood based on your help.