I am currently modelling and simulating a statcom,however I am having challenges controlling the capacitor voltage. My question is what affects the stability of the capacitor voltage.
It depends on the inter-relation between the voltage magnitude of the controlled bus, and the voltage set point (within the linear control region). You may read
and Chapter 7 in https://www.researchgate.net/publication/307926879_Dynamic_Security_of_Interconnected_Electric_Power_Systems_-_Volume_2_Dynamics_and_stability_of_conventional_and_renewable_energy_systems