I am designing a transmission tariff scheme where the transmission company pays charges to compensate generators for congestion costs. So I am looking for information of international ISOs that use this method.
Not as far as I know. If there is congestion in the network, the cost as a rule is borne by consumers/retailers for paying higher prices and lost revenue for cheaper generators who would have otherwise been dispatched. The market design is expected to address this by signaling the right price (e.g., locational/nodal marginal price or "LMP") and there may be financial mechanism in place for generators and retailers to hedge their risk (e.g., Financial Transmission Right). Transmission companies are by and large regulated everywhere. They build stuff if they can justify it (e.g., show the benefit of building a line would relieve congestion that has benefits exceeding costs) and earn a regulated return. But they are not the ones who would compensate the generators. There are constrained on/off payments in markets that are less sophisticated than LMP based market, but those are managed by the system operator who would collect uplift payment from retailers and pay the generators. It is a second best mechanism in a second best design.
During the period 2009-2014 in India, the Regulator penalised the Transmission company by cosidering the doubling of outage when ever any generation evacuation was constrained due to transmission outage , however there was no direct compensation to the generator . It is an oblique way to make the Transmission company responsible for congestion .