I am currently failing to find historic data on monthly exchange rates, and CPI for most African countries over the past 10 years for free. All I get is data from all countries except AFRICA. For Africa you all i get is data for a few countries.
Regarding monthly exchange rates for African countries, you have to be specific - Exchange rate of which country's currency against which currency. Exchange rate data can generally be obtained from the Central Bank's website, which may contain data about CPI too. So, you can explore the website of the Central Bank of your country. You can also try the UNCTAD website for inflation index related data.
The African Development Bank provides monthly (end of month) exchange rates for all African countries that have their own currency from January 2004 to the present. The link is: