for removing Ca and Mg from water, Reverse Osmosis is good option. How ever ion exchange and electrolysis are also useful. For domestic use Aqua guard is normally used.
It can be removed by using a water softener which works on the principle of ion exchange in which calcium and magnesium ions are exchanged with sodium or potassium ions.
Dear Fares Fenniche thanks for posting this interesting technical question on RG. The answer to your question depends on what scale this water purification should be done. For household use you can of course buy a commercial filter system which will gift you very soft water.
For a potentially useful general overview about this topic, please have a look at the the following article:
It is possible to effectively remove calcium and magnesium from the water supply by softening the water through ion exchange using what is called a water softener. This device exchanges calcium and magnesium with sodium. The water then passes through a bottle of ion exchange resins softening. Currently, there is a new method to soften water, without the use of resins, it is the CO2 water softener. This method is both ecological and economical, as it does not require regular regeneration and maintenance.