Well obviously, you'll need to work in an inert atmosphere (N2 or rare gas).  What are the other elements - do they form 100% fully miscible solid solutions? Look carefully at the phase diagrams. 

All subsequent processing and use will need to be in an inert atmosphere as Ca metal reacts very rapidly with oxygen and water in the atmosphere to produce first CaO and then Ca(OH)2.

Thanks Alan. Sure, protective inert atmosphere was the first idea, but it doesn´t work properly as theory predict. I tried also atmosphere saturated by Ca-vapour and still measure abnormal oxygen content in alloy by EDS in SEM microscope. 

Now I am concerned to find some other element which may decrease Ca reactivity. Do you think is that the way how to face this problem? 

Reading of phase diagrams was the starting point and they are full of intermetallics, more or less stable.

Then you have oxygen coming in from somewhere in in the calcium to begin with - Ca is used as an oxygen scavenger in a number of industries.. Perhaps a reducing atmosphere of hydrogen?  Perhaps someone from the atomic reactor industry can chime in  - they run with Na-K alloys in pipes.

Thank you for hint, Alan. I was considerate also hydrogen atmosphere, however I feel respect to hydrogen gas. Maybe its time to overcome this fear and get the first experience with hydrogen in furnace.

Maybe a gas mix?  For example, 10 to 30% H2 in 70% N2 ?

What furnace did you use? To avoid excessive evaporation and oxidation of Ca, could try Arc-melting or sintering the element powders under high pressure  ultrahigh purity argon.

For example: 

Based on patent EP0079765 B1by Raymond David Prengaman  (free on Google), in making the Pb-Ca-Al alloy without using of inert atmospheres or flux covers, this concept "oxidation shield" workout as "melting lead, heating the molten lead at a temperature from 549 to 660°C and stirring into the heated molten lead an eutectic calcium-aluminum alloy having an average content of 73% by weight calcium and 27% by weight aluminum". The calcium-aluminum eutectic can be alloyed at temperatures as low as 549°C formed by simply melting aluminum and thereupon adding the calcium". In this case, aluminum in the eutectic alloy protects physically the calcium from oxidation during alloying and only about 1.6% of calcium dross out as a result of surface oxidation.  It is the melting and casting process trick-like to overcome the calcium is in contact with air.

Best regards