Ca5.40Mg0,6C6O18 = Ca0.9Mg0.1 CO3 = calcite with increased Mg content. The boundary value between magnesian calcite and calcite is not clearly defined. Sometimes it is set to 0.1 or even 0.25 Mg.
It could be magnesium calcite or dolomite, or anything else. The crystal structure is not unique for material composition. XRD will tell you the structure and what composition might be assigned to these atomic bonds based on the peaks positions, but it is not sensitive to the elemental content. you need to match your XRD data to the reference XRD pattern of calcite and dolomite to know what your material is.