No. Compressive strength of concrete mainly depends on proportion of ingredients of concrete. Strong Coarse aggregates contributes higher strength if properly bonded with fine aggregates and cement paste.
In case of standard mortar, cement proportion is higher ( normally 1:3), Thickness of cement paste being more it fails during crushing. In concrete Coarse aggregates provides more interlocking action than sand particles as in case of mortar.
The strength of concrete and cement mortar, of course, depend on the type of cement used and the strength of the cement stone, but only to a small extent. The main influence on the strength of concrete has the packing density of grains of coarse and fine aggregates (the so-called matrix), as well as technology laying and compaction of the concrete mix. And of course, the ratio of water and cement.