Do you want to extract heat from the concrete itself (e.g. during casting- the heat of hydration)or just absorb the heat that is being transmitted from the other side of concrete blocks?
Another question would be how high are the temperatures? If its too high, concrete alone may not be the most suitable material. You may need to explore other ceramics with high refractive index.
As per my understanding, you do NOT mean the blocks to act as insulator (in that case, the cheapest solution is AIR - see for e.g. autoclaved light weight concrete).
You NEED some material that do not increase the temperature of concrete body by absorbing the heat energy. In this case, you will need material with high specific heat capacity. According to this table (https://www.engineeringtoolbox.com/specific-heat-capacity-d_391.html), most suitable candidates are timber and artificial wool; or water if you have some method to place inside the concrete block.