The procedure is similar to the mechanical homogenization but instead if imposing displacements, temperature gradients to the specific directions are needed. If the RVE is considered isotropic then the temperature gradient to one direction is enough.
The difficult part of RVE analysis is to apply the boundary conditions. The multiscale code SwiftComp eliminates all this difficulty by directly computing the thermal potential. It is pretty straight forward if you use SwiftComp. You only need to mesh the microstructure and providing the thermal conductivities for all the constituents.