It is hard to explain without the use of formulas. So, in the following, I will be using the symbol & to mean proportional and the symbol % for elevated to. Thereefore a & b % 2 means that the quantity a is proportional to the square of b. The symbol x means times.
Let then assume that the flow rate m (mass per unit time) is constant and that the fluid properties do not change with temperature.
The first reason to have small diameters is to have a turbulent flow in the pipe for a given m, i.e. a high Reynolds number Re. Generally, above about Re = 2300, the flow is turbulent. In turbulent flow the heat transfer coefficient h is much higher than in laminar flow. In fact, inlaminar flow, the energy exchange occurs at a molecular level while, in turbulent one, bunches of molecules (particles) are exchanged.
The Reynolds number can be put in the form:
Re = ro x V x D / mu & m/D where D is the pipe diameter and mu the dynamic viscosity coefficient.
So, for a given m, the smaller the D the larger the Re.
The second reason is the following one.
The convective heat tranfer Q (energy per unit time) can be written in terms of Newton's formula:
Q = S h (Twall - Tfluid) (1)
where S is the heat exchanging surface area and the higher the h, the smaller is the temperature difference to exchange a certain Q, so more efficient will be the heat exchanger.
In turbulent flow, the coefficient h can be derived by the Dittus and Boelter formula via the Nusselt number Nu:
Nu = h x D/lamda = k Pr % 0.6 x Re % 0.8 & 1/(D % 0.8) where lamda is the fluid thermal conductivity coefficient and Pr the fluid Prandtl number. So, finally
h & 1/(D % 1.8)
If you now substitute the above formula in (1) per unit pipe lenght you obtain:
Q & (Twall - Tfluid) / (D % 0.8)
so the smaller the diameter the more efficient the heat exchange.
Naturally, you have to spend more power to pump the fluid through the pipe because pressure losses get fairly higher.
another view, with high velocity can reduce the fouling factor and because of some difficulty in cleaning some types of pipes, it is better to put the fluid with high velocity in tube side to reduce the fouling factor.