suppose there are two material points, before deformation, they have coordinates (x) and (x+dx), the initial distance between these neighboring points are:
It is not the distance, it is the distance squared. The square is needed to convert the vector difference between the two positions, dx, into the scalar distance squared. If we take square root the, we will get the distance.
dx^2 is vector dx multiplied scalarly by itself and means dx1^2+dx2^2+dx3^2.
Square of distance is more preferable because there exists simple tensorial (bilinear) relationship exactly between squares of lengths of initial and deformed linear elements.
To convert the vectorial quantity in to scalar. Generally the points are specified with directions and the distance obtained by taking square root of individual square sums.