While measuring potential at an infinite point, it is assumed to be zero. While measuring potential over the earth's surface, again we take it as zero. How can we define zero two times?
'Potential' of what exactly? Mechanics potential energy, electrical / magnetic fields, gravity, etc. - describe the context and the specific frames of reference. The two points you mention, superficially, already have a problem because they mix a theoretical abstract mathematical basis ( 'infinite' ) and a measurable physical observation ( Earth's surface ).