How people infer the direction of earth's axis rotation because its invisible. How did they observe the earth's axis of rotation, how did they define and determine this axis?
Indeed, the Earth's rotation axis is invisible, but Earth's rotation is visible in the observations of stars and quasars (formerly optical observations, now radio observations), of the Moon (with laser ranging) and of satellites (various techniques, see https://www.iers.org/IERS/EN/Science/Techniques/techniques.html). From these observations it is possible to calculate the poistion of the axis and its changes (determinded by the so-called Earth Orientation Parameters, see https://www.iers.org/IERS/EN/Science/EarthRotation/EOP.html).
The definitions are quiet complicated, see Chapter 5 in the IERS Conventions (https://www.iers.org/IERS/EN/Publications/TechnicalNotes/tn36.html).
Thank you very much Wolfgang R. Dick for the answer. Can I find any video lecture explaining how these techniques you mentioned used to observe earth's rotation?
There is a video at the homepage of https://ggos.org/ demonstrating the different geodetic techniques (starting at about minute 2:30). However, this is not only about Earth rotation, and it does not go deep into the technology and methods. If I will find more videos, I will provide links.
Here are a few videos (at a very popular level) about using Very Long Baseline Interferometry (VLBI) in geodesy, including measuring the Earth's orientation:
https://www.youtube.com/watch?v=59Bl8cjNg-Y
https://www.youtube.com/watch?v=KheG0z2XH2w
https://www.youtube.com/watch?v=XcSZjT3oDSY
Here are more technical videos about the principles of radio interferometry:
Mathematically, the Earth Orientation Parameters (EOP) are transformation parameters between two reference systems, the International Terrestrial Reference System and the International Celestrial Reference System. Here are videos about these basics:
https://www.youtube.com/watch?v=vOUU5BAzKG0
https://www.youtube.com/watch?v=nyZ4NfSGUQA (in Spanish)
The simplest answer is, "by looking at the rotation of the stars around the pole of the sky". And being the simplest, it has allowed astronomers to determine the orientation of the Pole relative to the sky as accurately as their measurement techniques allowed for the best part of 7000 (and probably more) years.
Determining the change in the orientation of the Poles requires comparisons of the astronomical Pole over a period of time, and was first noticed by Hipparchos about 2400 years ago, based on a comparison of his observations of the positions of the stars relative to the declination circles defined by the daily movement of the stars with similar observations done by what he considered "ancient" astronomers (presumably 700 or so years earlier, but all the records were lost when the Great Library at Alexandria burned to the ground.
As far as geodesists would be concerned, where the axis comes out of the surface of the Earth requires much finer techniques than mere hand and eye observations, and the discovery of the Chandler Wobble (by Chandler, of course) is only a century or so old. A search for that topic should give you a good introduction to that aspect of the problem.
Outside the Earth "axis of rotation" on the sky is the continuation of the apparent line with zero angular accelerations to the intersection with the celestial sphere - Pole of the Sky (strictly speaking, this Pole depends on the points with the astronomical observations).
Inside the Earth "axis of rotation" is not precisely a straight line, since the points withs zero angular accelerations don't form the straight line due to the permanent diurnal mass movements, tides (and tectonic secular movements).