You should see which of force components (Fx, Fy, and Fz) recorded by dynamometr is in feed direction and which one is tangential to the mill rotation. Then put the values in the equation attached below.

Dear Dr. Mohammad Lotfi ........ In a dynamometer, Fx is the feed force, and Fy is the cutting force (tangential force). And I applied this above (well-known) relation...But friction coefficient values go above 1.0.........I don't know where is the issue

If the process is orthogonal, i guess, Fz is the feed force. If you put a picture from your setup, it will be clarified.

Here is the picture of the milling setup used in experiments Dr Mohammad Lotfi

The picture is not clear. But as you said the operation is orthogonal, i should say that in orthogonal cutting, depth of cut is equal or less than cutting edge width (width of an insert) which is along the cutting edge direction. Therefore, it remains two directions, vertical direction which is feed force value (the feed of cnc table is not important, because you should consider just one rotation cutting tool (one of inserts)) and another horizontal direction which is perpendicular to the depth direction is the tangential force.

In total, if you want to undrestand, you should simplify the process, in one rotation of one insert, so that my description will be clear for you.

I agree with Mohammad Lotfi. I made a quick sketch, which in my opinion may help you to understand Mahammad's phrase.

I suggest that you perform the orthogonal cutting experiments on a lathe, not on a mill. The reason is that the definition of orthogonal cutting is that the cutting direction should be normal to the cutting edge. In the milling process (with straight edge tool) this happens during a single moment while a cutting edge is engaging the workpiece. This point is very hard to detect and needs the synchronization of the milling tool and dynamometer readings.

You can simply measure the coefficient of friction using a lathe, a tool holder with 90 degrees of inclination angle, and a tube of your work-piece material.

Perhaps will help you the following work:

Article Friction in linear orthogonal cutting of beech wood (fagus s...

From my point of view, there are some strong differences between pin on disk and machining testing, to establish friction coefficient.

In machining testing there are plastic deformation, friction and high pressure contact. In pin on disk the contact pressure decreases continuously as pin wears.

All the specialized machining bibliography show that are normal find friction coefficient over 0.8 during machinig testing (see Viktor Ashtakov bibliography, as example).