see the links and paper

https://books.google.com.pk/books?id=GmHRBQAAQBAJ&pg=PA216&lpg=PA216&dq=the+derivation+of+circular+regression+matrix+for+circular+fit?&source=bl&ots=WotizfVneb&sig=vwzknLJ7TRMz3yOcUApR_NbxrMc&hl=en&sa=X&ved=0ahUKEwiFxNGo6vXMAhWC1RoKHSMsCS4Q6AEIHzAB#v=onepage&q=the%20derivation%20of%20circular%20regression%20matrix%20for%20circular%20fit%3F&f=false

http://math.stackexchange.com/questions/214661/circle-least-squares-fit

Tamer,

Attached is an excerpt from a document I wrote a while back.  It shows a derivation of your matrix equation and how to relate its solution to a circle with center (h,k) and radius A0.

Hallo Mr. Wilson,

May you please show me how we get the matrix, i am not able to get 3 by 3 Matrix.

Thanks in advance

It's a calculus problem. Take partial derivatives of f with respect to \tilde{h}, \tilde{k}, and b.  This gives three expressions.  Set each expression equal to 0 to get three equations.  Express these three equations as one matrix equation and you have it.

Thank you, i took the derivatives and multiplied bothe sides with the Matrix Transport, and it works very good. thank you for your time and help!