If you have an enzymatic reaction with substrate inhibition and you want to get the catalytic efficiency. What is a suitable way to calculate the value and would that be a real number? Is dividing the absolute values are enough in this case?
Standard non-linear regression is most commonly used when intial rate data has been measured. Equation can be found in the link. In this case the Vmax is defined as the theoretical vmax obtained in the absence of substrate inhibition. Dividing by active site conc can give you an equivalent Kcat. The results may vary when using a progress curve method. As ever, knowing an accurate active site conc is important.
You can calculate efficiency from these values so long as you provide the form of the equation used for regression for future comparison.
I agree with Jason R Smith that a good way to get the true Vmax for the kcat/Km calculation is to use nonlinear regression of the Michealis plot to a rate equation that includes substrate inhibition.
I would add as a caveat that you should be careful that the apparent substrate inhibition effect is really due to substrate inhibition, not some other phenomenon. As an example, high concentrations of ATP can cause apparent substrate inhibition in an ATPase assay if there isn't enough Mg2+. You could also get apparent inhibition by an impure substrate.
The above suggestions are indeed the best methods, although to obtain an accurate value for Km, Ks and Vmax you need to obtain robust data on both sides of inflection. Another approach is to determine the rates of enyme activity below Km, where the of rate vs [substrate] is essentially linear.