First, make sure to rule out any trivial explanation to your data (e.g., addition of high concentrations of your effector changes the pH of the solution and hence Vmax).Assuming your results are reproducible and not linked to some obvious artifact, I think you can consider your 'effector' molecule an inhibitor (although certainly not a purely competitive inhibitor) if it diminishes Vmax/KM (in other words, Vmax must increases less than KM). But formal treatments of these processes use sometimes more exotic definitions such as " mixed, dual modification". (see https://www.enzyme-modifier.ch/mechanisms/inhibition-and-nonessential-activation/)

It is not easy to decipher your results if you are not adding a pure inhibitor but rather a 'plant extract' as you state. Unless you have a pure inhibitor/effector/modulator it will be difficult to understand what is going on. You could have a competitive inhibitor in the extract that will increase Km and at the same time have another modulator in the extract that will alter the conformation of the active site, resulting in an increased Vmax. My suggestion is to purify the extract further before jumping to conclusions or trying to consider it a competitive inhibition.

hope it helps!

I think that Gustavo has provided a solution to the problem. On reading the question, my immediate reaction was that the inhibitory mixture probably contains a compound that stimulates the catalytic turnover of the enzyme in addition to an inhibitor hat acts competitively to raise Km. A degree of purification of the extract is needed if you hope to solve this problem.

Peter

I am agree with both previous answers. An alternative to additional purification steps, if there are not feasible, is to dilute your extract. This way, you will dilute also the other(s) compound(s) in your extract and, indeed decrease their effect. But purification will be more safety.

I believe that your plant extract also contain some activator in addition to inhibitors and hence the observation. This may be solved by dialysing and diluting extract. It may disappear on purification. Good luck.