Neostigmine is a synthetic quarternary ammonium compound, similar to physostigmine and rapid onset of action and can inhibit both true and pseudo-cholinesterases.
I think that Neostigmine inhibits catalysis of the hydrolysis of acetylcholine into choline and acetic acid. Thus in presence of neostigmine, acetylcholine released from axon terminals stays around the synaptic region for a longer time, resulting in prolonged activation of the muscle.
Dear Ali, please have a look of some papers (below an abstract from very old paper, but good one).
Furthermore, the previous administration of benzoquinonium abolished the antagonistic action to tubocurarine of normally effective doses of edrophonium and reduced that of choline. These results were similar to those previously obtained with neostigmine, physostigmine and ethyl pyrophosphate and suggested that there
was some similarity in the mechanism of action of all of these substances. Benzoquinonium, therefore, showed promise as a useful pharmacological tool for distinguishing compounds with this particular type of action. These anti-curare compounds did not appear to act by cholinesterase inhibition, nor by an increase in the sensitivity of the motor end-plates. In common with other workers, we suggest that there is a pre-synaptic mechanism of action.
As neostigmin inhibits the break down of acethylcholine, the excessive amounts of this neurotransmitter cause prolonged and exaggerated stimululation of the postsynaptic nicotinic receptors at motor endplate. This stimulation causes depolarization and contraction of skeletal muscles. The same action can be seen by organophosphates (as they also inhibit acethylcholinesterase enzyme). But this depolarization and uncontrolled contraction of the muscle is blocked by the skleletal muscle cells itself in a few minutes to protect theirselves from over stimulation (at the receptor level) and uncontrolled contraction then turns into skeletal muscle paralysis.
Persistent depolarization of end plate is caused by neostigmine(by increasing concentration of acetylcholine in synapse).This leads to relaxation of muscle as action potential cycle is not completed which is required for muscle contraction.
Hence,neostigmine acts as partial agonist,producing suboptimal effect or rather antagonistic effect.
Neostigmine is a reversible inhibitor of Acetylcholinesterase (AChE) and also, like acetylcholine, can directly produce agonist action (Thus a mixed acting agent) unlike physostigmine. This has been verified experimentally also. In an isolated frog rectus abdominis preparation, addition of neostigmine alone may produce some contraction, and it also potentiates Ach contractions. This is unlike physostigmine that only potentiates Ach effect. Generally, a partial agonist binds with the receptor and produces submaximal activity or efficacy ( some intrinsic activity ). This explanation can be applied on neostigmine as well!