Let's assume your algorithm is right

(http://en.wikipedia.org/wiki/Miller%E2%80%93Rabin_primality_test)

That is, you've got the one you've implemented in VHDL working in one of Scilab, Matlab, Octave, Python, C/C++, etc.. Now use that to generate input/output golden"test vectors" for different parts of your VHDL model. Usually I try and skip this part, but invariably when using an HDL like VHDL, I need to come back and "test" the intermediate values produced by the high-level language model against the HDL model I'm trying to get to work.

Given that, then you are trying to debug your VHDL implementation.

If by "functions" above, you mean VHDL functions, please understand their limitations.

They are not like C/C++ functions, and are a fairly advanced concept in the VHDL scheme of things. Look at http://www.eda.org/fphdl/ for examples of functions that are also synthesizable(I'm assuming that you want your VHDL model to be synthesizable, but are currently stuck on getting the simulation (perhaps using the ModelSim simulator) working correctly.

I'd recommend that if you are looking for speed from your VHDL model that you use the Entity-Architecture Register Transfer Level(RTL) methodology to design your parallel components, Embrace the clock, pipelining and their semantics, as that is is where your parallel speed will come from. I only use functions and procedures as semantic sugar until I'm comfortable that my basic design works with memory, logic, pipe lining, and clocks.

alan