I'm using the multiprocessing module (link below) in Python 2.7.3 to simulate identical chemical reaction systems in parallel. But I need these processes to talk to each other for passing and receiving the value of a state variable needed for their individual simulation in real time. So far I've been trying unsuccessfully to do this through pipes (also link below). Does anybody know a better way? any user-defined function? or at least a not-so-techie tutorial? Thanks!
http://pymotw.com/2/multiprocessing/basics.html
http://pymotw.com/2/pipes/index.html#module-pipes
Did you already try to use the mechanisms provided by the multiprocessing module?
https://docs.python.org/2/library/multiprocessing.html#sharing-state-between-processes
This should offer a more pythonic way to solve your problem compared to pipes.
instead of using pipe . you can try for parallel implementation of python
If you have possibility to modify the code of a process that you are executing from python (it is not clear from your question if the subprocess is also written in Python):
- You can use YAMI4. This library provides several communication mechanisms (unix sockets, tcp sockets, udp sockets), achieves high performance and it is lightweight. You can use as many processes as you can. The advantage of this library is that it is available in many programming languages and the processes do not need to run on the same PC providing you choose an appropriate communication mechanism (i.e. tcp/udp socket).
- Another possibility is to use standard communication libraries such as MPI (mpi4py) or openMP.
Otherwise:
- Please look here
http://stackoverflow.com/questions/9550338/asyncronous-bidirectional-io-redirection-for-a-child-process
you have to use select to determine if there are data available because the reading is blocking operation
http://www.inspirel.com/yami4/
http://stackoverflow.com/questions/9550338/asyncronous-bidirectional-io-redirection-for-a-child-process
You can look at some of the MPI based Python's (e.g. pyMPI or mpi4py) that allow direct point-to-point messages. If you have an environment that supports MPI this makes it very easy to launch and communicate even over distributed machines.
- Badges
- Science topic
- Similar topics
- Computer Science and Engineering
- Programming Languages
- Python