This is sometimes called SPMD or "Single Program, Multiple Data" parallelism. You may have to create multiple slave process for handling independent data streams. Since they may not communicate with one another, your application's IPC needs are fairly simple. Each slave needs to communicate with the master - wait/signal type of communication may be needed. You may need to think about how the data stream is read / written - whether there is a single I/O channel or multiple I/O channels. Understanding the processor architecture will help resolve this.

Best wishes.

hello C.P.Ravikumar sir, i am using jnetpcap libraries which is based on the libpcap (linux) and winpcap(windows), can i open two different streams on the same nic using  different stream filters so that one can analyze different streams of TCP in real-time. I am having quad core processor, and i want to assign capturing task for each stream to each processor. I want to use java fork/join for this purpose. I want to know that is this a good idea and also is it possible?

I am not familiar with the packages which you mention. But here is something for you to think about. Assume that you are TCP packets are arriving in series - p0, p1, p2, ... and you have two processes P1 and P2. The master can receive the packets and hand over the even numbered packets to P1 and the odd numbered packets to P2. I am assuming that P1 and P2 will independently be able to process their respective streams.

i will give it a try thanks

You are welcome. You have to check how you can keep the processes busy in doing useful work [processing the packets] and not waiting for packets. If you give them too little load, then they may end up waiting for work. So, although I mentioned odd and even numbered packets, you may have to experiment with the right amount of load to distribute to the slaves. In general, you could group the packets into groups of K and give the odd group to one slave and the even group to the other slave. 

basically i want to do like if one packet is being processed by one processor and at the same time next packet is coming to nic then after some timeout in buffer of nic the packets will start to loss, i want to prevent this by sending those packets to different cores. However, i will first give a try to simple odd even example only to check whether the code is working or not. Thanks again for your guidance

Just a couple of points I would like to mention in relation to your various queries:

a. If you wish to optimize package filtering vs. CPU overhead, Java is probably not the right choice of language. Its interpreted nature makes it really too slow for real-time handling on a loaded network interface, even with JIT compilation. C or C++ would be more in line with current operating system practice. There is an old, but still current tutorial by Tim Carstens on libpcap and C at this address: http://www.tcpdump.org/pcap.html .

b. Speaking specifically about the Linux operating system, the actual package capturing is performed by the kernel; the pcap library is simply an interface to the API made visible by the kernel. Files pcap_user.c (the user-land half) and pcap_kern.c (the kernel code itself) in subdirectory arch/um/drivers/ of the Linux kernel source code will give you some insight on the possibilities of the API at this level.

In my opinion, writing a multi-threaded application in C is not much more complicated than in Java, and would give you some hands-on experience with real system programming.

Cheers,

-Alan

thanks alan it was a nice application......

i had build the mechanism in java but it is not real time...

so for my second task i will look forward with your sugestion