Hi Rajesh

This is a fairly straightforward thing to do. Be aware that if your 2 desktops are running Windows OS of any description, you are likely to have some serious overhead losses. Much better to do this using a pair of Linux machines. You would ideally need a minimum of a dual high speed LAN port in each machine, or you could use a high speed LAN switch if that is problematic. I have given you a link below for a DIY supercomputer which will show you have to set it all up and test it. Another relatively simple option would be to set up a cloud environment, given the number of processing threads you will have. You could use Eucalyptus software, which is open source, and this will allow you to build a cloud setup which is compatible with AWS APIs. You might want to pay particular attentioin to your data storage, as this is likely to be the bottleneck for either option.

Anyway, whatever way you choose to go, you will have a lot of fun putting it all together.

Regards

Bob

http://www.webstreet.com/super_computer.htm

Dear Bob,

Thank you for answer.

I have one workstation(intel Xeon z820) with 24 cpu (12 physically present core with double hyper threading) and four Hp desktop with 4 cores. All have Ubuntu OS. I want to connect therm parallel for HPC. The Workstation has dual LAN but desktops have only one LAN. I have not much knowledge about cloud computing and all. Can you please tell me how to set up a cloud environment for parallel computing?

Regards

Rajesh 

Hi Rajesh

The advantage of cloud is that you can optimise the use of your resources. You can then allow difference users to use one or more processors, variable amounts of RAM and hard drive space, which can be spooled up and shut down as needed. Depending on how your cloud software is configured, you can usually allocate a number of processors, although for small scale systems, this will generally be restricted. Whereas an HPC will generally be configured to run with any number of processors from one, up to the maximum, thus allowing for much higher scale, and power to be used, but for a smaller number of users. If you don't know a lot about cloud, you might not want to start there to begin with.

What you do want to do is to think what it is you want our of your system, and what kind of applications you want to run on it, and how many users will be given access to systems at one time. Another big proviso to bear in mind, that you will have to ensure you run multi-thread enabled software for users. Conventional software will often run single threaded. So, for example, if you wanted to run an application that takes 54 hours to run single threaded, and you run it on an HPC system with 500 processors and 500GB of RAM, it will still take 54 hours to run. The other thing to consider is that depending on the type of application, you can sometimes also harness the power of your GPU processors as well, which can add considerable power, but not all GPU processors are enabled to run in this way.

It would be helpful if you were to think about these questions, and give me some feedback, and also very useful if you give me the exact specification of each of your machines. You can do that with the following commands:

Use the command line and run these commands for each of your machines:

"sudo apt-get install lshw"

"sudo lshw >machine1.txt"

Obviously, change the output file number to 2, 3, 4 and 5 for the other machines. Then copy all five files and add them to your reply to me. It would also be useful to know the specs of your GPU cards, make and model number, and the same for the hard drives and the RAM for each machine. Likewise the Network cards make and model number would also be useful.

Over the next few months, I will be building a mini testbed system using an HP microserver with a Xeon quad core, 16GB of RAM and a 500GB SSD. To this, I will be adding "cloud-in-a-box" software, and this will form the basis for a paper I plan to publish which you will be able to use to replicate the steps if you are interested in putting together a cloud system. It is then possible to add more resources over time, if you want to build it further.

Regards

Bob

Dear Sir,

Thank you for your detail instruction regarding HPC. I am happy to say you that I succeed in connecting two Ubuntu desktops parallel using the proceeder given trough link provided in your first response to this question.  The mpirun -n -f machinefile command works but i want to install some physics software like quantum espresso, VASP, CASINO etc., which have their own command to run calculation. So, these cases I find difficult to  install such a parallel code and run for parallel computing.  However, during the surfing I came across the "rocksclusters"  which can be done this job of parallel configuration. I want your suggestions about it if you have used it for HPC.

Thanking you,

Rajesh

http://www.rocksclusters.org/wordpress/