https://skydrive.live.com/view.aspx?resid=8A5F87204F4FC9F1!529&cid=8a5f87204f4fc9f1&app=Word&authkey=!

Architecture:

Host Machine - Windows Server 2008 R2 + SP1 + all post SP1 Updates

Roles: Active Directory Domain Services, DNS Server, Hyper-V, Web Server (IIS)

Software: SQL Server 2008 R2 x64, System Center Virtual Machine Manager 2008 R2 Server Components and Administrator Console, SCVMM Self Service Portal 2.0

Guest VM’s - Once this install is complete, you can create whatever guest VM’s you like to use for testing and demoing. In a future document I will detail a list of resources you may wish to create so you have a relevant test and demo environment.

Hardware Requirements:

I personally recommend using a desktop computer because of the drive options available. However, a high-end laptop can be used. I have performed this install to both hardware platforms in the following configurations:

Laptop: Lenovo W510 (quad processor + hyper-threading), 16gigs RAM, (1) 7200rpm SATA drive for host operating system, (1) 140gig Solid State Drive for guest VM storage

Software Requirements:

If you have a Technet or MSDN subscription you have everything you need. If you do not have a Technet or MSDN subscription you can use free trial software for everything. Just be mindful of the individual timebombs and make note of when things expire. Using the pieces below you should be able to run for 180 days from the day the Host machine OS is installed.

Windows Server 2008 R2 with SP1 Trial

System Center Virtual Machine Manager 2008 R2 with SP1 Trial

Microsoft SQL Server 2008 R2 Trial (get the 64bit version)

Microsoft System Center Virtual Machine Manager Self-Service Portal 2.0 with SP1

Suggested Pre-Reading/Learning:

An assumption is being made that you are familiar with installing and configuring Windows Server 2008 R2 and its related Roles and Features. If not, then you should bookmark and leverage the following -

Microsoft Technet Windows Server TechCenter

Additional Resources:

Microsoft SQL Server 2008 R2 TechCenter

System Center Virtual Machine Manager 2008 R2 TechCenter

System Center Virtual Machine Manager Self-Service Portal 2.0 TechCenter

Another Solution

Deploying a VM to Windows Azure

http://www.windowsazure.com/en-us/manage/windows/how-to-guides/connect-to-a-cloud-service/

http://www.windowsazure.com/en-us/manage/windows/tutorials/virtual-machine-from-gallery/

The first step in deploying your system image is to configure management and service certificates. When you upload a .vhd file to Windows Azure, it uses an x.509 certificate, saved as a .cer file, for authentication (referred to as a management certificate in the Windows Azure Management Portal). You also need a Personal Information Exchange certificate, saved as a .pfx file (referred to as a service certificate in the Windows Azure Management Portal), if you plan to remotely access the VM role instance.

You can self-sign these certificates. In other words, there’s no need to purchase them from a third-party provider. You can create them yourself with the makecert.exe tool. There are detailed instructions for creating your certificates in “How to Create a Certificate for a Role.”

Once you create your certificate, you can add it to the Windows Azure Management Portal. Select Hosted Services, Storage Accounts & CDN, then click Management Certificates or Hosted Services (for service certificates).

You need the subscription identifier to upload the VHD. You can copy and paste the information from the Management Portal to the command line. To find the subscription ID and the certificate thumbprint:

Log on to the Management Portal.

Click Hosted Services, Storage Accounts & CDN, and then click Management Certificates.

In the Properties pane, you can find the certificate thumbprint under Thumbprint. The subscription identifier is under Subscription ID.

Once you find the certificate thumbprint and subscription identifier, you can upload the VHD.

To upload the image:

Open a Windows Azure SDK Command Prompt window as an administrator.

Change the directory to which you saved the VHD image.

Set the connection string with the following command and replace SubscriptionId and CertThumbprint with the values from your subscription:

csupload Set-Connection "SubscriptionId=SubscriptionId; CertificateThumbprint=CertThumbprint"

Upload the VHD with the following command and replace the VHD file name and displayed name:

csupload Add-VMImage –LiteralPath VHDName.vhd –Location LocationValue –Name DisplayedNameInPortal

This will display a dialog box for the Windows Azure VHD Verification Tool. This tool verifies that the OS is in the proper state and ready for uploading to Windows Azure. Click OK to continue the uploading process.

Thank you for taking time to write an interesting solution. However, using proprietary software is out of the question for this particular case, even if it is free in a trial period. It will still at one point cease to be "free". I'd prefer an open source or any other free solution which doesn't include trial period try-outs or similar.

Hi Milan, what you describe sounds like you should be looking to use OpenNebula or Eucalyptus. What do you mean by "a VR server" ?

If you're trying to build a pool of resources which need to be orchestrated (ie, a private cloud), then OpenNebula is for you. See http://opennebula.org/

Depending on how you want to manage these resources, you may like Eucalyptus as well (http://www.eucalyptus.com/) Both are open source toolkits developed over many years.

Finally, many people in "the enterprise" are using OpenStack for building clouds. \

Howerver, I'm intruigued by the architecture you're looking at using (master-slave). Can you explain to me why you need a master/slave setup and why a p2p pool of resources would not be relevant ?

p2p is also acceptable. I suggested ss-m-cs architecture as a starting point. I haven't decided on it exclusively.

I will take a look at suggested solutions.

I need to run a virtual reality server (something similar to 2nd life) which would serve as a virtual environment for more then 15 users accessing through poa enabled clients who would interact in this virtual world with their avatars etc. Normal VR. The serve is demanding and I cannot have new dedicated hardware, so I have to use a network of PCs that are normally used for laboratory and practical lessons for classes at the University I work at.

That is the final requirement. Having a "virtual computer" with heightened performance to run the server.

According to my understand the first step for you is to build the cluster you can use either Rocks or Oscars and once the cluster is up test it using some simple MPI program than you should go to the virtualisation and other stuff.

Milan, after your more detailed description (thank you), I think the answer to your question lies in the actual application that you will be using. It sounds like you will have a central instance which a user will interact with, which will orchestrate a pool of resources in order to render individual users or environments. This is very different to HTC scheduling which something like ROCKS will help you deploy. It is also not a typical MPI application, since there are not atomic tasks to be executed, but more generally a pool of actions (perhaps calling them "threads" is good as an analogy) which need to communicate with each other in unpredictable ways. I would thefore reiterate the need to build a private cloud, but stress the fact that the VM's which you deploy on the cloud should be properly configured before their instantiation to work with the central VR service.

I'm not sure of the othes that I mentioned, but OpenNebula has a VM scheduling mechanism which can allow on-demand deployment of appliances.

I missed one important point previously - as mentioned by Abdullah Yousafzai above, you need to build the cluster (ie, deploy the base OS's on the "bare iron"). For this, depending on your hardware, you can use something that uses IPMI (http://en.wikipedia.org/wiki/Intelligent_Platform_Management_Interface) to remotely install the OS's from a local or remote OS repository. People that I work with largely use puppet (https://puppetlabs.com/)

Deploying the OS on nodes isn't the problem. I will ghost hard drives or boot them off removable devices or find appropriate ways to do this.

The OpenNebula project looks interesting. I've been reading about it yesterday and this morning as well.

Thank you for your suggestions up until now. They have really helped me with identifying different tasks and problems that may happen while making this idea.

I agree that a custom VR application written with MPI or even with raw TCP sockets would be the way to go. It doesn't really matter what OS you install, but Linux is free and fast. The key is finding out how to parallelize your VR application, I would think. Also, why install or use a virtual machine at all? Just run your custom, parallelized VR application on the actual hardware.

I'd like to use a solution that takes care of parallel processing on the system level, so that I do not need to make many changes to the actual application.

So, you have an application already that isn't cluster/grid/etc. based, and you're looking for a systems-level solution that could parallelize it across some desktop/compute nodes, while making as few changes to the application as possible?

If I understand you correctly, I don't think you'll have any luck with that, unless your current application is already parallelized in some way, and it will probably need to be a particular way:

If it is multi-threaded, things will be hard for you, but you may be able to use some kind of "NUMAlink" like tech to get one big system image to keep the threads happy.

If your current application is parallelized at the process level, you will have better luck, but even then you will probably have to change the interprocess communication (IPC) and synchronization methods. It would be easiest, I suppose, if your application processes already communicate over something like named pipes or files, so when they are running on different boxes, you could redirect those communication channels with zero changes to the VR application code.

So much depends on the specifics of your VR application. I honestly doubt anyone here can provide you a good answer without somewhat detailed understanding of the application.

Hey Milan...try Eucalyptus.

It is an environment to build private and hybrid cloud. A very easy tutorial for installation is available over internet.

I am also working on it. May be we can come up with a good results.