I have installed hadoop in a single machine, can anyone suggest any open source methods to create data nodes.
"Create"? I'm guessing what you really mean is "I have my first cluster node; how do I clone it to produce more nodes". I'm reasonably fond of OneSIS as a mechanism for producing clusters of nodes - basically it lets you customize any number of nodes derived from a readonly NFS-root. But there are many other ways to structure a cluster - kickstart-based installs, some sort of image copying, or simple low-tech manual installs.
If this doesn't address your question, perhaps you could provide more detail.
Thanks Mark Probably i m using the wrong word "Create", I have installed hadoop in a system and i treat it as server, for clustering i have installed in another machine also, by chaning in host file i have included the IP of another machine, is this the correct way?
I'm not really sure what you're asking. Adding a host to a host file merely provides a name-IP mapping, for software that needs it. (You could do this with DNS records instead if you have access to your domain's DNS server.) Most software doesn't actually require hostnames, if you don't want to bother though: using IP addresses directly may be just as convenient, especially if you're using a private range. (For instance, 10.*.*.* is private, so you might choose to make all your data nodes 10.0.1.* and but other nodes on 10.0.0.*.
I found Cloudera's CDH distribution one of the easiest way to install Hadoop on multiple machines. The basic version is free, but not open-source.
I have already installed Hadoop and hive in two machine, now I would like to make one machine as name node and one as data node
Refer Michael Noll blog for multinode cluster setup... You can attach multiple data nodes by just changing hosts file... it's easy and interesting... I am attaching both the files for your reference..( Single-Node as well as Multi-Node Cluster)...
- Badges
- Science topic