Hadoop cluster configuration 1 | Hadoop Tutorial pdf

Hadoop cluster configuration 1:

1. Login to your ubuntu

2. Start terminal

3. Firt edit your hostname
          command: sudo gedit /etc/hostname
          change name to "master" and save it.

4. command: hostname
          It will give old name, to get new name restart the service

5. command: sudo service hostname restart
          If you get unknown then type below command

6. sudo service hostname start

7. command: hostname
          Now it will show new name called master

8.command: sudo gedit /etc/hosts
          If you get any error then use below command
          $sudo vi /etc/hosts
          If you still get some problem then restart your VM
i.e. logoff and login again it will solve the issue and type below command
          $sudo gedit /etc/hosts

9. comment: ubuntu ip address
          #127.0.0.1 ubuntu
          Now get your IP address using command "ifconfig" and
paste in file and add some name
          eg: 198.126.34.9 master
          Save the file and close the file

10. command: hostname
You will able to see master

Now one machine is read

Now check sshd – it will be use to communicate between master and slave
11. command: which ssh
          output will be /usr/bin/ssh if you get some different output then
sshd is not installed in your machine.
You can install using below command

12. commad : sudo apt-get install ssh
          type Y

13. command : sudo apt-get install sshd

14. Check java is installed or not using command:jps
If not you can use below command to install Java
command: sudo apt-get install openjdk-6-jdk
          Y
Michael noll
(http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster)
15. $ sudo apt-get install python-software-properties
16. $ sudo add-apt-repository ppa:ferramroberto/java
17. $ sudo apt-get update
18. $ sudo apt-get install sun-java6-jdk
19. $ sudo update-java-alternatives -s java-6-sun

Start other instance of machine (yash2Ubuntu)

20. Take the IP address of master using command ifconfig
          e.g. 192.168.139.142
          Ping this IP address in slave machine
          Open terminal CTRL+ALT+T
          $ ping 192.168.139.142
          Should able to ping IP address in slave - require to communicate with master

21. Try to copy the IP address of slave and ping it in master
          yash1Ubuntu
          eg: $ ping 192.160.130.140
          It should ping in master to communicate with slave,
then only you can configure hadoop

22. Open Master(need to communicate through some source)
          command: sudo geedit /etc/hosts
          two things you need
          IP address of Master
          eg (
                192.168.139.142 yeshwanth1.yeshwanthsoft.com yeshwanth1 ->Master
                192.168.139.140 yeshwanth2.yeshwanthsoft.com yeshwanth2 ->Slave
          )
          How many nodes you have can be configured here.
          eg: 192.168.139.143 yeshwanth3.yeshwanthsoft.com yeshwanth3 ->Slave

          IP address of Slave
          Number of slaves will be added here

23. Now go to slave here also need to do same things
          $ sudo gedit /etc/hosts or sudo vi /etc/hosts or sudo vim /etc/hosts
(In vi mode you need to press "Insert key" to insert text)
 Add IP address of mater and slave
          eg (
                192.168.139.142 yeshwanth1.yeshwanthsoft.com yeshwanth1 ->Master
                192.168.139.140 yeshwanth2.yeshwanthsoft.com yeshwanth2 ->Slave
          )
Same IP addresses as present in master

24. Try to ping using name
          $ ping yeshwanth1
          It should able to ping to communicate master

$ sudo apt-get install python-software-properties
$ sudo add-apt-repository ppa:ferramroberto/java
$ sudo apt-get update
$ sudo apt-get install sun-java6-jdk
Y
$ sudo update-java-alternatives -s java-6-sun

25. CTLR+SHIFT+E to open new terminal
How to configure SSH in your system?

26. $ ssh-keygen -t rsa -P "" (it generates password less ssh key)

27. Second, you have to enable SSH access to your local machine with this newly created key.
$ cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys

28. $ ssh localhost
Yes
exit

29. $ ssh master
Yes
          exit

30. $ sudo opt-get install sun-java6-jdk
Y

31. Now goto master "yeshwanth1"
          $ ssh yeshwanth2
          If will ask for password.
          For password less configuration you need to do below things
          open HOME folder and press CTLR+H it shows hidden files.
          Open .ssh folder
          Open id_rsa.pub and copy the text

32. Now goto slave "yeshwanth2"
          open HOME folder and press CTLR+H it shows hidden files.
          open .ssh folder
          open authorized_key file
          Copy the public key of master

33. $ ssh yeshwanth1
          It will ask for password
          For password less configuration we need to Copy the public key(id_rsa.pub)
          of slave and past in authorized key(authorized_key) of master.

34. $ ssh yeshwanth2
          Now you can able connect master to slave without any password.