Configuring Hadoop Setup and Starting cluster services using Ansible Playbook..!!
Big Data is a big challenging problem which has two major factors having large Volume which is the large size of data and Velocity which is the speed of storing large amount of data in secured way. Not only the large size of data and management of speed but also cost and many more factors as well is a big problem. Hadoop works on Master slave Topology. But to setup all this consume lots of resources and cost.
All this Challenge can be solved by Master Slave Cluster for implementing this a software which introduces HADOOP. Hadoop works on Master slave Topology. So Hadoop is is the one who will create cluster via Networking, using HDFS Protocol which is Hadoop Distributed File System which will help in storing data in a distributed manner. But to setup all this consume lots of resources and cost So in This Automation World it is very much necessary to have a tool which by automation helps in configuring the setup so for automation ANSIBLE can be used which makes easier to configure these services.
Firstly for setting Hadoop so jdk file and hadoop rpm file is required so Ansible yum module can be used and command module will help to install forcefully. After that creating a directory in namenode and update the hdfs-site.xml configuration file and cre-site.xml file using template module. To format and start namenode use command module. Similarly setup datanode. To check that the services are started or not so in namenode and datanode we can use jps command.
All the process is shown here in video and following that we can setup Hadoop Setup using Ansible playbook..!!
Github URL- https://github.com/shrishtikapoor01/Hadoop-Setup-by-Ansible.git
So By this video Hadoop Setup can be easily done with the help of Ansible Playbook which makes the step of configuring faster and simpler..!!
