What is a NodeManager
A NodeManager agent is deployed on each of the processing nodes in the cluster. It is the counterpart to the ResourceManager at the node level. It communicates with the ResourceManager to update the node state and receive any job requests from it. It is also responsible for the lifecycle management and the reporting of various node metrics to the ResourceManager.
Decommissioning is a process that supports removing a component from the cluster. You must decommission a master or slave running on a host before removing the component or host from service. Decommissioning helps prevent potential loss of data or service disruption. Decommissioning is available for the following component types:
- DataNodes
- NodeManagers
- RegionServers
Decommissioning executes the following tasks:
- For DataNodes, safely replicates the HDFS data to other DataNodes in the cluster.
- For NodeManagers, stops accepting new job requests from the masters and stops the component.
- For RegionServers, turns on drain mode and stops the component.
Pre requisites
Make sure the configuration file for YARN i.e. /etc/hadoop/conf/yarn-site.xml should have the following property set:
Decommissioning NodeManager – Command Line
The steps to decommission a NodeManager is almost similar to removing a datanode. The file used here to include the NodeManager nodes to remove is /etc/hadoop/conf/yarn.exclude. Follow the steps outlined below to remove a NodeManager dn3.localdomain.
1. Login to the ResourceManager Node (dn2.localdomain in our case).
# ssh yarn@dn2.localdomain
You can find out the ResourceManager node from the YARN service page in amabri dashboard.
2. Add the NodeManager node you want to decomission in the file /etc/hadoop/conf/yarn.exclude
# vi /etc/hadoop/conf/yarn.exclude dn3.localdomain
3. Update the ResourceManager with the new set of NodeManagers. On the ResourceManager host (dn2.localdomain) machine, execute the following command with the user yarn:
[root@dn2 conf]# su - yarn
[yarn@dn3 ~]$ yarn rmadmin -refreshNodes 18/07/15 20:44:59 INFO client.RMProxy: Connecting to ResourceManager at dn2.localdomain/192.168.1.4:8141
4. Verify the decommission of NodeManager on the YARN services page.
Decommissioning NodeManager – Using Ambari-Server Web UI
1. To delete the NodeManager dn3.localdomain, goto the hosts page from the ambari dashboard and select the NodeManager node to be decommissioned.
Delete the NodeManager using the Actions dropdown as shown below.