Services and State with Ambari REST API

  • 时间:
  • 浏览:0

The possible states a service component can be in is listed below. To change or get the state of a service component the following API calls can be issued.

INITInitial/Clean state.

Hive ClientHIVE_CLIENTHIVE1+

7

curl -u admin:admin -H "X-Requested-By:ambari" -i -X PUT 

Application Timeline ServerAPP_TIMELINE_SERVERYARN1

Ranger User SyncRANGER_USERSYNCRANGER1

http://<HOST>:5050/api/v1/clusters/<CLUSTER_NAME>/hosts/<HOST_FQDN>/host_components/KNOX_GATEWAY

Falcon ServerFALCON_SERVERFALCON1

Journale Nodes for NameNode HAJOURNALNODEHDFS0+ (odd #)

Storm SupervisorSUPERVISORSTORM1+

DISABLED <-INSTALLED, INSTALL_FAILED, UNKNOWN

Nagios ServerNAGIOS_SERVERNAGIOS1

NameAmbari Component Name

Kafka BrokerKAFKA_BROKERKAFKA1+

curl -u admin:admin -H "X-Requested-by:ambari" -i -k -X PUT 

Zookeeper Failover ServiceZKFCHDFS0+

// Starting the new component

STOPPINGIn the process of stopping.

7

// Delete service

21

// Delete component

In this post we are going to look a little closer at the way the Ambari API can be used to manage Hadoop services. At the end of this post you will find a list of all the currently supported Hadoop services with all the needed master, slave and client components that can be manged and administrated within your HDP stack. Also this posts contains the possible states and state transitions a component might have which could become useful when facing problems like Host config is in invalid state.

UNINSTALLEDState when uninstall completed successfully.

9

16

3

Among the need to change the state of a service component a very common issue is the need to actually move one service – most likely a master service – to different host. Users faced with this problems often need to issue the existing API due to a lack of support by the existing UI – at least with older releases of Ambari Web UI. This is an example of moving the OOZIE_SERVER to a new host.

8

// Get service info

DISABLEDDisabled master’s backup state.

WebHcat ServerWEBHCAT_SERVERHIVE1

Desired StateStart State

10

ServiceCardinality

Oozie ClientOOZIE_CLIENTOOZIE1+

1

Spark Job History ServerSPARK_JOBHISTORYSERVERSPARK1

HCat ClientHCATHIVE1+

4

Allowed State transitions of a component:

10

23

http://<HOST>:5050/api/v1/clusters/<CLUSTER_NAME>/hosts

// Stop component KNOX_GATEWAY

UNKNOWNState could not be determined.

Sqoop ClientSQOOPSQOOP1+

// Getting the state of the component

Oozie ServerOOZIE_SERVEROOZIE1

7

20

http://<HOST>:5050/api/v1/clusters/<CLUSTER_NAME>/hosts

http://<HOST>:5050/api/v1/clusters/<CLUSTER_NAME>/services/KNOX

24

Removing a complete service from the cluster would require all the removal of all it’s components together with the service itself. For example for removing the Knox service we would need to remove the KNOX_GATEWAY component from the cluster, but before let’s get their state:

-d '{"RequestInfo": {"context": "Start Oozie","query":"HostRoles/component_name.in('OOZIE_SERVER')"}, 

Zookeeper ClientZOOKEEPER_CLIENTZOOKEEPER1+

26

"Body":{"HostRoles": {"state""INSTALLED"}}}' 

ServiceCardinality

/<OLD_HOST_FQDN>/host_components/OOZIE_SERVER

http://<HOST>:5050/api/v1/clusters/<CLUSTER_NAME>/hosts

Possible states of a component:

25

Nimbus ServerNIMBUSSTORM1

22

Ranger Admin ServerRANGER_ADMINRANGER1-3

INSTALL_FAILEDInstall failed.

curl -u admin:admin -H "X-Requested-by:ambari" -i -k -X PUT 

3

http://<HOST>:5050/api/v1/clusters/<CLUSTER_NAME>/services/KNOX/

INIT <-UNINSTALLED, INIT, WIPING_OUT

http://<HOST>:5050/api/v1/clusters/<CLUSTER_NAME>/hosts

curl -u admin:admin -H "X-Requested-by:ambari" -i -k -X DELETE

2

"Body":{"HostRoles": {"state""INSTALLED"}}}' 

curl -u admin:admin -H "X-Requested-by:ambari" -i -k -X GET

/<NEW_HOST_FQDN>/host_components/OOZIE_SERVER

Spark ClientSPARK_CLIENTSPARK1+

// Stop service KNOX

UPGRADINGIn the process of upgrading the deployed bits.

8

11

8

2

// Add component to host

http://<HOST>:5050/api/v1/clusters/<CLUSTER_NAME>/hosts/<HOST_FQDN>/host_components/YARN_CLIENT 

Ranger Key Management ServerRANGER_KMS_SERVERRANGER_KMS1+

ResourceMangerRESOURCEMANAGERYARN1-2

ZookeeperZOOKEEPER_SERVERZOOKEEPER1+ (odd #)

STARTINGIn the process of starting.

Phoneix Query ServerPHOENIX_QUERY_SERVERHBASE0+

The Ambari management tool for Hadoop offers among other handy tools a comprehensive REST API for cluster administration. Logically a cluster is divided into hosts, services and service components. While the UI might not always has support for all needed scenarios sure the REST API can be used to achieve it. For example moving a master component of a service from one host to another.

11

1

16

Storm REST ServerSTORM_REST_APISTORM1

-d '{"RequestInfo": {"context": "Install Oozie","query":"HostRoles/component_name.in('OOZIE_SERVER')"}, 

3

// Install component by achieving stated INSTALLED

http://<HOST>:5050/api/v1/clusters/<CLUSTER_NAME>/hosts

6

UNINSTALLED <-INSTALLED, UNINSTALLED, UNINSTALLING

HBase ClientHBASE_CLIENTHBASE1+

INSTALLEDState when install completed successfully.

-d '{"RequestInfo":{"context":"Install YARN_CLIENT"},

12

5

5

DRPC ServerDRPC_SERVERSTORM1

/<OLD_HOST_FQDN>/host_components/OOZIE_SERVER

http://<HOST>:5050/api/v1/clusters/<CLUSTER_NAME>/services/KNOX

HDFS ClientHDFS_CLIENTHDFS1+

14

http://<HOST>:5050/api/v1/clusters/<CLUSTER_NAME>/hosts/<HOST_FQDN>/host_components/YARN_CLIENT

Secondary NameNodeNFS_GATEWAYHDFS0+

Zeppelin ServerZEPPELIN_MASTERSPARK / HIVE1

4

YARN ClientYARN_CLIENTYARN1+

Cardinality

UNINSTALLINGIn the process of uninstalling.

13

curl -u admin:admin -H "X-Requested-By:ambari" -i -X PUT 

STARTED <-INSTALLED, STARTING, STARTED

Ganglia ServerGANGLIA_SERVERGANGLIA1

2

1

9

INSTALLED <-INIT, UNINSTALLED, INSTALLED, INSTALLING, STARTED, INSTALL_FAILED, UPGRADING, STOPPING, UNKNOWN, DISABLED

DataNodeDATANODEHDFS1+

// Stop the old component by putting it back in state INSTALLED

-d '{"ServiceComponentInfo": {"state": "INSTALLED"}}'

MapReduce ClientMAPREDUCE2_CLIENTMAPREDUCE21+

5

12

Node ManagerNODEMANAGERYARN1+

// Delete old component

INSTALLINGIn the process of installing.

17

STARTEDState when start completed successfully.

/<NEW_HOST_FQDN>/host_components/OOZIE_SERVER

WIPING_OUTIn the process of wiping out the install.

curl -u admin:admin -H "X-Requested-by:ambari" -i -k -X DELETE

4

curl -k -u admin:admin -H "X-Requested-By:ambari" -i -X GET

HBase MasterHBASE_MASTERHBASE1+

curl -u admin:admin -H "X-Requested-By:ambari" -i -X DELETE 

NameAmbari Component Name

17

Secondary NameNodeSECONDARY_NAMENODEHDFS1

/<NEW_HOST_FQDN>/host_components/OOZIE_SERVER

Mahout ClientMAHOUTMAHOUT0+

6

Ambari Metrics ServiceMETRICS_COLLECTORAMS1

Storm UISTORM_UI_SERVERSTORM1

HistoryServerHISTORYSERVERMAPREDUCE2 1

14

6

HBase RegionServerHBASE_REGIONSERVERHBASE1+

19

15

-d '{"RequestInfo": {"context": "Stop Oozie","query":"HostRoles/component_name.in('OOZIE_SERVER')"}, 

curl -k -u admin:admin -H "X-Requested-By:ambari" -i -X PUT 

Ganglia Metrics CollectorGANGLIA_MONITORGANGLIAALL

-d '{"ServiceInfo": {"state": "INSTALLED"}}'

HiveServer2HIVE_SERVERHIVE1-2

NameAmbari Component Name

StateDescription

curl -u admin:admin -H "X-Requested-By:ambari" -i -X PUT 

Hive MetastoreHIVE_METASTOREHIVE1-2

// Setting the state of the component to INSTALLED

Knox GatewayKNOX_GATEWAYKNOX1+

Falcon ClientFALCON_CLIENTFALCON1+

NameNodeNAMENODEHDFS1-2

"Body":{"HostRoles":{"state":"INSTALLED"}}}' 

18

A Hadoop cluster contains of multiple services that individually have either master, slave or client components. Below you will find a list of all currently supported components as part of a HDP stack divided in master, slave, or client groups. The cardinality notates the number of components that can exist at the same time in one cluster.

http://<HOST>:5050/api/v1/clusters/<CLUSTER_NAME>/services/KNOX/components/KNOX_GATEWAY

13

"Body":{"HostRoles": {"state""STARTED"}}}' 

Service

15

Ambari Metrics CollectorMETRICS_MONITORAMSALL

curl -u admin:admin -H "X-Requested-By:ambari" -i -X POST -d